News Shogun

Extension API

The overview page is a necessary prerequisite to understanding this page.

Index:

• LIBconf
• LIBsubst
• LIBpasswd
• LIBfilt
• LIBheader
• LIBtrack

LIBconf

LIBconf is intended as a general-purpose NDBM-like interface into a text file, although the dbm base has been augmented a bit by used and unused keywords, and much better diagnostics (and decremented by not having functions to set values for a key or remove individual keys).

It relies on no global variables, or even global postitions within its state variable, so it is very friendly to multithreaded applications, however it stores all data in a linear linked list, so searching for a keyword is a much more expensive ordeal than it should be.

Data types:

typedef struct {} CONF;  /* CONF is completely abstract */

typedef struct _CONFkeyword {
    char *keyword;
    char *value;
    int lineno;  /* Line in the config file where this keyword was loaded */
    int used;    /* has this keyword been reserved? */
} CONFkeyword;

typedef int (*CONFloadfunc)(char*, char*, char*, CONF*);

typedef struct _CONFkeyload {
    char *name;
    CONFloadfunc func;
}

Functions:

CONF *CONFload(char*)

Load keywords and values from the filename passed in.

void CONFsetload(CONFkeyload*, int)

Sets the hook functions for loading to the array of keyloaders passed in with the size.

void CONFfree(CONF*)

Free memory associated with a CONF in memory.

void CONFkeywordused(CONF *conf, char *keyword)

Marks keyword in CONF conf as used

char *CONFkeywordvalue(CONF *conf, char *keyword)

Returns the value associated with keyword in conf. Marks the keyword as used.

CONF *CONFcheckkeywords(CONF *conf)

Returns a copy of all unused keywords in conf.

CONFkeyword *CONFfirstkey(CONF *conf)

Returns the first keyword associated with conf.

CONFkeyword *CONFnextkey(CONF *conf, CONFkeyword *key)

Returns the keyword after key in conf.

char *CONFfilename(CONF *conf)

Returns the filename from which conf was loaded.

To register a hook function for loading parameters, allocate an array of parameter loaders and install them with CONFsetload(arr, sizeof(arr));. Every time the CONF runs into a name with a parenthesis in it, it searches for the string before the parenthesis in all the name elements in the array, calling the associated func with the name we've just found, the string between the parentheses, the value after the parentheses and the CONF that CONFload is working on.

back to top

LIBsubst

This is the news gizmo's text substitution library.. It is invoked on request, and is not used intrinsically by the config library. It parses text looking for strings of the format <%ID>, and when it runs into them, it calls a function associated with that ID with a return string. If no function associated with that ID exists, it looks for a variable with that ID set by a couple of functions in the program. If no variable was set by the program, it searches through a list of global variables to find the string. Functions have the prototype:

typedef void (*SUBSTfunc)(char *name, char *ret, size_t len, FILT *data);
typedef size_t (*SUBSTlenfunc)(char *name, FILT *data);

where name is the ID string referred to above, and ret and len are where the replacement string goes and how much space it's got. data gets sent when one of the functions below gets called.

The array substfuncs contains every ID and SUBSTfunc known to the library.

Functions:

void SUBSTgetvalue(char *id, char *ret, size_t len, FILT *data)

Invokes the SUBSTfunc associated with id in the array, the output goes into ret, and is no longer than len bytes long. data gets forwarded to the SUBSTfunc.

size_t SUBSTgetlength(char *id, FILT *data)

Invokes the SUBSTlenfunc associated with id to get the string length of the string SUBSTgetvalue would return. If there is no SUBSTlenfunc associated with id, it will invoke SUBSTgetvalue on a fairly large buffer and return the length of the string placed into the buffer. data gets forwarded to the SUBSTlenfunc.

void SUBSTchangetext(char *input, char *output, size_t len, FILT *data)

the input string is searched through for strings of the format described above, invoking SUBSTgetvalue on every one it runs into, and returning the resultant string (no longer than len) in output. data gets forwarded to SUBSTgetvalue.

size_t SUBSTlengthtext(char *text, FILT *data)

Same as above, except invokes SUBSTgetlength rather than SUBSTgetvalue.

extern void SUBSTaddsvar(FILT *filt, char *key, char *value)

Sets the text string <%key> to value during substitution.

back to top

LIBpasswd

This bit of the package has actually been revised a bit in the current version of the news gizmo. It will not only verify a password, but will also attempt to get a username without using a password. Provided authentication methods that don't use passwords are identd, and stripping a suffix from the remote host's hostname.

Password authentication in the standard INND distribution sucks. It allows people to either store their passwords in cleartext in a file viewable by more than root, or that the users are in the password file/database on the news machine. This library allows for extensible password authentication in INND, with support for multiple schemes of password authentication, and parameter passing to these schemes.

This modifies the format of the nnrp.access file (PATH_ACCESS in your config.dist) such that when the username field is '+', it is interpreted as a request to perform some non-standard kind of authentication. The precise kind of authentication is dependant on the password field.. IOW, the nnrp.access file now looks like this:

*:PR:+:PW(/path/to/passwd_database):*,!private.groups.*

where the PW in the password field is interpreted as a request to look in the passwd(5) format file between the parentheses.

Datatypes:

typedef int (*PERMauthfunc)(char *name, char *user, char *password, char *parameter, char *realuser, size_t len);

An authorization function. The 'name' parameter is the invocation name from the nnrp.access file, the 'user' and 'pass' parameters are the username and password of the user trying to authenticate (null if the user hasn't entered them), and the 'parameter' parameter is anything between the parantheses in the nnrp.access file. The 'realuser' parameter is the resolved username of the connecting user.

Functions:

extern int PERMcheckpassword(char *user, char *pass, char *param, char *defdomain, char *realuser, size_t len);

'param' is the actual password field in the nnrp.access file, 'user' and 'pass' retain the obvious meanings. 'realuser' is the username PERMcheckpassword has resolved, and is a memory buffer of size 'len'. 'defdomain' is the default domain name for a user. This can be overridden by the authentication functions, or by a possible suffix to a scheme function.

These functions return a constant defined in "passwd.h" of either

kPERMUserBad

The user is pretending to be someone else.

kPERMCantAuthenticate

Could not authenticate the user

kPERMUserOK

Properly authenticated the user, but did not find their namespace

kPERMUserWithDomainOK

Authenticated the user, found his namespace.

The function can indicate when it should be called by ORing together the following constants:

kPERMWantsPass

Invoke this function when a password is provided

kPERMWantsNoPass

Invoke this function when no password is provided

kPERMWantsAll

Always invoke this function

back to top

LIBfilt

The per-connection information API.

This part of the package defines an abstract data type which stores all the per-connection information for users. A FILT* is a list of data associated with a data type and a name. There can never exist two elements in the list with the same name and data type.

It also allows you to define special deletion functions to ensure that you never get memory leaks when something gets overwritten or deleted from the list.

In preparation for a feature I'd like to add soon (where certain limits apply only under certain circumstances), the FILT* now has a notion of a parent FILT*, from which it inherits or overrides data.

Data types:

typedef struct _FILT {} FILT; /* abstract FILT type */
typedef void (*FILTdelfunc)(int, void*, size_t); /* function to free a user
						  * defined data type */

Constants:

#define kFILTinttype	-1
#define kFILTstrtype	-2
#define kFILTfiletype	-3
#define kFILTfilttype	-4

Functions:

extern void FILTadddata(FILT*, int, char*, void*, size_t, int);
extern void *FILTgetdata(FILT*, int, char*, size_t*, int);
extern int FILThasdata(FILT*, int, char*, int);
extern void FILTdeldata(FILT*, int, char*, int);

extern void FILTaddstring(FILT*, char*, char*, int);
extern char *FILTgetstring(FILT*, char*, int);
extern int FILThasstring(FILT*, char*, int);
extern void FILTdelstring(FILT*, char*, int);

extern void FILTaddint(FILT*, char*, int, int);
extern int FILTgetint(FILT*, char*, int);
extern int FILThasint(FILT*, char*, int);
extern void FILTdelint(FILT*, char*, int);

extern void FILTaddfile(FILT*, char*, char*, FILE*, int);
extern FILE *FILTgetfile(FILT*, char*, int);
extern char *FILTgetfname(FILT*, char*, int);
extern int FILThasfile(FILT*, char*, int);
extern int FILTisopenfile(FILT*, char*, int);
extern void FILTclosefile(FILT*, char*, int);
extern void FILTdelfile(FILT*, char*, int);

extern void FILTaddfilt(FILT*, char*, FILT*, int);
extern FILT *FILTgetfilt(FILT*, char*, int);
extern int FILThasfilt(FILT*, char*, int);
extern void FILTdelfilt(FILT*, char*, int);

extern void FILTadddel(int, FILTdelfunc);

Everything should be basically self-explanatory.. The deletion functions are not associated with any particular connection, so FILTadddel modifies a global deletion function list. The last argument two the "get", "has" and "del" functions is a parameter indicating that we should also search the parent FILT*. The last argument to the "add" functions indicates that we want to add the data to the root FILT*.

There are also a few functions for initializing and destroying a connection, and higher level functions for accessing the current user:

int FILTinit()

Loads configuration and sets up global state.

FILT *FILTconnect(FILT *parent)

Allocates storage for and initializes the per-connection data structure. Inherits data from parent.

void FILTdestroy(FILT *filt)

Frees all the memory associated with filt

char *FILTgetuser(FILT *filt)

Returns the user associated with filt as set by FILTsetuser.

void FILTsetuser(FILT *filt, char *newuser)

Sets the user associated with filt to newuser.

LIBheader

This part of the package finds headers associated with the current post. It searches through a list of headers looking for the requested header, and if it can't be found calls a hook function to load the header. There is a function for inserting the header into the list, and for clearing out all previously loaded headers.

Data types:

typedef void (*FILTheadproc)(FILT*, char*);

Functions:

extern char *FILTgetheader(FILT*, char*);
extern void FILTaddheader(FILT*, char*, char*);
extern void FILTclearheaders(FILT*);
extern void FILTsetheadproc(FILTheadproc);

Most of these should be fairly self-explanatory. The FILTheadproc gets called when a header isn't in the local list but we still want to find the header. It should call FILTaddheader with the name of the header and the text.

LIBtrack

The general-purpose post tracking/blocking engine. What all the code was based around, really..

You can only extend this package in terms of what you can look for and block from posted messages. We provide functions for tracking posts and for blocking posts and crossposts. Tracking and detecting are intrinsically linked, so the same function is used for both (you can indicate you'd like to have some peristant storage in your function). This will not try to do more than it can.. Since we don't want to impose any more restrictions than we have to on the format of database entries, and since we've got Granny Larity in our database (24 hours data stored, refreshed every twelve), we give you the information you've stored in an array (currently there are two time periods, eventually that number may be configurable. There is a function to return the number of time periods.).

The format of the spam.limit file is:

username:scheme(parameter),scheme(parameter) ...

Where the schemes are defined in an array TRACKparams in the file 'track.c' and the format of parameters is defined by the scheme.

Storage in the post database follows the same format, but obviously the format or value of the parameters doesn't have to be the same. This allows us to track several different kinds of data (although presently only posts are tracked persistantly, there's nothing but size of the database entry stopping you from tracking actual newsgroups, or number of times the user has crossposted, or whatever you want), and not strictly define the format of tracked data.

Datatypes:

typedef int (*LIMparamfunc)(FILT *filt, char *name, char *value);
typedef void (*LIMdetectfunc)(FILT *filt, LIM *lim, char *name);
typedef void (*LIMsyncfunc)(FILT *filt, LIM *lim, char *name);

typedef struct _LIMent {
    char *param;	/* name of this tracker in the database */
    LIMparamfunc pf;	/* load configuration from a string */
    LIMdetectfunc df;	/* detect breaches of posting policy */
    int loaded;		/* has this been loaded from database before.
                         * Initialize to 0 */
} LIMent;

Functions:

void LIMloaduserlimits(FILT *filt)

Load limits for user 'user' from the value("limitfile") file.

void LIMtrackpost(FILT *filt)

Track post data from a user.. Hooks into all the detect functions.

Persistant storage functions:

char **LIMdata(LIM *lim, char *name)

Gets the old time-data associated with 'name' and the current user from the database

char *LIMnewdata(LIM *lim, char *name)

Gets the time-data that will be comitted to the directory after the synchronization functions are called.

void LIMsetdata(LIM *lim, char *name, char *data)

Sets the new time-data associated with 'name' and the current user to 'data'.

void LIMcleardata(LIM *lim, char *name)

Clears the new time-data associated with 'name' and the current user.

char *LIMstatdata(LIM *lim, char *name)

Gets the static (doesn't get refreshed) data associated with 'name' and the current user from the database.

char *LIMnewstatdata(LIM *lim, char *name)

Gets the data that will soon go into the database.

void LIMsetstatdata(LIM *lim, char *name, char *data)

Sets the static data associated with 'name' and the current user to 'data'.

void LIMclearstatdata(LIM *lim, char *name)

Empties out the static data associated with 'name' and the current user.

void LIMaddsync(LIM *lim, char *name, LIMsyncfunc sf)

Adds a synchronization function, that's called after all the detection functions are called, but before the database gets modified, and allows you to resolve requests for the same data, or abort your commitment to the database pending other circumstances.

void LIMblock(LIM *lim, char *name)

Indicates that the post should be blocked, and that name is what's blocking it.

void LIMclearblock(LIM *lim, char *name)

Indicates that we've changed our mind, and the post should not be blocked by this filter any more.

int LIMisblock(LIM *lim, char *name)

Returns a constant indicating if the filter associated with name blocked the post, and/or another filter did.

To store persistant post data, you will want to hook into the storage API defined above to put data into the database.

back to top