This software is currently in use on our servers for stopping spam and for making us more sound on 2senet. The current version is: [an error occurred while processing this directive]
In order to be a sound site on Usenet-2, an ISP has to be very diligent tracking posts from its users. It should be impossible for spam to come from its users, however spam detection software tends to be iffy at best. Mostly it relies on searching for strings in posts like "$$$", or having a list of spammers that you can block. The first method suffers because it is computationally expensive, and because it is not absolute. The second suffers because knowing your spammer usually means your spammer has sent out 20,000 posts and your newsmaster has had his mailbox flooded by thousands of angry usenetters.
This software relies on only one basic assumption for detection of spam: that spammers want their message seen as widely as possible.
What this software does is authenticate and track posts from a user. It is pure C code, so it is not as computationally expensive as spawning an entire perl process for post detection, and is fairly easy to integrate into any kind of data-transmission software. It is very extensible, and can persistantly store in a user's entry in a database any relevant post information for this user.
This package also comes with several extensions which, when put in place, basically ensure that your site can be Usenet-2 ``sound'' with no further effort on the News administrator's part required.
This package basically comes in two parts:
That all said, here's how it works:
When a user connects to the news server, an attempt is made to authenticate the user without requiring a password.. If this is not possible, the remote user must give a password before attempting to post. Once the user is authenticated, a bunch of post-related limits for the user are loaded.. These can include: hierarchies between which this user's crossposts can't occur, number of groups between which crossposts (or followups) can occur in a single message, total number of messages posted, &c.
The package isn't used again until the user attempts to post a message.
When the user attempts to post a message, post monitors related to the limits are invoked on the message. If anything about the user's post runs contrary to the administration's posting policy, the post is blocked and a descriptive error message is relayed to the user.
The post can also be logged to a holding directory, where an external daemon will run into it and send it back to the news server. This can happen, for example, when the user has posted enough times that you want to see his posts, but few enough times that you don't want to block it indefinately.
Mail can be sent back to the user and to the administrators which will describe the reason the post got slowed, and how to get the limit raised.
This package comes with several modules for user authentication/resolution, and for post tracking and blocking. Here is a brief list:
For the package to be worth anything at all, we have to be able to figure out precisely who the user at the other end of a connection is. This is configurable on a per-host basis, and every authentication scheme can take arguments which will help them to resolve the username.
Password file:
This authentication method requires both a username and a password. When
the user enters a username and password, the username is looked up in a
configurable password file which has lines of the format:
username:crypt(3)'ed password:ignored
where everything after the password is ignored.
If the encrypted password matches the password in the database, authentication succeeds.
Password database:
This is precisely the same as the Password file method
described above, except that the username is looked up in a database
where the key is the username, and the value is the crypt'ed password.
hostname
This resolution method does not take a password. What it does is search
for a suffix in the remote host's hostname and strip that suffix off.
The resulting string is interpreted as the username of the remote user.
remote ident server
This resolution method does not take a password. It will contact the
ident server of the remote host and ask it who the user talking to us is.
If the ident server sends back encrypted output, attempt to decrypt it
using a keyfile, and look up the resultant uid in a uid-to-username
database. This becomes the name of the remote user.
Now that we know who the remote user is, we have to figure out what kind of auditing and limiting we want to apply to their connection.
Total number of posts
This monitor will check the total number of messages this user has sent
out over a 24-hour period, and if they are greater than a certain number,
a message "User has exceeded posting limits" is returned to the user next
time they attempt to post. Post data is stored persistantly in a post
database, so this is not thwartable by the user resetting their connection
and posting in a new one.
Number of crossposts
This monitor will check the number of newsgroups in the newsgroups line,
and if they are greater than a certain number, the post is blocked and
the message "Crossposted to too many groups" is returned to the user.
Number of followups
This monitor will check the number of newsgroups in the Follow-up line
(if it exists, otherwise uses the Newsgroups line) of the message, and if
they are greater than a certain number, the post is blocked and the error
"Followups set to too many groups" is returned to the user.
Exclusive hierarchies
This monitor will go through the newsgroups in the newsgroups line, and if
it turns out that all the newsgroups don't fall into a defined hierarchy,
the message "Crossposted between mutually exclusive hierarchies" is
returned to the user. There is an implied "rest" hierarchy, which
consists of newsgroups not defined in the other hierarchies.
Emily Postnews
This monitor will look in the user database for entries specifying
posting permissions on certain hierarchies. If any newsgroups posted to
are explicitely disallowed, the message
"You don't have posting permission in %s" (where %s is replaced by the
newsgroups you've attempted to post to but aren't allowed). If there are
newsgroups that aren't explicitely accounted for, it will go through a
list of hierarchies and associated rules files looking for intersections
with the unaccounted for groups. If any hierarchies match, the same error
message as above is generated with the groups that have rules appended to
the %s part. The rule file associated with these hierarchies gets sent
out, with the text of your message suffixed.
We've finished writing a patch to the innd transfer server which will modify the format of hosts.nntp such that you can outright reject an article if it's destined for a particular hierarchy. Russ Allbery modified it, and here is his description of his modifications:
I didn't want the poison patch for hosts.nntp (and in fact exclusions in hosts.nntp in general) to result in entries into the history file even when REMEMBER_TRASH is set to yes. This is because, with Usenet II, one may receive sound articles via unsound paths before sound paths.
This patch is now a part of recent INN-2.0 beta snapshots. If you are running a post May 22 INN, you don't need to download this patch.
Find it at ftp://ftp.panix.com/pub/gizmo/poison.tar.gz .