Managing local e-mail delivery (211.2)

Candidates should be able to implement client e-mail management software to filter, sort and monitor incoming user e-mail.

Key Knowledge Areas

procmail configuration files, tools and utilities

Usage of procmail on both server and client side

Terms and Utilities

  • ~/.procmailrc

  • /etc/procmailrc

  • procmail

  • mbox and Maildir formats

Procmail

Procmail is a mail filtering utility that may be used for preprocessing and sorting of incoming mail. It can also be used to sort out mail from mailinglists, to filter spam and send auto-replies. The configuration of procmail is based on a file placed in the user's homedirectory. Procmail is rarely run from the command line (except for testing purposes) but it's an autonomous program which is normally invoked by MTA's (Mail Transport Agent) like Sendmail or Postfix.

Procmail follows the following scheme for reading its configuration (it reads both):

/etc/procmailrc
~/.procmailrc

Be careful using the system-wide /etc/procmailrc. It is usually read and processed as root. This fact means that a poorly designed recipe in that file could do serious damage. For instance, a typo could cause Procmail to overwrite an important system binary rather than use that binary to process a message. For this reason, you should keep system-wide Procmail processing to a minimum and instead focus on using ~/.procmailrc to process mail using individual accounts.

If Procmail isn't working try creating a .forward file in your homedir with | /path/to/procmail in it.

The Procmail configuration files (/etc/procmailrc and ~./procmailrc) contain Procmail recipes, which take the following form:

	:0 [flags] [:[lockfile]]
	[conditions]
	action
			

Each recipe begins with the string :0. Various flags may follow and can be combined.

Flags

Here are some of the most important flags of procmail. See also the man pages or use procmail -h to get an overview of all possible flags.

  • H - Matching is done to the message headers. (This is the default.)

  • B - Matching is done to the message body.

  • D - Matching is done in a case-sensitive manner. (The default is a case-insensitive match.)

  • c - Matching is done on a carbon copy of the message. The original is passed on for matching against subsequent recipes. This flag is generally used within nesting blocks.

  • w - Procmail waits for the action to complete. If it doesn't complete successfully, the message is matched against subsequent recipes.

  • W - The same as a flag of w, but it suppresses program failure messages.

Lockfile

The lockfile is the name of a file that Procmail uses to signal that it's working with a file. If Procmail sees a lockfile, it delays work on the affected file until the lockfile disappears. Ordinarily, a single colon (:) suffices for this function; Procmail then picks a lockfile name itself. You can specify a filename if you prefer, though.

Conditions

The conditions in a Procmail recipe are essentially ordinary regular expressions, but each conditions line begins with an asterisk (*). Most characters in a regular expression match against the same characters in the message, but there are exceptions:

  • ^ - A caret denotes the start of a line.

  • . - A dot matches any single character except for a new line.

  • .* - The combination of a dot and an asterisk denotes a string of any length.

  • ( | ) - The use of parentheses, with a vertical bar within it, denotes a match against the string on either side of the vertical bar.

  • \ - A backslash effectively undoes special formatting in the following character; for instance, to match an asterisk, you would specify the string \*.

  • ! - An exclamation mark reverses the sense of a match so that a recipe matches any message that does not meet the specified criteria.

Each recipe can have no, one, or several conditions. Using no conditions is usually done within nesting blocks or when experimenting with new recipes. If a recipe includes more than one condition all must match for the recipe to apply. Refer to the procmailrc man page for more information.

Actions

A Procmail recipe ends with a single line that tells it what to do - the action. An action line may be any of several things:

  • Filename Reference - The message will be stored in the specified mailbox name, either a file (mbox) or a directory (maildir). If it is a file the mail will be appended to it. To store messages in the maildir format, append a slash (/) to the end of the name. /dev/null can be used as well.

  • External Program - If the action line begins with a vertical bar (|), Procmail treats the line as a program to be executed.

  • Email Address - An exclamation mark (!) at the start of a line denotes an email address. Procmail sends the message to the specified address instead of delivering it locally.

  • Nesting Block - An action line that begins with an open curly brace ({) denotes a nested recipe. The nested recipe takes the same form as any other recipe, but it is used only if the surrounding recipe matches the message. The nested recipe ends with a close curly brace (}).

Read the procmail and procmailrc man pages to learn more about configuration options. The procmailex man page gives loads of examples.

Examples

A simple .procmailrc example:

	SHELL=/bin/sh
	MAILDIR=$HOME/Mail
	LOGFILE=$HOME/Mail/procmail.log

	:0:
	* ^Subject: test
	testing
				

The first three lines are just common parameters, not procmail specific. In this example mail with the subject test is placed into the directory testing. The :0: falls apart in three parts. The first colon indicates the start of a new recipe. The 0 is for historical reasons. The second colon means to use file locking where procmail will choose a lockfile name by itself. You have to use this when saving mail to a file. In this case the mail will be written into the directory testing.

The * indicates the beginning of a condition. The condition itself is a regular expression. This condition is matched against the headers of the mail. The last line testing has no special character. In this case, the simplest form of delivery is shown, saving the mail to the subfolder testing.

Next a slightly more complicated example, used to sort out mail messages sent from a mailing list called owner-info@listserv.com. In this example the filtering is on two conditions. The mail must be sent from owner-info@listserv.com and the subject must end with kernel info. When both conditions are met, the message is saved in a folder called maillist.linux.

	:0:
	* ^From owner-info@listserv\.com
	* ^Subject:.*kernel info
	maillist.linux  
				

Here is another example. In this case, mail, which meets the conditions, is forwarded to a secretary and saved in a specific folder of the destinated original user. Mail from the domain microsoft.com with the subject ending with licenses is forwarded to secretary@somewhere.edu. With the second rule the mail which meets the two conditions is saved in the folder or file forwarded.secretary.

	:0c   
	* ^From.*@microsoft\.com
	* ^Subject:.*licenses
	! secretary@somewhere.edu

	:0:   
	forwarded.secretary
				

In the next example you can see a nested action. All mail from peter about compilers is forwarded to william (and a copy of it is kept in petcompil).

	:0
	* ^From.*peter
	* ^Subject:.*compilers
	{
		:0 c
		! william@somewhere.edu

		:0
		petcompil
	}
				

Mbox and maildir storage formats

Mbox and maildir are e-mail storage formats. Postfix and Dovecot support the two mail storage formats where maildir is the recommended format.

Mbox format

Mbox is the traditional email storage format. In this format there is only one regular text file which serves as the mail user's mailbox file is created. Typically, the name of this file is /var/spool/mail/<user name>. Mbox locks the mailbox when an operation is performed on the mailbox. After the operation the mailbox is unlocked.

Advantages:
  • Mbox format is universally supported

  • Appending new mail is fast

  • Searching inside single mailbox is fast

Disadvantages:
  • Mbox is known for locking problems

  • The mbox format is prone to corruption

Maildir format

Maildir is the newer e-mail storage format. A directory maildir is created for each mail user, typically in the users' home directories. Under this maildir directory by default three more directories exist: new, cur and tmp.

Advantages:
  • Locating, retrieving and deleting a specific mail is fast, particularly when a mail folder contains hundreds of messages

  • Minimal to no file locking needed

  • Can be used on a network file system

  • Immune to mailbox corruption (assuming the hardware will not fail)

Disadvantages:
  • Some filesystems may not efficiently handle a large number of small files

  • Searching text, which requires all mail files to be opened is slow

Recipe differences between mbox and maildir for procmailrc

Before copying the recipes from this page into your procmailrc file, remember to adapt them to your particular maildir/mbox format, taking into consideration that the name of maildir folders end in "/". You do not need to lock the file when using maildir format (:0 instead of :0:).

In mbox the format is:

	:0:
	recipe
	directory_name
			

While in maildir it would be:

	:0
	recipe
	directory_name/
			
Copyright Snow B.V. The Netherlands