Chapter 11. E-Mail services (211)

This topic has a weight of 8 points and contains the following objectives:

Objective 211.1; Using e-mail servers (4 points)

Candidates should be able to manage an e-mail server, including the configuration of e-mail aliases, e-mail quotas and virtual e-mail domains. This objective includes configuring internal e-mail relays and monitoring e-mail servers.

Objective 211.2; Managing local e-mail delivery (2 points)

Candidates should be able to implement client e-mail management software to filter, sort and monitor incoming user e-mail.

Objective 211.3 Managing remote e-mail delivery (2 points)

Candidates should be able to install and configure POP and IMAP daemons.

Using e-mail servers (211.1)

Candidates should be able to manage an e-mail server, including the configuration of e-mail aliases, e-mail quotas and virtual e-mail domains. This objective includes configuring internal e-mail relays and monitoring e-mail servers.

Key Knowledge Areas

Configuration files for postfix

Basic knowledge of the SMTP protocol

Awareness of sendmail and exim

Terms and Utilities

  • Configuration files and commands for postfix

  • /etc/aliases

  • /etc/postfix/

  • /var/spool/postfix/

  • sendmail emulation layer commands

  • mail-related logs in /var/log

Basic knowledge of the SMTP protocol

Since April 2001 RFC2821 is in use for describing the SMTP protocol. This RFC obsoletes RFC821, RFC974 and updates RFC1123.

When an SMTP client has a message to transmit, it establishes a two-way transmission channel to an SMTP server. The responsibility of the SMTP client is to transfer it to one or more SMTP servers, or to report the failure to do so.

The means by which an e-mail message is presented to an SMTP client, and how the domain name to transfer the message to is determined, is a local matter that will not be addressed here.

Detailed information can be found at: RFC2821.

For demonstration and testing purposes an SMTP session can be held with telnet:

	telnet mx1.unix.nl 25

	Connected to mx1.unix.nl (213.154.248.146).
	Escape character is '^]'.
	220 mx1.unix.nl ESMTP Exim 4.63 Thu, 12 Aug 2010 08:50:30 +0200

	ehlo snow.nl

	250-mx1.unix.nl Hello mx1.unix.nl [81.23.226.83]
	250-SIZE 52428800
	250-PIPELINING
	250-AUTH PLAIN LOGIN
	250-STARTTLS
	250 HELP
		

Using telnet <smtp servername> 25 initial contact with the SMTP server is established. In the example above mx1.unix.nl was used as the server. The server responds with the maximum message size, the pipelining, and the authentication capabilities of the server, in this case AUTH PLAIN LOGIN. The server is also able to use STARTTLS, and HELP.

  • SIZE: when the size of the message the client wishes to send exceeds the set limit, the SMTP transaction will be aborted with an ERROR code pipelining. Also, when enabled, an SMTP client can transmit a group of SMTP commands (e.g., RSET, MAIL FROM:, RCPT TO:) without waiting for a response from the server.

  • AUTH PLAIN LOGIN: the SMTP server is capable of handling a plain password/username authentication. This can be handy for mobile devices using the SMTP server while originating from different IP addresses.

  • STARTTLS: The SMTP protocol itself doesn't include any form of encryption. With STARTTLS the communciation can be encrypted using certificates which is fully described in RFC3207.

Since we have established the initial connection we can now proceed:

	mail from: auser@snow.nl

	250 OK

	rcpt to: info@unix.nl

	250 Accepted
			

The server responds with 250 OK but when, for example, the MAIL FROM: command is followed by a misformatted or incomplete email address the server responds with 501: sender address must contain domain or a similar error. With RCPT TO: <email address> the destination of the message is given. If the recipient's address is accepted the server responds with: 250 Accepted.

Now the mailserver knows who we are and to whom we wish to transmit a message. The email client also knows what the SMTP server is capable of. We then proceed with the transmission:

	data

	354 Enter message, ending with "." on a line by itself

	mail from: Aiu <auser@snow.nl>
	From: <auser@snow.nl>
	To: Info <info@unix.nl>
	Subject: Demo messages
	Date: 12-02-2010 08:53:00
	MesssageID: 31231233@snow.nl

	This is a demonstration mesage.
	.

	250 OK id=1OjReS-0005kT-Jj

	quit
		

With the DATA command we proceed with the contents of the message. The server responds with: 354 Enter message, ending with "." on a line by itself.

Then the message contents is entered, starting with the message-headers. These headers are used by the email client. In this example the commands Mail From:, To:, Subject:, Date: and Message-ID: are used. Message-ID: is an unique identifier generated by the SMTP client. These headers are required as described in RFC822.

After the headers have been entered, a blank line indicates that the actual text of the message commences. The message ends, as the DATA command response has already indicated, with a . (without the quotes) on a line by itself. The server responds with: 250 OK id=1OjReS-0005kT-Jj. The id is a unique SMTP server identifier and can be used for troubleshooting.

Note

When fighting spam, some SMTP servers can check whether a message is RFC822 compliant. Regular SMTP clients are RFC822 compliant, but spammers often use a not so regular SMTP client and thus can send malformed messages.

Sendmail

Basic sendmail configuration steps are:

  1. Download Sendmail

  2. Set up Sendmail

  3. Configure Sendmail

  4. Build the Sendmail User Table

  5. Add your domain names to Sendmail

  6. Test your configuration file

Basic configuration of sendmail is described at: Sendmail basic installation.

An installation and operation guide for sendmail 8.12 can be found at: Sendmail installation and operation guide.

Sendmail can either be built from source or installed as sendmail binaries. With sendmail you must build a run-time configuration file. This is a file that sendmail reads when it starts up and it describes the mailers it knows about, how to parse addresses, how to rewrite the message header, and the settings of various options. Although the configuration file can be quite complex, a configuration can usually be built using an m4-based configuration language (m4 is a macro processing language). Assuming you have the standard sendmail distribution, see cf/README for further information.

The sendmail configuration files are processed by m4 to facilitate local customization; the directory cf of the sendmail distribution directory contains the source files. This directory contains several subdirectories:

cf

Both site-dependent and site-independent descriptions of hosts. These can be literal host names (e.g., ucbvax.mc) when the hosts are gateways or with more general descriptions (such as generic-solaris2.mc) as a general description of an SMTP-connected host running Solaris 2.x. File names ending with .mc (M4 Configuration) are the input descriptions; the output is in the corresponding .cf file. The general structure of these files is described below.

domain

Site-dependent subdomain descriptions. These are tied to the way your organization wants to do addressing. For example, domain/CS.Berkeley.EDU.m4 is our description for hosts in the CS.Berkeley.EDU subdomain. These are referenced using the DOMAIN m4 macro in the .mc file.

feature

Definitions of specific features that some particular host in your site might want. These are referenced using the FEATURE m4 macro. An example feature is use_cw_file that tells sendmail to read an /etc/mail/local-host-names file on startup to find the set of local names.

hack

Local hacks, referenced using the HACK m4 macro. Try to avoid these. The point of having them here is to make it clear that they smell.

m4

Site-independent m4(1) include files that have information common to all configuration files. This can be thought of as an #include directory.

mailer

Definitions of mailers, referenced using the MAILER m4 macro. The mailer types that are known in this distribution are fax, local, smtp, uucp, and usenet. For example, to include support for the UUCP-based mailers, use MAILER(uucp).

ostype

Definitions describing various operating system environments (such as the location of support files). These are referenced using the OSTYPE m4 macro.

sh

Shell files used by the m4 build process. You probably don't have to change these.

siteconfig

Local UUCP connectivity information. This directory has been replaced by the mailertable feature; any new configurations should use that feature to do UUCP (and other) routing. The use of the siteconfig directory is deprecated.

Make sure the m4 utility is available at the server. With the m4 utility the .mc macro files can be converted to a sendmail.cf file. The recommended method to configure sendmail is to edit the macro files. And not the sendmail.cf file itself.

Details of sendmail installation files can be seen at: Sendmail installation and operation guide.

Important sendmail files

/etc/mail/sendmail.cf - Main sendmail configuration file.

/etc/mail/access - The sendmail access database file can be created to accept or reject mail from selected domains, systems and users. Since /etc/mail/access is a database, after creating the text file, use makemap to create the database map.

To create a new access map:

	# makemap hash /etc/mail/access.db < /etc/mail/access
			

In the access file different actions can be defined:

OK

Accept mail even if other rules in the running ruleset would reject it, for example, if the domain name is unresolvable. "Accept" does not mean "relay", but at most acceptance for local recipients. Thus, OK allows less than RELAY.

RELAY

Accept mail addressed to the indicated domain or received from the indicated domain for relaying through your SMTP server. RELAY also serves as an implicit OK for the other checks.

REJECT

Reject the sender or recipient with a general purpose message.

DISCARD

Discard the message completely using the $#discard mailer. If it is used in check_compat, it affects only the designated recipient, not the whole message as it does in all other cases. This should only be used if really necessary.

SKIP

This can only be used for host/domain names and IP addresses/nets. It will abort the current search for this entry without accepting or rejecting it but causing the default action.

/etc/mail/local-host-names - Local host names can be defined in the local-host-names file. Add each domain that is to be considered a local account into /etc/mail/local-host-names.

/etc/mail/virtusertable - Used to map incoming email to a local account. With virtusertable, messages sent to one account can be distributed to two users. Also a "catch all" email address can be set up to route all mistyped email to one particular user to create a new virtusertable.

To generate the virtusertable database map:

	$ makemap hash /etc/mail/virtusertable < sourcefile
			

/etc/mail/genericstable - Used for outbound mail. Can be used to rewrite local usernames so they appear to have originated from a different host or domain.

/etc/mail/genericsdomain - Populate this file with the hostname --long command.

	# hostname --long > genericsdomain
			

/etc/mail/mailertable - Used to route email from remote systems.

/etc/mail/domaintable - Can be used to make a transition from an old domain name to a new one.

/etc/mail/aliases - Used to redirect mail for local recepients. Each line of /etc/aliases has the format of alias: user. Two system aliases must always be present: mailer_daemon: postmaster and postmaster: root. You can use aliases for all kind of daemons, for example use ntp: root. Now you can add a line to redirect all mail to root to a specific user of group of administrators, for example root: marc. newaliases needs to be run after any change in this file.

	# newaliases
			

If any update is made in one the of the configuration files sendmail needs to reload the files:

	# killall -HUP sendmail
			

Antirelaying

Starting with sendmail version 8.9, sendmail does not relay by default. When using an older sendmail version, make changes in the sendmail.cf or access file to make sure that sendmail does not relay. Antirelaying tips are described at: Controlling SMTP relaying tips - Sendmail.org.

Sendmail test option

Sendmail can be run in test mode. Use the -b and -t options to do this. You need to run this as root.

	# sendmail -bt
			

Sendmail and DNS

Make sure that the MX records for the MTA's are available in DNS. This can be checked with:

	$ dig MX somedomain.com
			

Manual entries in sendmail.cf

As mentioned before this is not the recommended way. Change the m4 files instead and use m4 to generate sendmail.cf.

Exim

Exim is a message transfer agent (MTA) developed at the University of Cambridge for use on Unix systems connected to the Internet. Exim can be installed in place of Sendmail, although the configuration of Exim is quite different. Also see exim.org for more detailed information.

The exim4-config_files man page contains descriptions for each of these files. Exim comes with an exim.conf template. Just edit this config file for your environment. The Exim Wiki on GitHub provides additional configuration help, FAQ, etc.

Postfix

Postfix is another, widely adopted, MTA. The default location for the postfix configuration files is /etc/postfix, there we find the two main configuration files main.cf and master.cf.

Postfix main.cf file format

The Postfix main.cf configuration file specifies a very small subset of all the parameters that control the operation of the Postfix mail system. Parameters not explicitly specified are left at their default values.

The general format of the main.cf file is as follows:

  • Each logical line is in the form "parameter = value". Whitespace around the "=" is ignored, as is whitespace at the end of a logical line.

  • Empty lines and whitespace-only lines are ignored, as are lines whose first non-whitespace character is a `#'.

  • A logical line starts with non-whitespace text. A line that starts with whitespace continues a logical line.

  • A parameter value may refer to other parameters.

  • When the same parameter is defined multiple times, only the last instance is remembered.

  • Otherwise, the order of main.cf parameter definitions does not matter.

See also Postfix Configuration Parameters. The remainder of this document is a description of all Postfix configuration parameters. Default values are shown after the parameter name in parentheses, and can be looked up with the postconf -d command.

One important option using Postfix is to disable the SMTP VRFY command on a publicly accessible mail server. This stops some techniques used to harvest email addresses. Use the following command to disable this:

	disable_vrfy_command = yes
				

After making changes to main.cf you need to reload postfix using: postfix reload.

Postfix master.cf file format

The Postfix mail system is implemented by small number of (mostly) client commands that are invoked by users, and by a larger number of services that run in the background. Postfix services are implemented by daemon processes. These run in the background under control of the master process. The master.cf configuration file defines how a client program connects to a service, and what daemon program runs when a service is requested.

The general format of the master.cf file is as follows:

  • Empty lines and whitespace-only lines are ignored, as are lines whose first non-whitespace character is a `#'.

  • A logical line starts with non-whitespace text. A line that starts with whitespace continues a logical line.

  • Each logical line defines a single Postfix service. Each service is identified by its name and type as described below. When multiple lines specify the same service name and type, only the last one is remembered. Otherwise, the order of master.cf service definitions does not matter.

Each logical line consists of eight fields separated by whitespace. These are described below in the order as they appear in the master.cf file. Where applicable a field of "-" requests that the built-in default value be used. For boolean fields specify "y" or "n" to override the default value.

Chroot option (default: Postfix >= 3.0: n, Postfix <3.0: y) - Whether or not the service runs chrooted to the mail queue directory (pathname is controlled by the queue_directory configuration variable in de main.cf file).

After making changes to the configuration of postfix you need to reload postfix using: postfix reload.

Postfix preparations

Before postfix can be used, it needs to know:

myorigin

The myorigin parameter specifies the domain that appears in outgoing email. This can be done using one of the following examples:

	myorigin = $myhostname
					

or

	myorigin = $mydomain
					

The $myhostname or $mydomain are replaced by postfix with the hostname or domain of the server it is running on.

mydestination

Postfix needs to know for which domain(s) it will receive mail. The parameter mydestination is used for this. More than one domain may be specified. The domain names can be separated using whitespace or a comma. Also, a pattern can be used to point to a lookup table (hash, btree, nis, ldap or mysql).

	mydestination = $mydomain, localhost.$mydomain, hash:/etc/postfix/moredomains
				

Note

You have to include $mydomain when the server is used as a mail server for the entire domain.

relay_domains

The default configuration of postfix will try to deliver incoming mail to authorized destinations only. The relay_domains parameter controls to which domains postfix will forward emails.

	relay_domains = 	(safe: never forward mail from strangers)
					

	relay_domains = $mydomain (forward mail to my domain and subdomains)
					

relayhost

By default postfix tries to deliver directly to the internet depending on the domain name of the destination address in the mail message. Using the relayhost parameter we can specify to use another SMTP server as relay:

	relayhost = 
					

This is the default, direct delivery to the internet, or using another ISP SMTP server:

	relayhost = mail.myisp.com
					

Logging

Postfix uses the syslog daemon for its logging. The syslog configuration itself is out of scope here. When /etc/syslog.conf is configured as in the example below, postfix's log is written to /var/log/maillog. Error messages are, in this example, redirected to the console.

	mail.err    /dev/console
	mail.debug  /var/log/maillog
			

Tip

Using egrep '<reject|warning|error|fatal|panic>' /var/log/maillog will help you to find any problems postfix encountered.

virtual domains

Generally a postfix server is the final destination for a limited number of domains. But postfix can also be configured to handle mail for additional domains which are different from, for example, the domain in which the postfix server is located. These destinations are called virtual hosts. Using the virtual_alias_domains parameter we can specify for which virtual hosts we wish to receive mail. The format of the parameters is the same as in the samples above. Separate multiple virtual hosts using a space or a comma. Also a link to a (hashed) file on disk is possible:

	virtual_alias_domains = example.com, anotherexample.com
				

or when using a hashed file (using the postmap utility):

	virtual_alias_domains = hash:/etc/postfix/virtual
				

The content of /etc/postfix/virtual can be:

	postmaster@example.com	peter
	info@anotherexample.com	gerda
	sales@example.com	petra
	@example.com		jim
				

In the above example peter receives the postmaster@example.com email. Gerda receives the info@anotherexample.com email and the sales@example.com goes to petra. The last line is a "catch all" rule, all email for example.com without a valid destination goes to jim.

Note

Use postmap /etc/postfix/virtual to create the hashed file and issue a postfix reload after updating the virtual file.

Sendmail emulation layer commands

Since Sendmail has been the de facto mail delivery standard on Unix like systems for years, replacement systems like Postfix have felt compelled to implement sendmail emulation layer commands in order to maintain compatibility with outside programs. That is to say that certain commands that were originally included with the sendmail package are available with alternatives like Postfix as well.

Mailq

mailq is available on most systems to check the mail queueu. It is equivalent to sendmail -bp, which works with Postfix too. Its native alternative is postqueue -p.

Newaliases

newaliases is required to generate a binary aliases file with both sendmail and postfix. Both employ /etc/aliases as the default input file.

On Postfix systems man sendmail will provide more details on the "Postfix to Sendmail compatibility interface".

/var/spool/mail

Specifies the default mail drop directory. By default, all mail is delivered to the /var/spool/mail/<username> file.

Copyright Snow B.V. The Netherlands