SMTPd restrictions, SPF, DKIM and greylisting

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

Postfix provides the ability to apply filters during the SMTP session. When an email arrives at your mail server the following steps are usually run through:

  • CLIENT:
    Incoming TCP connection on port 25
  • HELO
    The sending mail server tells you its name.
  • MAIL FROM
    The sender’s name and email address of the email
  • RCPT TO
    The recipient the mail is intended for
  • DATA
    In this phase of the SMTP session the actual email is transmitted
  • QUIT
    End of SMTP session

With the appropriate settings of SMTPd restrictions you can control what checks Postfix will run when an email is received. Those restrictions are defined in the /etc/postfix/main.cf configuration file and are called smtpd_…_restrictions. You can for example restrict certain IP addresses to even open an SMTP (TCP/25) connection to your mail server during the “client” phase. Or you can filter what recipients are allowed during the “rcpt to” phase. According to the above list of phases these restrictions apply:

  • smtpd_client_restrictions (default: empty)
  • smtpd_helo_restrictions (default: empty)
  • smtpd_sender_restrictions (default: empty)
  • smtpd_recipient_restrictions (default: permit_mynetworks, reject_unauth_destination)
  • smtpd_data_restrictions (default: empty)

So by default Postfix only applies checks during the “RCPT TO” phase to check if the email is either coming from your local network (the sending server’s IP address is part of the “mynetworks” setting) or if the recipient has a valid accounton your mail server. Otherwise the email would get rejected. It is important that you understand that the restrictions are checked in the respective phase. So it’s pointless to apply checks on the sender’s email address (MAIL FROM) in the HELO phase because that information is not yet known at that phase. Many mail server administrators just put all the checks into the smtpd_recipient_restrictions. The following sections give you some advice on how to configure your restrictions properly.

Realtime blacklists (RBL)

The most important technique nowadays to fight spammers are IP blacklists. There exist dozens of these lists that you can use freely. The operators of such lists have different policies regarding who is getting blacklisted. But most of them have spam probes on the internet that spammers fall for. The technique of spam traps means that otherwise unused email addresses are put on a web site in a hidden place or use them on mailing lists. Spammers then automatically spider (=scan) the internet for email addresses and then send their spam to the found addresses. If such a trap account gets email then that information is used by the operators of an IP blacklist to build a list of server that send spam. It is potentially dangerous to use IP blacklists though because you might block email from a legit server if that server was inadvertently blacklisted. My favorite blacklist “SORBS” for example has let me down several times by blacklistting entire internet providers for a period of hours or even days. So it’s a good idea to check their policies and see if you agree with it. In the end the risk is yours. Once you reject an email you cannot get it back. But I still strongly recommend you use blacklists or you will literaly drown in spam.

IP blacklists are specified by a domain name like “bl.spamcop.net”. To check if a certain IP address is listed in that zone you need to check if an entry for the reversed IP address exists in that zone. Assume that a mail server with the IP address 217.217.34.231 wants to send you an email. If you told Postfix to use the “bl.spamcop.net” blacklist then Postfix will reverse this IP address and check the DNS entry of 231.34.217.217.bl.spamcop.net. I just did that and got a result:

$> host 231.34.217.217.bl.spamcop.net
231.34.217.217.bl.spamcop.net has address 127.0.0.2

The fact that a result (127.0.0.2) was found means that the IP is blacklisted and that I should not accept emails from it. Postfix will then tell the sender that it triggered a blacklist and cancels the connection. The result returned almost always is 127.0.0.x where the X stands for the reason of the blacklisting. You will have to read the policy of the respective blacklist to see the reason. Let’s take another IP address that is not blacklisted – for example 85.214.93.191:

$> host 191.93.214.85.bl.spamcop.net
Host 191.93.214.85.bl.spamcop.net not found: 3(NXDOMAIN)

I did not get a result. So the IP is not blacklisted and I should accept email from it.

There are a few major blacklists that you should check out:

  • SORBS (dnsbl.sorbs.net)
    SORBS is very efficient but every few months tends to blacklist even large ISPs accidentally. I currently do not use it as I had too many user complaints.
  • SpamCop (bl.spamcop.net)
    Pretty reliable. On a sample day it blocked 29 mails on my server.
  • SpamHaus (zen.spamhaus.org)
    Also pretty reliable. On a sample day it blocked 147 mails on my server.
  • UCEprotect (dnsbl-1.uceprotect.net)
    Careful – the Class-2 and Class-3 lists are too strict and block even legit senders. On a sample day it blocked 33 mails on my server.

If you used these blacklists then the restrictions in your /etc/postfix/main.cf would be these:

smtpd_recipient_restrictions =
permit_mynetworks
reject_rbl_client dnsbl.sorbs.net
reject_rbl_client bl.spamcop.net
reject_rbl_client zen.spamhaus.org
reject_rbl_client dnsbl-1.uceprotect.net
reject_unauth_destination

(Note: You can also list the restrictions all in one line and seperated by commas. But I prefer the way to use multiple lines and indent the subsequent lines. It makes the main.cf file more readable.)

With these restrictions in place Postfix will run each of these checks after the “RCP TO” phase during the SMTP dialog.

  1. Is the sending IP address in your network range?
  2. Is it blacklisted by SORBS?
  3. Is it blacklisted by SpamHaus?
  4. Is it blacklisted by UCEprotect?
  5. Is your mail server responsible for the recipient’s domain? Or is the sender using SMTP authentication for relaying?

Further restrictions to consider

Postfix offers quite a lot of checks you can use in your smtpd_…_restrictions. Just to name a few that I use on my mail servers:

  • reject_unknown_client_hostname
    Runs a two-way DNS check of the IP address and hostname of the sending server. This can be potentially dangerous and reject legit email from misconfigured mail servers but nowadays every decent mail server is supposed to take care of their DNS settings.
  • reject_unknown_sender_domain
    Checks if the domain used in the sender’s email address actually exists. This runs a check for an MX and A record of the sending domain.
  • reject_unauth_pipelining
    Some spam bots just try to deliver emails as fast as possible. But formally the sending server must wait for the recipient server’s response in each SMTP step. If the recipient server allows pipelining then fast sending is allowed. This setting drops the connection if pipelining is attempted before it was offered by the destination server.

Further settings can be found in the Postfix documentation.

Are you blacklisted?

Of course you want to send out emails through your mail server. So you need to make sure that your server IP address is not blacklisted either. I suggest you check your server’s IP address through one of the blacklist test services:

(These services also give you an idea what other blacklists are available.)

If you are listed on any of these lists then you will want to get off the lists. How you do that depends on each blacklist’s policy. Some don’t even support manual removal from these lists. If your ISP assigned you a dynamic IP address then you are in a so called “dialup IP range” and are most likely blacklisted and it’s pointless to get off the list – read here for a solution. A lot of spam is coming from dynamic IP addresses. Either spammers send email from their ISP account. Or the large amount of virus-infested Windows PCs in the world is forwarding spam. So dynamic IP addresses are frowned upon by mail server administrators.

Source spoof prevention

The above section on IP blacklists helped you fight spam by rejecting it. Now let’s add another security measure to prevent source address spoofing. Spoofing means that someone fakes information. And the source address is the email address that appears to be the sender of an email. So it means that somebody pretends to be someone else by faking the email address of the sender. Maybe you haven’t given it much thought but basically everyone can send an email and pretend to be someone else. Perhaps you are even using multiple sender addresses yourself. What prevents someone from pretending he is you? There are two common options:

  1. SPF (Sender Policy Framework): Using SPF anyone can find out if an email was sent from an IP address (=mail server) that is allowed to act as a mail server for that sender domain. Technically with SPF you create a DNS entry for your domain and list the valid IP addresses there that are allowed to send email on your domain’s behalf.
  2. DKIM (DomainKeys Identified Mail): DKIM uses a private/public key pair to sign all outgoing emails automatically. By making your domain’s public key available through DNS every mail server on the internet can check if the signature is valid for your domain and nobody spoofed the email. Although SPF is more widespread than DKIM it has a major design flaw because it fails when you forward email (as the sending IP address changes and may not be a valid address listed in the SPF entry any more).

Setting your own SPF entry

The content of the SPF entry follows a certain syntax. To make things easy you can use the service at OpenSPF to set up your SPF entry. Enter your domain name there and fill out the fields. The web site will tell you how your SPF entry will have to look like. Add this to your DNS zone and you will both help others to fight spam as well as help yourself to prevent that anyone is sending unwanted emails on your behalf. Formerly a TXT resource record in the DNS zone was used but nowadays there is a distict SPF record type used for that purpose. After quite some controversy the “SPF” type of DNS records has been discontinued. So Using a TXT records is the way to go.

Checking other SPF entries

Many domains already have SPF entries. So you are encouraged to use that information for your own spam and phishing protection. Read the next section on greylisting to learn how to use the “tumgreyspf” service to use both the power of SPF and greylisting.

Forwarding breaks SPF

SPF has a major design flaw. It fails if someone forwards email to another server. Then the forwarding server is not listed in the SPF entry and the destination mail server will (correctly) reject the email. Less technically-savy users will not understand this limitation. So many mail administrators just take the SPF checks into account when looking for spam but they don’t ultimately reject the email. It’s up to you. SPF reliably fights spoofing and phishing for major domains like PayPal, eBay, Amazon or other large organisations that are often a target of phishers. In some cases it will just be overly careful.

Checking DKIM signatures of incoming email

DKIM was invented for a similar reason as SPF. It provides email authentication – to allow a receiving mail server to check that the email is authentic for a given sender domain. SPF just tells which IP are allowed to send email for a sender domain. But DKIM is working way more reliably because it adds a cryptographic signature to every outgoing email. To verify if an email is authentic a mail server can use the email’s signature and match it with the sender domain’s public key that can be obtained by DNS. Sounds too complicated? Actually it’s pretty simple if you understand the basics of public-key cryptography. Have you ever used PGP? Then you know that you had to create a keypair consisting of a private key you can use to sign emails and a public key that you can give others to encrypt emails for you or check your signatures. DKIM uses exactly that technique. The sending mail server has a keypair for each domain. When it sends out an email it will use the private key for the sender domain, take the email that is to be sent out and sign it with a cryptographic signature. That signature is added to the email (as an additional mail header) and the receiving mail server can take the email and the signature from the mail header and the public key of the sender domain to determine whether the signature was valid.

Adding automatic verification of other domains’ DKIM signatures is pretty easy. Just install the DKIM filter software:

apt-get install dkim-filter

Actually this software is called a “milter”. Milters are “mail filters” that were introduced as additional pieces of software for the ancient Sendmail mail server software. Milters speak a certain protocol that mail server can use to communicate with it and Postfix fortunately knows how to use milters, too. Right after the installation the “dkim-filter” process is already started and running in the background. But as Postfix is running in a chroot jail in /var/spool/postfix it cannot use the DKIM filter’s communication socket at its default location in /var/run/dkim-filter/dkim-filter.sock. You can fix the permissions and make DKIM run a socket in /var/spool/postfix instead but I prefer to rather run dkim-filter as a TCP service rather than a socket. Edit your /etc/default/dkim-filter file that contains its default settings and add

SOCKET=”inet:54321@localhost”

to it. Restart the dkim-filter process:

/etc/init.d/dkim-filter restart

Now dkim-filter is listening to TCP port 54321 on the localhost interface. To make Postfix use this milter run these commands to add the appropriate milter definitions to your /etc/postfix/main.cf file:

postconf -e smtpd_milters=inet:127.0.0.1:54321
postconf -e non_smtpd_milters=inet:127.0.0.1:54321

Now Postfix will run all emails (both incoming and outgoing) through your DKIM filter and check DKIM signatures (if found) on incoming emails. It will not sign outgoing emails though – this is covered in the next section.

A side note: you can decide what should happen if there is a problem with the DKIM filter. By default Postfix will temporarily reject emails until you fixed the issue. It’s up to you. But I prefer to just accept emails if there is a milter problem. So I set:

postconf -e milter_default_action=accept

For more information on milters please consult the Postfix documentation on milters.

Setting up DKIM for sending emails

The actual idea of DKIM is to make a domain’s mail server sign outgoing emails automatically. To do that you need to create a cryptographic keypair first. Fortunately you do not have to buy a certificate because publishing the key in your DNS zone is enough. The tool you need is “dkim-genkey” and it requires the bit-length of the key and the domain name. You can optionally use a selector to use multiple keys. By default the selector is called “default”. I suggest you use the domain name as your selector which will name your key files correctly automatically.

It’s best to create a seperate directory to store the keys. I keep mine in /etc/postfix/dkim:

mkdir /etc/postfix/dkim
cd /etc/postfix/dkim

Then create the keypair there:

dkim-genkey -b 1024 -d example.org -s example.org

In your current directory you will now find two files.

  • “example.org.private” is the private key that the DKIM milter will use to sign all outgoing emails
  • “example.org.txt” is the public key that you are supposed to add to your domain’s DNS zone

The configuration file for the DKIM milter is /etc/dkim-filter.conf. Edit that file. The only change you need is to add a line for the key list file like:

KeyList     /etc/dkim-keys.conf

Create a new file /etc/dkim-keys.conf that you just pointed to. Its contents define which key is used for which sender address. Or to be specific the syntax is:

sender-pattern:signing-domain:keypath

as pointed out in the “man dkim-filter.conf” man page. So if your domain were example.org you would write:

*@example.org:example.org:/etc/postfix/dkim/example.org

Add one line per (sender) domain that you wish to sign outgoing emails for.

Caveat: dkim-filter adds “.private” to the keypath you specify. So the above line is correct. Do not specify /etc/postfix/dkim/example.org.private or signing will fail.

Restart your DKIM milter:

/etc/init.d/dkim-filter restart

Now outgoing emails should get a DKIM signature automatically. Send an email to another mail server and check the headers of the email. You should see a header like this:

DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=example.org; s=example.org t=1307571770; bh=OtZOIOSkkhL1t+kR5KOE6HvvjUjLkDE+70agcKmcjJg=; h=Message-ID:Date:From:MIME-Version:To:Subject:Content-Type: Content-Transfer-Encoding; b=RlxE9IRiOJrOzM4JbJFIp/YmK4XBRJFti79rL0vzGkppAC1AJi2CdtwGFT/sTovt/iKikrdyQ7M7JhdlC9u3bh8rrvhLA53HZCmu6WvHvv059ysdpjUhrktEqZFrFpgds2wqCkzF4ar/ly3TzYZmoZJO+C8j2uU5L3cKzESfGw4=

The meaning of each parameter can be found on the Wikipedia page. or in the less human-friendly format definition in the RFC 4871.

However the recipient cannot tell if this is a correct signature because they don’t have the public key to verify this signature with. Edit your DNS zone for your domain and just add the TXT record that “dkim-genkey” created for you in /etc/postfix/dkim/example.org.txt. On my test server it looked like this:

example.org._domainkey IN TXT “v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDKPFhCADCGcxDchGHFqvSoQYuW0epBT0gJuCwDMkX0uPejeeMdBNDcaGo8AyZdXJTFKqGK5kao2OvpDsH6XeKMBjPt/CnyBm4PNwlwNrkJTBc15xjD6Swmlk457+Ioz/tbpBA3b3RnC8NsqiknLQ3JxDkE7fXfji7Uds5+swWwJQIDAQAB” ; —– DKIM example.org for example.org

Verify that your DNS zone for your domain really contains the right entry by querying for a TXT record with your domain name and the selector you used:

dig +short example.org._domainkey.example.org txt

And it should return your public key:

“v=DKIM1\; g=*\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDKPFhCADCGcxDchGHFqvSoQYuW0epBT0gJuCwDMkX0uPejeeMdBNDcaGo8AyZdXJTFKqGK5kao2OvpDsH6XeKMBjPt/CnyBm4PNwlwNrkJTBc15xjD6Swmlk457+Ioz/tbpBA3b3RnC8NsqiknLQ3JxDkE7fXfji7Uds5+swWwJQIDAQAB”

Now all outgoing emails for …@example.org will get a DKIM signature added automatically and recipients can verify them. Any mail server relying on DKIM signature checks can now check whether emails from your domain are legit.

I assume that you have set up the DKIM milter in Postfix as described in the previous section. Setting “smtpd_milters” and “non_smtpd_milters” in your /etc/postfix/main.cf will now do both the verification of DKIM signatures as well as the signing.

Greylisting

Another way to help you reduce the amount of spam you receive is using greylisting. Whitelisting means that a certain kind of email is always accepted (delivery status  2.x.x). Blacklisting is the opposite and always blocks certain emails (delivery status 5.x.x). Greylisting is something in between and temporarily refuses to accept the email for a couple of minutes. The idea is to tell virus infested computers trying to spam you from actual mail servers. Viruses (or rather: spam worms) just try to deliver a spam email once. But real mail servers have a queue of outgoing emails and upon a temporary error they will try to re-deliver the email. If the sending mail server has never sent us an email then greylisting will send back a temporary error code (delivery status 4.x.x) which signals the sending server to try delivering the email again in a few minutes. Your server will take a note of the IP address of the sending server and after a set period (a common value is 10 minutes) the blockade is lifted and upon the next delivery attempt the email will be accepted. Also the greylisting service will take a note in its cache and further delivery attempts will be allowed instantly.

The obvious drawback is that emails are often delayed by several minutes – especially if you just start using greylisting and the cache of trusted sending servers is empty. It is your decision whether your users will accept that. In my personal experience greylisting is not as effective nowadays as it has been in the beginning. It appears like malware now just tries to deliver spam emails several times. But it’s one measure to maybe block some spam emails.

There are several ways to implement greylisting. The way I do it is using the “tumgreyspf” software. Start by installing it:

apt-get install tumgreyspf

To make Postfix use tumgreyspf you have to define a service for it in the /etc/postfix/master.cf:

tumgreyspf unix  –      n       n       –       –       spawn
user=tumgreyspf argv=/usr/bin/tumgreyspf

You also need to add it to your smtpd_recipient_restrictions of your /etc/postfix/main.cf file. Example:

smtpd_recipient_restrictions =
permit_mynetworks
permit_sasl_authenticated
reject_rbl_client bl.spamcop.net
check_policy_service unix:private/tumgreyspf
reject_unauth_destination

These are example restrictions. Do not just copy/paste it. The example is just meant to show you a good place to add the policy check in your smtpd_recipient_restrictions.

Finally restart Postfix:

postfix reload

To customize the behavior of tumgreyspf you can alter the files in /etc/tumgreyspf.

(This procedure is also documented in the /usr/share/doc/tumgreyspf/README.Debian file.)

Managing your mail server

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

Managing control information of a mail server in a databaes gives a lot of flexibility compared to raw text files. But let’s be honest for a moment. Most reader of this tutorial won’t set up a mail server with thousands of users to make money with. They are just curious what it’s like to run a mail server and wanted to learn a great deal. Are you like them? Then you probably just want to manage a domain and a handful of users and don’t like to edit database rows through SQL statements, right? Don’t despair – there are two ready web interfaces to manage your mail server.

Ole Jungclaussen’s IMA

Ole has written his own web interface that you can get here – there is also a demo available.

Mail Admin Tool

Steffan Slot created a PHP based mangement interface that you can find at http://mat.ssdata.dk/

MailAdm

Robert Kuntz is working on MailAdm to replace all other tools. Check out a demo version.

Scott Moody’s management interface

Scott Moody has contributed another simple web interface consisting of just one PHP file that helps you manage your mail server. Visit his web site if you are interested.

mailadmin

Ben Clarke also wrote a free and open source web administration interface. You can find his work at GitHub.

Authenticated SMTP

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

Relaying

Before we dive into SMTP authentication I want you to understand what relaying actually means. When Postfix receives an email and needs to forward it to another server this is called relaying.

Incoming email

When someone on the internet sends an email to john@example.org some other mail server will deliver the email using SMTP to your mail server. Postfix will determine that it’s responsible for email addresses in the example.org domain and accept the email. John can then use POP3 or IMAP to fetch the email from your server.

Outgoing email (without authentication)

John is on the internet somewhere and wants to send an email to lisa@example.com. As your mail server is not responsible for the “example.com” domain it would have to forward the email to the responsible mail server. So your server receives John’s email and forwards (relays) it to the mail server that is responsible for …@example.com email addresses. This may seem like a harmless case your mail server will deny that:

Why? Because anyone can claim to be John and make your mail server forward mail. If an attacker (like a spammer) would send millions of spam emails through your server then other organisations will accuse you of spamming. Your mail server would be what people call an “open relay”. This is not what you want because your mail server’s IP address would get blacklisted and you will have serious trouble ever sending out mail again. So without any proof that John is actually John your server will reject the email.

Outgoing email (with authentication)

So how does John send his email? He needs to use authenticated SMTP. This is similar to the previous case but John’s email program will send his username and password first.

mynetworks

In addition to using SMTP authentication you can tell Postfix to always relay email for certain IP addresses. The mynetworks setting contains the list of IP networks or IP addresses that you trust. Usually you define your own local network here. The reason John had to authenticate in the above example is because he is not sending the email from your local network (usually).

Enabling SMTP authentication in Postfix

Authenticated SMTP with Postfix has been a hassle in the past. It was done through the SASL (Simple Authentication and Security Layer) library that was once part of the Cyrus mail server. It was nearly impossible to debug and threw error messages that were gibberish and misleading. Fortunately nowadays we can make Postfix ask the Dovecot server to verify the username and password. And as you already configured Dovecot’s authentication part this is really easy now. Postfix just needs some extra configuration.

postconf -e smtpd_sasl_type=dovecot
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_sasl_auth_enable=yes

postconf -e smtpd_tls_security_level=may
postconf -e smtpd_tls_auth_only=yes
postconf -e smtpd_tls_cert_file=/etc/ssl/certs/mailserver.pem
postconf -e smtpd_tls_key_file=/etc/ssl/private/mailserver.pem
postconf -e smtpd_recipient_restrictions=” \

permit_mynetworks \
permit_sasl_authenticated \
reject_unauth_destination”

smtpd_sasl_auth_enable enables SMTP authentication altogether. And the smtpd_recipient_restrictionsdefine rules that are checked after the remote user sends the RCPT TO: line during the SMTP dialog. In this case relaying is allowed if:

  • permit_mynetworks: the user is in the local network (mynetworks) or
  • permit_sasl_authenticated: if the user is authenticated or
  • reject_unauth_destination: the mail is destined to a user of a domain that is a local or virtual domain on this system (mydestination, virtual_alias_domains or virtual_mailbox_domains).

There are further restrictions (smtpd_client_restrictions, smtpd_helo_restrictions,smtpd_sender_restrictions) that get checked during the different states of the SMTP dialog (IP connection, HELO/EHLO command, MAIL FROM command) but for now you should put all restrictions into thesmtpd_recipient_restrictions.

Enabling encryption

The smtpd_tls_security_level is set to “may” to enable TLS encryption when sending the email, the user name and especially the password. You don’t want your users to send their password unencrypted over the internet. Ideally you would set from “may” to “encrypt” which would forbid sending such information unencrypted. While it sounds tempting it would break reception of emails from other mail servers on the internet. So unless you operate different mail servers for servers and users you must not use “encrypt”. However users are still able to ignore encryption and still send their password unencrypted. So we take another step…

Enforcing encryption for authentication data

Although emails may still get delivered unencrypted from other servers we should make sure that users don’t accidentally (or out of ignorance) send their passwords unencrypted over the internet. That’s what we enforce using “smtpd_tls_auth_only=yes”. Now we should be safe.

How SMTP authentication works

Are you curious how SMTP authentication looks on a protocol level? Let’s go through that. In previous versions of this guide we used “telnet” to connect to TCP port 25 and speak SMTP. But now we enforce encryption and can’t do SMTP authentication unencrypted. Let’s try the unencrypted part:

$> telnet localhost smtp

The server will let you in:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mailtest ESMTP Postfix (Debian/GNU)

Say hello:

ehlo example.com

Postfix will present a list of features that are available during the SMTP dialog:

250-mailtest
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Let me briefly explain what these lines mean:

  • PIPELINING
    This is a feature to speed up SMTP communication. Usually the remote system has to wait for a response to every command it sends. Pipelining allows the remote server to send bulks of commands without waiting for a response. Postfix will just store these commands and execute them one by one. If you told Postfix to forbid pipelining it would disconnect the remote server when it tries to send bulks of commands without waiting for the proper reply. It is mainly a feature against spamming programs that don’t behave.
  • SIZE 10240000
    The remote server is allowed to send emails up to 10 MB large. This has long been a common maximum size for emails. However nowadays 40 MB or even more are more common sizes because emails have grown larger.
  • VRFY
    Allows remote servers to verify a given name or email address. For example the remote server could send “VRFY john” and your server might respond “250 John Doe <john@example.org>”. It is a kind of dictionary lookup. It poses a security risk because it allows spammers to harvest valid email addresses. So Postfix will only allow it once the remote system has authenticated.
  • ETRN
    A command that a remote system can send to flush the Postfix queue of mails for a certain domain. It can be used if the remote system had technical problems and failed to receive email for a while. Then it could send an ETRN command to make your server start sending outstanding emails for that domain. It is rarely used.
  • STARTTLS
    This tells the remote system that it might start switching from this unencrypted to an encrypted connection by sending the “STARTTLS” command. It will then start negotiating a TLS-encrypted connection. You could compare it to an HTTP connection that suddenly switches over to an encrypted HTTPS connection. The advantage is that you can start talking SMTP on TCP port 25 and don’t have to open up a second TCP port like 465 which is the “SSMTP” (secure SMTP) port and only accepts encrypted connections.
  • ENHANCEDSTATUSCODES
    This enables more three-digit return codes for various conditions. See the RFC2034 if you are curious.
  • 8BITMIME
    In ancient times SMTP only processed 7-bit characters. You couldn’t transfer special characters like “Ä” or “ß” without special encoding. 8BITMIME allows a transmission of emails using 8-bit characters. Still many emails are specially encoded using ISO8859-1 or UTF-8.
  • DSN
    It enables DSNs (delivery status notofications) that allows the sender to control the messages that Postfix creates when an email could not be delivered as intended

However one important line is missing here that would allow us authentication:

250-AUTH PLAIN LOGIN

We told Postfix to only allow authentication when the connection is encrypted. So we are not offered authentication over this plaintext connection. (You could allow unencrypted authentication by temporarily setting “postconf -e smtpd_tls_security_level=” and trying again. You would then see this line.)

Are you still connected? Okay, good. So we need an encrypted connection using TLS. You could enter STARTTLS:

STARTTLS

And the server would reply:

220 2.0.0 Ready to start TLS

 

However now it’s getting complicated because you would have to speak TLS which is not human-friendly at all. But we can use OpenSSL instead. Enter “QUIT” to terminate the SMTP connection. Instead run:

$> openssl s_client -connect yourmailserver:25 -starttls smtp

You will see a lot of output. OpenSSL has connected to TCP port 25 and issued a STARTTLS command to switch to an encrypted connection. So whatever you type now will get encrypted. Enter:

ehlo example.com

And Postfix will send a list of capabilities that will look like this:

250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

 

And now that we are using an encrypted connection Postfix offers us to authenticate. So let us send the authentication string with a Base64-encoded password:

AUTH PLAIN am9obkBleGFtcGxlLm9yZwBqb2huQGV4YW1wbGUub3JnAHN1bW1lcnN1bg==

The server should accept that authentication:

235 2.7.0 Authentication successful

Disconnect from Postfix:

QUIT

Goodbye:

221 2.0.0 Bye

Authentication works. Well done.

Note

If you have set John’s password to something other than ‘summersun’ you need Base64-encode it yourself. Use:

$> perl -MMIME::Base64 -e \
‘print encode_base64(“john\@example.org\0john\@example.org\0password”)’;

Now you can test sending email with authentication enabled. To make even your local network untrusted temporarily you can set:

$> postconf -e mynetworks=

Now you should be forced to use SMTP authentication even in your local network.

Troubleshooting

Should you get a warning message in your /var/log/mail.log that reads “warning: TLS library problem: 24125:error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca:s3_pkt.c:1256:SSL alert number 48:” then you are using a self-signed certificate and the email application refused the certificate. In this case the users need to add an exception so the certificate gets trusted.

If you cannot send email from the mail client and get “Relay access denied” as an error message then it did not properly send a username and password. Re-check the settings of your mail program.

Testing email delivery

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

By now you will have spent considerable time digging through configuration files and trying to understand new terms. And you may be worried whether all you did actually leads to a working mail server. So before we continue with the configuration let’s take a break and verify that all you did works so far.

At this point the /var/vmail directory should be empty yet. You can get a list of all files and directories within by running:

find /var/vmail

There is probably nothing except perhaps a “lost+found” directory if /var/vmail is on a separate partition.

If you added test data to the database in the section about database preparation as you were told then you have “example.org” configured as a virtual domain and “john@example.org” as a user in that domain. Open a new terminal window and run

tail -f /var/log/mail.log

to see what the mail server is doing. Now let’s send an email to John. A simple way to create and send an email is to pipe text to the “mail” command. Do that in another terminal window:

echo test | mail john@example.org

If all works as expected your mail.log will show a lot of technical information about the email delivery. Like this:

Oct 20 17:42:32 sirius postfix/pickup[22273]: A6F8EB55: uid=0 from=<root>
Oct 20 17:42:32 sirius postfix/cleanup[22582]: A6F8EB55: message-id=<20131020154232.A6F8EB55@sirius.localdomain>
Oct 20 17:42:32 sirius postfix/qmgr[22274]: A6F8EB55: from=<root@sirius.workaround.org>, size=287, nrcpt=1 (queue active)
Oct 20 17:42:32 sirius dovecot: auth-worker(22635): mysql(127.0.0.1): Connected to database mailserver
Oct 20 17:42:32 sirius dovecot: lda(john@example.org): msgid=<20131020154232.A6F8EB55@sirius.localdomain>: saved mail to INBOX
Oct 20 17:42:32 sirius postfix/pipe[22586]: A6F8EB55: to=<john@example.org>, relay=dovecot, delay=0.14, delays=0.04/0/0/0.1, dsn=2.0.0, status=sent (delivered via dovecot service)
Oct 20 17:42:32 sirius postfix/qmgr[22274]: A6F8EB55: removed

Your output will look slightly different. But focus on the parts that are printed in bold letters. They should be exactly the same in your log file. If everything worked as expected Postfix has accepted the email and forwarded it to Dovecot which in turn wrote the email in John’s maildir. Look again:

find /var/vmail

Dovecot has now created a directory structure for John and created several directories and extra files:

/var/vmail/
/var/vmail/example.org
/var/vmail/example.org/john
/var/vmail/example.org/john/Maildir
/var/vmail/example.org/john/Maildir/dovecot.index.log
/var/vmail/example.org/john/Maildir/cur
/var/vmail/example.org/john/Maildir/dovecot-uidvalidity.5263f9e8
/var/vmail/example.org/john/Maildir/new
/var/vmail/example.org/john/Maildir/new/1382283752.M737526P22632.sirius,S=352,W=362
/var/vmail/example.org/john/Maildir/dovecot-uidvalidity
/var/vmail/example.org/john/Maildir/tmp
/var/vmail/example.org/john/Maildir/dovecot-uidlist
/var/vmail/example.org/john/Maildir/dovecot.index.cache

Your files may have slightly different numbers.

Access the email on disk using different methods

Shell style

So John finally has finally received a new email into his inbox (that I have highlighted in bold letters). Just open that file and take a look – it is just a standard email with headers and a body. On my system it reads:

Return-Path: <root@sirius.workaround.org>
Delivered-To: john@example.org
Received: by sirius.localdomain (Postfix, from userid 0)
id A6F8EB55; Sun, 20 Oct 2013 17:42:32 +0200 (CEST)
To: john@example.org
Message-Id: <20131020154232.A6F8EB55@sirius.localdomain>
Date: Sun, 20 Oct 2013 17:42:32 +0200 (CEST)
From: root@sirius.workaround.org (root)

test

If anything went wrong then carefully check the last lines of your /var/log/mail.log. It will very likely point you to the problem. Or read the troubleshooting section. Or just add a comment to this page and ask other readers for help.

(If you are curious how to send an email to your mail server using a manual SMTP session then read the respectivesection in the Lenny tutorial. It’s also good as an additional test because the “mail” command bypasses a few of Postfix’s security features.)

Mutt style

You can also use a slightly more comfortable tool to access Maildirs that will come handy for you as a mail server administrator: “mutt”.

mutt -f /var/vmail/example.org/john/Maildir

(You may get asked to create /root/Mail – this is standard procedure. Just press Enter.)

What you see now are the contents of John’s mailbox:

Press enter and you can read the email number 1:

So there is your test email. Press “q” to quit “mutt”.

IMAP style (mutt)

Actually we just cheated a little as we have accessed John’s inbox directly on disk. A better test is to use POP3 or IMAP. And fortunately “mutt” supports IMAP:

mutt -f imap://john@example.org@localhost

You may be prompted to confirm that you are connecting to a mail server with an untrusted SSL certificate. That’s okay. In the end you should see the index and email just like in the screenshots above. (The password is “summersun” – remember?) That worked? Great. Otherwise check your /var/log/mail.log for error messages.

The folder name looks a bit weird, doesn’t it. Let me explain it:

  • imap://
    The protocol
  • john@example.org
    The login name (as we use the email address as the user name, too)
  • @
    The seperator between the user name and the host name of the mail server
  • localhost
    The mail server

​Maybe it makes a bit more sense now. 🙂

If you still can’t get enough and want to run a manual POP3 and IMAP session using TELNET then check out the Lenny tutorial on a complete example.

IMAP style (Thunderbird)

Your users are not likely to use “mutt” to read their email – unless they are immortal console nerds. So of course you can use a graphical mail client to access John’s emails. If you use Thunderbird (or “icedove” on Debian) then feel free to configure a new email account using IMAP and connect to the server. Remember that the user name and email address are both “john@example.org” and the password is “summersun”.

POP3 versus IMAP

If you wonder what the difference between POP3 and IMAP is:

  • POP3 (Post Office Protocol) is a simple protocol that lets you fetch email from a single mailbox. It is usually used to collect all emails, though you can also leave them on the server but this is a bit of a hack and you can’t create multiple folders on the server to sort your mail. It saves space on the mail server because the email gets moved to the user’s hard disk on their computer. But they won’t be able to access the same email from another computer. Besides you cannot create multiple folders on the server to sort your mail. There is just the inbox. This variant is antiquated and not exactly user-friendly.
  • IMAP (Internet Messaging Application Protocol) is predominantly focused upon leaving your mail on the server but you can also collect it like POP3. The inbox is where your incoming emails are stored but users can also maintain folders and move emails to them. But users can move emails to different directories. IMAP is useful when you want to access your email from different locations without losing mail because you fetched it from another location. The drawback is that lazy users leave their mail on the server thus filling up your server’s hard disk (unless you use quotas).

Connecting Postfix to the database

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

In the previous chapter you have fed the MySQL database. Now it’s time to make use of it. The entry point for all email on your system is Postfix. So we need to tell Postfix where to find the database-stored information. Let’s start by telling it which virtual domains you have.

virtual_mailbox_domains

As described earlier a mapping in Postfix is just a table that contains a left-hand side (LHS) and a right-hand side (RHS). To make Postfix use MySQL to define a mapping we need a ‘cf’ file (configuration file). Start by creating a file called /etc/postfix/mysql-virtual-mailbox-domains.cf for the virtual_mailbox_domains mapping that contains:

user = mailuser
password = fLxsWdf5ABLqwhZr​    <– use your own database password here
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_domains WHERE name=’%s’

Imagine that Postfix received an email for somebody@example.org and wants to find out if example.org is a virtual mailbox domain. It will run the above SQL query and replace ‘%s’ by ‘example.org’. If it finds such an entry in the virtual_domains table it will return a ‘1’. Actually it does not matter what exactly is returned as long as there is a result.

Note: You may be tempted to write “localhost” instead of “127.0.0.1”. Don’t do that because there is indeed a difference in this context. “localhost” will make Postfix look for the MySQL socket file and it can’t find it within it’s chroot jail at /var/spool/postfix because it is at /var/run/mysqld/mysqld.sock by default. But if you tell Postfix to use 127.0.0.1 as described here you make Postfix use a TCP connection to port 3306 on localhost which is working even if Postfix is jailed.

Now you need to make Postfix use this database mapping:

postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

(The “postconf -e" command conveniently adds configuration lines to your /etc/postfix/main.cf file. It also activates the new setting instantly so you do not have to reload the Postfix process.)

Postfix will now search your virtual_domains table to find out if “example.org” is a virtual mailbox domain. Let us see if this works. You have set up the “example.org” domain in the previous chapter already. So we can query Postfix now to see if it will find the domain in the database:

postmap -q example.org mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

You should get ‘1’ as a result. Your first mapping is working. Great. Let’s get to the second one.

virtual_mailbox_maps

You will now define the virtual_mailbox_maps which is mapping email addresses (left-hand side) to the location of the user’s mailbox on your harddisk (right-hand side). If you saved incoming email to the hard disk using Postfix’s built-in virtual delivery agent then it would be queried to find out the mailbox path. But in our setup the actual delivery is done by Dovecot’s LDA (local delivery agent) so Postfix does not really care about the path. Postfix just needs to check if a certain email address is valid. Similar to the above you need an SQL query that searches for an email address and returns “1”.

Next you will need to create a “.cf” file to tell Postfix about the SQL query for this table. In addition to the email address it is also important to get the user’s password later on. As the path of the user’s mailbox is fixed it is not important to get that information from the database. The directory structure will always be /var/vmail/$DOMAIN/$USER. So for John’s example it would be/var/vmail/example.org/john.

Now things are a bit simpler and you can finally create a “.cf” file at /etc/postfix/mysql-virtual-mailbox-maps.cf that is as simple as:

user = mailuser
password = fLxsWdf5ABLqwhZr    <– use your own database password here
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_users WHERE email=’%s’

Tell Postfix that this mapping file is supposed to be used for the virtual_mailbox_maps mapping:

postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

Test if Postfix is happy with this mapping by asking it where the mailbox directory of our john@example.org user would be:

postmap -q john@example.org mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

You should get “1" back which means that john@example.org is an existing virtual mailbox user on your server. Later in the Dovecot configuration part you will also use the email and password fields but Postfix does not need them here. Great, there is just one mapping left to define:

virtual_alias_maps

The virtual_alias_maps mapping is used for forwarding emails from one email address to another. It is possible to name multiple destinations. In the database this is achieved by using different rows.

Create another “.cf” file at /etc/postfix/mysql-virtual-alias-maps.cf:

user = mailuser
password = fLxsWdf5ABLqwhZr    <– use your own database password here
hosts = 127.0.0.1
dbname = mailserver
query = SELECT destination FROM virtual_aliases WHERE source=’%s’

Make Postfix use this database mapping:

postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf

Test if the mapping file works as expected:

postmap -q jack@example.org mysql:/etc/postfix/mysql-virtual-alias-maps.cf

You should see the expected destination:

john@example.org

Optional: Black magic you need if you intend to allow catch-all aliases

As explained earlier in the tutorial there is way to alias all email for a domain to a certain destination email address. This is called a  “catchall” alias. Catchalls catch all emails for a domain if there is no specific virtual user for that email address. Catchalls are evil – seriously. It is tempting to generally forward all email addresses to one person if e.g. your marketing department requests a new email aliases every week. But the drawback is that you will get even more insane amounts of spam because spammers will send their crap to any address of your domain. Or perhaps a sender mixed up the proper spelling of a recipient but the mail server will forward the email instead of rejecting it for a good reason. So think twice before using catchalls.

A catchall alias looks like “@example.org” and forwards email for the whole domain to one account. We have created the ‘john@example.org’ user and would like to forward all other email on the domain to ‘kerstin@example.com’. So we would add a catchall alias like:

sourcedestination
@example.orgkerstin@example.com

Now imagine what happens when Postfix receives an email for ‘john@example.org’. Postfix will first check the aliases in the virtual_alias_maps table. (It does not look at the virtual_mailbox_maps table at the moment.) It finds the catchall entry as above and since there is no more specific alias the catchall account matches and the email is redirected to ‘kerstin@example.com’. John will never get any email. This is probably not what you wanted. So you need to make the table look like this instead:

emaildestination
@example.orgkerstin@example.com
john@example.orgjohn@example.org

More specific aliases have precedence over general catchall aliases. Postfix will lookup all these mappings for each of:

  • john@example.org (most specific)
  • john (only works if “example.org” is the $myorigin domain)
  • @example.org (catchall – least specific)

Postfix will find an entry for ‘john@example.org’ first and sees that email should be “forwarded” to ‘john@example.org’ – the same email address. This trickery may sound weird but it is needed if you plan to use catchall accounts. So the virtual_alias_mapsmapping must obey both the “view_aliases” view and this “john-to-himself” mapping. This is outlined in the virtual(5) man page in theTABLE SEARCH ORDER section.

Create a “.cf” file /etc/postfix/mysql-email2email.cf for the latter mapping:

user = mailuser
password = fLxsWdf5ABLqwhZr    <– use your own database password here
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM virtual_users WHERE email=’%s’

Check that you get John’s email address back when you ask Postfix if there are any aliases for him:

postmap -q john@example.org mysql:/etc/postfix/mysql-email2email.cf

The result should be the same address:

john@example.org

Now you need to tell Postfix that these two mappings should be searched by adding this line to your main.cf:

postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf

The order of the two mappings is not important here.

You did it! All mappings are set up and the database is generally ready to be filled with domains and users. Make sure that only ‘root’ and the ‘postfix’ user can read the “.cf” files – after all your database password is stored there:

chgrp postfix /etc/postfix/mysql-*.cf
​chmod u=rw,g=r,o= /etc/postfix/mysql-*.cf

Tracking last logins with Dovecot

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

Per request I have added a section to the Wheezy ISPmail tutorial about how to track last logins with the Dovecot mail server. This can be useful if you operate a semi-public mail server and want to expire email accounts that have not been used in a while. (Not to confuse with the expire Dovecot plugin whose purpose is to automatically delete emails in certain mail folders like the Trash.)

Webmail access using Roundcube

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

Apparently an increasing number of users does not want to up an email client like Thunderbird. Instead they believe that email is supposed to be managed using their web browsers. To each their own. So let’s set up RoundCube as a webmail software to make them happy. RoundCube is a web application written in PHP that provides a user-friendly web interface that speaks IMAP with any mail server.

Enabling RoundCube through the Apache web server

You should already have the ’roundcube’ package installed. The /usr/share/doc/roundcube-core/README.Debian file gives you additional information on how to make RoundCube available from your Apache web server.

If you are not using Apache virtual hosts (just one web site)

Basically if you just use the web server on your mail server to provide a webmail interface you can uncomment the “Alias” directives in the /etc/apache2/conf.d/roundcube file and restart Apache:

apache2ctl restart

If you are using Apache virtual hosts (co-exist with other web sites on your server)

If you have are running multiple virtual hosts on your server then do not enable the two “Alias” directives in /etc/apache2/conf.d/roundcube but instead put them into the <VirtualHost> section of the desired virtual host. For example you could set up a web server at https://webmail.example.org/ and add a

Redirect permanent / https://YOUR-MAILSERVER/roundcube

statement to your HTTP instance (listening on TCP port 80). Then restart Apache:

apache2ctl restart

Testing the login

Point your web browser to the /roundcube/ path of your mail server (e.g. https://YOUR-MAILSERVER/roundcube/). If all worked as expected you will be greated by RoundCube’s login dialog:

You should already be able to login using the username “john@example.org” and the password “summersun”. As a server enter “localhost” – we will get rid of that field in a minute. Click on “Login” and you should get to your inbox:

Fine-tuning the configuration

Let’s edit the main configuration file at /etc/roundcube/main.inc.php and incorporate these changes:

$rcmail_config[‘default_host’] = ‘localhost’;
(Disables the “Server” field and always uses the local server using IMAP.) 

$rcmail_config[‘force_https’] = true;
(Makes sure that you do not accidentally let users send their login data over an insecure HTTP connection.)

I recommend you go through the various settings to customize RoundCube with your logo and think about security and convenience features. All features are well-documented.

Sieve configuration

Sieve is a functionality that allows to use server-side rules to deal with emails on the server. Dovecot supports such rules and you have already set it up to listen on a network port for requests using the “managesieve” protocol that allows you to add, remove or edit such rules. RoundCube fortunately speaks “managesieve” – you just have to tell it how to talk to Dovecot. Start by copying the default configuration file to the required position:

cp /usr/share/roundcube/plugins/managesieve/config.inc.php.dist /etc/roundcube/plugins/managesieve/config.inc.php

Then edit the /etc/roundcube/plugins/managesieve/config.inc.php file and set the correct TCP port that Dovecot expects managesieve connections on:

$rcmail_config[‘managesieve_port’] = 4190;

 

Also enable the “managesieve” RoundCube plugin by editing this line to the /etc/roundcube/main.inc.php file:

$rcmail_config[‘plugins’] = array(‘managesieve’);

 

Now when you click on “Settings” in the top right-hand corner of the RoundCube web interface you will see four tabs where the last is called “Filters”. Click on it:

These are the rules that Dovecot obeys for the current email account. Well – there are none yet. But you can add any filter rules in the “Filters” pane by clicking on the “+” icon. That lets you add rules.

The “Filter name” is just a human-readable name that helps you remember what you intended when you created this rule (e.g. “move Tina’s emails to the ‘tina’ folder”). The “For incoming mail” section lets you define criteria that need to match if you want this rule to run. And the “…execute the following actions” depicts the actions that Dovecot will do with such emails.

Or lets consider another example:

This filter acts on all incoming email from the sender email address “tina@example.org” that has “cat pictures” in the subject line. Dovecot will then instantly send a copy of the message to “jill@example.com”. You can use these filters to sort emails to folders, delete certain bogus emails, respond to them or to active an out-of-office notification.

The filter rules are save in the mail directory of the respective user. For john@example.org such rules get saved to: “/var/vmail/example.org/john/sieve/managesieve.sieve”. The above rule would look like this in machine-readable format:

require [“copy”];
# rule:[Forward cat pictures to Jill]
if allof (header :contains “From” “tina@example.org”, header :contains “Subject” “cat pictures”)
{
redirect :copy “jill@example.com”;
}

So much for server-side rules. Have fun with them.

Malware scanning using AMaViSd

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

(I have not tested this kind of AMaViSd setup in production thoroughly. Proceed with prejudice. Please comment if everything worked for you. Thanks.)

(This section is completely optional. If you don’t want to filter incoming email for malware then feel free to skip it. Note that using AMaViSd and ClamAV as described here will increase your RAM usage by up to 500 MB and will considerably lower the througput of emails.)

In a previous section we already dealt with restrictions during the SMTP dialog. You learned about real-time black lists (RBLs) which help you block most of the incoming spam. There is another piece of software that can help you fight back internet goons: AMaViSd (A Mail Virus Scanner daemon). It has two main features:

  • reject spam emails or send them to a quarantine
  • reject malware emails

Fighting spam with AMaViSd is arguably not optimal. Quarantined emails seem to be confuse users and create work for system administrators. And the spam detection rate appears to be rather low compared to bayesian filters. You would have to train AMaViSd (which then uses SpamAssassin for spam detection) using serious amounts of ham (good) and spam (bad) emails. And the typical user won’t be able to tell spam from the mailing list newsletter they subscribed to but can’t remember. Spam detection is a very individual thing so it should be left to users and their email programs. Feed a mail client like Thunderbird one month of ham and spam emails and you will get a nearly 100% accuracy of spam detection. So to cut it short: this guide will not cover spam detection using AMaViSd. Sorry.

However malware is still a problem. Windows is losing market share thanks to MacOS and more user-friendly Linux desktops. But there are still many Windows users around and as MacOS is gaining shares it will probably get attacked, too. AMaViSd can fortunately help here. It makes use of any virus scanner that has a command-line interface. And there is even a free virus scanner called ClamAV that detect computer viruses, trojan horses and even phishing emails. Let’s make use of that.

Installation

Install AMaViS and its suggested packages:

$> apt-get install amavisd-new clamav-daemon clamav-testfiles clamav-freshclam

This setup is a tad complex and can go wrong. So if you are adding content filtering to your live server I strongly suggest you enable the soft bounce feature in Postfix. If Postfix intends to reject or bounce an email then this feature will rather keep the mail in the queue and try again later. Whenever you do major changes in your mail server setup I recommend you enable it:

$> postconf -e soft_bounce=yes
$> postfix reload

Alright – so how does AMaViSd plug into Postfix? Basically AMaViSd runs permanently as a daemon process and listens on a TCP port (10024) for incoming SMTP connections. Postfix will send all incoming email to that port. AMaViSd then runs all configured malware tests and decides what to do with the email. If it consider the email clean it will pass it back to Postfix – but on a different TCP port (10025) where Postfix will accept any email for delivery.

However by default Postfix doesn’t listen on port 10025. Neither does it know of AMaViS. So first you need to edit /etc/postfix/master.cf and add these two services:

amavis unix    –       –       n       –       5     smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o smtp_tls_note_starttls_offer=no

127.0.0.1:10025 inet n    –       n       –       –     smtpd
-o content_filter=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o smtpd_restriction_classes=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
-o local_header_rewrite_clients=
-o smtpd_milters=
-o local_recipient_maps=
-o relay_recipient_maps=

 

Note that all lines but the first lines of each service need to be indented by spaces.

(You can find detailed information on these services and the general configuration in the documentation in the file /usr/share/doc/amavisd-new/README.postfix.gz on your mail server.)

Restart Postfix to make it pick up these two new services:

postfix reload

Now you need to tell Postfix to use the “amavis” service as a content filter. Making Postfix forward emails to AMaViS is done by setting the content_filter setting. Also set the “receive_override_options” setting that will be explained later by running these shell commands:

postconf -e content_filter=amavis:[127.0.0.1]:10024
postconf -e receive_override_options=no_address_mappings

service postfix restart

The purpose is probably obvious. All email is forwarded to the “amavis” service defined in the /etc/postfix/master.cf using TCP port 10024 on the IP address 127.0.0.1 (localhost).

Why filter outbound email?

You may wonder why AMaViSd even scans outgoing email. But nowadays I would indeed recommend to allow scanning any email because it helps determine a virus infestation of a local PC. Ultimately you decide. There are different approaches described in the AMaViS documentation.

Configuring AMaViS

AMaViS is configured using different files in /etc/amavis/conf.d. You may want to check what is set there. Above all you need to change the 15-content_filter_mode file to enable virus and spam scanning. Confusingly the default settings are:

#@bypass_virus_checks_maps = (
#   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

#@bypass_spam_checks_maps = (
#   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

To enable scanning you have to remove the “#” signs in front of these four lines. It sounds like it would then bypass the scanning but instead it enables it.

The default scanner is ClamAV which is a free and decent anti-virus software. To use it you will have to get a group membership right:

adduser clamav amavis
/etc/init.d/clamav-daemon restart

The configuration files in /etc/amavis/conf.d are run in sorted order. A setting in “50-user” will override the same setting in “01-debian”. So if you feel you want to override any seeetings please put them into “50-user”. Make sure that your 50-user file ends with “1;” or else AMaViS will not start up properly.

Restart AMaViS if you have made changes to the config files:

/etc/init.d/amavis restart

Make sure that AMaViS is listening on TCP port 10024:

netstat -nap | grep 10024

You should get this output:

tcp  0   0 127.0.0.1:10024     0.0.0.0:*    LISTEN   12345/amavisd

If you get such a line then AMaViS is running and waiting for incoming SMTP sessions. Otherwise check your /var/log/mail.log file – perhaps you have made a mistake in the configuration files.

This should get you started. For a more detailed documentation please see the README.Postfix that provided by AMaViS.

Testing

Now that everything is set up you will want to test your spam scanning process. As you have installed the “clamav-testfiles” package you can use those samples to test ClamAV.

First try to scan the sample files in /usr/share/clamav-testfiles:

$> clamdscan /usr/share/clamav-testfiles/

ClamAV will then scan each of the sample files by querying the ClamAV daemon running in the background. Using the daemon (“clamdscan” instead of “clamscan”) saves you loading the malware database every time. You should see:

/usr/share/clamav-testfiles/clam.pdf: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.chm: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.sis: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.impl.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-nsis.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.tar.gz: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bz2.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.7z: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-yc.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ole.doc: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.newc.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-aspack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ppt: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ea05.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ea06.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.html: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-upack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.bz2: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bin-be.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.cab: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.mbox.base64: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.arj: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.binhex: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-fsg.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.mbox.uu: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.odc.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_cache_emax.tgz: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.tnef: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-mew.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-upx.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-pespin.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_ISmsi_ext.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.mail: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-wwpack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_IScab_ext.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_ISmsi_int.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-petite.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.szdd: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.d64.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_IScab_int.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bin-le.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.rtf: ClamAV-Test-File FOUND

———– SCAN SUMMARY ———–
Infected files: 44
Time: 1.478 sec (0 m 1 s)

If your output looks like this then ClamAV has successfully found the malware patterns in the test files. If you instead get “ERROR: Can’t connect to clamd: No such file or directory” the please make sure that the ClamAV daemon is running by executing “service clamav-daemon start”.

Now let’s try running an infested email through Postfix using the swaks tool to see if AMaViSd and ClamAV are used correctly:

swaks -t john@example.org –attach – –server localhost –suppress-data < /usr/share/clamav-testfiles/clam.exe

In addition to many lines in your /var/log/mail.log file dealing with “postfix” you should see one line dealing with “amavis”. It will look like this:

 29 23:16:23 sirius amavis[4558]: (04558-02) Blocked INFECTED (ClamAV-Test-File) {DiscardedOutbound,Quarantined}, LOCAL [::1]:44226 [::1] <root@mailserver.example.org> -> <john@example.org>, quarantine: C/virus-ClS0k87Qzm4Q, Queue-ID: 6608ED6B, Message-ID: <20131029221623.6608ED6B@sirius.localdomain>, mail_id: ClS0k87Qzm4Q, Hits: -, size: 1547, 128 ms

This proves that the ClamAV malware scanner is working and found a test pattern. Also the email was moved to /var/lib/amavis/virusmails/C/virus-ClS0k87Qzm4Q for your inspection. To learn how to deal with quarantined emails please see the AMaViS documentation. In a nutshell you could release this potentially dangerous file from the quarantine and deliver it nonetheless using “amavisd-release C/virus-ClS0k87Qzm4Q“.

Nearly done

Did you enable soft bounce earlier? Everything works? If the

mailq

command shows that there are still emails in the queue that need to be delivered then re-queue them first:

postsuper -r ALL

Re-scheduling means that Postfix reconsiders what to do with every email in the queue. Transport (routing) and content filtering information stick to an email. So even if you reconfigure Postfix then emails in the queue would not pick that up until you re-queue them.

Now flush the queue

postfix flush

and the emails should get delivered. The /var/log/mail.log will give you information on what happens.

If everything works as you expect then switch off soft bounce mode again and you are done:

postconf -e soft_bounce=no

Want more?

If you seriously want to run AMaViS on a production mail server then please spend time with its documentation. It’s a complex piece of software that can integrate a lot of third-party software like virus scanners. It can be tweaked a lot. It can automatically add DKIM signatures on outgoing email. It has a quarantine system so you can moderate spam emails. It just offers many additional features outside of the scope of this tutorial. If you believe you have a valuable addition to this tutorial then please send a comment on this page. Thank you.

Just one word on spam scanning. SpamAssassin (as being used by AMaViS) uses different rules to look for spam but can also learn to distinguish ham from spam emails. To do that properly it needs to scan an equal amount of several hundred ham and spam emails before it can really do its job. And as far as I know there is no way to learn spam per-user. So whatever you teach SpamAssassin is good or bad will apply for all your users.

Sysadmin niceties

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

This page deals with a few miscellaneous issues that system administrators should consider.

Logrotate

On a busy mail server your /var/log/mail.log will grow quickly. That file not grow indefinitely thanks to the logrotate software that gets installed by default. Every day (controlled by cron – see the /etc/cron.daily/logrotate file) log files will get rotated. That means the old file “mail.log” gets renamed to “mail.log.1” and the Syslog daemon is restarted so it opens a new empty mail.log file. Upon the next rotation cycle the “mail.log.1” gets “mail.log.2” which will get compressed with GZip so it becomes a much smaller “mail.log.2.gz” file.

This is controlled by the /etc/logrotate.d/rsyslog file. By default is defines the rotation cycle a “weekly” and “rotate 4”. So your mail.log is rotated once a week and after four rotation cycles the oldest file is deleted. Busy mail servers may need a daily rotation instead – so change the “weekly” to “daily”. And if you have to provide log files for more than four rotation cycles then increase the number from “rotate 4” to e.g. “rotate 90” to make it 90 days.

Message queue and queue IDs

Understanding the Postfix mail queue is very useful. Postfix will put every accepted email into its queue before another Postfix process picks it up and tries to deliver it. You can see what emails are currently in your queue by running

mailq

in the shell. On a busy mail server you may have hundreds of email being stuck for various reasons. Maybe an ISP is throttling your mail server because you are sending unusual amounts of emails (easily happens when sending newsletters) or the destination mail server is currently unreachable. An example entry in the queue may look like this:

04D8CA8C442     2399 Tue May 31 09:49:11  MAILER-DAEMON
(connect to mx.example.com[146.44.5.213]:25: Connection timed out)
info@example.com

The “04D8CA8C442” is the queue ID. To see what happened with this email you can grep through your /var/log/mail.log file:

grep 04D8CA8C442 /var/log/mail.log

The output could be:

May 31 09:49:11 mx1 postfix/smtpd[4428]: 04D8CA8C442: client=web.localnet[10.10.41.3]
May 31 09:49:11 mx1 postfix/cleanup[4473]: 04D8CA8C442: message-id=<20110531094911.0A73631F59D@newsletter.example.net>
May 31 09:49:11 mx1 postfix/qmgr[25512]: 04D8CA8C442: from=<>, size=2399, nrcpt=1 (queue active)
May 31 09:49:41 mx1 postfix/smtp[4945]: 04D8CA8C442: to=<info@example.com>, relay=none, delay=30, delays=0.05/0.06/30/0, dsn=4.4.1, status=deferred (connect to mx.example.com[146.44.5.213]:25: Connection timed out)

The different Postfix processes that handled the email are shown after the “postfix/…”. The “smtpd” received the email from the web.localnet server. Then the “cleanup” process put it into the mail queue. Next the “qmgr” moved the email into the active queue and the “smtp” tried to deliver it. The remote mail server mx.example.com could not be connected to so Postfix kept the email in the queue. The DSN (delivery status notification) code was 4.4.1. All codes starting with 4 are temporary errors – Postfix will retry to deliver the email. Codes starting with 5 are permanent errors and Postfix will instantly bounce the email and inform the sender of the delivery failure. Codes starting with 2 are successes.

If the delivery was successful the “postfix/smtp” line will look like this:

Jun  5 12:41:10 mail postfix/smtp[12044]: 2171CA860E0: to=<jonathan@example.com>, relay=mx.example.com[191.13.14.2]:25, delay=0.32, delays=0.04/0/0.17/0.1, dsn=2.0.0,
status=sent (250 2.0.0 Ok: queued as B0BAC736372)

The last part of the message contains a string that the remove mail server returns after the successful delivery. So if you see that an email got lost between your and the remote (mx.example.com) mail server this log line helps a lot. You see that the email was once handled in your queue as ID 2171CA860E0 and then delivered to mx.example.com which put it into its own queue as queue ID B0BAC736372. So you can try to contact the postmaster of the remote server and ask what happend to this email.

Getting statistics

Reading your mail log file barely gives you interesting information at a glance. Especially on a busy mail server you will barely get an idea what is going on except for counting the lines of the log file. I prefer to run “pflogsumm” (Postfix Log Summary) on my mail logs. You can easily install this software:

apt-get install pflogsumm

The just run this program on your log file:

pflogsumm /var/log/mail.log

What you get is general counts like:

messages

129   received
331   delivered
1   forwarded
8   deferred  (59  deferrals)
3   bounced
586   rejected (63%)
0   reject warnings
0   held
0   discarded (0%)

3108k  bytes received
3967k  bytes delivered
36   senders
28   sending hosts/domains
229   recipients
147   recipient hosts/domains

as well as per-hour traffic summaries (how many emails were sent and received during what hours). You also get a sorted list of sender and recipient domains. Aside from general statistics pflogsumm also extracts errors and warnings so you get a quick overview of why emails were deferred or rejected/bounced and the effectiveness of the RBLs (real-time blacklists) that you use.

Take a look at the example cron entries in /usr/share/doc/pflogsumm/examples on your system. They help you get an automatic daily and weekly report.

Blocking script kiddies with fail2ban

You are surely aware that the internet is not the friendly place it once used to be. Fortunately most of the attackers you will face at your mail server are stupid and adventurous script kiddies. They will use dumb scripts to try a few username/password combinations to get access to mailboxes. In your /var/log/mail.log this will look like this:

dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<mp3>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<oscar>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<webpage>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<webpage>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<bbuser>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<telecom>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<root>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104
dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<dbadmin>, method=PLAIN, rip=61.144.24.114, lip=87.152.157.104

We are not using user names like “oscar” but rather “oscar@example.com” so this embarrassing attempt will not work anyway. But why take the chance? We can use the “fail2ban” software to deal with brute force attacks like this easily. First install the fail2ban software on your server:

apt-get install fail2ban

Fail2ban starts a daemon process that can watch various log files, look for certain attack patterns there (defined by regular expressions) and act accordingly. We can teach fail2ban to look for “auth failed” lines like above and add a firewall rule (using iptables by default) to block access from the kid’s IP address. By default fail2ban will run such a blocking action it it finds three attack attempts within ten minutes and will then block the attacker for ten minutes. These values are configurable.

Add a file “/etc/fail2ban/filter.d/dovecot-pop3imap.conf” containing:

[Definition]
failregex = (?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
ignoreregex =

This adds a regular expression telling fail2ban what a failed login attempt looks like. The P<host> part at “rip” (remote IP) will contain the respective IP address of the attacker. fail2ban passes it to the blocking action to ban this very IP address.

Then edit the “/etc/fail2ban/jail.conf” file. First enable the “postfix” part that is disabled by default:

[postfix]
enabled  = true
port     = smtp,ssmtp
filter   = postfix
logpath  = /var/log/mail.log

Also add a section for the Dovecot configuration you added earlier.

[dovecot-pop3imap]
enabled = true
port = pop3,pop3s,imap,imaps
filter = dovecot-pop3imap
logpath = /var/log/mail.log

Restart fail2ban:

/etc/init.d/fail2ban restart

Now you can wait for kids to try their luck. But to check that everything works you can also fake an attack by sending manually crafted log entries:

logger -p mail.info -t dovecot “imap-login: Aborted login (auth failed, 2 attempts): user=<hanswaltergeorgfoo>, method=PLAIN, rip=10.20.30.40, lip=1.2.3.4, TLS”

Run the above command three times so that fail2ban triggers its action. If everything went as desires then your /var/log/fail2ban.log will read:

fail2ban.actions: WARNING [dovecot-pop3imap] Ban 10.20.30.40

Now attackers can do no more than three attacks within ten minutes. Nice, isn’t it?

Tracking last logins

Sometimes you may want to track when a user logged in last. Perhaps you have to deal with a lot of users who don’t use your mail server any more but you are still keeping their ancient mail accounts. To get rid of them you need to tell inactive from active users. In this case you tell Dovecot to run a shell script every time that a user logs in. These are the change you will need:

Edit your /etc/dovecot/conf.d/10-master.conf file. Look for the section called “service imap” that is usually empty. Make it:

service imap {
executable = imap imap-postlogin
}

This will connecto to a UNIX listener at every user’s login. So right below define this listener:

service imap-postlogin {
executable = script-login /etc/dovecot/scripts/last-login.sh
user = $default_internal_user
unix_listener imap-postlogin {
}
}

Create the /etc/dovecot/scripts directory (I like to keep my scripts away from my configuration files) and create /etc/dovecot/scripts/last-login.sh like this:

#!/bin/sh
echo “UPDATE virtual_users SET lastlogin=now() WHERE email=’$USER'” | mysql -h 127.0.0.1 -u mailadmin -pPASSWORD mailserver
exec “$@”

This script will update the “lastlogin” field of a user’s account on every login. Give it appropriate permissions so that Dovecot can execute it but nobody else can access the script and the secret password:

chgrp dovecot /etc/dovecot/scripts/last-login.sh
chmod u=rwx,g=rx,o= /etc/dovecot/scripts/last-login.sh

So you will need to create that field using this SQL query:

mysql> ALTER TABLE virtual_users ADD ‘lastlogin’ DATETIME;

As your “mailuser” MySQL user only has SELECT privileges you should create another user (I call it “mailadmin”) that has write access to the databsae to update it:

mysql> GRANT SELECT,INSERT,UPDATE,DELETE ON `mailserver`.* TO ‘mailadmin’@’127.0.0.1’ IDENTIFIED BY ‘PASSWORD’;

Finalls restart your Dovecot services and you are done.

Troubleshooting

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

General troubleshooting tips

  • Run “postfix check” to make Postfix look for obvious configuration errors. If it returns no output then no problem was found.
  • Read your /var/log/mail.log and look for warnings and errors.

Common problems and solutions

ClamAV fails to scan for viruses

May  7 12:42:28 debian amavis[1834]: (01834-01) (!)run_av (ClamAV-clamd) FAILED – unexpected , output=”/var/lib/amavis/tmp/amavis-20110507T124228-01834/parts: lstat() failed: Permission denied. ERROR\n”

You forgot to run

adduser clamav amavis

Missing indentation in the master.cf

The /etc/postfix/master.cf file needs proper indentation. The first line of each service starts in the first column. Additional lines of the same service need to be indented by spaces.

Postfix keeps unwanted emails in the queue

Check that you do not keep soft_bounce enabled. If “postconf soft_bounce” shows “yes” then run “postconf soft_bounce=no”.

I get “Permission denied” from Dovecot in the mail.log file

You have the permissions wrong. Run:

chown -R vmail:vmail /var/vmail

Postfix says: “Mail loops back to myself”

Postfix tried to send an email to the internet because it did not feel responsible for the recipient’s domain. However DNS records pointed it back to itself. So either your DNS record for a certain domain is wrong (unlikely) or your Postfix refuses to receive email for the given domain (very likely). Check that the recipient’s domain is listed in the “virtual_domains” table in the MySQL database. And verify that your CF files are working correctly.

General questions

Why is the database not normalized (email addresses with domain in virtual_users table)?

It’s possible to normalize the database and use the JOIN syntax to get the domain name from the virtual_domains table. But that would lead to string operations when Postfix and Dovecot look up a certain email address. This quickly becomes a performance penalty when you have many users. So in this case performance wsas rated higher than strict normalisation.

Why are the passwords stored in unsalted MD5 hashes? Isn’t that insecure?

Yes, that’s indeed less secure than using salted hashes. The password storage format has been kept unchanged though for server administrators that kept upgrading their Debian mail server from previous Debian versions.

If you are interested in making your mail server more secure and don’t have any legacy users then consider the Dovecot documentation on passwords schemes to use saltes hashes.

© 2021 workaround.org - Proudly powered by theme Octo