SMTPd restrictions, SPF, DKIM and greylisting

There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!

Postfix provides the ability to apply filters during the SMTP session. When an email arrives at your mail server the following steps are usually run through:

  • CLIENT:
    Incoming TCP connection on port 25
  • HELO
    The sending mail server tells you its name.
  • MAIL FROM
    The sender’s name and email address of the email
  • RCPT TO
    The recipient the mail is intended for
  • DATA
    In this phase of the SMTP session the actual email is transmitted
  • QUIT
    End of SMTP session

With the appropriate settings of SMTPd restrictions you can control what checks Postfix will run when an email is received. Those restrictions are defined in the /etc/postfix/main.cf configuration file and are called smtpd_…_restrictions. You can for example restrict certain IP addresses to even open an SMTP (TCP/25) connection to your mail server during the “client” phase. Or you can filter what recipients are allowed during the “rcpt to” phase. According to the above list of phases these restrictions apply:

  • smtpd_client_restrictions (default: empty)
  • smtpd_helo_restrictions (default: empty)
  • smtpd_sender_restrictions (default: empty)
  • smtpd_recipient_restrictions (default: permit_mynetworks, reject_unauth_destination)
  • smtpd_data_restrictions (default: empty)

So by default Postfix only applies checks during the “RCPT TO” phase to check if the email is either coming from your local network (the sending server’s IP address is part of the “mynetworks” setting) or if the recipient has a valid accounton your mail server. Otherwise the email would get rejected. It is important that you understand that the restrictions are checked in the respective phase. So it’s pointless to apply checks on the sender’s email address (MAIL FROM) in the HELO phase because that information is not yet known at that phase. Many mail server administrators just put all the checks into the smtpd_recipient_restrictions. The following sections give you some advice on how to configure your restrictions properly.

Realtime blacklists (RBL)

The most important technique nowadays to fight spammers are IP blacklists. There exist dozens of these lists that you can use freely. The operators of such lists have different policies regarding who is getting blacklisted. But most of them have spam probes on the internet that spammers fall for. The technique of spam traps means that otherwise unused email addresses are put on a web site in a hidden place or use them on mailing lists. Spammers then automatically spider (=scan) the internet for email addresses and then send their spam to the found addresses. If such a trap account gets email then that information is used by the operators of an IP blacklist to build a list of server that send spam. It is potentially dangerous to use IP blacklists though because you might block email from a legit server if that server was inadvertently blacklisted. My favorite blacklist “SORBS” for example has let me down several times by blacklistting entire internet providers for a period of hours or even days. So it’s a good idea to check their policies and see if you agree with it. In the end the risk is yours. Once you reject an email you cannot get it back. But I still strongly recommend you use blacklists or you will literaly drown in spam.

IP blacklists are specified by a domain name like “bl.spamcop.net”. To check if a certain IP address is listed in that zone you need to check if an entry for the reversed IP address exists in that zone. Assume that a mail server with the IP address 217.217.34.231 wants to send you an email. If you told Postfix to use the “bl.spamcop.net” blacklist then Postfix will reverse this IP address and check the DNS entry of 231.34.217.217.bl.spamcop.net. I just did that and got a result:

$> host 231.34.217.217.bl.spamcop.net
231.34.217.217.bl.spamcop.net has address 127.0.0.2

The fact that a result (127.0.0.2) was found means that the IP is blacklisted and that I should not accept emails from it. Postfix will then tell the sender that it triggered a blacklist and cancels the connection. The result returned almost always is 127.0.0.x where the X stands for the reason of the blacklisting. You will have to read the policy of the respective blacklist to see the reason. Let’s take another IP address that is not blacklisted – for example 85.214.93.191:

$> host 191.93.214.85.bl.spamcop.net
Host 191.93.214.85.bl.spamcop.net not found: 3(NXDOMAIN)

I did not get a result. So the IP is not blacklisted and I should accept email from it.

There are a few major blacklists that you should check out:

  • SORBS (dnsbl.sorbs.net)
    SORBS is very efficient but every few months tends to blacklist even large ISPs accidentally. I currently do not use it as I had too many user complaints.
  • SpamCop (bl.spamcop.net)
    Pretty reliable. On a sample day it blocked 29 mails on my server.
  • SpamHaus (zen.spamhaus.org)
    Also pretty reliable. On a sample day it blocked 147 mails on my server.

If you used these blacklists then the restrictions in your /etc/postfix/main.cf would be these:

smtpd_recipient_restrictions =
permit_mynetworks
reject_rbl_client dnsbl.sorbs.net
reject_rbl_client bl.spamcop.net
reject_rbl_client zen.spamhaus.org
reject_unauth_destination

(Note: You can also list the restrictions all in one line and seperated by commas. But I prefer the way to use multiple lines and indent the subsequent lines. It makes the main.cf file more readable.)

With these restrictions in place Postfix will run each of these checks after the “RCP TO” phase during the SMTP dialog.

  1. Is the sending IP address in your network range?
  2. Is it blacklisted by SORBS?
  3. Is it blacklisted by SpamHaus?
  4. Is your mail server responsible for the recipient’s domain? Or is the sender using SMTP authentication for relaying?

Further restrictions to consider

Postfix offers quite a lot of checks you can use in your smtpd_…_restrictions. Just to name a few that I use on my mail servers:

  • reject_unknown_client_hostname
    Runs a two-way DNS check of the IP address and hostname of the sending server. This can be potentially dangerous and reject legit email from misconfigured mail servers but nowadays every decent mail server is supposed to take care of their DNS settings.
  • reject_unknown_sender_domain
    Checks if the domain used in the sender’s email address actually exists. This runs a check for an MX and A record of the sending domain.
  • reject_unauth_pipelining
    Some spam bots just try to deliver emails as fast as possible. But formally the sending server must wait for the recipient server’s response in each SMTP step. If the recipient server allows pipelining then fast sending is allowed. This setting drops the connection if pipelining is attempted before it was offered by the destination server.

Further settings can be found in the Postfix documentation.

Are you blacklisted?

Of course you want to send out emails through your mail server. So you need to make sure that your server IP address is not blacklisted either. I suggest you check your server’s IP address through one of the blacklist test services:

(These services also give you an idea what other blacklists are available.)

If you are listed on any of these lists then you will want to get off the lists. How you do that depends on each blacklist’s policy. Some don’t even support manual removal from these lists. If your ISP assigned you a dynamic IP address then you are in a so called “dialup IP range” and are most likely blacklisted and it’s pointless to get off the list – read here for a solution. A lot of spam is coming from dynamic IP addresses. Either spammers send email from their ISP account. Or the large amount of virus-infested Windows PCs in the world is forwarding spam. So dynamic IP addresses are frowned upon by mail server administrators.

Source spoof prevention

The above section on IP blacklists helped you fight spam by rejecting it. Now let’s add another security measure to prevent source address spoofing. Spoofing means that someone fakes information. And the source address is the email address that appears to be the sender of an email. So it means that somebody pretends to be someone else by faking the email address of the sender. Maybe you haven’t given it much thought but basically everyone can send an email and pretend to be someone else. Perhaps you are even using multiple sender addresses yourself. What prevents someone from pretending he is you? There are two common options:

  1. SPF (Sender Policy Framework): Using SPF anyone can find out if an email was sent from an IP address (=mail server) that is allowed to act as a mail server for that sender domain. Technically with SPF you create a DNS entry for your domain and list the valid IP addresses there that are allowed to send email on your domain’s behalf.
  2. DKIM (DomainKeys Identified Mail): DKIM uses a private/public key pair to sign all outgoing emails automatically. By making your domain’s public key available through DNS every mail server on the internet can check if the signature is valid for your domain and nobody spoofed the email. Although SPF is more widespread than DKIM it has a major design flaw because it fails when you forward email (as the sending IP address changes and may not be a valid address listed in the SPF entry any more).

Setting your own SPF entry

The content of the SPF entry follows a certain syntax. To make things easy you can use the service at OpenSPF to set up your SPF entry. Enter your domain name there and fill out the fields. The web site will tell you how your SPF entry will have to look like. Add this to your DNS zone and you will both help others to fight spam as well as help yourself to prevent that anyone is sending unwanted emails on your behalf. Formerly a TXT resource record in the DNS zone was used but nowadays there is a distict SPF record type used for that purpose. After quite some controversy the “SPF” type of DNS records has been discontinued. So Using a TXT records is the way to go.

Checking other SPF entries

Many domains already have SPF entries. So you are encouraged to use that information for your own spam and phishing protection. Read the next section on greylisting to learn how to use the “tumgreyspf” service to use both the power of SPF and greylisting.

Forwarding breaks SPF

SPF has a major design flaw. It fails if someone forwards email to another server. Then the forwarding server is not listed in the SPF entry and the destination mail server will (correctly) reject the email. Less technically-savy users will not understand this limitation. So many mail administrators just take the SPF checks into account when looking for spam but they don’t ultimately reject the email. It’s up to you. SPF reliably fights spoofing and phishing for major domains like PayPal, eBay, Amazon or other large organisations that are often a target of phishers. In some cases it will just be overly careful.

Checking DKIM signatures of incoming email

DKIM was invented for a similar reason as SPF. It provides email authentication – to allow a receiving mail server to check that the email is authentic for a given sender domain. SPF just tells which IP are allowed to send email for a sender domain. But DKIM is working way more reliably because it adds a cryptographic signature to every outgoing email. To verify if an email is authentic a mail server can use the email’s signature and match it with the sender domain’s public key that can be obtained by DNS. Sounds too complicated? Actually it’s pretty simple if you understand the basics of public-key cryptography. Have you ever used PGP? Then you know that you had to create a keypair consisting of a private key you can use to sign emails and a public key that you can give others to encrypt emails for you or check your signatures. DKIM uses exactly that technique. The sending mail server has a keypair for each domain. When it sends out an email it will use the private key for the sender domain, take the email that is to be sent out and sign it with a cryptographic signature. That signature is added to the email (as an additional mail header) and the receiving mail server can take the email and the signature from the mail header and the public key of the sender domain to determine whether the signature was valid.

Adding automatic verification of other domains’ DKIM signatures is pretty easy. Just install the DKIM filter software:

apt-get install dkim-filter

Actually this software is called a “milter”. Milters are “mail filters” that were introduced as additional pieces of software for the ancient Sendmail mail server software. Milters speak a certain protocol that mail server can use to communicate with it and Postfix fortunately knows how to use milters, too. Right after the installation the “dkim-filter” process is already started and running in the background. But as Postfix is running in a chroot jail in /var/spool/postfix it cannot use the DKIM filter’s communication socket at its default location in /var/run/dkim-filter/dkim-filter.sock. You can fix the permissions and make DKIM run a socket in /var/spool/postfix instead but I prefer to rather run dkim-filter as a TCP service rather than a socket. Edit your /etc/default/dkim-filter file that contains its default settings and add

SOCKET=”inet:54321@localhost”

to it. Restart the dkim-filter process:

/etc/init.d/dkim-filter restart

Now dkim-filter is listening to TCP port 54321 on the localhost interface. To make Postfix use this milter run these commands to add the appropriate milter definitions to your /etc/postfix/main.cf file:

postconf -e smtpd_milters=inet:127.0.0.1:54321
postconf -e non_smtpd_milters=inet:127.0.0.1:54321

Now Postfix will run all emails (both incoming and outgoing) through your DKIM filter and check DKIM signatures (if found) on incoming emails. It will not sign outgoing emails though – this is covered in the next section.

A side note: you can decide what should happen if there is a problem with the DKIM filter. By default Postfix will temporarily reject emails until you fixed the issue. It’s up to you. But I prefer to just accept emails if there is a milter problem. So I set:

postconf -e milter_default_action=accept

For more information on milters please consult the Postfix documentation on milters.

Setting up DKIM for sending emails

The actual idea of DKIM is to make a domain’s mail server sign outgoing emails automatically. To do that you need to create a cryptographic keypair first. Fortunately you do not have to buy a certificate because publishing the key in your DNS zone is enough. The tool you need is “dkim-genkey” and it requires the bit-length of the key and the domain name. You can optionally use a selector to use multiple keys. By default the selector is called “default”. I suggest you use the domain name as your selector which will name your key files correctly automatically.

It’s best to create a seperate directory to store the keys. I keep mine in /etc/postfix/dkim:

mkdir /etc/postfix/dkim
cd /etc/postfix/dkim

Then create the keypair there:

dkim-genkey -b 1024 -d example.org -s example.org

In your current directory you will now find two files.

  • “example.org.private” is the private key that the DKIM milter will use to sign all outgoing emails
  • “example.org.txt” is the public key that you are supposed to add to your domain’s DNS zone

The configuration file for the DKIM milter is /etc/dkim-filter.conf. Edit that file. The only change you need is to add a line for the key list file like:

KeyList     /etc/dkim-keys.conf

Create a new file /etc/dkim-keys.conf that you just pointed to. Its contents define which key is used for which sender address. Or to be specific the syntax is:

sender-pattern:signing-domain:keypath

as pointed out in the “man dkim-filter.conf” man page. So if your domain were example.org you would write:

*@example.org:example.org:/etc/postfix/dkim/example.org

Add one line per (sender) domain that you wish to sign outgoing emails for.

Caveat: dkim-filter adds “.private” to the keypath you specify. So the above line is correct. Do not specify /etc/postfix/dkim/example.org.private or signing will fail.

Restart your DKIM milter:

/etc/init.d/dkim-filter restart

Now outgoing emails should get a DKIM signature automatically. Send an email to another mail server and check the headers of the email. You should see a header like this:

DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=example.org; s=example.org t=1307571770; bh=OtZOIOSkkhL1t+kR5KOE6HvvjUjLkDE+70agcKmcjJg=; h=Message-ID:Date:From:MIME-Version:To:Subject:Content-Type: Content-Transfer-Encoding; b=RlxE9IRiOJrOzM4JbJFIp/YmK4XBRJFti79rL0vzGkppAC1AJi2CdtwGFT/sTovt/iKikrdyQ7M7JhdlC9u3bh8rrvhLA53HZCmu6WvHvv059ysdpjUhrktEqZFrFpgds2wqCkzF4ar/ly3TzYZmoZJO+C8j2uU5L3cKzESfGw4=

The meaning of each parameter can be found on the Wikipedia page. or in the less human-friendly format definition in the RFC 4871.

However the recipient cannot tell if this is a correct signature because they don’t have the public key to verify this signature with. Edit your DNS zone for your domain and just add the TXT record that “dkim-genkey” created for you in /etc/postfix/dkim/example.org.txt. On my test server it looked like this:

example.org._domainkey IN TXT “v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDKPFhCADCGcxDchGHFqvSoQYuW0epBT0gJuCwDMkX0uPejeeMdBNDcaGo8AyZdXJTFKqGK5kao2OvpDsH6XeKMBjPt/CnyBm4PNwlwNrkJTBc15xjD6Swmlk457+Ioz/tbpBA3b3RnC8NsqiknLQ3JxDkE7fXfji7Uds5+swWwJQIDAQAB” ; —– DKIM example.org for example.org

Verify that your DNS zone for your domain really contains the right entry by querying for a TXT record with your domain name and the selector you used:

dig +short example.org._domainkey.example.org txt

And it should return your public key:

“v=DKIM1\; g=*\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDKPFhCADCGcxDchGHFqvSoQYuW0epBT0gJuCwDMkX0uPejeeMdBNDcaGo8AyZdXJTFKqGK5kao2OvpDsH6XeKMBjPt/CnyBm4PNwlwNrkJTBc15xjD6Swmlk457+Ioz/tbpBA3b3RnC8NsqiknLQ3JxDkE7fXfji7Uds5+swWwJQIDAQAB”

Now all outgoing emails for …@example.org will get a DKIM signature added automatically and recipients can verify them. Any mail server relying on DKIM signature checks can now check whether emails from your domain are legit.

I assume that you have set up the DKIM milter in Postfix as described in the previous section. Setting “smtpd_milters” and “non_smtpd_milters” in your /etc/postfix/main.cf will now do both the verification of DKIM signatures as well as the signing.

Greylisting

Another way to help you reduce the amount of spam you receive is using greylisting. Whitelisting means that a certain kind of email is always accepted (delivery status  2.x.x). Blacklisting is the opposite and always blocks certain emails (delivery status 5.x.x). Greylisting is something in between and temporarily refuses to accept the email for a couple of minutes. The idea is to tell virus infested computers trying to spam you from actual mail servers. Viruses (or rather: spam worms) just try to deliver a spam email once. But real mail servers have a queue of outgoing emails and upon a temporary error they will try to re-deliver the email. If the sending mail server has never sent us an email then greylisting will send back a temporary error code (delivery status 4.x.x) which signals the sending server to try delivering the email again in a few minutes. Your server will take a note of the IP address of the sending server and after a set period (a common value is 10 minutes) the blockade is lifted and upon the next delivery attempt the email will be accepted. Also the greylisting service will take a note in its cache and further delivery attempts will be allowed instantly.

The obvious drawback is that emails are often delayed by several minutes – especially if you just start using greylisting and the cache of trusted sending servers is empty. It is your decision whether your users will accept that. In my personal experience greylisting is not as effective nowadays as it has been in the beginning. It appears like malware now just tries to deliver spam emails several times. But it’s one measure to maybe block some spam emails.

There are several ways to implement greylisting. The way I do it is using the “tumgreyspf” software. Start by installing it:

apt-get install tumgreyspf

To make Postfix use tumgreyspf you have to define a service for it in the /etc/postfix/master.cf:

tumgreyspf unix  –      n       n       –       –       spawn
user=tumgreyspf argv=/usr/bin/tumgreyspf

You also need to add it to your smtpd_recipient_restrictions of your /etc/postfix/main.cf file. Example:

smtpd_recipient_restrictions =
permit_mynetworks
permit_sasl_authenticated
reject_rbl_client bl.spamcop.net
check_policy_service unix:private/tumgreyspf
reject_unauth_destination

These are example restrictions. Do not just copy/paste it. The example is just meant to show you a good place to add the policy check in your smtpd_recipient_restrictions.

Finally restart Postfix:

postfix reload

To customize the behavior of tumgreyspf you can alter the files in /etc/tumgreyspf.

(This procedure is also documented in the /usr/share/doc/tumgreyspf/README.Debian file.)

Scroll to Top