Prevent spoofing with DKIM

Email sender spoofing is the act of pretending to be in control of someone else’s email address. This is a common problem with phishing. Frequently scammers send emails with a sender address of something@paypal.com and hope that the recipient falls for it and trusts them. In fact SMTP does not care which sender address you send. Many mail service providers enforce that you send emails only using your own email address. But some do not. And spammer and scammers obviously do not care at all.

Example without DKIM

So a new method was conceived that added a cryptographic signature to the header of an email that the recipient could check to verify the authenticity of the sender and the integrity of the email. The signature is created using a private key that only the sending mail server has. It can then be verified by the recipient by downloading the corresponding public key from the DNS zone of the sending domain and running a signature check. This works very similar to PGP or S/MIME signing – just on a domain level. Your mail server can sign all outgoing emails automatically. The method used nowadays is called Domain Keys Identified Mail – or short: DKIM.

Let’s take an example. I have just sent an email from GMail to my personal email account on my own mail server. Google uses DKIM signing so the email got this additional header from Google’s mail servers:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=/FpkZfBuKR0WWcH2fFcr9M4qgX4Z4+/0dX4dpqycK28=;
 b=fZ4Efy1TuXAl1ho2twkEuNiVP8k5GRKqlol/f/dTawaxciAv1bwinkbuTCpK4T3SoL
 ZFhdh/p82MYiIt75V5eGFtSMQfocVIKfsusggj3ZKlwUNvSbB/jd4fn3SKwGjLSyKMT0
 agJFblxF+ydGzRKYzJNPfFYhdQY3YZcMEria87SpqgVgqECCMRvuT10w7KQGIx7AsvDy
 F2coh2TnX400sRidMVF1S/9QhD85dE3xSie3ZdTfWBsP00Y7xWbos7MlSl1MnsVvXPtQ
 7bnNBVdcn1tP9x8IeNKp6qEYoLYzEeLZVp8eLB0F0AoWIgRb348FaIvMF9jYvO/h/cQ+
 YP5Q==

I need Google’s DKIM public key to verify that signature. It is stored in their DNS zone as a TXT record of “20161025._domainkey.google.com”. The “20161025” is the key selector that is mentioned in the signature as “s=20161025“. You can use any number of keys as long as you create the signatures with the matching private key.  The “_domainkey” part is the standard subdomain for DKIM keys. So let’s get that TXT record:

dig +short 20161025._domainkey.google.com txt

“k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwXNZF1j8sJPDleRjf9SPBNem0ik58kF1ilC1nUgKAttl9v7FX9hXJXPmLNhVtSKVZ8yruaeOZLeIxtgtk1s81zzIE5Mj0AiGn2wlFt4kYfqlDfYe95YLQHjynu4i7vj1Tj” “ksf62btcCbL+3XhbK+oD5PlqYhXHWuzoKoEp5L4lCihgkONvU/oy7NNeE6quqfF/y0YSLwF2WVA2Kd8L6R0Ar2dYT/3wZCFknI7xhvPqh9HNcIWBELGPwtXcsHbX1wvBlCgNQAUcdJrf2YWzAwqmZ564/1ipL1IMk1nafPJk75ktumVNz6ORuIn3jbZWp9rRpnaeI9cu/8KfSKH2EY9QIDAQAB”

This is the public key that I can use to verify the signature. An automated check can be done using the “opendkim-testmsg” tool as described later. I can run it and paste the entire email including headers and body into it. If it doesn’t complain with an error message then the signature is correct.

Sounds good? Okay, let’s implement that for your email domain, too.

Creating a keypair

As explained above you need a private key that your mail server will use and a public key that gets added to your DNS zone. rspamd can create DKIM keys already. You may want to install “dig” though which allows to query DNS records. It works like “nslookup” but is more versatile.

apt install dnsutils

rspamd has its built-in DKIM signing module enabled by default. If you put your key file into /var/lib/rspamd/dkim/ using a certain naming scheme it will pick it up automatically. Create that directory to store keys in:

mkdir /var/lib/rspamd/dkim
chown _rspamd:_rspamd /var/lib/rspamd/dkim

Create your keypair:

rspamadm dkim_keygen -d example.org -s 2018022301

The selector (-s) I chose is 2018022301 because that’s the day I created it. The first (01) key on 2018-02-23. It doesn’t matter though – you can name it anything you want. If you are lazy and unconcerned you could even use “dkim” as the selector and later spare some work by not needing DKIM maps that define which key is supposed to be used for each domain. “dkim” is the default selector if you do not use maps. However if you need to replace your key later without invalidating previously sent emails you are in trouble. So I recommend you rather use maps as explained further below. It gives you more flexibility and is pretty easy to do.

The output will look like this:

-----BEGIN PRIVATE KEY-----
MIICeAIBADANBgkqhkiG9w0BAQEFAASCAmIwggJeAgEAAoGBALF6ce6mRgs+YUxs
mNX11AgIpSXbEf9p7FJFTAK7DtMn12R12og37q3LDhUqGdbxHT9SVvOVIUuZWAxT
S5STBO1WrIMeG1JTstX9KRViZSy3h1NQXWv7w++ECbu+hsxgp3C9Gw6rbsAG1lzw
/ihTZdceqprotTyziw3GzXaEtb2zAgMBAAECgYEAltFCuFa79xoPfO5cApZglGMk
VEMRzEaq+E7DBR8Ivg+BTnGIP3Godi+zy/B2/NzwybqjAWOdY6zv0amG9L7q04sM
BnhM06S+DFbmetEV0CKGg+yGQ3QRIxm5ZeFj7OOcssPxzyTrzI3MU2vhDkOrKycE
kaiWu4PI2TwvBF7PtGECQQDaq4rJfd8IITtL84qqt6DxWNb45cuay8oWnq2kjU0k
nkcLhpvCdO5a037AUnBmpQzQWxsE1OMhl54oV6QY8EhPAkEAz8a2q2wshFuiSBjV
BU3kXELdjqxEoti+zBuuT8QldCUntHbG1Vyo8A6ObVOnbbGEJmmRqjajjBV3Imvj
+NW3XQJATiqR8xLmnh5a9vx47iu0AXITbmeej5gXhlx0j+Te1Z8qoEke9rUb2yXN
ImyVt2t6mCRpFSIQixqKs7sYcr256wJBALFbSDZPojhhF8c/Gybp9FIk6YaN2hC/
vy3vOv3vyqC3zoqnENazv6kaY81CqRpjexBm8OZAuZ4r/IOvYOvZUikCQQCtWZtZ
8oYlVB2R4OMF5CMrjEkMNW9J5wZwPj7NHdbXLTtzDoI+oCrjus9foffe3T5ADFQ6
x6ZDbufbaRujG8/B
-----END PRIVATE KEY-----
2018022301._domainkey IN TXT ( "v=DKIM1; k=rsa; "
 "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCxenHupkYLPmF … QAB" ) ;

The first part is the private key. And that includes “…BEGIN…” and “…END…” lines. This key must be kept secret and will only be used by your mail server to sign outgoing emails.

The second part is the DNS record you need to add to your domain’s DNS zone. Let’s start with that.

Adding the DNS record

Before you start signing your emails you must make sure that the public key is properly present in your DNS zone for the domain you are sending emails from. Otherwise the recipient will be unable to verify the signature and may incorrectly assume that the email was spoofed.

Take a look at the 2018022301.txt file. It will look something like this:

2018022301._domainkey IN TXT ( “v=DKIM1; h=sha256; k=rsa; “p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCxenHupkYLPmFMbJjV9dQICKUl2xH/aexSRUwCuw7TJ9dkddqIN+6tyw4VKhnW8R0/UlbzlSFLmVgMU0uUkwTtVqyDHhtSU7LV/SkVYmUst4dTUF1r+8PvhAm7vobMYKdwvRsOq27ABtZc8P4oU2XXHqqa6LU8s4sNxs12hLW9swIDAQAB” ) ;

If you are running your own DNS server you should be able to copy this entire file and put it into your DNS zone. However if your internet provider offers you just a web interface to manage your domains then create a new TXT record with a host name of “2018022301._domainkey” and put the string within the double-quotes into it as the value. In my example:

2018022301._domainkey ➠p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCxenHupkYLPmFMbJjV9dQICKUl2xH/aexSRUwCuw7TJ9dkddqIN+6tyw4VKhnW8R0/UlbzlSFLmVgMU0uUkwTtVqyDHhtSU7LV/SkVYmUst4dTUF1r+8PvhAm7vobMYKdwvRsOq27ABtZc8P4oU2XXHqqa6LU8s4sNxs12hLW9swIDAQAB

Depending on your ISP it may take a while until the new record is visible on the internet. You can use dig to verify that:

dig +trace 2018022301._domainkey.example.org txt

If you get the TXT entry like as follows then you are ready to enable DKIM signing in rspamd for that domain:

TXT “p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCxenHupkYLPmFMbJjV9dQICKUl2xH/aexSRUwCuw7TJ9dkddqIN+6tyw4VKhnW8R0/UlbzlSFLmVgMU0uUkwTtVqyDHhtSU7LV/SkVYmUst4dTUF1r+8PvhAm7vobMYKdwvRsOq27ABtZc8P4oU2XXHqqa6LU8s4sNxs12hLW9swIDAQAB” “” from server foo.bar in 24 ms.

Enabling DKIM maps in rspamd

As explained above it is advised to use DKIM maps. It’s nothing fancy. Just a simple file defining which selector you want to use for a certain domain. rspamd will assume that your selector is always “dkim” unless specified otherwise in a map. If you used “dkim” then you may get into trouble when you later want to replace your key. DNS is a sluggish system and propagating a new DKIM public key may take a day. Emails signed with a newer key may get rejected while the DNS record is not yet known everywhere in the world.

Using maps is simple. First we need to change the selector_map setting of the dkim_signing module. To do that create a new file in /etc/rspamd/local.d/dkim_signing.conf and make it contain just these two lines:

path = "/var/lib/rspamd/dkim/$domain.$selector.key";
selector_map = "/etc/rspamd/dkim_selectors.map";

The configuration is pretty self-explaining. rspamd will look for the domain-to-key mapping in the dkim_selectors.map file. Create that file and make it contain this line:

example.org 2018022301

That’s all really. rspamd now knows that whenever it sees an outgoing email from anyone@example.org it will get the DKIM private key from /var/lib/rspamd/dkim/example.org.2018022301.key and use it to sign the email.

Reload the configuration:

service rspamd reload

This method works well if you just have a few domains that virtually never change. If you are rather serving random customer domains you should consider putting the keys into a Redis database instead as described in the documentation. There is not yet a way to manage DKIM keys in a database like MySQL.

Adding the domain key to rspamd

Take the private key that was created earlier (the multi-line string including “…BEGIN PRIVATE KEYS…” and “…END PRIVATE KEY…”) and put it into a file at the location where rspamd will look for it:

/var/lib/rspamd/dkim/example.org.2018022301.key

The name of the file has to be  DOMAIN + dot + SELECTOR + “.key” like above. If you name the file incorrectly you will get an error in your rspamd.log file like “lua_dkim_sign_handler: cannot load dkim key /var/lib/rspamd/dkim/example.org.dkim.key“.

Make sure that only _rspamd can read it:

chown _rspamd /var/lib/rspamd/dkim/*
chmod u=r,go= /var/lib/rspamd/dkim/*

rspamd will automatically pick up the files and does not need to be restarted.

Send a test email

If you have another email account at another location then you could just send a test email there via your mail server. If you take a look at the received email it should have a DKIM header now like:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.org;
	s=2018022301; t=1519400838;
	h=from:subject:date:message-id:to:mime-version:content-type:content-transfer-encoding;
	bh=kpYel1IlDvqXEUc0SyIpXbMte3XpQOCXHl+zTyHQvGc=;
	b=NEUyWUoeKEoKAYTY8g04o73j+wrYUcEGSq7uwpbsAGo0OzuuIBluEfG1MbGF/Tf6yxxJB4
	gTDD3sqb19EsQxv39QsAwgddAz01Osw5LKU0MjLZpxw6NA8zLllJUsrNdNQAYSII9ip4xX
	ImU7+KFOEF+gmxR5aseUt5H6JT/aOmhPE9xsSyg9wLf0Bikyy5Cgh+Ay7AHQLMZogbTi9W
	dAPpZZcZs0pTwhcard6SaesypJ+xZNna+BA+C1vXrGDc+9stYZVi+Zufh6zlZo1E/sQSRL
	jowB1mjv1vjINRY30aq0rh4dT8RHe38/PKFf8vQHOSOKvjIKv984UeOTIFIUHw==
DKIM signature verification

To verify the signature install the opendkim-tools package, copy the entire test email (including headers and body), run opendkim-testmsg in your shell and paste the email (finish with CTRL-D).

If you get no output then the signature verified correctly. But if you get something like “opendkim-testmsg: dkim_eom(): Unable to verify” then double-check your DNS record.

You can also use the dkimvalidator.com service to verify that your signatures are working well.

SPF and DMARC

Adding DKIM signatures is a good first step. But you can take it further by telling receiving mail servers that they should not accept any email from your domain without a valid signature or from servers that you do no operate. There are two concepts that aim to help. The older SPF and the newer DMARC. Either of them means creating a machine-readable string in a predefined format and adding a TXT record to your DNS zone. Receiving mail servers can check those records and take your advice (as the domain owner) what to do if the criteria of the email are not met. It could accept the email anyway or flag it as spam or reject it altogether.

Let’s take a look at a typical SPF record:

"v=spf1 ip4:157.97.194.11 mx ~all"

What it means:

  1. this is an SPF record of version 1 of the standard (there is currently no other version)
  2. please accept emails from the IP address 157.97.194.11
  3. alternatively accept emails from any server that is mentioned in our domain’s MX record (the server(s) that receive email for your domain)
  4. any other email should be considered suspicious – it might be spam or worse

There are websites like SPFwizard that help you create your SPF string to add to your DNS domain. Keep in mind though:

  • You should know which mail servers send email from your domain. Do not forget to include mailing list or newsletter services that send in your name.
  • Start with “~all” to mark emails as spam that do not meet the criteria. If all goes well switch to “-all” after a few weeks if you like.
  • Note that forwarding emails from your domain may break SPF because suddenly the email appears to be coming from an IP address that is not authorized. This has been a common problem for mailing lists and is gradually being fixed by resending the email from the domain of the mailing list service.

I mentioned that DMARC is the newer standard. So why use SPF anyway? Because some email providers value your effort if you use SPF, too. Technically it’s sufficient to specify a DMARC entry. In my opinion restricting the IP addresses allowed to send is a little dangerous and a little inflexible. It is fare more interesting to require that emails from your domain have a valid DKIM signature. Such a record may roughly look like:

"v=DMARC1; p=reject; adkim=s"

However to create a proper DMARC entry I suggest you use one of the web sites that aid you there and explain the restrictions and extra features.

Fighting brute force attacks

So many bad guys everywhere. Paranoia you say? Well, take a look at your mail log. While I was preparing this guide I could barely follow it. The mail server I used for testing was by far not ready and I already got visits by a bunch of chinese IP addresses trying to guess accounts of email users. Scoundrels constantly scan the internet for mail servers and do not tire to find a way to abuse them. Let’s do something about that before they actually guess email accounts successfully. (more…)

Success stories

I would appreciate if you leave a comment here telling others if your mail server is running. Perhaps even a word about the purpose of the mail server and the number of users you have. That allows others to see that their time is well-spent and indeed leads to a working mail server. Thanks for your feedback.

(Have I mentioned the donation button at the bottom? 🙂 )

Troubleshooting your mail server

General troubleshooting tips

  • Run “postfix check” to make Postfix look for obvious configuration errors. If it returns no output then no problem was found.
  • Read your /var/log/mail.log and look for warnings and errors.

Postfix keeps unwanted emails in the queue

Check that you do not keep soft_bounce enabled. If “postconf soft_bounce” shows “yes” then run “postconf soft_bounce=no”. To remove all emails from your queue (dangerous!) run “postsuper -d ALL”.

I get “Permission denied” from Dovecot in the mail.log file

You have the permissions wrong. Run:

chown -R vmail:vmail /var/vmail

Postfix says: “Mail loops back to myself”

Postfix tried to send an email to the internet because it did not feel responsible for the recipient’s domain. However DNS records pointed it back to itself. So either your DNS record for a certain domain is wrong (unlikely) or your Postfix refuses to receive email for the given domain (very likely). Check that the recipient’s domain is listed in the “virtual_domains” table in the MySQL database. And verify that your CF files are working correctly

Filtering out viruses and malware

My usual rant: I consider virus scanners snake oil. If anyone wants to seriously infect a computer by sending an infected attachment then virus scanners won’t help you. They are based on patterns of known malware and (if you are lucky) a working heuristic algorithm trying to detect unusual behavior. It has been proven time and again that a custom-made virus – even a slightly altered variant or a well-known virus – goes easily undetected by your virus scanner. So why do they even exist? FUD. IT managers know well about the weaknesses of this approach. But if any Wind*ws computer gets infected they can tell senior management that they employed all common measures. I have talked to several managers and they all tell the same story. Seems like the digital religion of the 21st century. 🙂 Anyway – if you decide that you want virus scanning let’s add it to your mail server.

Pros and cons of ClamAV

As usual I prefer to stay with open-source components. So the obvious choice is the famous ClamAV virus scanner. But its detection rate is ridiculously low with its default signatures. I heard of a study telling that in their tests ClamAV showed a 100% detection rate. Another gave it rank 2 of all the scanners they tested. I myself have been scanning multiple infected emails with it (that have been found by another software) and it did not even find one infection.

So I started researching the alternatives – like a free-beer virus scanner from Sophos that worked well with the amavis-milter in the past. Its detection rate is pretty good. And the rspamd developers even said that their antivirus module supports Sophos. But no matter what I tried – I couldn’t get the two to work together. And to be honest… the documentation of rspamd’s antivirus module is incomplete to put say the least.

Then suddenly a friendly nerd pointed me to the SaneSecurity project that adds anti-virus signatures to ClamAV to make it actually useful. And this finally became the recommended solution for the ISPmail guide. In a commercial setup where your customers rely on better virus scanning you probably cannot avoid spending money on a commercial variant though. Why do I tell you all this? Mainly as an excuse that it took two weeks to just write this page. 🙂

ClamAV

Let’s get going and start ClamAV. First install the necessary package:

apt install clamav clamav-daemon

The daemon is a permanent process running on your system waiting for connections from other programs. That makes it faster than starting a process to scan email attachments time and again. The default configuration works well so there it nothing to do.

Check if the daemon is running already:

service clamav-daemon status

Look for the “Active” line in the output. Does it say “Active: active (running)”? Very good. Or does it say “Active: inactive (dead)”? Don’t worry. Most likely it has not downloaded the antivirus patterns yet. ClamAV comes with the “freshclam” daemon that does these updates automatically. But after a fresh installation you may want to save time and download the patterns manually once. Stop the freshclam daemon and do an update:

service clamav-daemon stop
freshclam
service clamav-daemon start

Run “service clamav-daemon status” again and check that it is running now.

Did you get a message like…?

WARNING: Your ClamAV installation is OUTDATED!
WARNING: Local version: 0.99.2 Recommended version: 0.99.3
DON'T PANIC! Read http://www.clamav.net/documents/upgrading-clamav

Do as it says: don’t panic. 🙂 Debian Stretch comes with version 0.99.2 which is perfectly fine. You do not need to do anything about it.

Let rspamd use ClamAV

Finally you have to tell rspamd how to talk to the clamav daemon. Unfortunately the documentation is a bit misleading or at least incomplete which confused me at first. The ClamAV daemon is by default creating a socket file on your server at /var/run/clamav/clamd.ctl. Sockets look like files and work as interfaces for interprocess communication. rspamd can communicate through that socket to ask ClamAV if a certain email contains a virus.

The default configuration can be seen in /etc/rspamd/modules.d/antivirus.conf. Let’s improve it a bit. Create a new file /etc/rspamd/override.d/antivirus.conf with this content to replace the default section:

clamav {
 attachments_only = false;
 symbol = "CLAM_VIRUS";
 type = "clamav";
 action = "reject";
 servers = "/var/run/clamav/clamd.ctl";
}

So what does this mean?

  • attachment_only… actually defines whether just attachments are scanned or whether the entire email including images gets scanned. This setting appears to be broken and skip attachments, so I recommen setting it to false.
  • symbol… If a virus is found by ClamAV then rspamd adds a symbol named CLAM_VIRUS. You may remember that these symbols can be assigned scores or added as email headers to see which criteria matched. You can call it whatever you want.
  • type… sets a few defaults in the antivirus module that tell rspamd how to talk to ClamAV
  • action… is an optional parameter that we use to instantly reject the email if a virus is found. We do not add a value to the spam score. Instead we refuse to accept the email during the incoming SMTP connection.
  • servers… defines the way to communicate with ClamAV. In this case we define the path of the socket that the ClamAV daemon opened for us.

Restart rspamd:

service rspamd reload

Virus scanning should now be working. Let’s see if it works as expected.

Testing

As mentioned earlier there exists a test signature for virus scanners called EICAR. It is not a virus – just a file that can be fed to your virus scanner to see if it is generally working. Download the test file from their website:

wget https://secure.eicar.org/eicar.com

If you installed the swaks tool as suggested earlier you can use it to send a test email with EICAR attached:

swaks --to john@example.org --attach - --server localhost < eicar.com

One of the last lines should read…

<** 554 5.7.1 clamav: virus found: \”Eicar-Test-Signature\”

This proves that Postfix send the email through rspamd which in turn checked the email using ClamAV which again found the test signature and rejected the email.

Better detection signatures

As said at the beginning the detection rate of the signatures provided by ClamAV are pretty ridiculous. We do not want to spoon feed our vacuum cleaner – we want a clean living room. Fortunately the SaneSecurity people help make ClamAV actually find real-life viruses. The have been providing virus signatures for over a decade and claim that they find over 97% of all viruses while ClamAV usually just finds about 14%.

Friendly nerds have created an automated update script that can be found at Github. Setting it up takes a few steps but is then completely automatic. First get their newest installation archive:

wget https://github.com/extremeshok/clamav-unofficial-sigs/archive/master.tar.gz

Unpack it:

tar xvzf master.tar.gz

The archive has been unpacked into a directory called clamav-unofficial-sigs-master. Go there:

cd clamav-unofficial-sigs-master

Then copy the script itself to /usr/local/sbin where administrative user-installed scripts live:

cp clamav-unofficial-sigs.sh /usr/local/sbin/

Create a new directory for the configuration files:

mkdir /etc/clamav-unofficial-sigs

Copy the configuration template for Debian Stretch to that directory:

cp config/os.debian9.conf /etc/clamav-unofficial-sigs/os.conf

Copy the two other required files:

cp config/{master.conf,user.conf} /etc/clamav-unofficial-sigs/

To enable the update script you need to edit the /etc/clamav-unofficial-sigs/user.conf file and at the end set:

user_configuration_complete="yes"

Install a logrotate configuration to compress and archive older log files created from the script:

/usr/local/sbin/clamav-unofficial-sigs.sh --install-logrotate

Install the manual page (man 8 clamav-unofficial-sigs) for the script:

/usr/local/sbin/clamav-unofficial-sigs.sh --install-man

Copy the systemd configuration for recurring updates (works like cron):

cp systemd/* /etc/systemd/

Try updating the signatures manually:

/usr/local/sbin/clamav-unofficial-sigs.sh

If the update worked successfully you can now check which additional signatures ClamAV recognizes:

clamscan --debug 2>&1 /dev/null | grep "loaded"

The output should look like:

LibClamAV debug: /var/lib/clamav/sigwhitelist.ign2 loaded
LibClamAV debug: daily.info loaded
LibClamAV debug: daily.cfg loaded
LibClamAV debug: daily.ign loaded
LibClamAV debug: daily.crb loaded
LibClamAV debug: daily.hdb loaded
LibClamAV debug: daily.pdb loaded
LibClamAV debug: daily.sfp loaded
LibClamAV debug: daily.fp loaded
LibClamAV debug: daily.ldb loaded
LibClamAV debug: daily.msb loaded
LibClamAV debug: daily.ftm loaded
LibClamAV debug: daily.hsb loaded
LibClamAV debug: daily.cdb loaded
LibClamAV debug: daily.ign2 loaded
LibClamAV debug: daily.idb loaded
LibClamAV debug: daily.wdb loaded
LibClamAV debug: daily.mdb loaded
LibClamAV debug: daily.ndb loaded
LibClamAV debug: /var/lib/clamav/daily.cld loaded
LibClamAV debug: /var/lib/clamav/hackingteam.hsb loaded
LibClamAV debug: main.info loaded
LibClamAV debug: main.hdb loaded
LibClamAV debug: main.hsb loaded
LibClamAV debug: main.mdb loaded
LibClamAV debug: main.msb loaded
LibClamAV debug: main.ndb loaded
LibClamAV debug: main.fp loaded
LibClamAV debug: main.sfp loaded
LibClamAV debug: main.crb loaded
LibClamAV debug: /var/lib/clamav/main.cvd loaded
LibClamAV debug: /var/lib/clamav/spamimg.hdb loaded
LibClamAV debug: /var/lib/clamav/bofhland_cracked_URL.ndb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.zeus_js
LibClamAV debug: cli_loadyara: loaded 1 of 1 yara signatures from /var/lib/clamav/EK_Zeus.yar
LibClamAV debug: /var/lib/clamav/EK_Zeus.yar loaded
LibClamAV debug: /var/lib/clamav/sanesecurity.ftm loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.AnglerEKredirector
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_flash
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_flash2
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_flash4
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_flash5
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_flash_uncompressed
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_html
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_html2
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.angler_js
LibClamAV debug: cli_loadyara: loaded 10 of 10 yara signatures from /var/lib/clamav/EK_Angler.yar
LibClamAV debug: /var/lib/clamav/EK_Angler.yar loaded
LibClamAV debug: /var/lib/clamav/scam.ndb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_css
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_css2
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_htm
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_js
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_js2
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_js3
LibClamAV debug: load_oneyara: successfully loaded YARA.zeroaccess_js4
LibClamAV debug: cli_loadyara: loaded 7 of 7 yara signatures from /var/lib/clamav/EK_ZeroAcces.yar
LibClamAV debug: /var/lib/clamav/EK_ZeroAcces.yar loaded
LibClamAV debug: /var/lib/clamav/blurl.ndb loaded
LibClamAV debug: /var/lib/clamav/antidebug_antivm.yar loaded
LibClamAV debug: /var/lib/clamav/spamattach.hdb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html10
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html11
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html2
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html3
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html4
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html5
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html6
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html7
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html8
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_html9
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_jar2
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_jar3
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_pdf
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_pdf2
LibClamAV debug: load_oneyara: successfully loaded YARA.phoenix_pdf3
LibClamAV debug: cli_loadyara: loaded 17 of 17 yara signatures from /var/lib/clamav/EK_Phoenix.yar
LibClamAV debug: /var/lib/clamav/EK_Phoenix.yar loaded
LibClamAV debug: /var/lib/clamav/winnow_malware.hdb loaded
LibClamAV debug: bytecode.info loaded
LibClamAV debug: 3986185.cbc loaded
LibClamAV debug: 3986187.cbc loaded
LibClamAV debug: 3986188.cbc loaded
LibClamAV debug: 3986206.cbc loaded
LibClamAV debug: 3986212.cbc loaded
LibClamAV debug: 3986214.cbc loaded
LibClamAV debug: 3986215.cbc loaded
LibClamAV debug: 3986216.cbc loaded
LibClamAV debug: 3986217.cbc loaded
LibClamAV debug: 3986218.cbc loaded
LibClamAV debug: 3986219.cbc loaded
LibClamAV debug: 3986220.cbc loaded
LibClamAV debug: 3986221.cbc loaded
LibClamAV debug: 3986222.cbc loaded
LibClamAV debug: 3986223.cbc loaded
LibClamAV debug: 3986224.cbc loaded
LibClamAV debug: 3986229.cbc loaded
LibClamAV debug: 3986230.cbc loaded
LibClamAV debug: 3986231.cbc loaded
LibClamAV debug: 3986232.cbc loaded
LibClamAV debug: 3986233.cbc loaded
LibClamAV debug: 3986234.cbc loaded
LibClamAV debug: 3986235.cbc loaded
LibClamAV debug: 3986236.cbc loaded
LibClamAV debug: 3986242.cbc loaded
LibClamAV debug: 3986244.cbc loaded
LibClamAV debug: 3986249.cbc loaded
LibClamAV debug: 3986259.cbc loaded
LibClamAV debug: 3986282.cbc loaded
LibClamAV debug: 3986283.cbc loaded
LibClamAV debug: 3986289.cbc loaded
LibClamAV debug: 3986292.cbc loaded
LibClamAV debug: 3986301.cbc loaded
LibClamAV debug: 3986303.cbc loaded
LibClamAV debug: 3986305.cbc loaded
LibClamAV debug: 3986306.cbc loaded
LibClamAV debug: 3986310.cbc loaded
LibClamAV debug: 3986321.cbc loaded
LibClamAV debug: 3986322.cbc loaded
LibClamAV debug: 3986326.cbc loaded
LibClamAV debug: 3986327.cbc loaded
LibClamAV debug: 3986328.cbc loaded
LibClamAV debug: 3986334.cbc loaded
LibClamAV debug: 3986337.cbc loaded
LibClamAV debug: 4306126.cbc loaded
LibClamAV debug: 4306157.cbc loaded
LibClamAV debug: 4307467.cbc loaded
LibClamAV debug: 4310114.cbc loaded
LibClamAV debug: 4416867.cbc loaded
LibClamAV debug: 4510302.cbc loaded
LibClamAV debug: 4526683.cbc loaded
LibClamAV debug: 4553522.cbc loaded
LibClamAV debug: 4970075.cbc loaded
LibClamAV debug: 5044126.cbc loaded
LibClamAV debug: 5588995.cbc loaded
LibClamAV debug: 5819336.cbc loaded
LibClamAV debug: 5999914.cbc loaded
LibClamAV debug: 5999936.cbc loaded
LibClamAV debug: 6300337.cbc loaded
LibClamAV debug: 6311970.cbc loaded
LibClamAV debug: 6316126.cbc loaded
LibClamAV debug: 6324281.cbc loaded
LibClamAV debug: 6327695.cbc loaded
LibClamAV debug: 6329916.cbc loaded
LibClamAV debug: 6329917.cbc loaded
LibClamAV debug: 6333894.cbc loaded
LibClamAV debug: 6334209.cbc loaded
LibClamAV debug: 6335427.cbc loaded
LibClamAV debug: 6335443.cbc loaded
LibClamAV debug: 6335540.cbc loaded
LibClamAV debug: 6335560.cbc loaded
LibClamAV debug: 6335564.cbc loaded
LibClamAV debug: 6335669.cbc loaded
LibClamAV debug: 6336035.cbc loaded
LibClamAV debug: 6336074.cbc loaded
LibClamAV debug: 6336630.cbc loaded
LibClamAV debug: 6336737.cbc loaded
LibClamAV debug: /var/lib/clamav/bytecode.cvd loaded
LibClamAV debug: /var/lib/clamav/winnow_bad_cw.hdb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.zerox88_js2
LibClamAV debug: load_oneyara: successfully loaded YARA.zerox88_js3
LibClamAV debug: cli_loadyara: loaded 2 of 2 yara signatures from /var/lib/clamav/EK_Zerox88.yar
LibClamAV debug: /var/lib/clamav/EK_Zerox88.yar loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.bleedinglife2_adobe_2010_1297_exploit
LibClamAV debug: load_oneyara: successfully loaded YARA.bleedinglife2_adobe_2010_2884_exploit
LibClamAV debug: load_oneyara: successfully loaded YARA.bleedinglife2_jar2
LibClamAV debug: load_oneyara: successfully loaded YARA.bleedinglife2_java_2010_0842_exploit
LibClamAV debug: cli_loadyara: loaded 4 of 4 yara signatures from /var/lib/clamav/EK_BleedingLife.yar
LibClamAV debug: /var/lib/clamav/EK_BleedingLife.yar loaded
LibClamAV debug: /var/lib/clamav/malwarehash.hsb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.OITC_pdf_with_emb_docm
LibClamAV debug: load_oneyara: successfully loaded YARA.INDICATOR_IMPLANT_Loader
LibClamAV debug: load_oneyara: successfully loaded YARA.INDICATOR_Implant_Loader2
LibClamAV debug: load_oneyara: generic string: [File {0} has been uploaded in {1}] => [46696c65207b307d20686173206265656e2075706c6f6164656420696e207b317d]
LibClamAV debug: load_oneyara: successfully loaded YARA.IMPLANT2_3
LibClamAV debug: load_oneyara: successfully loaded YARA.CryptoWall_Resume_phish
LibClamAV debug: load_oneyara: successfully loaded YARA.java_JSocket_20151217
LibClamAV debug: load_oneyara: successfully loaded YARA.detect_powershell_precursor_downloader
LibClamAV debug: load_oneyara: successfully loaded YARA.kmon_cred_phish
LibClamAV debug: load_oneyara: successfully loaded YARA.rtf_phishing_script_lines
LibClamAV debug: cli_loadyara: loaded 9 of 9 yara signatures from /var/lib/clamav/winnow_malware.yara
LibClamAV debug: /var/lib/clamav/winnow_malware.yara loaded
LibClamAV debug: /var/lib/clamav/bofhland_malware_attach.hdb loaded
LibClamAV debug: /var/lib/clamav/phishtank.ndb loaded
LibClamAV debug: /var/lib/clamav/winnow_extended_malware.hdb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_jar2
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_jar3
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_pdf
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole_basic
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole1_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_css
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm10
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm11
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm12
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm3
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm4
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm5
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm6
LibClamAV debug: load_oneyara: successfully loaded YARA.blackhole2_htm8
LibClamAV debug: cli_loadyara: loaded 16 of 16 yara signatures from /var/lib/clamav/EK_Blackhole.yar
LibClamAV debug: /var/lib/clamav/EK_Blackhole.yar loaded
LibClamAV debug: /var/lib/clamav/foxhole_filename.cdb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.Sanesecurity_TestSig_Type4_Hdr_2
LibClamAV debug: load_oneyara: successfully loaded YARA.Sanesecurity_TestSig_Type3_Bdy_4
LibClamAV debug: load_oneyara: successfully loaded YARA.Sanesecurity_TestSig_Type4_Bdy_3
LibClamAV debug: load_oneyara: successfully loaded YARA.Sanesecurity_PhishingTestSig_1
LibClamAV debug: cli_loadyara: loaded 4 of 4 yara signatures from /var/lib/clamav/Sanesecurity_sigtest.yara
LibClamAV debug: /var/lib/clamav/Sanesecurity_sigtest.yara loaded
LibClamAV debug: /var/lib/clamav/bofhland_malware_URL.ndb loaded
LibClamAV debug: /var/lib/clamav/phish.ndb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.Sanesecurity_Spam_test
LibClamAV debug: load_oneyara: successfully loaded YARA.Sanesecurity_Spam_pornspam
LibClamAV debug: cli_loadyara: loaded 2 of 2 yara signatures from /var/lib/clamav/Sanesecurity_spam.yara
LibClamAV debug: /var/lib/clamav/Sanesecurity_spam.yara loaded
LibClamAV debug: /var/lib/clamav/junk.ndb loaded
LibClamAV debug: /var/lib/clamav/winnow_malware_links.ndb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.sakura_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.sakura_jar2
LibClamAV debug: cli_loadyara: loaded 2 of 2 yara signatures from /var/lib/clamav/EK_Sakura.yar
LibClamAV debug: /var/lib/clamav/EK_Sakura.yar loaded
LibClamAV debug: /var/lib/clamav/porcupine.ndb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.crimepack_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.crimepack_jar3
LibClamAV debug: cli_loadyara: loaded 2 of 2 yara signatures from /var/lib/clamav/EK_Crimepack.yar
LibClamAV debug: /var/lib/clamav/EK_Crimepack.yar loaded
LibClamAV debug: /var/lib/clamav/rfxn.hdb loaded
LibClamAV debug: /var/lib/clamav/jurlbl.ndb loaded
LibClamAV debug: /var/lib/clamav/porcupine.hsb loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_htm
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_js
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_js2
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_js_flash
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_js_java
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_js_quicktime
LibClamAV debug: load_oneyara: successfully loaded YARA.fragus_js_vml
LibClamAV debug: cli_loadyara: loaded 7 of 7 yara signatures from /var/lib/clamav/EK_Fragus.yar
LibClamAV debug: /var/lib/clamav/EK_Fragus.yar loaded
LibClamAV debug: load_oneyara: successfully loaded YARA.eleonore_jar
LibClamAV debug: load_oneyara: successfully loaded YARA.eleonore_jar2
LibClamAV debug: load_oneyara: successfully loaded YARA.eleonore_jar3
LibClamAV debug: load_oneyara: successfully loaded YARA.eleonore_js
LibClamAV debug: load_oneyara: successfully loaded YARA.eleonore_js2
LibClamAV debug: load_oneyara: successfully loaded YARA.eleonore_js3
LibClamAV debug: cli_loadyara: loaded 6 of 6 yara signatures from /var/lib/clamav/EK_Eleonore.yar
LibClamAV debug: /var/lib/clamav/EK_Eleonore.yar loaded
LibClamAV debug: /var/lib/clamav/foxhole_generic.cdb loaded
LibClamAV debug: /var/lib/clamav/winnow.attachments.hdb loaded
LibClamAV debug: /var/lib/clamav/bofhland_phishing_URL.ndb loaded
LibClamAV debug: /var/lib/clamav/rogue.hdb loaded
LibClamAV debug: /var/lib/clamav/rfxn.ndb loaded

Filtering out spam with rspamd

You have a perfectly working mail server by now. But before you go live let’s do something about the insane amount of spam. In previous editions of this guide I used and recommended SpamAssassin. However I have found a piece of software that is more versatile, scales better and is still easy to integrate: rspamd. rspamd has a (maybe biased) comparison on their home page.

rspamd is a permanent process that runs on your mail server. It listens to connections from Postfix using the milter (=mail filter) protocol. Every time an email enters ryour system, Postfix will send it to rspamd to have its content checked. rspamd runs a lot of checks on the email and computes a total score. The higher the score – the more likely the email should be considered unsolicited.

Make Postfix use rspamd

Let’s tell Postfix to send all incoming email through rspamd. Run these commands on the shell:

postconf smtpd_milters=inet:127.0.0.1:11332
postconf non_smtpd_milters=inet:127.0.0.1:11332
postconf milter_protocol=6
postconf milter_mail_macros="i {mail_addr} {client_addr} {client_name} {auth_authen}"

Postfix will now connect to rspamd that is listening to TCP port 11332 on localhost (127.0.0.1) and pass the email over that connection. The smtpd_milters setting defines that connection for emails that came into the system from the internet via the SMTP protocol. The non_smtpd_milters setting is optional – it makes Postfix have all emails checked that originate from the system itself. Finally the milter_mail_macros setting defines several variables that rspamd expects for better spam detection. rspamd then runs its checks and tells Postfix whether the email should pass or get rejected.

Testing spam detection

For testing we can use a sample spam email that comes with SpamAssassin. It is called GTUBE (Generic Test for Unsolicited Bulk Email). It contains a certain articial pattern that is recognized as spam by SpamAssassin. Do you know EICAR.COM to test virus scanners? This is the same thing for spam.

I suggest that you download the file on the server:

wget http://spamassassin.apache.org/gtube/gtube.txt

…and send that email to our test user John…

sendmail john@example.org < gtube.txt

Check your /var/log/mail.log. You will find something like this:

Jan 13 09:21:01 mail postfix/cleanup[19945]: 95B7A42212: milter-reject: END-OF-MESSAGE from localhost[127.0.0.1]: 5.7.1 Gtube pattern; from=<root@mail.example.org> to=<john@example.org>
Jan 13 09:21:01 jen postfix/cleanup[19945]: 95B7A42212: to=<john@example.org>, relay=none, delay=0.18, delays=0.18/0/0/0, dsn=5.7.1, status=bounced (Gtube pattern)

The interesting parts are those printed in bold letters. “milter-reject” tells that the milter (rspamd) recommended to Postfix to reject the email. It gave the reason “5.7.1 Gtube pattern”. In mail communication you often find these three digit codes. They are defined in RFC 3463. The first digit is most important:

  • 2 = Success
  • 4 = Transient failure (temporary problem – come back later)
  • 5 = Permanent failure (do not try again in this way)

So 5.7.1 tells us that the result code is a permanent failure in delivery. If you looked up the RFC you would find that the 7 stands for an issue regarding the security policy. So it’s not a technical failure but instead a security-relevant component on the system has rejected the email. That’s what rspamd did. It even told us the reason: “Gtube pattern”. So you see that rspamd knows about the Gtube spam test pattern and reacts as expected.

Accordingly you won’t see this email in John’s inbox. This is a great advantage of using milters by the way. Imagine Postfix receiving a spam email and confirming its reception. What should it do when it finds out that it’s unwanted email? According to the SMTP protocol it must not throw away any emails. Would you create a bounce message telling the sender that you did not accept the email? That would be a bad idea because the pretended sender address is very likely not the real sender. You would send the bounce to an innocent person thus creating so called backscatter and make it even worse. So the right approach is to check the email while the sending server is still connected to your Postfix. This allows Postfix to reject the email with a 5.x.x error code and let the other side figure out what to do.

Score metrics (optional)

rspamd will however not reject all spam email. As it computes a score of the spam propability you can tell it which scores you would accept, flag as spam or make the incoming email get rejected. Take a look at the /etc/rspamd/metrics.conf file. There are tons of scores defined for various conditions. At the beginning of the file you will find:

 actions {
 reject = 15;
 add_header = 6;
 greylist = 4;
 }

These are the default actions. If rspamd computes a score of at least 15 then the email will get rejected just like the Gtube pattern in the previous test. Any score above 6 will add header lines like “X-Spam: Yes” so that your mail software can detect them and maybe file the email to a different folder. Any score above 4 will trigger greylisting which is a mechanism that temporarily rejects the email with a 4.x.x code and waits if the sending server will try again. After a waiting time of a few minutes greylisting will accept the email. The idea is to reject email from systems that do not have a sending queue. Malware like on infected Wind*ws computers used to try sending an email just once which triggered greylisting and successfully rejected the spammer. But even malware programmers have learned and may try again after a few minutes thus circumventing greylisting. Your mileage may vary.

If you like to change these defaults then create a new file in /etc/rspamd/override.d/metrics.conf contaning:

actions {
 reject = 150;
 add_header = 2;
 greylist = 5;
}

This would virtually never reject an email. And it would flag any email with a score of at least 2 as spam. Greylisting would happen at a score above 5. These are not necessarily sane values – they are just meant as an example.

Please take a moment to understand how to change rspamd defaults. You can either create files in /etc/rspamd/local.d/… which will replace entire sections; or create files in /etc/rspamd/override.d/… which will change only small parts of the configuration. There is a helpful page in the rspamd documentation that contains examples. Whatever you do – never change the /etc/rspamd/* files directly because a software update will try to replace them.

Of course restart rspamd after any configuration changes:

service rspamd reload

To check if rspamd has picked up your configuration use this command to see the current configuration:

rspamadm configdump

You may test your configuration using

rspamadm configtest

Alternatively you may check if all required rspamd processes are running…

pgrep -a rspam

22510 rspamd: main process 
22511 rspamd: rspamd_proxy process 
22512 rspamd: controller process 
22513 rspamd: normal process 
22514 rspamd: normal process 
22515 rspamd: hs_helper process

Adding headers

As you may know an email contains of the header and the body. Your users will only see header information like the subject, the sender, the recipient and the date and time the email was sent. But there is way more information like the router the email travelled or extended headers added by the various mail server on the way to the destination. Such extended headers begin with an “X-“. rspamd can add such headers to help you filter out spam. For that purpose create a new configuration override file at /etc/rspamd/override.d/milter_headers.conf with this content:

extended_spam_headers = true;

As documented it will add these headers:

X-Rspamd-Server: mail
Authentication-Results: dmarc=fail reason="No valid SPF, No valid DKIM" …
X-Rspamd-Queue-Id: C22E55A005B3
X-Spamd-Result: default: False [11.55 / 15.00]
 R_PARTS_DIFFER(0.27)[63.4%]
 FROM_NO_DN(0.00)[]
 RCVD_COUNT_ZERO(0.00)[0]
 R_DKIM_NA(0.00)[]
 FUZZY_DENIED(12.00)[1:19305c7fdd:1.00:bin,1:35699594fd:0.91:txt]
 RBL_SENDERSCORE(2.00)[55.181.23.94.bl.score.senderscore.com]
 ARC_NA(0.00)[]
 RCPT_COUNT_ONE(0.00)[1]
 RCVD_TLS_ALL(0.00)[]
 FROM_EQ_ENVFROM(0.00)[]
 R_SPF_SOFTFAIL(0.00)[~all]
 BAYES_HAM(-2.71)[98.75%]
 TO_MATCH_ENVRCPT_ALL(0.00)[]
 MIME_GOOD(-0.10)[multipart/related,multipart/alternative,text/plain]
 MID_RHS_MATCH_FROM(0.00)[]
 ASN(0.00)[asn:16276, ipnet:94.23.0.0/16, country:FR]
 TO_DN_NONE(0.00)[]
 DMARC_POLICY_SOFTFAIL(0.10)[Chronopost.fr : No valid SPF, No valid DKIM,none]
 SUBJECT_ENDS_EXCLAIM(0.00)[]
X-Spam: Yes

Each of the uppercase symbols like FROM_HAS_DN means that a certain detection routing of rspamd was triggered. It does not necessarily mean something bad about the email. For example R_SPF_ALLOW has a negative score that lowers the total score because it is something good about the email. There are a several symbols with a 0.00 score. These do not change the score but show you what rspamd has found. But if you consider certain criteria good or bad then you can define your own scores for them.

The last line here is especially interesting because next on our list is…

Sending spam to the Junk folder

Your users will not realize that their spam emails have an added “X-Spam: Yes” header. Their emails just appear like normal in their inbox. So let’s aid them by moving spam to a seperate Junk folder beneath their inbox automatically. Dovecot has support for Sieve filters which are basically scripts that run automatically whenever an email coming in.

John could log into Roundcube and configure a new Sieve filter for himself that would save any emails to his “Junk” folder if the header line “X-Spam: Yes” was found. This rule would be useful for all your users though so let’s find a general solution.

Note: SpamAssassin as used in previous versions of this guide used a slightly different header. “X-Spam-Flag: YES”. Make sure to change your Sieve filters accordingly.

Dovecot supports global Sieve filters that apply to all users. Edit the file /etc/dovecot/conf.d/90-sieve.conf. Look for the “sieve_after” lines. They are commented out. So add a new line there:

sieve_after = /etc/dovecot/sieve-after

The “sieve after” filters are executed after the user’s filters. John can define his own filter rules. And after that Dovecot will run any filter rules it finds in files in /etc/dovecot/sieve-after. Create that directory:

mkdir /etc/dovecot/sieve-after

And create a new file /etc/dovecot/sieve-after/spam-to-folder.sieve reading:

require ["fileinto","mailbox"];

if header :contains "X-Spam" "Yes" {
 fileinto :create "INBOX.Junk";
 stop;
}

The “require” lines include functionality to move emails into certain folders (fileinto) and to create folders if they don’t exist yet (mailbox). Then if SpamAssassin marked a header as spam it is moved into the INBOX.Junk folder which just appears as “Junk” to the user underneath their inbox.

Dovecot cannot deal with such human-readable files though. So we need to compile it:

sievec /etc/dovecot/sieve-after/spam-to-folder.sieve

That generated a machine-readable file /etc/dovecot/sieve-after/spam-to-folder.svbin.

Restart Dovecot:

service dovecot restart

Now all your users will automatically get spam emails moved to their Junk folder. Nice – isn’t it?

Learning existing spam

If you followed previous ISPmail guides then you will have used SpamAssassin for a while. You may have trained it with ham (good) and spam (unwanted) emails for a while which fueled the built-in Bayes database to increase its detection reliability. The bad news is that you cannot use that training data in rspamd because it uses a different algorithm. There are three ways to get started with rspamd…

(a) No Bayes training

This is not as bad as it sounds. rspamd has way more functionality to determine if an email is ham or spam. I recommend that you enable auto learning. Just create a new file /etc/rspamd/override.d/classifier-bayes.conf and make it contain:

autolearn = true;

This is a very conservative approach though. It learns emails as spam if they are so bad that they get rejected. And it learns emails as ham if they have a negative (=very good) score. The rspamd documentation has further examples how to fine-tune auto learning.

(b) Using pre-built statistics

rspamd provides a sample database with over 3000 learned emails. They are from 2015 and perhaps are not a good start because the nature of spam changes over time.

Stop rspamd:

service rspamd stop

Take their pre-built statistics and put the *sqlite files into your /var/lib/rspamd directory. Fix the ownership of those files by running…

chown _rspamd._rspamd /var/lib/rspamd/*sqlite

And finally start rspamd again…

service rspamd start

Verify that the database is now filled by running…

rspamc stat

At the end of the output you will see something like…

Statfile: BAYES_SPAM type: sqlite3; length: 41.78M; free blocks: 0; total blocks: 596k; free: 0.00%; learned: 1880; users: 1; languages: 3
Statfile: BAYES_HAM type: sqlite3; length: 51.12M; free blocks: 0; total blocks: 684k; free: 0.00%; learned: 1578; users: 1; languages: 3
Total learns: 3458

(c) Re-learn your existing ham and spam

You have been running your mail server for a while? And you have a good amount of ham and spam emails? Then let’s use those to train rspamd. It is important to train both ham and spam emails. The rspamc command will allow you to feed entire directories/folders of emails to the learning process. Fortunately we are using the Maildir format to store emails which rspamd can understand. An example to train spam:

rspamc learn_spam /var/vmail/example.org/john/Maildir/.INBOX.Junk/cur

And this would be an example to train ham:

rspamc learn_ham /var/vmail/example.org/john/Maildir/cur".

Of course the quality of spam detection will depend on how good the source data is. So do not train spam emails from users who randomly put mails into their Junk folder that is not actually spam.

Check the number of emails you learned by running…

rspamc stat

Bayes spam checking will not work before it learned at least 200 spam and ham emails. Teaching rspamd fewer emails or just spam emails will not work. This is defined by the min_learns variable defined in /etc/rspamd/statistic.conf.

In the output you will find lines beginning with “Statfile” like these…

Statfile: BAYES_SPAM type: sqlite3; length: 41.78M; free blocks: 0; total blocks: 0; free: 0.00%; learned: 0; users: 1; languages: 1
Statfile: BAYES_HAM type: sqlite3; length: 51.12M; free blocks: 0; total blocks: 0; free: 0.00%; learned: 0; users: 1; languages: 1
Total learns: 0

This is what you usually start with. The more emails you feed into the training process the better the detection rate will be. Some emails however may not be long enough or be too similar to previously trained emails. So don’t worry if you are training 1000 emails but just get a count of 500 emails here.

Learning from user actions

Now we are getting to something really cool. Let’s tell Dovecot that moving emails into the Junk folder teaches rspamd instantly that the email is spam. And train an email as ham if it is moved out of the Junk folder. We will add triggers (actually “sieve scripts”) to the action of moving emails via IMAP.

There used to be a nice Dovecot plugin called “Antispam” but unfortunately it became deprecated. Bummer. The currently recommended way is to use the “IMAPSieve” plugin instead. There is nothing to install – it comes with the Dovecot packages. We just need to configure it.

A word of warning though: Currently the spam learning database is global for all users. If one user deliberately trains it with gibberish then it will ruin your detection rate. So the approach described here is suitably only for users who can tell ham from spam and can be trusted. It is also possible with rspamd to teach spam for each user but that’s currently not within the scope of this guide and may even be problematic because most users do not get enough ham and spam emails to make spam detection work properly. If you are still interested in per-user spam training see the respective rspamd documentation.

First order of business is enabling the IMAPSieve plugin for the IMAP protocol/service in Dovecot. Edit the /etc/dovecot/conf.d/20-imap.conf file and look for the line reading “mail_plugins”. Turn it into:

mail_plugins = $mail_plugins imap_sieve

We also need to edit Dovecot’s Sieve configuration to enable two plugins that are required for our task. Sieve is a scripting language that automates things in conjunction with emails and folders. Edit the file /etc/dovecot/conf.d/90-sieve.conf. First alter the line reading “sieve_plugins” to look like:

 sieve_plugins = sieve_imapsieve sieve_extprograms

And anywhere within the { … } section add these lines:

# From elsewhere to Junk folder
imapsieve_mailbox1_name = Junk
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/learn-spam.sieve

# From Junk folder to elsewhere
imapsieve_mailbox2_name = *
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_before = file:/etc/dovecot/sieve/learn-ham.sieve

sieve_pipe_bin_dir = /etc/dovecot/sieve
sieve_global_extensions = +vnd.dovecot.pipe

The first rule tells Dovecot to run the Sieve rules as defined in the /etc/dovecot/sieve/learn-spam.sieve file whenever an email is moved into a user’s “Junk” folder. We will create that Sieve script in a minute.

The second rule sets the other way. Whenever an email is moved from the “Junk” folder to any (*) folder then the /etc/dovecot/sieve/learn-ham.sieve Sieve script is called.

The “sieve_pipe_bin_dir” setting defines where executable scripts are allowed to reside. We will put our simple learning scripts there. And finally the “sieve_global_extensions” setting enables the pipe plugin that allows sending email to external commands.

Note: if you come from previous versions of this guide, your mail directories may still use dots (“.”) instead of paths (“/”) to compose the folder structure. A clear sign is that you find directories like “INBOX.Junk” instead of just “Junk” in mail directories within /var/vmail. In that case replace the word “Junk” in the section above by “INBOX.Junk”.

Next up let’s create the Sieve scripts that we told Dovecot about. Create a new directory /etc/dovecot/sieve to put our new files in:

mkdir /etc/dovecot/sieve

Then create the file /etc/dovecot/sieve/learn-spam.sieve and let it contain:

require ["vnd.dovecot.pipe", "copy", "imapsieve"];
pipe :copy "rspamd-learn-spam.sh";

Let’s do the same for /etc/dovecot/sieve/learn-ham.sieve

require ["vnd.dovecot.pipe", "copy", "imapsieve"];
pipe :copy "rspamd-learn-ham.sh";

These two scripts need to be compiled – that is turning them into machine-readable code:

sievec /etc/dovecot/sieve/learn-spam.sieve
sievec /etc/dovecot/sieve/learn-ham.sieve

This creates two new files “learn-ham.svbin” and “learn-spam.svbin” that look like gibberish inside but are now in a format that Dovecot’s Sieve plugin can understand.

Let’s fix the permissions of these files, too, while we are at it:

chmod u=rw,go= /etc/dovecot/sieve/learn-{spam,ham}.sieve
chown vmail.vmail /etc/dovecot/sieve/learn-{spam,ham}.sieve

And the last step is to create the simple shell scripts that do the actual spam/ham training. The first file is /etc/dovecot/sieve/rspamd-learn-spam.sh which contains:

#!/bin/sh
exec /usr/bin/rspamc learn_spam

That looks simple, doesn’t it? Nothing more is actually needed. The spam email is piped to this script and rspamd learns it as a spam email and adjusts its spam detection database accordingly.

The second script teaches ham and is called /etc/dovecot/sieve/rspamd-learn-ham.sh. Make it contain:

#!/bin/sh
exec /usr/bin/rspamc learn_ham

These two shell scripts must be executable so go fix that:

chmod u=rwx,go= /etc/dovecot/sieve/rspamd-learn-{spam,ham}.sh
chown vmail.vmail /etc/dovecot/sieve/rspamd-learn-{spam,ham}.sh

I hope you haven’t lost your mind yet. It’s really just a chain of things to happen. Let’s reiterate how this process works:

  1. a user moves a spam email into their “Junk” folder
  2. Dovecot realizes that this triggers the Sieve rule “imapsieve_mailbox1” so it calls the Sieve script /etc/dovecot/sieve/learn-spam.sieve (in fact the *.svbin version of the script)
  3. Sieve will take the email and send (“pipe”) it to the executable rspamd-learn-spam.sh shell script
  4. the script in turn runs the email through the “/usr/bin/rspamc learn_spam” command

This works equally for the other way or learning ham emails of course.

I am sure you are eager to try it out. Yes, it’s about time. However to see that it actually works I suggest you edit the /etc/dovecot/conf.d/10-logging.conf file and set “mail_debug=yes”. That will add a lot more detail to the /var/log/mail.log file but on a busy server may also lead to headaches. 🙂

Restart Dovecot…

service dovecot restart

…and watch the /var/log/mail.log file…

tail -f /var/log/mail.log

Now open your IMAP client (Thunderbird, Evolution, Roundcube, mutt or whatever you prefer) and drag an email to your Junk folder. The mail log should read something along the lines of…

imap(john@example.org): Debug: imapsieve: mailbox INBOX.Junk: MOVE event
imap(john@example.org): Debug: imapsieve: Matched static mailbox rule [1]
imap(john@example.org): Debug: sieve: file storage: Using Sieve script path: /etc/dovecot/sieve/learn-spam.sieve
imap(john@example.org): Debug: sieve: file script: Opened script `learn-spam’ from `/etc/dovecot/sieve/learn-spam.sieve’
imap(john@example.org): Debug: sieve: Opening script 1 of 1 from `/etc/dovecot/sieve/learn-spam.sieve’
imap(john@example.org): Debug: sieve: Loading script /etc/dovecot/sieve/learn-spam.sieve
imap(john@example.org): Debug: sieve: Script binary /etc/dovecot/sieve/learn-spam.svbin successfully loaded
imap(john@example.org): Debug: sieve: binary save: not saving binary /etc/dovecot/sieve/learn-spam.svbin, because it is already stored
imap(john@example.org): Debug: sieve: Executing script from `/etc/dovecot/sieve/learn-spam.svbin’
imap(john@example.org): Debug: sieve: action pipe: running program: rspamd-learn-spam.sh
imap(john@example.org): Debug: Mailbox INBOX.Junk: Opened mail UID=3978 because: mail stream
imap(john@example.org): Debug: waiting for program `/etc/dovecot/sieve/rspamd-learn-spam.sh’ to finish after 0 msecs
imap(john@example.org): sieve: pipe action: piped message to program `rspamd-learn-spam.sh’

Look for errors and warnings. If at the end you see that Dovecot called the “rspamd-learn-spam.sh” script then you should be fine.

And finally if you pull an email out of the “Junk” folder you should see mailbox rule [2] be called and the email being learned as ham:

imap(john@example.org): Debug: imapsieve: mailbox INBOX: MOVE event
imap(john@example.org): Debug: imapsieve: Matched static mailbox rule [2]
imap(john@example.org): Debug: sieve: file storage: Using Sieve script path: /etc/dovecot/sieve/learn-ham.sieve
imap(john@example.org): Debug: sieve: file script: Opened script `learn-ham’ from `/etc/dovecot/sieve/learn-ham.sieve’
imap(john@example.org): Debug: sieve: Opening script 1 of 1 from `/etc/dovecot/sieve/learn-ham.sieve’
imap(john@example.org): Debug: sieve: Loading script /etc/dovecot/sieve/learn-ham.sieve
imap(john@example.org): Debug: sieve: Script binary /etc/dovecot/sieve/learn-ham.svbin successfully loaded
imap(john@example.org): Debug: sieve: binary save: not saving binary /etc/dovecot/sieve/learn-ham.svbin, because it is already stored
imap(john@example.org): Debug: sieve: Executing script from `/etc/dovecot/sieve/learn-ham.svbin’
imap(john@example.org): Debug: sieve: action pipe: running program: rspamd-learn-ham.sh
imap(john@example.org): Debug: Mailbox INBOX: Opened mail UID=28412 because: mail stream
imap(john@example.org): Debug: waiting for program `/etc/dovecot/sieve/rspamd-learn-ham.sh’ to finish after 0 msecs
imap(john@example.org): sieve: pipe action: piped message to program `rspamd-learn-ham.sh’

That’s it. Nifty, isn’t it?

Autoexpunge

Andi Olsen pointed out that Dovecot has introduced a feature to automatically delete emails in a folder that reach a certain age. This is especially useful for the “Trash” and “Junk” folders. To enable this feature just edit the /etc/dovecot/conf.d/15-mailboxes.conf file and add the autoexpunge parameter where desired. Example:

mailbox Junk {
  special_use = \Junk
  auto = subscribe
  autoexpunge = 30d
}
mailbox Trash {
  special_use = \Trash
  auto = subscribe
  autoexpunge = 30d
}

Restart Dovecot after any configuration change.

Logging

rspamd keeps a verbose log of its actions in /var/log/rspamd/rspamd.log. If a user complains that a certain email got blocked or at least flagged as spam then take a look at this log. You can match the /var/log/mail.log with it by comparing the Postfix queue ID. Those are the 12-digit hexadecimal number like “95CE05A00547“. Those IDs can be found in the rspamd.log, too:

2018-01-14 06:39:45 #10424(normal) <40985d>; task; rspamd_task_write_log: id: <undef>, qid: <95CE05A00547>, ip: 12.13.51.194, from: <…>, (default: F (no action): [3.40/15.00] [MISSING_MID(2.50){},MISSING_DATE(1.00){},MIME_GOOD(-0.10){text/plain;},ARC_NA(0.00){},ASN(0.00){asn:8220, ipnet:212.123.192.0/18, country:GB;},FROM_EQ_ENVFROM(0.00){},FROM_NO_DN(0.00){},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_ZERO(0.00){0;},RCVD_TLS_ALL(0.00){},TO_DN_NONE(0.00){},TO_DOM_EQ_FROM_DOM(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 181, time: 16.000ms real, 6.385ms virtual, dns req: 0, digest: <69b289a82827c11f759837c033cd800a>, rcpts: <…>, mime_rcpt: <…>

The web interface

rspamd’s web interface

rspamd comes with a neat bonus feature: a web interface. It allows you to check emails for spam, get statistics and fine-tune scores. It is already installed and enabled by default and expects HTTP requests on port 11334 on the localhost interface. I suggest you add a simple proxy configuration to your already working HTTPS-enabled web mail configuration to get access.

First you need to enable Apache’s module for HTTP proxying:

a2enmod proxy_http

You can either create a new virtual host configuration or just edit the /etc/apache2/sites-available/webmail.example.org-https.conf file. Anywhere within the VirtualHost tags add:

 ProxyPass "/rspamd" "http://localhost:11334"
 ProxyPassReverse "/rspamd" "http://localhost:11334"

This piece of configuration will forward any requests to https://webmail.example.org/rspamd to localhost:11334 and thus give you access to the rspamd web interface.

The interface is password protected. Let’s generate a new access password:

pwgen 15 1

This gives you a password like “eiLi1lueTh9mia4”. You could put that password in an rspamd configuration file. But cleartext passwords in configuration files are not quite elegant. Let’s create a hash of the password:

rspamadm pw
Enter passphrase: …
$2$icoahes75e7g9wxapnrbmqnpuzjoq7z…

Feed it the password that pwgen created for you and you will get a long hashed password. This procedure by the way is also documented on the rspamd pages.

Create a new configuration file /etc/rspamd/local.d/worker-controller.inc and put your hashed password in there:

password = "$2$icoahes75e7g9wxapnrbmqnpuzjoq7z…"

That’s it for the configuration. Finally restart both rspamd and Apache to load your changed configuration:

service rspamd reload
service apache2 restart

If everything went as expected you should now be able to access the rspamd web interface at https://webmail.example.org/rspamd

Anything else?

If you consider using rspamd for a larger environment then please take your time reading the good amount of documentation they provide. rspamd scales very well but requires additional setup like using Redis for faster spam analysis across multiple mail gateways.

Managing users, aliases and domains

Maybe you already know what you have to do to create mail domains and mail users. After all I tried to explain the database schema in the section that dealt with preparing the database. But if that wasn’t clear enough let me explain what you need to do to manage your mail accounts.

Web interfaces

If you don’t like using SQL queries to manage your mail server you may like to install a web-based management software. Several developers contributed web interfaces for an earlier version of this guide and they will probably still work because the database schema has not changed. With a little experience in web development you should be able to develop your own management system. I have documented a few common SQL queries further down.

These web interfaces have been provided to you by other diligent hands:

ISPmail Admin

Homepage: http://ima.jungclaussen.com/
Demo: http://ima.jungclaussen.com/demo/

ima_screenshot

GRSoft Virtual Mail Manager

Peter Gutwein has updated his PHP-based web interface to create strong password hashes as recommended in this Jessie guide. It supports english, german, spanish, french, italian, russian and swedish.

Homepage -> http://www.grs-service.ch/pub/grs_mminstallation.html

Here are screenshots of the working application:

grsoft-vmm-screenshot1grsoft-vmm-screenshot2grsoft-vmm-screenshot3grsoft-vmm-screenshot4grsoft-vmm-screenshot5grsoft-vmm-screenshot6grsoft-vmm-screenshot7grsoft-vmm-screenshot8grsoft-vmm-screenshot9

Managing the database directly

Common tasks / PHPMyAdmin

This table explains what changes are required in the database for your everyday tasks. You can click your way through PHPMyAdmin following the instructions in this table:

Create a mail domainInsert a new row into the virtual_domains table and set the “name” to the name of the new domain.
Delete a mail domainDelete the row from the virtual_domains table that has the right “name”. All aliases and users will automatically be deleted, too. However the mailboxes will stay on disk at /var/vmail/… and you need to delete them manually.
Create a mail userFind out the “id” of the right domain from the virtual_domains table. The insert a new row into the virtual_users table. Set the domain_id to the value you just looked up in the virtual_domains table. Set the “email” field to the complete email address of the new user. Create a new password in a shell using the “dovecot pw -s SHA256-CRYPT” command and insert the result into the “password” field.
Change the password of a userFind the row in the virtual_users table by looking for the right “email” field. Create a new password in a shell using the “dovecot pw -s SHA256-CRYPT” command and insert the result into the “password” field.
Delete a mail userFind the row in the virtual_users table by looking for the right “email” field and delete it. The mailbox will stay on disk at /var/vmail/… and you need to delete it manually
Create a mail forwardingYou can forward emails from one (source) email to other addresses (destinations) – even outside of your mail server. Find out the “id” of the right domain (the part after the “@” of the source email address) from the virtual_domains table. Create a new row in the virtual_aliases table for each destination (if you have multiple destination addresses). Set the “source” field to the complete source email address. And set the “destination” field to the respective complete destination email address.
Delete a mail forwarddingFind all rows in the virtual_aliases table by looking for the right “source” email address. Remove all rows that you lead to “destination” addresses you don’t want to forward email to.

SQL queries

Create a mail domainINSERT INTO virtual_domains (name) VALUES (“example.org”);
Delete a mail domainDELETE FROM virtual_domains where name=’example.org’;
Create a mail userINSERT INTO virtual_users (domain_id, email, password) VALUES ( (SELECT id FROM virtual_domains WHERE name=’example.org’), ‘john@example.org’,CONCAT(‘{SHA256-CRYPT}’, ENCRYPT (‘new password’, CONCAT(‘$5$’, SUBSTRING(SHA(RAND()), -16)))));
Change the password of a userUPDATE virtual_users SET password=CONCAT(‘{SHA256-CRYPT}’, ENCRYPT (‘new password’, CONCAT(‘$5$’, SUBSTRING(SHA(RAND()), -16)))) WHERE email=’email@address’;
Delete a mail userDELETE FROM virtual_users WHERE email=’john@example.org’;
Create a mail forwardingINSERT INTO virtual_aliases (domain_id, source, destination) VALUES ( (SELECT id FROM virtual_domains WHERE name=’example.org’), ‘melissa@example.org’, ‘juila@example.net’);
Delete a mail forwardingDELETE FROM virtual_aliases WHERE source=’melissa@example.org’;

DNS MX records

About MX entries

So now you have your working mail server. But how do other mail servers get to know you? The answer lies in the most important service on the internet: DNS. Assume that a mail server somewhere on the other end of the internet wants to send an email to john@example.org. The server must find out the name (and IP address) of the mail server responsible for the example.org domain. This is how that works:

ispmail-jessie-bigpicture-mx

The remote server queries its DNS server for the MX (Mail eXchanger) record of the “example.org” domain. (If no MX record was found it tries again and asks for A (address) records. That’s a fallback solution.) Let’s run a query for a real domain to get an idea. Let’s ask which servers receive email for …@gmail.com.

$> host -t MX gmail.com
gmail.com mail is handled by 10 alt1.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 20 alt2.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 30 alt3.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 40 alt4.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 5 gmail-smtp-in.l.google.com
.

So as a result we get 5 different MX records. Each of them consists of a priority and the host name of the mail server. A mail server would pick the entry with the highest priority (=the lowest number) and establish an SMTP connection to that host. In this example that would be the priority 5 server gmail-smtp-in.l.google.com. If that server could not be reached then the next best server with priority 10 would be used and so on. So all you have to do in your own DNS zone is add an MX entry pointing to your mail server. If you want to run a backup mail server (which is outside of the scope of this tutorial) then you can add a second entry with a an equal or lower priority.

A mistake some people make is using an IP address in MX records. That is not allowed. An MX record always points to a host name. You will have to add an A record for your MX record to point to the actual IP address of your mail server.

(In the above example it is very unlikely that a mail server will ever have to use the server with priority 40. Adventurous system administrators can add such a low-priority entry and see who connects to it. An interesting observation is that spammers often try these servers first – hoping that it is just for backup purposes and less restrictive or lazily configured than the main server. If you see someone connecting to the lowest-priority address first without having tried a higher-priority mail server then you can be pretty certain that it’s not a friend who’s knocking at your door.)

Fallback to A entries

It’s best to explicitly name mail servers explicitly for your domain in the MX records. If you can’t do that for whatever reason then the remote mail server will just do an A record lookup for the IP address and then send email there. If you just run one server for both the web service and the email service then you can do that. But if the web server for your domain is located at another IP address than your mail server then this won’t work.

Relaying with SMTP authentication

Relaying

Your mail server is almost ready for use. But one puzzle piece is missing. Your users can already fetch emails but they need to be able to send emails, too. Mail server know how to send an email to another mail server by looking up MX records. And the receiving mail server will happily accept any email that is destined for a valid recipient. Mail clients however are expected to send email to their mail server first which then forwards email to the internet. When Postfix receives an email from a user and forwards it to another server this is called relaying. Postfix acts as a relay.

So why do the mail client not work the same way? Couldn’t they also just look up the MX record and send the email to the destination server? Technically, yes. But users are often sitting at home using a DSL line. And such IP networks are usually blocked off from sending mail. Some ISPs block the SMTP port on their routers. And most receiving mail servers check real-time blacklists to block DSL IP addresses. The reason is that home users are more likely to have infected Wind*ws PCs that send out spam.

Am I going too fast? Okay, let’s walk through the different scenarios one by one.

Incoming email

When someone on the internet sends an email to john@example.org, some other mail server will deliver the email using SMTP to your mail server. Postfix will determine that it’s responsible for email addresses in the example.org domain and accept the email. John can then use POP3 or IMAP to fetch the email from your server.

Outgoing email (without authentication)

John is on the internet somewhere and wants to send an email to lisa@example.com. Your mail server is not responsible for the “example.com” domain so it receives John’s email and would have to forward (relay) it to the mail server that is responsible for …@example.com email addresses. This may seem like a harmless scenario but your mail server must deny that:

Why? Because anyone can claim to be John and make your mail server forward mail. If an attacker (like a spammer) would send millions of spam emails in John’s name through your server then other organisations will accuse you as the operator of the mail server of spamming. Your mail server would be what people call an open relay. This is not what you want because your mail server would get blacklisted and you will not be able to send out mail to most other servers. So without any proof that John is actually John your server must reject the email.

Outgoing email (with authentication)

So how does John prove his identity? He needs to use authenticated SMTP. This is similar to the previous case but John’s email program will also send his username and password.

Of course we are making sure that his authentication happens over an encrypted connection so John’s password is blared out.

Postfix setting “mynetworks”

In addition to using SMTP authentication you can tell Postfix to always relay email for certain IP addresses. The mynetworks setting contains the list of IP networks or IP addresses that you trust. Usually you define your own local network here. The reason John had to authenticate in the above example is because he is not sending the email from your local network.

If your users are using the webmail interface running on your mail server then they will not need to authenticate. Roundcube sends email to localhost which is trusted according to the mynetworks setting.

Make Postfix use Dovecot for authentication

Enabling SMTP authentication in Postfix is surprisingly easy. You already configured Dovecot’s regarding user authentication. So let’s just make Postfix utilize that by telling it to ask the Dovecot server to verify the username and password. Postfix just needs some extra configuration. Run these commands on the shell:

postconf smtpd_sasl_type=dovecot
postconf smtpd_sasl_path=private/auth
postconf smtpd_sasl_auth_enable=yes

This enables SMTP authentication and tells Postfix that it can talk to Dovecot through a socket file located at /var/spool/postfix/private/auth. Do you remember that Postfix runs in a sandboxed chroot directory? That’s at /var/spool/postfix. It cannot access any files outside of that directory. But fortunately in a previous section you edited the /etc/dovecot/conf.d/10-master.conf file and made Dovecot place a socket file into /var/spool/postfix/private/auth to allow communication from Postfix.

Enable encryption

The following settings enable encryption, define the key and certificate and enforce that authentication must only occur over encrypted connections:

postconf smtpd_tls_security_level=may
postconf smtpd_tls_auth_only=yes
postconf smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.example.org/fullchain.pem
postconf smtpd_tls_key_file=/etc/letsencrypt/live/webmail.example.org/privkey.pem
postconf smtp_tls_security_level=may

Note #1: In the past you may have configured your smtpd_recipient_restrictions to restrict relaying to authenticated users. Postfix nowadays has setting called “smtpd_relay_restrictions” that deals with relaying requests in the “RCPT TO” phase of the SMTP dialog. So essentially it works like the good old “smtpd_recipient_restrictions” but is checked first. smtpd_relay_restrictions has a reasonable default so authenticated relaying works automatically.

Note #2: Postfix 2.10 has changed the behavior of the smtpd_*_restrictions altogether. If a restriction makes Postfix reject an email it will wait until after the “RCPT TO” line. Read the Postfix documentation for a description of the reasons for that change.

The smtpd_tls_security_level is set to “may” to allow encrypted communication when Postfix receives an email. This not only applies to users sending emails but also remote mail servers that send email to you. It may sound tempting to change that to “encrypt” which would enforce encryption and reject any attempts to create an unencrypted SMTP connection. But unfortunately there are still mail servers in the world that can’t encrypt and would not be able to deliver email to you. So “may” is the proper setting. Same goes for smtp_tls_security_level for outgoing email sent to other mail servers.

Although emails may still get delivered unencrypted from other servers we should make sure that users don’t accidentally (or out of ignorance) send their passwords unencrypted over the internet. That’s what we enforce “smtpd_tls_auth_only=yes”. Now our users should be safe.

How SMTP authentication works

Are you curious how SMTP authentication looks on a protocol level? Let’s go through that. In previous versions of this guide we used “telnet” to connect to TCP port 25 and speak SMTP. But now we enforce encryption and can’t do SMTP authentication unencrypted. Let’s try the unencrypted part:

telnet localhost smtp

The server will let you in:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mail ESMTP Postfix (Debian/GNU)

Say hello:

ehlo example.com

Postfix will present a list of features that are available during the SMTP dialog:

250-mailtest
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Let me briefly explain what these lines mean:

  • PIPELINING
    This is a feature to speed up SMTP communication. Usually the remote system has to wait for a response to every command it sends. Pipelining allows the remote server to send bulks of commands without waiting for a response. Postfix will just store these commands and execute them one by one. If you told Postfix to forbid pipelining it would disconnect the remote server when it tries to send bulks of commands without waiting for the proper reply. It is mainly a feature against spamming programs that don’t behave.
  • SIZE 10240000
    The remote server is allowed to send emails up to 10 MB large. This has long been a common maximum size for emails. However nowadays 40 MB or even more are more common sizes because emails have grown larger.
  • VRFY
    Allows remote servers to verify a given name or email address. For example the remote server could send “VRFY john” and your server might respond “250 John Doe <john@example.org>”. It can be used to verify that a certain recipient email address is deliverable
  • ETRN
    A command that a remote system can send to flush the Postfix queue of mails for a certain domain. It can be used if the remote system had technical problems and failed to receive email for a while. Then it could send an ETRN command to make your server start sending outstanding emails for that domain. It is rarely used.
  • STARTTLS
    This tells the remote system that it might start switching from this unencrypted to an encrypted connection by sending the “STARTTLS” command. It will then start negotiating a TLS-encrypted connection. You could compare it to an HTTP connection that suddenly switches over to an encrypted HTTPS connection. The advantage is that you can start talking SMTP on TCP port 25 and don’t have to open up a second TCP port like 465 which is the “SSMTP” (secure SMTP) port and only accepts encrypted connections.
  • ENHANCEDSTATUSCODES
    This enables more three-digit return codes for various conditions. See the RFC2034 if you are curious.
  • 8BITMIME
    In ancient times SMTP only processed 7-bit characters. You couldn’t transfer special characters like “Ä” or “ß” without special encoding. 8BITMIME allows a transmission of emails using 8-bit characters. Still many emails are specially encoded using ISO8859-1 or UTF-8.
  • DSN
    It enables DSNs (delivery status notofications) that allows the sender to control the messages that Postfix creates when an email could not be delivered as intended

However one important line is missing here that would allow us to send our username and password:

250-AUTH PLAIN LOGIN

We told Postfix to only allow authentication when the connection is encrypted. So we are not offered authentication over this plaintext connection.

Are you still connected? Okay, good. So we need an encrypted connection using TLS. You could enter STARTTLS:

STARTTLS

And the server would reply:

220 2.0.0 Ready to start TLS

However now it’s getting complicated because you would have to speak TLS encryption which is not a language that humans speak. So let’s quit this using the “QUIT” command.

But we can use OpenSSL to help us with the decryption. Now run:

openssl s_client -connect localhost:25 -starttls smtp

You will see a lot of output. OpenSSL has connected to TCP port 25 and issued a STARTTLS command to switch to an encrypted connection. So whatever you type now will get encrypted. Enter:

ehlo example.com

And Postfix will send a list of capabilities that will look like this:

250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

And now that we are using an encrypted connection Postfix offers us to authenticate. So let us send the authentication string with a Base64-encoded password:

AUTH PLAIN am9obkBleGFtcGxlLm9yZwBqb2huQGV4YW1wbGUub3JnAHN1bW1lcnN1bg==

The server should accept that authentication:

235 2.7.0 Authentication successful

Excellent. You are logged in through SMTP. Disconnect from Postfix:

QUIT

Goodbye:

221 2.0.0 Bye

Authentication works. Well done.

Base64-encoded passwords

You may wonder how I got to the long cryptic string that apparently contained John’s email address and his password. It is just a Base64-encoded (not encrypted!) version of the string “john@example.org<NULL-BYTE>john@example.org<NULL-BYTE>summersun”.

One way to create that string is using Perl:

perl -MMIME::Base64 -e \
'print encode_base64("john\@example.org\0john\@example.org\0summersun")';

The submission port (optional)

Although I have been talking about SMTP on port 25 to relay mails there is actually a better way: using the submission port on TCP port 587 (as described in RFC 4409). The idea is to use port 25 for transporting email (MTA) from server to server and port 587 for submitting (MSA) email from a user to a mail server. As many users still use SMTP by default you will likely have to run both ports.

Edit your /etc/postfix/master.cf file and add a service for the submission port like this (mind the indentation from the second line on). Don’t use the suggested and commented-out submission service in the master.cf – just ignore it.

submission inet n - - - - smtpd
 -o syslog_name=postfix/submission
 -o smtpd_tls_security_level=encrypt
 -o smtpd_sasl_auth_enable=yes
 -o smtpd_sasl_type=dovecot
 -o smtpd_sasl_path=private/auth
 -o smtpd_sasl_security_options=noanonymous
 -o smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf
 -o smtpd_sender_restrictions=reject_sender_login_mismatch
 -o smtpd_sasl_local_domain=$myhostname
 -o smtpd_client_restrictions=permit_sasl_authenticated,reject
 -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject

Basically this new service uses the “smtpd” daemon which is the piece of software that responds if you open an SMTP connection on TCP port 25. But it gets a few extra options set…

  • in the /var/log/mail.log mark the connections to the submission port as “postfix/submission/smtpd” (syslog)
  • enforce encryption on this port (security level)
  • enforce user authentication (sasl)
  • make sure that an authenticated user can only send emails in their own name (sender_login_maps)

Restart the Postfix server:

service postfix restart

Your users can now use the submission port to send email.

Protecting against forged sender addresses

Wait a minute. We can make sure that a user can only send emails using their own sender address? Cool. How does that work? Take a look at this line we just used above when we added the submission service:

smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf

The smtpd_sender_login_maps is a Postfix mapping with two columns:

  1. the email address the user wants to use as a sender address
  2. the username that was used when authenticating at the mail server

The reason I am just using the /etc/postfix/mysql-email2email.cf mapping is that it fits perfectly here. That mapping maps the email address to itself if it exists. Confused? Look at the SQL query that the “email2email” mapping runs:

SELECT email FROM virtual_users WHERE email='%s'

So when a user wants to relay an email Postfix checks the virtual_users table looking for the sender address. Say that John wants to send out an email from his email address “john@example.org”. Postfix will check the virtual_users table if there are any rows with an email field like that. It will find one and return the email field – which is just again the email address. Postfix expects to get back the username that was used to login. We are using the email address as a login username – so that mapping works. Only if John would try to send an email with a forged sender address then Postfix will see that it does not match his account and reject the email. Your mail.log would read something like:

NOQUEUE: reject: RCPT from foo.bar[…]: 553 5.7.1 <forged@email.address>: Sender address rejected: not owned by user john@example.org; from=<forged@email.address> to=<…>

It is your own decision whether you want to use the smtpd_sender_restrictions like that.

Testing email delivery

So far you have spent considerable time with theory and configuration. Are you worried whether all you did actually leads to a working mail server? Before we do the final steps let’s take a break and verify that everything you did now works as expected.

At this point the /var/vmail directory should be empty or maybe contain an “example.org” directory if you played with the john@example.org account previously. You can get a list of all files and directories within by running:

find /var/vmail

Although there are not actually any emails on the server yet, you may still get somethinng along the lines of:

/var/vmail/
/var/vmail/lost+found
/var/vmail/example.org
/var/vmail/example.org/john
/var/vmail/example.org/john/Maildir
/var/vmail/example.org/john/Maildir/new
/var/vmail/example.org/john/Maildir/subscriptions
/var/vmail/example.org/john/Maildir/dovecot-uidvalidity
/var/vmail/example.org/john/Maildir/tmp
/var/vmail/example.org/john/Maildir/dovecot-uidvalidity.5a537e85
/var/vmail/example.org/john/Maildir/dovecot.mailbox.log
/var/vmail/example.org/john/Maildir/dovecot-uidlist
/var/vmail/example.org/john/Maildir/dovecot.index.log
/var/vmail/example.org/john/Maildir/cur
/var/vmail/example.org/john/Maildir/.INBOX.Trash
/var/vmail/example.org/john/Maildir/.INBOX.Trash/new
/var/vmail/example.org/john/Maildir/.INBOX.Trash/maildirfolder
/var/vmail/example.org/john/Maildir/.INBOX.Trash/tmp
/var/vmail/example.org/john/Maildir/.INBOX.Trash/dovecot-uidlist
/var/vmail/example.org/john/Maildir/.INBOX.Trash/dovecot.index.log
/var/vmail/example.org/john/Maildir/.INBOX.Trash/cur
/var/vmail/example.org/john/Maildir/.INBOX.Junk
/var/vmail/example.org/john/Maildir/.INBOX.Junk/new
/var/vmail/example.org/john/Maildir/.INBOX.Junk/maildirfolder
/var/vmail/example.org/john/Maildir/.INBOX.Junk/tmp
/var/vmail/example.org/john/Maildir/.INBOX.Junk/dovecot-uidlist
/var/vmail/example.org/john/Maildir/.INBOX.Junk/dovecot.index.log
/var/vmail/example.org/john/Maildir/.INBOX.Junk/cur

Basically the schema you see here is /var/vmail/DOMAIN/USER/Maildir/…

Every IMAP mail folder then has three subdirectories:

  • new – every file here is an email that was stored in this mail folder but not yet read
  • cur – the same but for email that has been read already
  • tmp – for temporary actions by the mail server

According to our Dovecot configuration the mail folders are nested like this:

  • …/Maildir/new/… – the main inbox
  • …/Maildir/.INBOX.reddit/new/… – the “reddit” mail folder below the inbox
  • …/Maildir/.INBOX.servers.inga/new/… – the “servers”/”inga” mail folder below the inbox

If you do not use the legacy scheme with “separator = .” your directories will rather look like:

  • …/Maildir/new/… – the main inbox
  • …/Maildir/INBOX/reddit/new/… – the “reddit” mail folder below the inbox
  • …/Maildir/INBOX/servers/inga/new/… – the “servers”/”inga” mail folder below the inbox

Send a test email

It is time to send a new emai linto the system. Open a new terminal window and run

tail -f /var/log/mail.log

to see what the mail server is doing. Now let’s send an email to John. A simple way to create and send an email is to pipe text to the “mail” command. Do that in another terminal window:

date | sendmail john@example.org

If all works as expected your mail.log will show a lot of technical information about the email delivery. Like this:

Jan 9 09:10:47 jen postfix/pickup[2370]: 30B4B41A98: uid=0 from=<root>
Jan 9 09:10:47 jen postfix/cleanup[2633]: 
  30B4B41A98: message-id=<20180109081047.30B4B41A98@mail.example.org>
Jan 9 09:10:47 jen postfix/qmgr[30330]: 
  30B4B41A98: from=<root@mail.example.org>, size=287, nrcpt=1 (queue active)
Jan 9 09:10:47 jen dovecot: lmtp(2636): Connect from local
Jan 9 09:10:47 jen dovecot: lmtp(john@example.org):
  iHHHEwd5VFpMCgAA3BOsLQ: msgid=<20180109081047.30B4B41A98@mail.example.org>:
  saved mail to INBOX
Jan 9 09:10:47 jen dovecot: lmtp(2636): Disconnect from local: Successful quit
Jan 9 09:10:47 jen postfix/lmtp[2635]:
  30B4B41A98: to=<john@example.org>, relay=mail.example.org[private/dovecot-lmtp],
  delay=0.28, delays=0.06/0.03/0.08/0.11, dsn=2.0.0, status=sent
  (250 2.0.0 <john@example.org> iHHHEwd5VFpMCgAA3BOsLQ Saved)

Your output will look slightly differently. Just focus on the parts that are printed in bold letters. They should be exactly the same in your log file. If everything worked as expected Postfix has accepted the email and forwarded it to Dovecot which in turn wrote the email in John’s maildir. If you get any errors in the log file then try to understand the error message and find the cause of the problem before you proceed.

Look again:

find /var/vmail

Dovecot has now created a directory structure for John and created a new file:

/var/vmail/
[…]
/var/vmail/example.org/john/Maildir/new/1515485447.M404984P2636.mail,S=510,W=522
[…]

The file may have a different name on your system – that’s okay. The file name printed in bold letter depicts the actual email file. It is the only file in the “new” folder.

Accessing the email as a file on disk

The file just contains the email:

Return-Path: <root@mail.example.org>
Delivered-To: john@example.org
Received: from mail.example.org
 by mail.example.org (Dovecot) with LMTP id iHHHEwd5VFpMCgAA3BOsLQ
 for <john@example.org>; Tue, 09 Jan 2018 09:10:47 +0100
Received: bymail.example.org (Postfix, from userid 0)
 id 30B4B41A98; Tue, 9 Jan 2018 09:10:47 +0100 (CET)
Message-Id: <20180109081047.30B4B41A98@mail.example.org>
Date: Tue, 9 Jan 2018 09:10:47 +0100 (CET)
From: root@mail.example.org (root)

Di 9. Jan 09:10:47 CET 2018

If anything went wrong then carefully check the last lines of your /var/log/mail.log. It will very likely point you to the problem. Or read the troubleshooting section. Or just add a comment to this page and ask other readers for help.

You can also use a slightly more comfortable tool to access Maildirs that will come handy for you as a mail server administrator: “mutt”.

mutt -f /var/vmail/example.org/john/Maildir

(You may get asked to create /root/Mail – this is standard procedure. Just press Enter.)

What you see now are the contents of John’s mailbox:

ispmail-jessie-mutt-index

Using mutt is a nice way to check mailboxes while you are logged in to the mail server.

To reiterate what happens when you receive an email:

  1. Postfix receives the email (using the “sendmail” command in this example – but usually through the network using the SMTP protocol from other servers)
  2. Postfix talks to Dovecot via LMTP and hands over the email
  3. Dovecot write the email file to disk

Accessing the email via IMAP (Roundcube)

Now that the email has been delivered you can talk to Dovecot using the IMAP protocol to retrieve your email again. Are you still logged in via the Roundcube webmail interface? Then just reload and you will see the email:

 

Accessing the email via IMAP (mutt)

In the last section we used mutt to access the Maildir directly on disk. But remember that mutt also speaks IMAP:

mutt -f imaps://john@example.org@localhost

Does it show you the email now? Great.

Access the email through IMAP (desktop client)

Your users are not likely to use “mutt” to read their email – unless they are immortal console nerds. So of course you can use a graphical mail client to access John’s emails. If you use Thunderbird, Evolution, A**le Mail or some other IMAP client then feel free to configure a new IMAP account and connect to the server. Remember that the user name and email address are both “john@example.org” and the password is “summersun”.

POP3 versus IMAP

Aside from talking IMAP Dovecot also knows how to speak POP3. But let’s face it – POP3 is dead. I do not offer it to my users any more. But as your users may ask let’s quickly review the differences of the two protocols:

  • POP3 (Post Office Protocol) is a simple protocol that lets you fetch email from a single mailbox. It is usually used to collect all emails, though you can also leave them on the server but this is a bit of a hack and you can’t create multiple folders on the server to sort your mail. It saves space on the mail server because the email gets moved to the user’s hard disk on their computer. But they won’t be able to access the same email from another computer. Besides you cannot create multiple folders on the server to sort your mail. There is just the inbox. This variant is antiquated and not exactly user-friendly.
  • IMAP (Internet Messaging Application Protocol) is predominantly focused upon leaving your mail on the server but you can also collect it like POP3. The inbox is where your incoming emails are stored but users can also maintain folders and move emails to them. But users can move emails to different directories. IMAP is useful when you want to access your email from different locations without losing mail because you fetched it from another location. The drawback is that lazy users leave their mail on the server thus filling up your server’s hard disk (unless you use quotas).

© 2021 workaround.org - Proudly powered by theme Octo