I managed to introduce quite a few improvements that make this tutorial better than any previous issue:
- Clearer explanations. More diagrams and screenshots. More side notes and links to deepen your knowledge.
- A newer Roundcube version. The webmail software has been upgraded from version 0.9 to the prettier version 1.1. Debian Jessie officially does not include Roundcube because its package maintainers did not manage to come up with decent packages when the release deadline was approaching. But fortunately the jessie-backports repository provides fresh Roundcube packages – so we can use them.
- Way more secure passwords. Dovecot handles the authentication for us. In all previous tutorials that used Dovecot I used unsalted MD5 hashes for the user passwords. That is admittely a very weak way to store passwords. Should someone accidentally get access to your encrypted passwords it would be rather simple to reverse-compute the plaintext passwords from that. Previous tutorials kept using MD5 though so that upgrades would not break anything for your existing users. From now on the new hashing method is salted SHA256 which is almost impossible to crack today. However existing users can still use their old MD5-hashed passwords. Once they change their passwords they will automatically get an SHA256 hash. Your users will not even notice that.
- Spam handling improved a lot. Previously I fully relied on real-time blacklists (RBL). So if a sending mail server on the internet would have been listed all email from them would have been rejected. It turned out that blacklists are often maintained automatically so it may accidentally happen that a legit mail server gets listed and thus blocked. Your users won’t be happy at all about it no matter how often you tell them about spam fighting. So this tutorial relies on SpamAssassin to query multiple blacklists (besides a lot of other checks) to determine if an email is spam. But even if it is definitely spam it will be delivered to the user – but into their Junk folder. And we will automatically subscribe them to their Junk folder so they will notice spam emails easily. Your users will never bother you again to track lost emails while you still have a much better spam detection rate than before.
- No greylisting. Spammers nowadays seem to be too smart to fall for greylisting. Greylisting delays incoming email but hardly has any effect on spam rejection. So I decided to leave it out. Besides users appreciate if they get their email as quickly as possible. Most of them will not be happy if they need to wait several minutes for a new email.
- Your users can finally change their own email password. Roundcube provides a password plugin that is integrated into the webmail interface that can do the change in the database.
- Communication between Postfix and Dovecot is now using the LMTP protocol instead of the LDA (local delivery agent). This increases delivery performance, uses less system resources and can handle emails with multiple recipients at one go.
- For experienced users I provide an Ansible playbook that helps you set up a mail server within minutes. But please work through this tutorial patiently before trying the playbook. It is intended for advanced system administrators who aim to set up new mail servers frequently.
I understand why no graylisting anymore. But whats about SPF and DKIM?
Writing such a guide is a highly opinionated thing. Others may argue that graylisting and SPF save them from a great deal of spam. I prefer to keep things simple. Users want their email instantly so I don’t use graylisting. And users may forward their email creating difficult situations where SPF is inadvertently blocking of emails. SPF was often error-prone and still too few organisations seem to use it.
I consider DKIM a very important technique though because it addresses the spoofing problem in a cryptographic way which sounds very right. The page about DKIM is already on my todo list. It may be a week or two though until it is good enough to be published.
I do actually sill use both SPF and DKIM
SPF remains the same as before as it’s essentially just an entry on your DNS zone file.
DKIM is slightly more complex, but I now use opendkim, which is available as a Debian package straight from the repositories. I followed this tutorial:
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy
which works equally as well on Jessie.
Just be very careful about the three essential files; the KeyTable, SigningTable and TrustedHosts, only in as much as their syntax can be easily misconfigured, if you’re not very careful. Needless to say, a misconfiguration will lead to a loss of email service. I know, I’ve been there! The saving grace is you will see the errors in your mail.log file, although sometimes (particularly if you have more than one error), it’s not that clear how to resolve them. However, the fact that I managed it and am far from being a email guru, suggests that it’s possible 😉
You may also find the public (txt) certificates generated by the opendkim-genkey application, need to be modified; NOT the key itself, but the way the key is generated into separate lines by parenthesis and double quotes. You can find an example of this in the following:
http://www.opendkim.org/opendkim-README
about half way down.
There are two potential problems here; first, not all registrars/ISP’s accept DKIM signatures at all and second, if they do, some don’t accept them in this format and their automated systems reject the key and won’t let you add it to your DNS zone record. They often want them as a single line.
In this case you need to get rid of the parenthesis and the double quotes other than those at the beginning and end of the file. So from; “v=DKIM1………..etc…….to end of key” making sure there are no spaces left between the key characters, no parenthesis and no quotes other than the beginning and the end.
They’re the prime reason I now run my own DNS servers, as it does away with hassle of trying to get around these problems. That said, whether you run your own servers or not, the configuration needs to be accurate.
I’ve also found that in the UK, some major email providers are now rejecting emails if they are not signed. So if you want your server to mix with the majors, you may not have any choice but to implement these services.
Finally, remember any DNS change needs time to propagate around the Internet. So even if you’ve done all the right things, it maybe a few hours before you can see the fruits of your labours.
One think I can recommend to look into is `rspamd`. I’m using it now instead of spamassassin, and the performance and spam detection have proven to be significantly better! It was a touch job to configure due to messy documentation, but it’s worth the effort!
I’ve been using linux of many distros for decades. I’m an OK ~ average admin but no expert admin.
Never had the need to setup email, but I do now. Installed postfix and opendkim as per this guide for starters: https://easyengine.io/tutorials/mail/dkim-postfix-ubuntu/
I have a basic question about what a proper hostname should be. I have a single VPS machine to support my domain. It is very common to see references like “mail.domain.com” in these tutorials (and I’ve read many) but only limited discussion about subdomains and hostnames, even for a simple case like mine. Technically “mail” is a host according to IP convention, but it is often used or referred to as a subdomain, especially with shared hosting environments.
Also, there is no well recognized standard or convention regarding the hostname value (short name or FQDN) and these affect how the various DNS records are formated.
Would you care to address how to configure a host for proper opendkim signing of outgoing emails for this common case of a single VPS host that needs to send email?
Thx!
Hi Christoph,
We’re using your tutorial since years. Now we’ve recognized a problem.
If there’s a coming a mail from an external server to an alias pointing to a gmail address, we’ve problems with the original spf record of the sending domain. Because in the mind of the receiving server, the sending server is our postfix.
We’ve read about srs (sender rewriting scheme) … is it possible to add that to postfix?
Thank you in advance
Reto
HI Christoph, I have just started with trying to set up my own mail server on a root server. I am using Ubuntu 16.04 server. Does your script also work for that version?
Thank you for great work and an answer.
/gerd
wretye5ryabcd.com
First, excuse my English, please. I manage a mail server over debian etch since 2010. It’s time to update it and I’m looking for a good tutorial that would let me maintain all functionalities I have currently on it. One of them is graylisting. I added graylisting to my server in 2014 and the result was really amazing. It decreased spam in about an 80%. I think that if emails delayed a little in arriving to recipient is not so traumatic. Email service was not developed to make messages arrive instantly. There are other software that makes that work much better or even a phone call is more effective if you want to communicate something immediately. Do you think graylisting systems work today or they have lost its utility at present?
Hi, Pedro. Some say that greylisting still works well. I can’t tell it scientifically but it feels ot me that spammers are not that stupid any more. Even the millions of infected Windows PC just retry sending the email and bypass greylisting. It probably depends on what your users are actually used to. At my employer we used fetchmail for a while and that was ridiculously unprofessional. However it made the users wait for an email for up to 15 minutes. It’s rather common nowadays to have someone on the phone telling you “I just sent you the information” – “oh, let me check” and expect the email to be delivered in 1-5 seconds.
In the next issue of this guide I will recommend using rspam which uses greylisting for email that appear suspicious but are not definitely spam. Let’s see what results we get with that.
Hi, Christoph. I don’t need scientific evidence. Your experience is useful enough for me. I feel the same way too about spammers’ shrewdness. I’m going to put my trust in your guide and will keep an eye on my spam to study the usefulness of the modern anti-spam techniques. Thank you very much for your help.
hi,
Merry xmas! I am able to integrate prosody (xmpp) with ISPmail using the setup done by Christoph here. If there is an interest please let me know I would need to write it up about 1-2 page of installation procedures
Nice