I would appreciate if you leave a comment here telling others if your mail server is running. Perhaps even a word about the purpose of the mail server and the number of users you have. That allows others to see that their time is well-spent and indeed leads to a working mail server. Thanks for your feedback.
A huge thank-you to you Christoph. I’ve spun down my old mail server, and everything is working splendidly on this new build. It’s just a small operation with 6 domains and a handful of users, but working well all the same.
Yay, thanks for your feedback. That makes at least two of us who made it. 🙂
I’m not fully through, but I have used some older guide to set my mailserver up and it did work just fine. I did not follow the instructions exactly, because I did not set up a new server with Debian 10 and instead used what I migrated from and left some settings as they were. But at the moment my mailserver does work again and has a neat new spamfilter (replaced the spamassasin). I will look at the rest of the tutorial at a later time.
If I’m honest: I don’t know why I even use that overkill of a mailserver. It runs on a server at home and I’m the only user and nobody ever sends me mail there, because nobody knows the address. But I use it to get status mails from my server (cron jobs, updates, etc.). Not really needed, but I do it for the fun of it. I plan to try D-Push or Z-Push to get mails pushed onto my mobile, next.
Just FYI if you want to add that to your tutorial anytime. Push-Mail with D-Push works realtively easy. I had to apt install d-push (which installed z-push and everything and even enabled the configurations in the webserver).
I had to add some paths to the php_admin_value open_basedir in my apache configuration. The paths I had to add were /usr/share/z-push/:/etc/share/z-push/:/var/log/z-push/:/var/lib/z-push/
After that some configurations are required in /etc/z-push. You can add the correct timezone in z-push-conf.php in the define ‘TIMEZONE’. And in the same time I had to set the Define ‘BACKEND_PROVIDER’ to ‘BackendIMAP’.
After that some configuration in the imap.conf.php is required. Mainly the server data. I had to change the IMAP_PORT to 993 and the IMAP_OPTIONS to ‘/ssl/novalidate-cert’ to support SSL. Then there is the Define IMAP_FOLDER_CONFIGURED which has to be set to true, after checking the Defines below that, are matching the folder structure in your IMAP. (I don’t know if I have a standard config there, so I’m not giving my config here, but if I open the folder in roundcube the folder name is displayed as variable _mbox in the URL.
Restarted the Webserver and checked if I could access https:///Microsoft-Server-ActiveSync This should ask for username and password (use the data for an E-Mail-Address). After that you should get an error message stating “GET not supported”. If another error message is displayed the error should give hints about wrong configuration. And if you only get an error 500 or something like that, the Logs are your friends.
At this point the Server should support Pushmail. Now the mobile has to support it, too. I am using K-9 mail for android and it works fine.
I hope this is enough for anyone wanting the feature to get started.
Holger:
I’m sure this is obvious to people who know what they are doing, but where in the apache config did you add those paths?
I went through the Buster guide and I’m very happy that it’s really easy and fast-forward.
I subscribed last year professional email boxes to an ISP and when I used mx-tester, the quality of outbound mail was 3/10 and unlikely not to be delivered elsewhere than spam boxes.
I run “real” SMTP because my project needs more than sending newsletters and an email API is just not the right call for that.
As per use case, I will have a small business run with one domain and 10-20 users and i’m pretty confident it’ll do the job well without me buying a Google Suite access or cloud-base solutions. I’m not a big believer of having personal data “somewhere” in the cloud, stored in conditions I can’t check.
So, this guide is a great opportunity to catch up on Email hosting.
Big Thanks Christoph !
P.S.: For those who looks for an interface you could pick up a GUI or take a look at a web API i’m starting : https://github.com/Gompali/ispmailadmin-api. It’s still beta but may work as soon as possible. Feel free to contribute.
Thank you very much for the feedback. The API looks promising. I will surely take a look soon.
Yep. Now it works. 😉
Just a general thank you and a success story. With only a few bumps on the way I made it, and moved our small but ancient (really: qmail+vpopmail circa 2004 vintage) server to ISPmail (imapsync helped enormously – the only way to migrate from something that different). Works like a charm, really excellent tutorial.
Amazing work.
I owe you.
Regards
Ludek
Hi Christoph,
I have setup my Family email server in 2005 with Sarge. Since then, I was always doing inplace Upgrades. These days I found your updated buster guide and decided to start now again with a blank install. To learn something new, I use postgres as a backend. I hope migration will be done in some days, as I only spent here and there an hour on it.
For me this is one of the best guides ever seen, as it serves the need of professionals, but also newbies by explaining the context and giving additional background Information.
Many thanks
Matthias
PS: I will keep my existing dbox format and not migrate to Maildir;-)
Hi Cristoph,
Thank you very, very much for this amazing documentation. I just configured my VPS here and it’s working flawlessly. In fact, this documentation is so good that I’ll redo another VPS that I manage here and it’s a little messy. 🙂
I loved the inclusion of the theory of how mail servers works! And you have a very clear language. I’m a Brazilian Portuguese native speaker and understood everything.
Hope you continue to update this guide for the next Debian releases. 🙂
Best regards!
Thanks for this awesome tutorial! I have my mail server up and running on my Raspberry Pi 4 with an attached SSD for mail backup purposes. I use an additional Raspberry Pi as a local DNS server (with Pihole and Unbound) that handles the mail server’s spam queries (instead of sending them out to my ISP’s DNS servers). I’m a residential Comcast customer so I had to set up a relay through their smtp server to send outgoing mail, and I have to use dynu.com as an incoming relay because Comcast blocks port 25. But your server set-up has proven to be rock solid despite these local workarounds. I particularly appreciate the info on DKIM and SPF record set-ups. Those are the little touches that actually make the mail server usable in the real world. Thanks!
Hello,
This guide is all i was looking for to learn linux.
It introduced me to mailservers, tought me how they work + made me configure it.
I am pleased to say that my mailserver is now running smoothly.
It was done just for learning purposes, but man this guide helped alot.
PS The email address used to post this comment is hosted on the newly installed server.
Thanks a lot!
@Christoph: do you have experience about adding Dovecot full text search that you can share with the community? This is probably “the one missing feature” to more easily say goodbye to gmail et. al.
Indeed. I wonder how that dropped off my todo list. Let me see what I can provide.
Seems that I managed to get the Solr search engine plugin working. I’m currently indexing my family’s mail server with 27 GB of emails. Let’s see where that ends. 🙂 A quick text with a single mailbox and a full-text search from Thunderbird was lightning fast.
Hi Christoph, I would like to see your choosen configuration for this. I pasted mine here https://pastebin.com/raw/1pSJK6ca . Could you share yours?
Funny. Mine is very similar. I have tried with Tomcat and Jetty first though until I learned I don’t need that. And I was shocked to see that the Solr version in Debian is from the stone ages. Besides the Buster version has been vulnerable for over a year because of CVE-2019-0193.
I only did a few details differently: https://pastebin.com/raw/acS77Y5z
What bothers me is that I don’t see Dovecot talk to Solr (in tshark) when I search from Thunderbird. I still need to figure out if it’s using everything correctly.
Exciting… thank you for looking into this! Can’t wait to learn if this will make it into your guide!!! Thank you again!!
Vielen dank Christoph! My brand new e-mail server is running better then ever on a clean Ubuntu 20.04 server with all previous emails migrated from the old server. It’s only managing 4 domains for the family and a handful of friends and accepting mails for a long list of aliases, so nothing too fancy.
Your tutorial was fantastic. Not only did you explain What to do and How, but also Why. The result was I learned a great deal of new stuff and truly enjoyed the whole learning process.
DKIM was new to me and makes the server much more usable than my previous setup.
Thanks again for your time and effort, you really created a high quality tutorial here, by far the best I have come across on the net…
That’s feedback that makes my day. 🙂 Thank you for taking the time to report your success.
Most of the servers I deal with have just 10-30 users, too, by the way. But don’t worry. We will be ready when managers realize that moving into the cloud was utter idiocy and we will help them get their emails back under control. 🙂
Many thanks Christoph !
Your tutorial is just perfect. I realy like the way you explain things. You always justify your choices, that’s respectfull for your readers.
I use your tutorial to run a self-hosting server in my house. The part that i realy appreciate is the security stuff (firewalling, fail2ban…)
THANK YOU !
First of all, thanks Christoph!
I completed your tutorial on a Raspberry Pi 4 and it worked like it should. But after I completed the tutorial I found out that my ISP was blocking port 25, I should have checked it up front.
So I decided to rent a cheap VPS and followed the tutorial again. Everything works like a charm. The hardest part is not following the tutorial, but when you’re done, you need to check if you mail is coming through. I can really advice others to use https://www.mail-tester.com to check if you’ve done everything correctly (like DKIM, DMARC & SPF). When I was finished with the tutorial, I tested my mail delivery with mail-tester.com and I started with an score of 0.2. If you made small mistakes with DKIM, DMARC or SPF, your mail will probably never end up in another mailbox. After some changes I scored a 10 on the mail-tester website. Those things are a little bit different on every domain name registrar but the basics are the same.
I think in theory, the VPS company / owner can always access your files, that’s why I also implemented the server-based mail encryption. If you run your own server from home, you probably don’t need this. But it’s a nice extra security feature.
Thanks again,
J.
I have automated this guide in Puppet. I will share a github repository once i seperated the code from my own setup.
This is a very nice guide – i was using another guide for the last 5 years or so, but wanted to upgrade to latest ubuntu and use rspamd (vs spamassassin). I came across this guide and it looked very complete with everything i was looking for, so gave it a try. I’m up and running in 1 evening. My use case is simply for my own email although i could see trying to do more with it…
i did make some modifications which all seem to work:
– i used ubuntu 20.04 instead of debian buster – just because i’ve been using ubuntu for the last 10 years. only thing i had to do was add a php repo so i could install php7.3 (otherwise ubuntu would have installed php7.4).
– i skipped adminer and quotas (i don’t need quotas for myself) and instead installed postfixadmin (PFA) – only thing i had to do to make PFA work was modify the sql queries as PFA’s database structure is different than what you set up… i just inspected your sql queries and found the appropriate variables/tables in PFA’s sql database. One nice thing about PFA is that you don’t have to prepare the database manually – it gets done automatically when you use PFA to setup the domain & users.
– i replaced roundcube with rainloop… simply a personal preference. it worked out of the box pretty much – and has native sieve support – i created a filter and found the sieve file in my vmail/user folder, tested it and it works.
– i also used the encryption option – something i had always wanted to implement. I had to tweak PFA to add a custom field the sql database – PFA supports custom field additions and was pretty straightforward to setup. (i left some questions about automating encryption in that section of the guide).
thanks for a great guide!
jay
Thanks a lot for the detailed feedback. An evening sounds like you are indeed an experienced sysadmin. Would you mind sharing the changes you made to PostfixAdmin? It has been requested a lot but I never managed to get it configured properly to work with this database schema. I’m sure it would make a lot of people happy.
few things related to “an evening” – it was a long one and i did read the guide over the preceding days so i had a head start before touching the keyboard, also that didn’t include the encryption..
on PostfixAdmin setup – i started with the guide i used previously – https://www.exratione.com/2019/02/a-mailserver-on-ubuntu-18-04-postfix-dovecot-mysql/ – see the section labeled “Install Postfix Admin 3.2 and the MySQL Schema”. this seems to follow steps 1-5 from https://github.com/postfixadmin/postfixadmin/blob/master/INSTALL.TXT. So i did first have to set up mail database and users which i followed from your guide here- https://workaround.org/ispmail/buster/prepare-the-database/, i just didn’t have to set up the tables.
so based on that, i modified /etc/dovecot/dovecot-sql.conf.ext in the following ways:
1) default_pass_scheme = MD5-CRYPT. (PostfixAdmin defaults to md5crypt – there do seem to be other encryption options in PostfixAdmin but not specifically BLF-CRYPT though one option is to specify the dovecot encryption scheme – i meant to go back and try this…)
2) i had to modify the sql queries:
2a) user_query = SELECT username as user, \
‘/var/vmail/%d/%n’ AS home, \
5000 AS uid, 5000 AS gid \
FROM mailbox WHERE username=’%u’
(i left out the concat line since i’m not using quotas)
2b) password_query = SELECT password FROM mailbox WHERE username=’%u’
2c) iterate_query = SELECT username AS user FROM mailbox
Then in the https://workaround.org/ispmail/buster/making-postfix-get-its-information-from-the-mysql-database/ section, i made modified the sql queries to these:
1) virtual_mailbox_domains sql query: query = SELECT 1 FROM domain WHERE domain=’%s’
2) virtual_mailbox_maps sql query: query = SELECT 1 FROM mailbox WHERE username=’%s’
3) virtual_alias_maps query: query = SELECT goto FROM alias WHERE address=’%s’
after i did all that, i noticed that in /srv/postfixadmin/config.inc.php there’s a section you can copy to /srv/postfixadmin/config.local.php to remap table names – so you could probably remap the table names to match yours..
to make encryption work w/ PostfixAdmin, you need another column in the mailbox table – that equates to the crypt column @Eelke describes. PostfixAdmin supports custom fields (you have to name them like x_NEW_FIELD_NAME). to do this add the following to /srv/postfixadmin/config.local.php:
function x_struct_mailbox_modify($struct) {
$struct[‘x_crypt’] = pacol(1, 1, 1, ‘int’, ”, ”, 2, array(), 0, 0, “”, “”, “”);
return $struct;
}
$CONF[‘mailbox_struct_hook’] = ‘x_struct_mailbox_modify’;
i don’t know a thing about php, but i was able to piece above together from instructions in /srv/postfixadmin/config.inc.php & some other stuff i searched online, so i can’t really advise what many of pacol(…) parameters specificy but ‘int’ is the column type, & 2 is the default value for x_crypt which gets mapped to userdb_mail_crypt_save_version in the query statement in /etc/dovecot/dovecot-sql.conf.ext. So, if using this mailcrypt option you have to modify the query i showed above to this (so 2b becomes):
password_query = SELECT password, x_crypt as userdb_mail_crypt_save_version, ‘%w’ AS userdb_mail_crypt_private_password FROM mailbox WHERE username=’%u’
this just makes PostfixAdmin support the additional column… you have to manually add the column in sql – e.g., some thing like: alter table mailbox add x_crypt int(11) not null after quota; once thats done though, new mailboxes added get setup with x_crypt with a default value of 2.
So if the user uses PostfixAdmin to modify their password, you have to follow that with the doveadm mailbox cryptokey password -u john2@example.org -n newpass -o oldpass (i left a suggestion/question in the encryption section on how one might automate that – for detection, PosftixAdmin supports an addition to run a script after mailbox is edited…)
that was pretty much it… i think all of the other postfix & dovecot options related to working with PostfixAdmin came from either defaults or resulted from your guide.
Here’s a fuller writeup i did – https://gist.github.com/yajrendrag/203b0172fee96a8b002a026362d27bf2
This covers both basic setup and and use of the optional encryption module. I figured out a few more things since initial response – i’m missing a SQL statement above and i was able to fully automate encryption key updates when a user changes their password via the PostfixAdmin GUI. Also, while i didn’t implement quotas, the gist assumes you’ll want to use quota enforcement – the database set up by PostfixAdmin contains a quota field and the modified SQL queries should leverage that.
Hi Christoph
Thanks for sharing this tutorial!
Poking around to get DKIM and DMARC working the days before, but this one give we the wanted lift-off 🙂
Got today a 10/10 via https://www.mail-tester.com !
One question:
How can I configure that postfix also read /etc/aliases and I can easly fill when I made an new mailman list?
Hi,
thank you very much for your substantial guide. 😀
I’ve put up my mailserver mostly to learn how it all works and how it is done.
I’ll just see what I could do with it in the future.
I have been in a helpdesk job for the last 13 years and I am now somewhat semi sysadmin, so I thought it is mandatory to learn the intestines of E-Mail.
I also study for my masters degree in computer science in my free time and aim to get into infromation security in the future. So knowing mailservers is a must, i guess.
Thank you again for teaching me. 🙂
P.S. I didn’t include the mailadress of my own mailserver. Not because I don’t trust you, but because I don’t exactly know how long it will run and how reliable my own work is.
It’s me again. I forgot I also wanted to include my setup FYI.
My setup is the following:
My mailserver is located on a proxmox PVE at my home. So how do I send E-Mails, when i have an dial-up IP-address?
I have been renting a tiny 1€ VPS. My mailserver connects to it via wireguard vpn.
The VPS Server then routes all traffic on the needed ports to my Mailserver at home.
The mailserver has a road warrior config, so it routes, in return, all traffic to the internet through the VPN. The VPS does a dnat on incoming traffic and a snat on outgoing traffic.
So I can use the nice IP-Address from the VPS and still have my mailserver at home.
I like it. 😀
Thanks for taking the time to share your story. A 1€ VPS is pretty cheap indeed.
By the way I don’t like to harvest people’s email addresses. I wish I could tell WordPress not to ask that. 🙁
Thank you very much for your tutorials 🙂 My e-mail server based on your tutorial has been running great for the past two years.
Hey Christoph,
thank you very much for this perfect guide!
I set up my mail server some years ago following the Stretch guide. I learned a lot and the server was running perfectly.
Last year I was eagerly waiting for the new Buster guide and as soon as it was available I started to set up a new server. Unfortunately my spare time melted away because other projects needed my attention and I couldn’t finish the setup. Two weeks ago I started again on a new server. After some days everything was up and running and I migrated to the new server. I have one small problem with the DKIM signatures, see the comment on the appropiate page, but this is just some minor problem.
I’m really happy with this guide and I can’t even imagine how much time you put into all the testing, writing and correcting. I can’t thank you enough and would like to let you know that your guide is really appreciated.
Thank you very much!
Arne
Hi. I set up my first mailserver back in ~2012 following your Debian squeeze guide. It served me well for 8 years until my VPS host announced the end of OpenVZ VPS on Jan 1st, 2021.
So I rented a KVM VPS and followed this Buster guide from scratch, copied over my ancient vmail directory from my old mailserver, and as of today I swapped IPs and the new mailserver on Debian Buster is working perfectly in it’s place.
Thank you so much for the guides. They’re a unique and invaluable resource for the ‘net. They’re made my life much easier and better.
Excellent guide, helped me big time, many thanks Christoph.
Hi Christoph
I just wanted to say that I have never before come across such a comprehensive and thorough tutorial. It’s people like you and all your contributors who are willing to put in the time and effort making open source and Linux so satisfying. A big thanks from NZ.
Thanks for this awesome resource, Christoph! I have been running a mail server just for myself since 2016, when a friend at a Linux user’s group pointed me to your Debian Jessie tutorial. I’ve been really pleased with how the setup has served me, despite only dabbling in admin tasks for it now and then. It’s great that you make it doable even for us amateurs.
I was caught a bit off guard by Ubuntu 16.04 LTS losing support, but the Buster guide got me up and running on a new machine. I can confirm that it’s a great guide for 20.04 LTS as well. I think I’m going to try to make a quad- or biannual habit of rebuilding it when it’s time for a new operating system to make sure I stay on top of it.
Thanks for keeping the instructions for managing users in mysql, by the way. I keep my setup pretty simple, so I prefer to skip the web portals, and that list of queries is a very handy reference.
Thanks for the feedback. Great to hear you got back on track. I often get asked why I don’t introduce the much cooler Nginx and PostgreSQL. But reading your comment I know why I try to keep the changes as minimal as possible.
Consider using Debian instead of Ubuntu. In that way you won’t have to adapt the recipe.
Has anyone had any problems or success using Mobile Outlook, latest version from AppStore on latest public release of iOS with IMAP. I can setup with same settings on Apple Mail, Canary, other mail apps on same iphone and ipad and have no problems at all from start of setup. I have disabled TLS 1 and 1.1 on server, no idea unless Outlook does not use STARTTLS as I see no options to change that or force it?
Hey there! I love your guides! I have been using them for years and years, and they never fail to impress. The amount of learning in here is crazy. I rarely take the guide as is, but it serves as a solid basis to start from everytime I’m refreshing my servers or switching providers!
I usually use Ubuntu LTS servers instead of debian, no quota support, and still use iptables for some reason :/
Thanks a lot man, you rock!
Hi Christoph,
Thank you very much for this guide which worked perfectly on Debian 9 and now on Debian 10!
I only have one question, how can I add the use of a “no-reply” mail account? I have read about a devnull in /etc/aliases and about using access with discard but I really don’t know how to set it up.
Do you (or anyone reading me) know how can I succeed with that?
Best regards