Upgrade or fresh installation?
I recommend that you set up a new server and make sure that the new mail server is working correctly before you start migrating existing email users to it. You may argue that Debian can be upgraded easily using “apt-get dist-upgrade” but that is very dangerous on a live mail server. Automatic configuration changes may have evil side effects and you risk losing emails. At least you are causing a downtime for your users.
If you follow my advice then get a new server and install Debian Bookworm on it. Once the user mailboxes are migrated and all works well you can tear down the old server.
Once your new server is installed come back here.
Read before you type
Follow this guide to the end. Only then start attempting a migration. Inform your users about the migration and set a time when you intend to move email accounts to the new server. The change will require a DNS change which takes a while to be visible worldwide so your users will have a period when incoming email is delayed. If you proceed careful though not a single email will be lost.
So you have your new server up and running and did everything to make it a working mail server? You really read all the pages in this guide and did what they told you? Okay, then let’s start.
Turn down the TTL of your MX record
The DNS “MX” record for your domain contains the hostname of your mail server. When switching to the new server you need to change the MX record. Every DNS record has a TTL (time-to-live) that defines the period of time that a record will stay valid even after you change it. Usually that TTL is rather high like 86400 seconds (=1 day). This information is used by caching name servers that they can use the cached values for a day. Turn that TTL down temporarily to 60 seconds so that the rest of the internet will pick up your change quicker. However it will take a day until everyone else on the internet picks up your TTL change.
Migrate the mailserver database
You need to copy the database that contains the control data about your email domains and accounts. Log into the old (Bullseye) server as root and back up the mailserver database. That is as easy as running…
mysqldump mailserver > mailserver.sql
Copy that file to the new server (using scp) and import it there:
mysql mailserver < mailserver.sql
Obviously any database changes on the old server from now on will have to be done on the new server as well until the migration is done.
Roundcube contacts
If your users are using Roundcube as a webmail interface then you should migrate their data like their contact lists. Dump the SQL from the old server:
mysqldump roundcube > roundcube.sql
Copy that file to the new server and import it:
mysql roundcube < roundcube.sql
One caveat though. To distinguish multiple mail servers Roundcube stores the server’s name in the mail_host column of the users table. So as a last step change that column if your new mail server has a new FQDN/hostname by running this SQL query on the new roundcube database:
UPDATE users SET mail_host='new.mail.server';
Migrate rspamd spam training data
If you have been using rspamd with the Redis backend then copy over the Redis database from your previous server. Details are found in the rspamd chapter.
Migrate the Maildirs hot
Fortunately Dovecot uses the maildir format that stores emails as plain files on disk. Login to the new (Bookworm) server and use rsync to copy over the mails from the old (Bullseye) mail server:
rsync -va oldserver:/var/vmail/ /var/vmail/
(Note the trailing slashes. Type them exactly as shown above or your files will end up in wrong places.)
There is no need to shut down Dovecot on your production Bullseye server. Copying the files while Dovecot is running will not break anything. This is called a “hot copy”. It may not be consistent but it will save time during the final synchronization.
Copy certificates
Copy over everything in /etc/letsencrypt and /var/lib/rspamd/dkim from your old to the new server.
rsync -va oldserver:/etc/letsencrypt/ /etc/letsencrypt/ rsync -va oldserver:/var/lib/rspamd/dkim/ /var/lib/rspamd/dkim/
Downtime
You told your users about the downtime, right? The time has come? Okay. Shut down Dovecot on both servers.
Migrate the emails cold
Let’s synchronize again. rsync will only copy those files that have changed which makes it much faster than the first sync. On your new server run:
rsync -va --delete oldserver:/var/vmail/ /var/vmail/
(The “--delete” option makes sure that files that have been removed from the old server will also be deleted from the new server. So if a user has deleted an email it will be deleted on the new server as well.)
The new Dovecot version uses a slightly different indexing mechanism. So force rebuilding the users’ indices:
doveadm force-resync -A '*'
Switch the DNS records
For all your domains you will have to change the DNS “MX” or “A” record to point to your new server.
Enable soft_bounce
Accidents happen. And you don’t want to lose emails. So run this command to enable your safety net on the new server:
postconf soft_bounce=yes
This makes Postfix always keep emails in the queue that it would otherwise reject. So you can fix any errors and the queue will empty. Start Postfix and Dovecot on the new server. Watch your /var/log/mail.log and run “mailq” from time to time to see what emails get stuck in the queue. If you are certain that emails can be removed from the queue then use “postsuper -d QUEUE-ID” (as shown in the “mailq” output).
Once you are certain that emails are properly received and sent you can switch off the soft_bounce mode again:
postconf soft_bounce=no
Shut down the old server
If possible do a final backup of the old server. If users are not complaining then dismiss the old system after a week.
Hello, thank you for all your work. I have this question, on migrating: right now i have a server named “hermes” and in order not to annoy my users, i wish for the new server to keep using the name “hermes”. But – unless i am not getting it properly – this can not be done, since while preparing the new server, i will have to issue a certificate, set up dns entries etc, but there can not be two “hermes” co-existing on the internet, right?
I would suggest that you set it up on a new name. Maybe “artemis”. 🙂 Copy over everything in /etc/letsencrypt and /var/lib/rspamd/dkim from hermes to artemis. And when you are ready for the switch just make hermes a CNAME record to artemis.
I can not believe that i did not think about this approach, all these years. I was staying awake, in order to finish everything up while customers were sleeping, got rather good at it in the meantime. I am saved. 🙂
Hello, I’m trying to follow this advice but I struggle with the certificates. If I create – to follow that example – an CNAME entry hermes, pointing to artemis, then the IP will be resolved for hermes to be the same as artemis. However, the certificates are created for artemis and will therefore be invalid for hermis. How did you solve this problem?
Hi Oliver. Just create the certificate for the name that the user requested. If the user intended to talk to “hermes” then create the certificate for “hermes”. Even if the DNS points to “artemis” in the background. DNS happens on a lower level. The client program just wants to see a certificate that matches what it asked for. Makes sense?
Hi Christoph, thank you for you quick response. Yes, It makes absolutely sens, but I’m not sure if I understand the full picture here. What do I enter as the MX Record? The MX must not refer to a CNAME, if I understand correctly.
Wouldn’t that mean I would have:
A – artemis.example.org – IP
CNAME – hermis.example.org – artemis.example.org
MX – artemis.example.org – IP
and wouldn’t certificates then have to be delivered for both, because other mailserver will try to connect to artemis (from the MX) and clients will stick to hermes as before.
The boring details aber DNS are in RFC 2181, section 10.3. 🙂 MX records must only point to A or AAAA records. No CNAMES are allowed.
But, yes, I see your point. And I’m not 100% sure what the best approach would be. I observed that many (other) mail servers do not care about the common name on your certificate. They seemed to be happy if they get any change to encrypt the SMTP connection.
I have a similar case here. And I use a wildcard certificate for *.workaround.org to be more flexible. My DNS records is managed at the german ISP Hetzner and I’m using their API to bring Certbot and them together to keep the wildcard certificate updated. Wildcard certificates at Let’s Encrypt require changing a TXT record in the DNS zone. So you will want to automate that. HTTP challenges (verifying your domain at /.well-known/…) are not supported.
A slightly simpler way might be to request a certificate for two FQDNs. You will have to prove that you own both of them. But that should work as well. Just use multiple domains when creating the certificate requests… like “-d artemis.example.org -d hermes.example.org” when using the Certbot.
Hehe, glad it helped. Some hosting providers offer to reserve IP addresses that can be assigned to your virtual servers. In that case you could even do the switchover on an IP level instead DNS.
Anway it’s good that you asked the question. I have added that part to the this page.
Hi Christoph, thanks again for your quick reply. One certificate for two FQDNs works fine. Interesting that some mail servers don’t care about the common name.
Anyway, my setup works with the solution you suggested.
Thanks for your and help!
Hi Christoph, I have this error :
# doveadm force-resync -A ‘*’
Error: auth-master: userdb list: User listing returned failure
doveadm: Error: Failed to iterate through some users
An idea ?
Hello,
… I know that a new server with a fresh installation of Debian 12 is highly recommended …
But has someone tried a dist-upgrade from Debian 11 to 12? What were the problems with Workaround ISPmail and how could they be solved?
> someone tried a dist-upgrade from Debian 11 to 12?
I did; And followed an advise I found here in the comments (sorry, I don’t remember where exactly) and then all was good since months …
But is it worth doing this so late, since it we will have next Debian 13 in the beginning of 2025, aren’t we? Years are flying… 😉 (just kidding!)
Trick for faster switching and lower downtime:
* First rsync all the email data to the new system while old one remains in operation. Do this right before switch.
* Now when you execute the switch, do a final rsync, which should be fast: you already compied 99% of the data.
* this will minimize downtime.
Isn’t that the recommended procedure?