This chapter of our journey leads us to Dovecot – the software that…
- gets emails from Postfix and saves them to disk
- executes user-based “sieve” filter rules (can be used to e.g. move emails to different folders based on certain criteria or send automated vacation responses)
- allows the user to fetch emails using POP3 or IMAP
Before we get to the actual configuration for security reasons I recommend that you create a new system user that will own all virtual mailboxes. The following shell commands will create a system group “vmail” with GID (group ID) 5000 and a system user “vmail” with UID (user ID) 5000. (Make sure that UID and GID are not yet used or choose another – the number can be anything between 1000 and 65000 that is not yet used):
groupadd -g 5000 vmail useradd -g vmail -u 5000 vmail -d /var/vmail -m
If the /var/vmail directory was already there because you assigned it a dedicated mount point then you should make sure that the permissions are set correctly:
chown -R vmail.vmail /var/vmail
The configuration files for Dovecot are found in /etc/dovecot/conf.d/. All these files are loaded by Dovecot. This is done by this magical line in the dovecot.conf file:
!include conf.d/*.conf
It loads all files in /etc/dovecot/conf.d/ that end on “.conf” in sorted order. So “10-auth.conf” is loaded first and “90-quota.conf” is loaded last. The big advantage is that you can edit or replace parts of the configuration without having to overwrite the entire configuration. The main /etc/dovecot/dovecot.conf file does not require any changes. Those other files in conf.d/ however need to be edited…
conf.d/
10-auth.conf
If your users are still using Outlook Express (Windows XP) or Microsoft Mail (Windows Vista) then you need to add the “LOGIN” authentication mechanism in addition to the standard “PLAIN” mechanism:
auth_mechanisms = plain login
These are plaintext (unencrypted) ways to transmit a mail user’s password. But don’t worry. By default Dovecot sets “disable_plaintext_auth = yes” which ensures that authentication is only accepted over TLS-encrypted connections.
At the end of this file you will find various authentication backends that Dovecot uses. By default it will use system users (that are listed in /etc/passwd). But we want to use the MySQL database backend so go ahead and change this block to:
#!include auth-system.conf.ext !include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext #!include auth-static.conf.ext
Now edit the SQL configuration file:
auth-sql.conf.ext
Change the “userdb” section to:
userdb {
driver = static
args = uid=vmail gid=vmail home=/var/vmail/%d/%n
}
The placeholder %d stands for the domain part and %n for the user part if you have an email address like “user@domain”. Dovecot’s variables are documented on their website.
10-mail.conf
Change the mail_location setting to:
mail_location = maildir:/var/vmail/%d/%n/Maildir
This is the directory where Dovecot will look for the emails of a specific user. So for john@example.org this leads to “/var/vmail/example.org/john/Maildir”.
Further down you will find sections defining the namespaces. These namespaces are folder structures that your email program sees when connecting to the mail server. If you use POP3 you can only access the “inbox” – which is where all incoming email is stores. Using the IMAP protocol you get access to a hierarchy of folders and subfolders. And you can even use folders shared between users. Or public folder that could be accessed by anyone – even anonymously.
Look for the “namespace inbox” section. If you already have emails stored on your server from previous versions of this ISPmail guide you need to change:
separator = .
here. By default the seperator is “/” which creates a directory structure like “/var/vmail/example.org/john/Maildir/INBOX/staff/marketing/simon”. This is perfectly fine. But previous ISPmail guides used “.” as a seperator so the mentioned folder would rather have been “/var/vmail/example.org/john/Maildir/.INBOX.staff.marketing.simon”.
10-master.conf
This configuration file deals with services that allow communication with other processes. For example it enables or disables POP3 or IMAP. Don’t worry about the standard unencrypted TCP ports 110 (for POP3) and 143 (for IMAP). They can be kept accessible. If a user connects to these ports they will have to issue a STARTTLS command to switch into encrypted mode before they are allowed to send their password. There is basically no difference between using an plaintext port like 110 for POP3 and then using STARTTLS – or connecting to the encrypted 995 port for POP3S (=secure). See the Dovecot documentation for another explanation.
So most settings are sane here and do not have to be changed. However one change is required in the “service auth” section because we want Postfix to allow Dovecot as an authentication service. This is what has to be entered:
# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0660
user = postfix
group = postfix
}
Why that strange path? Well, Postfix runs in a chroot environment located at /var/spool/postfix. It can’t access anything outside of that directory. So to allow communication with Postfix we tell Dovecot to use the communication socket file in that chroot area.
10-ssl.conf
Earlier in this guide you created both a key and a certificate file to encrypt the communication with POP3, IMAPs and HTTPS between the users and your mail server. You need to tell Dovecot where to find these files:
ssl_cert = </etc/letsencrypt/live/webmail.example.org/fullchain.pem ssl_key = </etc/letsencrypt/live/webmail.example.org/privkey.pem
And enable TLS/SSL encryption by setting:
ssl = yes
You could also set “ssl=required” but as Dovecot disallows sending plaintext passwords over unencrypted connections anyway we don’t actually need it. See the Dovecot documentation on SSL encryption for more information.
15-mailboxes.conf
You should also add these lines within your “namespace inbox” section:
mailbox INBOX.Junk {
auto = subscribe
special_use = \Junk
}
mailbox INBOX.Trash {
auto = subscribe
special_use = \Trash
}
It ensures that a “Junk” and a “Trash” folder are created within the inbox when a user logs in. The user is also automatically subscribed to these folders. (When using IMAP you can choose which folder you want to see by subscribing them.) The “Junk” folder is required so that we can move spam emails to it – we will configure that later. And the “Trash” folder is required so that users using the Roundcube web mailer can actually delete emails.
The names used in “special_use” refer to the special-use mailboxes as described in the RFC 6154.
/etc/dovecot/dovecot-sql.conf.ext
This file is referred to by /etc/dovecot/conf.d/auth-sql.conf.ext. It tells Dovecot how to access the MySQL database and where to find the information about email users/accounts. You will find it well documented although all configuration directives are commented out. Add these lines at the bottom of the file:
driver = mysql connect = host=127.0.0.1 dbname=mailserver user=mailuser password=ChangeMe <- use your database access password instead default_pass_scheme = SHA256-CRYPT password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';
What these lines mean:
- driver: the kind of database
- connect: where to find the MySQL database and how to use it (username, password)
- default_pass_scheme: the format in which the passwords are stored (we use strong salted SHA256 hashes)
- password_query: an SQL statement that returns the user (=the email address) and the password (SHA256 hash) from the database where “%u” is the login user name (=we use the email address as the user name of an email account). Whenever Dovecot needs to check an email user’s password if will run the above query and verify the password against the hash in the database.
Finishing move
You should also make sure that only root can access the SQL configuration file so nobody else is reading your database access passwords:
chown root:root /etc/dovecot/dovecot-sql.conf.ext chmod go= /etc/dovecot/dovecot-sql.conf.ext
Restart Dovecot from the shell:
service dovecot restart
Look at your /var/log/mail.log logfile. You should see:
... dovecot: master: Dovecot v2.2.27 (c0f36b0) starting up for imap, lmtp, sieve, pop3 (core dumps disabled)
If you get any error messages please double-check your configuration files.
Hi Christoph,
Concerning 10-ssl.conf it seems that your text has not be updated since the jessie version. Could you explain how to set up 10-ssl.conf with the let’sEncrypt certificate? Thanks.
François
Thanks for the hint, François. I wonder how I missed to change that.
Hi Christoph
The section about the separator “separator = .” implies that only legacy installations should use the dot as separator but later it is used in the config too – maybe a bit of clarification would help.
Hmmm. What I wrote:
“By default the seperator is “/” which creates a directory structure like “/var/vmail/example.org/john/Maildir/INBOX/staff/marketing/simon”. This is perfectly fine. But previous ISPmail guides used “.” as a seperator”
Where do you mean “later”? I currently don’t see where I mention that.
Hi Christoph,
are you sure about the directory structure?
According to https://wiki.dovecot.org/Namespaces#Hierarchy_separators it only sets the mailbox structure, which must not be the same like the directory structure?
I tested it and only with LAYOUT=fs the folders are really created without the .foo.bar structure.
Regards,
Andy
sorry for not being more clear on this – I meant later in the guide not on this page – specifically in the rspamd config you include INBOX.Junk which if I am not wrong would only exist in the legacy notation using the dot.
Is it expected that at the time you modify 10-master.conf, the file /var/spool/postfix/private/auth is not present?
In fact the configuration change in 10-master.conf will make Dovecot *create* that socket there so that Postfix can access it. Before the change the socket shouldn’t be there. Is that what you mean?
Right, I noticed that later with the lmtp socket. Thanks 🙂
Also a question on namespaces inbox; by default there’s already “Junk” and “Trash” mailboxes – I guess those can be left in?
Could you try with the default settings? I believe that because I’m on a legacy test installation here I needed INBOX.Junk. But with a fresh installation and “separator=/” it might be actually correct to use “Junk”.
Ah, the difference seems to be that “Junk” just puts the folder outside “INBOX”. For personal preference, I just added “auto = subscribe” to the defaults. (which worked in roundcube and email clients)
In my auth-sql.conf.ext file was the passdb commented.
After adding
passdb {
driver = sql
# Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
args = /etc/dovecot/dovecot-sql.conf.ext
}
I was able to connect to the IMAP account.
Hi,
In 15-mailboxes.conf there’s already :
# NOTE: Assumes “namespace inbox” has been defined in 10-mail.conf.
namespace inbox {
# These mailboxes are widely used and could perhaps be created automatically:
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Trash {
special_use = \Trash
}
Ok, I just modified the subscription. No pb
Hi Christoph,
While I wait for the section “enforcing mail quotas” I tried the following configuration and it worked, sure there are some improvements to make. Thanks for the guide!!!!
mysql> ALTER TABLE `virtual_users` ADD `quota_kb` INT NOT NULL
Enabling quota plugins
conf.d/10-mail.conf:
mail_plugins = $mail_plugins quota
conf.d/20-imap.conf:
protocol imap {
mail_plugins = $mail_plugins imap_quota
/etc/dovecot/conf.d/auth-sql.conf.ext
uncomment sql:
userdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
comment static:
#userdb {
# driver = static
# args = uid=vmail gid=vmail home=/var/vmail/%d/%n
#}
Add user_query on: /etc/dovecot/dovecot-sql.conf.ext
user_query = SELECT CONCAT(‘/var/vmail/’,CONCAT(SUBSTRING_INDEX(email,’@’,-1),’/’,SUBSTRING_INDEX(email,’@’,1))) AS home, 5000 AS uid, 5000 AS gid, CONCAT(‘*:storage=’,quota_kb) AS quota_rule FROM virtual_users WHERE email=’%u’;
https://workaround.org/wp-content/uploads/2008/12/ISPmail-tutorial-for-Debian-Lenny.pdf
https://wiki2.dovecot.org/Quota
https://wiki2.dovecot.org/Quota/Configuration
Beware of the backquotes here, they should be replaced with single quotes (apostrophes) otherwise the query will fail.
So your user DB setting works for delivering mail, but doveadm cannot enumerate the users on the system which is needed to get mailbox replication working https://wiki.dovecot.org/Replication
I recognise that’s a feature that may be a little more advanced than you target audience needs, but I found it useful.
I set the userdb section to read
userdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
and in the dovecot-sql.conf.ext I set the user query and iterate query as
user_query = SELECT CONCAT(‘/var/vmail/’,(SELECT name FROM virtual_domains WHERE id=domain_id),’/’,REPLACE(‘%u’,CONCAT(‘@’,(SELECT name FROM virtual_domains WHERE id=domain_id)),”)) AS home, 5000 AS uid, 5000 AS gid FROM virtual_users WHERE email =’%u’;
iterate_query = SELECT email AS username FROM virtual_users;
This allows doveadm to find all users set on the system using the command `doveadm users ‘*’`
The dovecot page linked explains how to set up replication once that’s working if you’re interested.
I would highly recommend doing the following in the 10-mail.conf instead:
mail_home = /var/vmail/%d/%n
mail_location = maildir:~/Maildir
If you do not define a mail_home, you will end up with conflicts with the sieve folder and the .dovecot.sieve files created for the filters in Roundcube.
Especially if the separator is a dot (.)
Thanks a lot, Andi! This tip save my day!
The default dovecot SSL configuration – even in the latest version – is not secure. I strongly recommend adding these three lines to the guide (in 10-ssl.conf):
ssl_protocols = !SSLv3 !TLSv1 !TLSv1.1
ssl_prefer_server_ciphers = yes
ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!SEED
This disabled SSLv3, TLSv1 and TLSv1.1 (i.e. it only allows TLS 1.2).
It also disables lots of old insecure ciphers.
It is unclear if the separator in 10-mail.conf should be set = . or / not specified.
Is it necessary to set Junk and Trash folder within the INBOX?
INBOX.Junk
INBOX.Trash
Can I set it like this?
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
I prefer to have them outside of INBOX too, so I did as you did and it works fine.
Just remember to adjust for this in the rest of the guide to avoid conflicts.
Thanks for great tutorial!
After reading this I feel one part is missing and I’d really appreciate if you could update or at least explain it. I’m trying to add custom sieve rules for my virtual users that can be managed via DB. I see some bit and pieces in pigeonhole documentation: https://wiki.dovecot.org/Pigeonhole/Sieve/Configuration/Dict but unfortunately it is not clear how it can be achieved.
Any help will be appreciated.
For anyone interested in setting up a Master user (user who can log into other mailboxes), it can be done with just a few extra changes to the configuration. This is based on documentation from Dovecot (https://wiki.dovecot.org/Authentication/MasterUsers). I chose to use an existing user, instead of creating a different user or the master password.
First, update the “virtual_users” table with a column of “master_user” tinyint(1) and a default of “0”.
Set the value of this column to 1 for any master users you want, and everyone else should be set to 0.
Copy the existing ‘/etc/dovecot-sql.conf.ext’ file to ‘/etc/dovecot-sql-master.conf.ext’, and add “and master_user = true” to the end of the password_query line.
It should look like this:
“password_query = SELECT email as user, password FROM virtual_users WHERE email=’%u’ and master_user = true;”
In the /etc/dovecot/conf.d/auth-master.conf.ext file, comment/delete everything out and add the following (which is brought over and updated from the existing “auth-sql.conf.ext” file). The “master = yes” is the only difference in the passdb section, and the rest of the file’s contents aren’t needed:
auth_master_user_separator = *
passdb {
driver = sql
master = yes
args = /etc/dovecot/dovecot-sql-master.conf.ext
}
Finally, in 10-auth.conf, uncomment this line and restart dovecot:
!include auth-master.conf.ext
To user your new master user(s), in roundcube/mail-client, specify the username as ‘user_to_login_as@domain.com*your_master_user@domain.com’ and the password of the master user in the password field.
Hi Christoph,
In Roundcube i have the Spam and Trash map indented. On my old Courier mail system (iredmail) i don’t have that, all the maps are the same not indented.
I configured ISPmail on a new clean Debian stretch VM. Since you wrote: only if you are migrating from a previous ISPmail you should set separator = . in 10-mail.conf.
Since this is a clean install i left it commented out.
I tested it by setting it to separator = . restarted dovecot but no difference the Spam and Trash map are still indented.
Is this normal are am i doing something wrong?
In debug mode i also see this:
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Debug: Loading modules from directory: /usr/lib/dovecot/modules
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Debug: Effective uid=5000, gid=5000, home=/var/vmail/example.org/john@example.org
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Debug: Home dir not found: /var/vmail/example.org/john@example.org
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/vmail/example.org/john/Maildir
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Debug: maildir++: root=/var/vmail/example.org/john/Maildir, index=, indexpvt=, control=, inbox=/var/vmail/example.org/john/Maildir, alt=
Jun 3 00:00:05 server2 dovecot: imap(john@example.org): Logged out in=50 out=511
Hi Christoph,
I’ve defined two virtual aliases (‘postmaster@mydomain.com’ and ‘admin@mydomain.com’) for a single virtual user ‘webmaster@mydomain.com’ and created a password for it too. Now, when I try to login via Roundcube, the password is accepted only for the virtual user and denied for both virtual aliases. The password for the two virtual aliases should be inherited from the associated virtual user right?
I think I’m wrong… Roundcube login will only work for the virtual user because its password_query command does not involve the virtual_aliases table at all.
I’m trying to build a mail server for several domains. In /etc/dovecot/conf.d/10-ssl.conf you suggest
ssl_cert = </etc/letsencrypt/live/webmail.example.org/fullchain.pem
ssl_key = </etc/letsencrypt/live/webmail.example.org/privkey.pem
But I've now registered LetsEncrypt certs for three groups ( I got the method wrong at first). Is there a workaround so the correct cert can be aligned with the correct domains?
I'm assuming I can't simply put the three pairs of keys into a list.
I found the answer by posting the question on the LetsEncrypt forum: simply draw up a new csr including all of the domains and subdomains, for which a single cert will be issued. Repeat the exercise when adding new domains or subdomains.
I had errors in the mail log file after running “service dovecot restart” and couldn’t work out what they meant. Finally, almost in desperation, I ran “service postfix restart”, then re-ran “service dovecot restart” again. Problem solved.
After updating to Debian 10 Buster, I could no longer retrieve my mail with the help of an email client such as Thunderbird. Roundcube, on the other hand, worked as it should.
A look in /var/log/mail.log gave “imap-login: Error: Failed to initialize SSL server context: Can’t load DH parameters: error: 1408518A: SSL routines: ssl3_ctx_ctrl: dh key too small: user = ”
The solution was to add ssl_dh = </usr/share/dovecot/dh.pem at the end of /etc/dovecot/conf.d/10-ssl.conf and then restart dovecot.
After a new installation from scratch which specifies “separator = /” under “namespace inbox” within /etc/dovecot/conf.d/10-mail.conf new folders created under Inbox within Roundcube is still created as .INBOX.test rather than /INBOX/test within the files system.
Is there something more to only specifying the separator within 10-mail.conf ?