Setting up Dovecot

This chapter of our journey leads us to Dovecot – the software that…

  • gets emails from Postfix and saves them to disk
  • executes user-based sieve filter rules (can be used to e.g. move emails to different folders based on certain criteria or to send automated vacation responses)
  • allows the user to fetch emails using POP3 or IMAP

Before we get to the actual configuration for security reasons I recommend that you create a new system user that will own all virtual mailboxes. The following shell commands will create a system group “vmail” with GID (group ID) 5000 and a system user “vmail” with UID (user ID) 5000. (Make sure that UID and GID are not yet used or choose another – the number can be anything between 1000 and 65000 that is not yet used):

groupadd -g 5000 vmail
useradd -g vmail -u 5000 vmail -d /var/vmail -m

If the /var/vmail directory was already there because you assigned it a dedicated mount point then you should make sure that the permissions are set correctly:

chown -R vmail:vmail /var/vmail

The configuration files for Dovecot are found in /etc/dovecot/conf.d/. All these files are loaded by Dovecot. This is done by this magical line in the dovecot.conf file:

!include conf.d/*.conf

It loads all files in /etc/dovecot/conf.d/ that end on “.conf” in alphanumerical order. So “10-auth.conf” is loaded first and “90-sieve-extprograms.conf” is loaded last. The big advantage is that you can edit or replace parts of the configuration without having to overwrite the entire configuration. The main /etc/dovecot/dovecot.conf file does not require any changes. Those other files in conf.d/ however need a few changes…

conf.d/

10-auth.conf

The most common authentication mechanism is called PLAIN. However if you have Outl**k users then you need to add the LOGIN mechanism, too.:

auth_mechanisms = plain login

These are plaintext (unencrypted) ways to transmit a mail user’s password. But don’t worry. By default Dovecot sets “disable_plaintext_auth = yes” which ensures that authentication is only accepted over TLS-encrypted connections.

At the end of this file you will find various authentication backends that Dovecot ships with. By default it will use system users (that are listed in /etc/passwd). But we want to use the MariaDB database backend so go ahead and change this block to:

#!include auth-system.conf.ext
!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
#!include auth-static.conf.ext

Now edit the SQL configuration file:

auth-sql.conf.ext

Now Dovecot reads the auth-sql.conf.ext which defines how to find user information in your database. Open the file. You will find two sections:

  • userdb: where to find a user’s mailbox in the file system
  • passdb: which password hash the user has

By default Dovecot will run two queries at your database. One for the userdb that gets information like the user ID, group ID, home directory and quota. And another for the passdb that gets the hashed password.

No more static

Previous versions of the ISPmail guide used the static method for userdb. That conflicts with using the quota plugin though. So we need to use an SQL query for the userdb again.

So please change the “userdb” section to:

userdb {
   driver = sql
   args = /etc/dovecot/dovecot-sql.conf.ext
}

Leave the passdb section as it is. We will deal with the configuration file it refers to (/etc/dovecot/dovecot-sql.conf.ext) later.

10-mail.conf

Change the mail_location setting to:

mail_location = maildir:~/Maildir

This is the directory where Dovecot will look for the emails of a specific user. The tilde character (~) means the user’s home directory as defined in the previous section.

Further down you will find sections defining the namespaces. Those are folder structures that your email program sees when connecting to the mail server. If you use POP3 you can only access the “inbox” – which is where all incoming email is stored. Using the IMAP protocol you get access to a hierarchy of folders and subfolders. And you can even share folders between users. Or use a public folder that can be accessed by anyone – even anonymously.

Look for the “namespace inbox” section. If you already have emails stored on your server from previous versions of this ISPmail guide you need to change:

separator = .

here. By default the separator is “/” which creates a directory structure like “/var/vmail/example.org/john/Maildir/INBOX/staff/marketing/simon”. This is perfectly fine. But previous ISPmail guides used “.” as a separator so the mentioned folder would rather have been “/var/vmail/example.org/john/Maildir/.INBOX.staff.marketing.simon”.

Also edit the “mail_plugins” line to enable the quota plugin we will configure later and turn it into:

mail_plugins = quota

10-master.conf

This configuration file deals with services that allow communication with other processes. For example it enables or disables POP3 or IMAP. Don’t worry about the standard unencrypted TCP ports 110 (for POP3) and 143 (for IMAP). They can be kept accessible. If a user connects to these ports they will have to issue a STARTTLS command to switch into encrypted mode before they are allowed to send their password. There is basically no difference between using an plaintext port like 110 for POP3 and then using STARTTLS – or connecting to the encrypted 995 port for POP3S (=secure). See the Dovecot documentation for another explanation.

So most settings are sane here and do not have to be changed. However one change is required in the “service auth” section because we want Postfix to allow Dovecot as an authentication service. This is what has to be entered:

# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
  mode = 0660
  user = postfix
  group = postfix
}

Why that strange path? Well, Postfix runs in a chroot environment located at /var/spool/postfix. It can’t access anything outside of that directory. So to allow communication with Postfix we tell Dovecot to place a communication socket into that chroot.

10-ssl.conf

Earlier in this guide you created both a key and a certificate file to encrypt the communication with POP3, IMAPs and HTTPS between the users and your mail server. You need to tell Dovecot where to find these files:

ssl_cert = </etc/letsencrypt/live/webmail.example.org/fullchain.pem
ssl_key = </etc/letsencrypt/live/webmail.example.org/privkey.pem

And enforce TLS encryption by setting:

ssl = required

See the Dovecot documentation on SSL encryption for more information.

/etc/dovecot/dovecot-sql.conf.ext

This file is referred to by /etc/dovecot/conf.d/auth-sql.conf.ext. Do you remember from earlier? It tries to get passdb information (the user’s password hash) from there. You will find it well documented although all configuration directives are commented out. Add these lines at the bottom of the file:

driver = mysql
connect = host=127.0.0.1 \
  dbname=mailserver \
  user=mailserver \
  password=x893dNj4stkHy1MKQq0USWBaX4ZZdq
user_query = SELECT email as user, \
  concat('*:bytes=', quota) AS quota_rule, \
  '/var/vmail/%d/%n' AS home, \
  5000 AS uid, 5000 AS gid \
  FROM virtual_users WHERE email='%u'
password_query = SELECT password FROM virtual_users WHERE email='%u'
iterate_query = SELECT email AS user FROM virtual_users

What these lines mean:

  • driver: the kind of database. MariaDB is the same kind as MySQL.
  • connect: where to find the MySQL database and how to use it (username, password)
  • user_query: an SQL query that returns the user name (=the email address), the quota, the home directory, user ID and group ID.
  • password_query: this SQL query just gets the password hash from the database
  • iterate_query: ‘doveadm’ uses this query to get a list of all users. Without this query “doveadm user ‘*'” would not work.

So the user_query gets several pieces of information from the database. Let’s look at it one by one:

  • email AS user
    It gets the the email field from the database which corresponds to the user name. Dovecot expects it in the user field so we set an alias to “user”.
  • userdb_quota_rule
    This the user’s quota in bytes. Think of it as the maximum possible space on disk that the user must occupy. As documented Dovecot expects the quota in a special format like “*:bytes=10000” if the user should not be able to store more than 10,000 bytes.
  • userdb_home
    This leads to the directory where all emails and various control file of this user are located. The placeholder ‘%d’ replaces the domain and ‘%n’ the user part. So for John that makes it “/var/vmail/example.org/john”.
  • userdb_uid and userdb_gid
    Those are the user ID and group ID of vmail user – 5000 for both. Dovecot uses it to set the permissions of files it creates. As all users share the same system user “vmail” this is just a static number.

Fix permissions

Make sure that only root can access the SQL configuration file so nobody else is reading your database access passwords:

chown root:root /etc/dovecot/dovecot-sql.conf.ext
chmod go= /etc/dovecot/dovecot-sql.conf.ext

Restart Dovecot from the shell:

systemctl restart dovecot

Look at your /var/log/mail.log logfile. You should see:

... Dovecot v2.3.4.1 (f79e8e7e4) starting up for imap, lmtp, sieve, pop3 (core dumps disabled)

If you get any error messages please double-check your configuration files.

65 thoughts on “Setting up Dovecot”

  1. Hans Busch

    Hallo Christoph, please look at: /etc/dovecot/dovecot-sql.conf.ext you make a double connection to the database. Think one is a copy paste from the stretch guide 😉

  2. Hello Christoph,
    For auth-sql.conf.ext, in fact, I left the entire file free of changes. It was in the state you mentionned.

      1. I can confirm that on a fresh debian buster installation no edits where necessary to auth-sql.conf.ext as the file was in that exact state.

  3. Take a look at the /var/spool/postfix/private directory. If all went as intended you will find a socket file called quota-status there. Otherwise please check the /var/log/mail.log file for errors.

    Well, i followed your guide with attention but i don’t see quota-status.

    root@xxx:/var/spool/postfix/private# cat /var/log/mail.log
    Dec 29 22:27:37 xxx postfix/postfix-script[3369]: starting the Postfix mail system
    Dec 29 22:27:37 xxx postfix/master[3371]: daemon started — version 3.4.7, configuration /etc/postfix
    Dec 29 22:31:53 xxx dovecot: master: Dovecot v2.3.4.1 (f79e8e7e4) starting up without any protocols (core dumps disabled)
    Dec 29 22:31:58 xxx dovecot: master: Warning: Killed with signal 15 (by pid=15924 uid=0 code=kill)
    Dec 29 22:31:59 xxx dovecot: master: Dovecot v2.3.4.1 (f79e8e7e4) starting up for imap, lmtp, sieve, pop3 (core dumps disabled)
    Dec 30 00:33:46 xxx dovecot: master: Warning: Killed with signal 15 (by pid=18307 uid=0 code=kill)

    1. Thomas Gauweiler

      I had the same problem as Gompali. After adding the “service quota-status …” section in 90-quota.conf the dovecot service did no more start.
      A look at “service dovecot status” shows:
      doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/90-quota.conf line 95: Expecting ‘{‘

      I could get it running when I removed the newline between these two lines:
      unix_listener /var/spool/postfix/private/quota-status
      {

      Regards, Thomas

  4. Pascal Suter

    i am following your guide (more or less) on ubuntu 18.04 and experienced a problem with the “service quota-status” section as written above. dovecot expects the opening squirly bracket after the “unix_listener” line seems to be expected to be on the same line as the unix_listener directive.. i get this error when i restart dovecot in my syslog:

    doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/90-quota.conf line 95: Expecting ‘{‘

    i moved the { to the end of the the unix_listener line and it now seems to work.

    1. Florian Effenberger

      Had the same problem as Pascal, his bugfix worked for me. 🙂

      Another question: Is it intentional to edit the existing /etc/dovecot/conf.d files instead of creating an override file with all custom changes inside? The latter would be helpful in case the defaults change with a package update, and gives a more comprehensive overview of the own settings. However, not sure if there are drawbacks to that approach.

      1. Christoph Haas

        I considered that, too. In fact I’m a huge fan of not changing the upstream files. In this case I found the documentation in the source files very helpful and wanted to avoid a mess in an additional file. But your solution sounds sane as well. 🙂

      2. Florian Effenberger

        Thanks! 🙂 I will try to setup with override files – if it works, happy to share the documentation with you!

  5. For quota it says Bytes but if I want to set “2147483648” Bytes = “2048” Megabytes this does not work… what do I do wrong?

    1. Christoph Haas

      Usually space on disk is measured in MB while RAM is measured in MiB. So if you want to allow 2 GB of quota that should be 2000000000.

  6. Pascal Suter

    I’m having some permission issues with the quota-warning function. when a warning should be sent out, i get this error message:

    Dec 30 23:04:24 mail dovecot: lmtp(mymail@mydomain.ch): Error: quota: net_connect_unix(/var/run/dovecot/quota-warning) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +r perm: /var/run/dovecot/quota-warning, we’re not in group 116(dovecot), dir owned by 0:0 mode=0755)

    again, i’m trying this on ubuntu 18.04, so maybe this is only an issue in my setup. i’ve also tried to add the vmail user to the dovecot group but even though i restarted both dovecot and postfix i still get the same errors.

    adding global read and write permissions to /var/run/dovecot/quota-warning solved the issue, however i don’t feel good about this, not knowing the security implications this may have.

  7. Pascal Suter

    ..and a small change request: after the title “Test It” you suggest one should test quotas now using swaks. this will fail at this state because we haven’t configured the lmtp connection between postfix and dovecot yet (see next section).. so if someone follows the advise and tests this right here, he will see errors from postfix saing:

    “fatal: bad string length 0 < 1: virtual_mailbox_base ="

    this is quite misleading (at least i fell for it ;)) before finally realizing, that one piece of the puzzle is still missing 🙂
    so i suggest you mention that one can test this LATER ON using swaks.

    1. Christoph Haas

      Oops… sounds like I shouldn’t move sections around without testing everything. 🙂

    1. Christoph Haas

      Sorry for the confusion. I moved the quotas page to a seperate and later position in the guide because there were dependencies as pointed out by Pascal above.

  8. Why do you still use the Maildir format?

    I see a lot better performance with mdbox. What do you think about the different formats like Maildir and sdbox and mdbox?

    best regards
    wittiko

    1. Christoph Haas

      I considered that, too. The reason to stay with Maildir was mainly due to laziness (change as little as possible compared to previous versions) and to lack of time to actually try it out. But I see how some users have tens of thousands of emails in a single folder (mainly their inbox… I wonder how those people survive) and reading those folder is very slow.

      It seems that according to https://wiki2.dovecot.org/MailboxFormat/dbox#Alternate_storage you can keep the old format and receive new emails in the mdbox format. Although I would prefer to have a clear migration path from Maildir to mdbox. Have you come across anything like that?

      1. Alt storage is for mixing capacity and performance disks for the mailstore.
        Altmove moves the old mails to the altstorage. This is not we need when converting maildir to (m)dbox.

        I did the migration some years ago. I had no problems and use hourly snpshots for backup with replication to another datacenter.

        I would do following:

        One Server:
        1) Create a snapshot of the existing data (just to be sure)
        2) Doing initial dsync
        3) Stop delivering Mails and kicking all users
        4) Doing another Dsync
        5) Update database fields or edit dovecot file
        6) Reactivate mail dilivery

        If you have a lot of users you can do this on per user – but then you need to query the location from the database.

        PS: I would use views for dovecot and postfix and add a boolean field enabled. So you can also disable domains/mailboxes/aliases without deleting them.
        Thanks for your great tutorial.

      2. In 10-mail.conf, if you use the “”/”” seperator, with :
        mail_location = maildir:~/Maildir
        You will have :
        Maildir/.folder/
        Maildir/.folder.subfolder/
        With mail_location = maildir:~/Maildir:LAYOUT=fs
        You will have :
        Maildir/folder/
        Maildir/folder/subfolder/
        Isn’t it better ?
        https://wiki2.dovecot.org/MailLocation/Maildir

        1. I think there should be no difference.

          The performance should be more or less the same. Maybe some Mailboxes with a lot of subfolders could be faster with the 2nd method.

          However Maildir is slow per design so i don’t think this is doing a big difference.

  9. On my setup I prohibit TLSv1.1 with (10-ssl.conf):
    ssl_min_protocol = TLSv1.2

    Also I am using the imp_quota plugin (20-imap.conf):
    protocol imap { mail_plugins = $mail_plugins imap_quota }

  10. Matthias Kloas

    I propose to add an iterate query to the file /etc/dovecot/dovecot-sql.conf.ext :

    # Query to get a list of all usernames
    iterate_query = SELECT email AS user FROM virtual_users

    This is useful for doveadm. Without it the command
    doveadm user ‘*’
    returns an error instead of a list of all users.

  11. Florian Effenberger

    I wonder if it makes sense to limit the vmail user a bit more, by

    1. making it a system user
    2. giving nologin as shell
    3. explicitly have a disabled login

    For additional security, /var/vmail could be made chmod 700, I think?

    For the service auth section, does it make sense to set mode = 0600 for additional security too?

    1. Christoph Haas

      A system user sounds reasonable. You mean with a UID below 1000, right? I’m taking a note at least for the next version of the guide.

      Nologin and disabled login sounds good. Will add that.

      I have 0700 on /var/vmail on my server. So that seems to be fine. I can’t think of a service that needs to access the directory using group permissions.

      Regarding the settings of the auth socket… I think there was a reason that I didn’t set it to 0600. But I can’t quite remember what it was. Currently my test server is down and I don’t want to try it on a production system. Have you tried it by any chance?

      1. Something like:
        useradd –home-dir /var/vmail –create-home –system –shell /usr/sbin/nologin –user-group vmail

  12. At the end of “/etc/dovecot/dovecot-sql.conf.ext” you talk about “userdb_uid and user_gid”. I think those should be “uid and gid”. That’s what you use in the SQL.

  13. Is there a specific reason to use “host=127.0.0.1” instead of “host=localhost” in “/etc/dovecot/dovecot-sql.conf.ext”? In couldn’t think of one and the documentation tells nothing about that.

  14. Christoph Haas

    Yes, same reason as with Postfix. 🙂 See the “prepare the database” page for a detailed explanation.

  15. I have a question. According to the dovecot documentation, we could use the prefetch driver instead of SQL; this way a single lookup can return information necessary for both userdb and passdb. Is there any specific reason why you chose the SQL driver instead of prefetch? Does it perhaps have any side effect?

    1. Christoph Haas

      Thanks. There is a buggy wiring in my brain regarding that word because my english teacher has already been correcting that 35 years ago. 🙂

  16. Charles Atkinson

    In the 10-ssl.conf section, the “Earlier in this guide” link is to the stretch page

  17. Hello – first off, these are great guides and I wanted to THANK YOU for publishing these for all of us.

    Next, it appears that the “dovecot-sql.conf.ext” is changed quite a bit from the previous guides and that the mail location and quota have been added to this file. My questions are:

    1) Do I need to use quota if not desired?

    2) Is there any reason I cannot just use the previous versions of the “dovecot-sql.conf.ext” file and define my mail location as in previous guides?

    3) If the mail location is fetched from DB via the “dovecot-sql.conf.ext” file, then what effect does the “mail_location = maildir:~/Maildir” statement have?

    Thanks in advance for your time on these questions.

    1. Christoph Haas

      Hey Tony.
      1) No.
      2) To be honest I don’t remember the exact settings of those many guides. But you should be fine if you stay with the way the previous guide did it.
      3) You get the “home” from the database. That is the “~” part of the mail_location.

  18. dovecot-sql.conf.ext says that password_query needs to return password and (user or (username and domain)), yet you return just the password. Why? It even gives example queries:

    # Example:
    # password_query = SELECT userid AS user, pw AS password \
    # FROM users WHERE userid = ‘%u’ AND active = ‘Y’
    #
    #password_query = \
    # SELECT username, domain, password \
    # FROM users WHERE username = ‘%n’ AND domain = ‘%d’

      1. You have quoted the wrong section. It says that for the userdb section, yet I’m talking about the passdb section.

        1. Christoph Haas

          You are right. Still the username and domain do not need to be fetched from the database because they are derived from the email address.

  19. It’s better not to hardcode 5000 as uid/gid in the SQL query but set the default uid/gid in 10-mail.conf using the user and group names:

    mail_uid = vmail
    mail_gid = vmail

    and not return any uid/gid overrides in the SQL.

    1. Correction.
      Turns out there are rare cases (bugs?) when dovecot needs uid/gid but doesn’t fallback to using mail_uid/mail_gid. So do hardcode uid/gid in the SQL query, however not as a number but as a name, e.g.:

      ‘vmail’ as uid,
      ‘vmail’ as gid

      Also set:

      mail_uid = vmail
      mail_gid = vmail

      in 10-mail.conf too.

  20. Quota rule support b/k/M/G/T/% suffixes,It’s better not to using “b”as default,try this:

    user_query = SELECT email as user, \
    concat(‘*:bytes=’, quota, ‘M’) AS quota_rule, \
    ‘/var/vmail/%d/%n’ AS home, \
    5000 AS uid, 5000 AS gid \
    FROM virtual_users WHERE email=’%u’

  21. Hi Christoph,
    Is there a reason why you haven’t included editing 15-mailboxes.conf to include the auto = subscribe option to the mail folders (as per previous guides)? I was at the rspamd section and suddenly I realized I had no Junk folder (or any other folders) in roundcube, only inbox.
    They do get created automatically when adding a mail client but it may be confusing for someone that is following this guide for the first time not to seeing any mail folders in roundcube.

  22. Hans-Werner

    Hello,
    my name is hans-werner and I have a question. What are the correct file and folder permissions in /var/vmail/ ?

    I have 0755 for folders and 0644 for files, but should it not be 750 for folders and 640 for files?

    cu
    Hawe

  23. Rita Burrows

    I’m using the guide step by step. This is the furthest I’ve ever been! I am getting the following error when I test iMAP:

    Sep 20 18:04:27 raspberrypi dovecot: imap-login: Disconnected (auth failed, 1 attempts in 6 secs): user=, method=PLAIN, rip=86.176.179.xx, lip=192.168.1.xxx, TLS, session=

    I replaced some details if x’s for privacy. The certificates was fine. I tired with my own email address and john@example.org. Both came back with ‘Login failed’.

    Any suggestions?

  24. Just ran into problems after upgrading dovecot to 2.3.7.2. I now get the error `dh key too small`.
    It looks like since v2.3 you need a pem size of at least 4096.

    1. For anyone with the same issue. I fixed it by generating a dh.pem file: `openssl dhparam 4096 > dh.pem` (will take very long!).
      Then I added it in the `10-ssl.conf` as `ssl_dh = <[path]/dh.pem`.

  25. Hello,

    I’m using two dovecot instances with dsync – how do i delete the users mail data (maildir) properly with doveadm when a virtual user gets removed from the db?

    thanks,
    Alex

  26. Great tutorial Christoph.

    I encountered an issue not documented in your guide, found when testing with roundcube.
    In dovecot-sql-conf.ext there is a default password scheme parameter as follows:
    default_pass_scheme = MD5
    This default setting caused problem when I changed test user password through Roundcube. I had to change this to:
    default_pass_scheme = CRYPT

    Thought I would pass along in case other readers run into this. Thanks!

    1. >In dovecot-sql-conf.ext there is a default password scheme parameter as follows:
      >default_pass_scheme = MD5

      Where in the Buster tutorial do you see that?

      I don’t think MD5 (PLAIN-MD5 to be more precise) has been used since Wheezy.

      1. OnceAceGuy

        @harshness – It is not in the tutorial, that is why I added the comment 🙂

        I did a fresh install of Buster yesterday, all default packages from Deb repositories. In my dovecot-sql-conf.ext file the offensive parameter is on line 81. Note that IT IS commented out! But the value is MD5. I can only assume this is the default value if you make no changes. I should have been more clear that the parameter is commented out by default.

        I discovered this when testing Roundcube in the “Webmail using Roundcube” section. The paragraph right above ‘sieve plugin’ section talks about changing password from within Roundcube, logging out, then logging back in again. This is when I found out it was using MD5 (stated clearly in /var/log/mail.log).

  27. In the codeblock for dovecot-sql.conf.ext, it looks like the password=[example password] is on a new line, when in fact it belongs to the connect line.

    driver = mysql
    connect = host=127.0.0.1 dbname=mailserver user=mailserver password=x893dNj4stkHy1MKQq0USWBaX4ZZdq
    user_query = SELECT email as user, \

  28. Hello,
    i have an error when im trying to send an email :
    Feb 16 19:41:27 mail postfix/smtps/smtpd[10263]: warning: SASL PLAIN authentication failed:
    Feb 16 19:41:33 mail postfix/smtps/smtpd[10263]: warning: SASL LOGIN authentication failed: GFzc3dvcmQ6
    Thx.
    Best regards

  29. So, after beating my head against an error, I finally figured it out:

    Everything went swimmingly, right up until I tried to log in, when I got a server error. Checking the log gave me: dovecot: imap(foo@bar.com): Error: Namespace ”:
    No home directory for system user. Can’t expand ~Maildir for mail root dir in: ~Maildir

    Turns out that in 10-mail.conf, I needed to change ‘mail_priviledged_group = mail’ to ‘… = vmail’, so dovecot had the permissions to create the mailbox

  30. Hello,
    Is it possibleto configure pop3 port on Microsoft Outlook 2019 client with option starttls. When I choose pop3, the only option is SSL/TLS. Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

Scroll to Top