There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!
Let us now configure Dovecot which will do several things for us:
- get emails from Postfix and save them to disk
- execute user-based “sieve” filter rules (can be used to put away emails to different folders)
- allow the user to fetch emails using POP3 or IMAP
Before we get to the actual configuration for security reasons I recommend that you create a new system user that will own all virtual mailboxes. The following shell commands will create a system group “vmail” with GID (group ID) 5000 and a system user “vmail” with UID (user ID) 5000. (Make sure that UID and GID are not yet used or choose another – the number can be anything between 1000 and 65000 that is not yet used):
groupadd -g 5000 vmail
useradd -g vmail -u 5000 vmail -d /var/vmail -m
Also make sure that this directory has the proper permissions:
chown -R vmail:vmail /var/vmail
chmod u+w /var/vmail
The configuration files for Dovecot are found under /etc/dovecot. In previous guides there were only two configuration files. But in Debian Wheezy’s Dovecot version you will find a new directory “conf.d” that contains about 30 configuration files. All these files are joined together to form the complete Dovecot configuration. This is done by this magical line in the dovecot.conf file:
!include conf.d/*.conf
It loads all files in /etc/dovecot/conf.d/ that end on “.conf” in sorted order. So “10-auth.conf” is loaded first and “90-quota.conf” is loaded last. The big advantage is that you can edit or replace parts of the configuration without having to overwrite the entire configuration. The disadvantage is that if you are upgrading your mail server from a previous Debian version that you need to re-configure everything in Dovecot. The main /etc/dovecot/dovecot.conf file does not require any changes. These other files in conf.d/ however need to be edited…
conf.d/
10-auth.conf
If you expect your users to use Micro$oft Outlook as an email application you should enable the “LOGIN” authentication mechanism in addition to the standard “PLAIN” mechanism:
auth_mechanisms = plain login
These are plaintext (unencrypted) ways to transmit a mail user’s password. But don’t worry. By default Dovecot sets “disable_plaintext_auth = yes” which ensures that every connection is encrypted using TLS.
At the end of this file you will find various authentication backends that Dovecot uses. By default it will use system users (that are listed in /etc/passwd). But we want to use the MySQL database backend so go ahead and change this block to:
#!include auth-system.conf.ext
!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
#!include auth-static.conf.ext
Now edit the SQL configuration file:
auth-sql.conf.ext
Comment out (add “#” in front) the standard “userdb” section.
Then add:
userdb {
driver = static
args = uid=vmail gid=vmail home=/var/vmail/%d/%n
}
(Warning: There must only be one ‘userdb’ section.)
(Careful: there is a section that looks very similar. But it uses “%u” instead of “%d/%n”. This would expect a file system of /var/vmail/USER@DOMAIN instead of /var/vmail/DOMAIN/USER.)
10-mail.conf
Change the mail_location setting to:
mail_location = maildir:/var/vmail/%d/%n/Maildir
This is the directory where Dovecot will look for the emails of a specific user. The “%d” variable is the domain part and “%n” is the user part of an email address. So for john@example.org this leads to “/var/vmail/example.org/john/Maildir”.
(The maildir format specifies that every email is represented by a single file on disk using a defined file name format. An older and less flexible format is the “mbox” format that stores all emails in one folder into a single file which makes deleting emails slow because large mbox files will have to be read and writen entirely.)
Further down you will find sections defining the namespaces. These namespaces are folder structures that your email program sees when connecting to the mail server. If you use POP3 you can only access the “inbox” – which is where all incoming email is stores. Using the IMAP protocol you get access to a hierarchy of folders and subfolders. And you can even use folders shared between users. Or public folder that could be accessed by anyone – even anonymously.
Look for the “namespace inbox” section. If you already have emails stored on your server from previous versions of this ISPmail guide you need to change:
separator = .
here. By default the seperator is “/” which creates a directory structure like “/var/vmail/example.org/john/Maildir/INBOX/staff/marketing/simon”. This is perfectly fine. But previous ISPmail guides used “.” as a seperator so the mentioned folder would rather have been “/var/vmail/example.org/john/Maildir/.INBOX.staff.marketing.simon”.
10-master.conf
This configuration file deals with services that allow communication with other processes. For example it enables or disables POP3 or IMAP. Don’t worry about the standard unencrypted TCP ports 110 (for POP3) and 143 (for IMAP). They can be kept accessible. If a user connects to these ports they will have to issue a STARTTLS command to switch into encrypted mode before they are allowed to send their password. There is basically no difference between using an plaintext port like 110 for POP3 and then using STARTTLS – or connecting to the encrypted 995 port for POP3S (=secure). See the Dovecot documentation for another explanation.
So most settings are sane here and do not have to be changed. However one change is required in the “service auth” section because we want Postfix to allow Dovecot as an authentication service. This is what has to be entered:
# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0660
user = postfix
group = postfix
}
10-ssl.conf
Earlier in this guide you created both a key and a certificate file to encrypt the communication with POP3, IMAPs and HTTPS between the users and your mail server. You need to tell Dovecot where to find these files:
ssl_cert = </etc/ssl/certs/mailserver.pem
ssl_key = </etc/ssl/private/mailserver.pem
15-lda.conf
In this configuration file you should enable the “sieve” plugin. Simply put sieve is a way to manage server-side email rules. A rule consists of conditions and actions. For example if the sender address matches “steve@example.org” you could tell Dovecot to move such emails to your “staff/sales/steve” folder. These rules are stored on the Dovecot server and executed automatically. They do not have to be run from a user’s mail client. Whether you connect from your smartphone your laptop or use the webmail access – the rules always work.
Locate the “protocol lda” section at the end of the file. Add “sieve” to the list of mail plugins:
protocol lda {
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = $mail_plugins sieve
}
/etc/dovecot/dovecot-sql.conf.ext
This file is referred to by /etc/dovecot/conf.d/auth-sql.conf.ext. It tells Dovecot how to access the MySQL database and where to find the information about email users/accounts. You will find it well documented. Make sure that it contains these lines (you can just add them at the end of the file if you like):
driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser password=fLxsWdf5ABLqwhZr <— use your database access password instead
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_users WHERE email=’%u’;
What these lines mean:
- driver: the kind of database
- connect: where to find the MySQL database and how to use it (username, password)
- default_pass_scheme: the format in which the passwords are stored (we use unsaltes MD5 hashes)
- password_query: an SQL statement that returns the user (=the email address) and the password (MD5 hash) from the database where “%u” is the login user name (=we use the email address as the user name of an email account). Whenever Dovecot needs to check an email user’s password if will run the above query. It will create an MD5 hash of the user’s password and compare that to the hash it found in the “virtual_users” database table.
Finishing move
Before starting Dovecot go fix file system permissions for the /etc/dovecot/dovecot.conf file so that only the vmail user can access the Dovecot configuration. The reason is that Postfix starts the delivery agent with vmail permissions:
chgrp vmail /etc/dovecot/dovecot.conf
chmod g+r /etc/dovecot/dovecot.conf
You should also make sure that only root can access the SQL configuration file so nobody else is reading your database access passwords:
chown root:root /etc/dovecot/dovecot-sql.conf.ext
chmod go= /etc/dovecot/dovecot-sql.conf.ext
Restart Dovecot from the shell:
service dovecot restart
Look at your /var/log/mail.log logfile. You should see:
… dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
If you get any error messages please re-read this page and double-check your configuration files.
I was happy to find the Wheezy-version of this great tutorial, as it’s exactly what I need migrating my old server running Debian 5 to a new one with Debian 7.
The only real challenge in that lies with the Dovecot configuration that changed completely with the switch to 2.x. Sadly that’s the one part in this version of the tutorial that’s not up to date yet.
Any chance to get an update on that in the near future?
Apparently you found the work in progress. 🙂 It shouldn't be online and it isn't linked yet anywhere on workaround.org. However I needed to make it visible to organize the chapter. A stupid shortcoming of the Drupal CMS. Please do not use the ISPmail guide yet. I'm still working on it and the Dovecot page is yet just copy-paste from the previous version.
The final version should be available soon from a link on the home page.
Yep[!], ich saß auch mit offenem Mund vor der Glotze, als ich den Fortschritt des neuen Tutorials bemerkte. War es von den Vorgänger-Tutorials schon gewohnt, etwas länger zu warten…”Good things takes time…”.
Als ich dann zu lesen begann, erlebte ich ein Déjà-vu, dabei blieb es aber nicht nur beim offenen Mund, nein, mein Unterkiefer prallte auf die Schreibtischoberfläche:
„AMaViS is no longer used“.
Ein Schlag mitten ins Gesicht! Eine Tasse Kaffee musste her – ich glaube, ich hatte ein Time out, einen Buffer over run, einen blue Screen (nein, nur keine Fenstersoftware).
Lob für die getane Arbeit hast du von meiner Seite bereits erhalten und als Verehrer deiner „ISP Mail Tutorials“ – seit der Sarge-Epoche – nehme ich mir schlichtweg das Recht, an dieser Stelle ordentlich Kritik zu üben.
“AMaViS is no longer used. Many sysadmins found that it does not scale well, was cryptic to configure and did not stop spam emails reliably for each individual user. It may still be useful as a virus scanner if you want to protect Windows users but it is no longer covered in this tutorial. ”
Für mich vehemente Widersprüche zur Einleitung des Tutorials, wo geschrieben steht:
“…What this tutorial is *NOT* about….” und weiter “…If you just need a no-cost mail server … then this is the wrong place for you…” im Widerspruch zu “…Many sysadmins found that it does not scale well…” dann solltest du allenfalls die Hardwareempfehlungen im Tutorial passend abändern, AMaViS etwas mehr Platz im Tutorial schenken und entsprechend auf die Konfiguration eingehen; AMaViS wurde in den Dokus bislang vernachlässigt, seine Vielfältigkeit gerade mal angegestreift.
…und weiter “… I personally recommend that you understand what you are doing. And that’s what the ISPmail guides are for….” im Widerspruch zu “… AMaViS…was cryptic to configure…”
…und weiter “… and did not stop spam emails reliably for each individual user. It may still be useful as a virus scanner if you want to protect Windows users…”.
Mittels AMaViS´s Policy Banks (eventuell mit mySQL Anbindung) / Postfix transportmaps bieten sehr individuelle Möglichkeiten der Mailsteuerung, bis hinab zur Gruppen/Benutzerebene. Das Virenproblem betrifft nicht nur Windowsbenutzer, welche nach wie vor einen Großteil der Anwender stellen, sondern mittlerweile auch Android-Devices und unter Bezugnahme der Contentfiltering von ausführbaren Dateien(anhängen) erweitert sich der Einsatzbereich noch etwas.
Fazit: Im folgenden „ISP Mail Tutorial“ wird unter anderen auf die Möglichkeiten der Contentfilterung (AMaVis als Pre-Queue / Content-Filter), der Erkennung und Abweisung von infizierten Mails (Viren) durch Einbindung von z.B. ClamAV, der Spamfilterung mittels Spamassassin, Razor/Pyrazor und das Zusammenspiel der Mailflussstreuerung mittels Postfix (tranportmaps) und AMaViS-Policy-Banks sowie das Nebenprodukt der DKIM-Signierung (durch AMaVis) und der DKIM-Bewertung (durch Spamassassin) nicht eingegangen.
Das „famous ISP-style“ mutiert zum „Mail“ Tutorial.
Eigentlich hatte ich gehofft, dass einige der in den Vorgänger Tutorials recht stiefmütterlich behandelnden Abschnitte nun endlich etwas mehr Aufmerksam gewidmet wird, leider ist das Gegenteil der Fall.
Was mir sonst noch aufgefallen ist: die Einlieferung der Mails von Clients (MUA´s) wird ausschließlich per Port 25/465 dokumentiert, auf die Möglichkeit, dies per SUBMISSION (587 TLS/SSL) zu tun und eine mögliche Mailflusssteuerung mittels AMaViS -Policy-Banks und Postfix wird nicht nur gänzlich verzichtet, sondern nicht einmal erwähnt.
Der Abschnitt „Postfix SSL“ ist nicht vollständig, speziell bei Debian sollten die Standarteinstellungen geprüft werden, auf “Perfect Forward Secrecy (PFS) für Postfix” wird nicht eingegangen (Dovecot [?] ), Postfix “proxymap” werden noch immer nicht angewendet.
So, jetzt ist der Dampf raus…!
Sage nichts – ich kenne deine Situation: Viel Aufwand, alles Freizeit, keine Kohle, jeder will alles und das schon letzte Woche und niemanden kann Recht gemacht werden.
Wahrscheinlich gibt es auch andere Leser/Kollegen/Leidgenossen/Spezialisten/Nerds und Freaks die hier lesen und profitieren. Vielleicht können wir dir unsere bescheidene Hilfe anbieten.
JustMy2Cents
Alois
Alois, danke für deinen extrem umfangreichen Kommentar. Ich nehme ihn gar nicht negativ, sondern denke tatsächlich gerade darauf herum, was genau im Tutorial enthalten sein sollte. Ich stehe mit AMaViS auf Kriegsfuß – ich gebe es zu. Die Spam-Erkennung ist in allen Installationen, die ich gesehen habe, einfach nur miserabel. Spam-Emails scheinen zu vielfältig zu sein, als dass man eine globale Erkennung für ein paar hundert User zuverlässig hinbekommt. Und wenn man die User am Lernprozess mit bedingten Wahrscheinlichkeiten teilnehmen lässt, wird die Qualität der Erkennung noch schlechter, weil sie oft einen Newsletter nicht von echtem Spam unterscheiden können. Ein anderes Thema ist vermutlich die Malware-Erkennung. Hier wäre ich durchaus bereit, mich noch einmal in Ruhe mit AMaViS zu beschäftigen. Das wird für ISPs dennoch kein Thema sein, weil AMaViS einfach nicht performant genug arbeitet, um nicht-triviale Dosen von Emails über das Gateway zu schicken und dabei zu scannen. Aber der Guide ist ja auch eher für den gehobenen Privatanwender. 🙂
Was sind deine Erfahrungen mit AMaViSd? Was würdest du dir konkret wünschen? Und könntest du mich vielleicht sogar unterstützen?
Dass ich mit der Veröffentlichung immer Monate hinterherhänge, ärgert mich persönlich auch. Ich konnte es die letzten zwei Male schon wegen Projekten nicht anders einrichten. Und regelmäßig frage ich mich, ob ich die Serie nicht einfach einstellen sollte. Wenn es dann nicht wieder Stimmen gäbe, die verzweifelt auf der Suche nach Dovecot-Konfigurationen und Pfad-Problemen in der Prozesskommunikation sind. Und einige dieser Probleme habe ich bei Kunden bereits gelöst. Manchmal mutiert das ISPmail-Tutorial zu einem Zombie-Projekt, das immer noch gerade wieder nachts im Nebel aus den Gräbern steigt. 🙂
Wenn ich mit meinem bescheidenen Wissen etwas beitragen kann, sehr gerne!
Melde mich per Mail.
Ym2C Alois
Bittebitte, lieber Christoph,
stelle nicht die Arbeit an neuen Versionen dieses Tutorials ein.
Ich schätze Dein Tutorial sehr. Fundierte Information, und vor allem: gut geschrieben.
Eine Perle im Internet!
Wenn das ein Zombie ist, dann riecht er noch ziemlich gut.
Ich habe mich oft gefragt, wie ich Dir zurückgeben oder Dich unterstützen kann.
Oh, da unten ist ja ein Flattr-Button, der ist wohl neu. Hmmmm….
Gerade beschäftige ich mich mit Autokonfiguration von Mailclients, vielleicht kann ich damit beitragen?
BitteBitte mit Sahne und Zucker obendrauf!
Hallo Christoph,
ich muss zugeben, dass ich erst vor einigen Tagen auf deine Tutorial-Serie gestoßen bin.
Seit über einem Jahr nun bin ich im Besitz eines VPS, der sich anfangs zwar wie ein Spielzeug gefühlt haben muss, aber inzwischen stabil läuft. Sogar ein Zweiter ist inzwischen hinzugekommen, um etwas Redundanz zu schaffen.
Nach längerer Beschäftigung ist es dank der umfangreichen Hilfen im Netz wirklich kein Problem mehr, einen nginx-Webserver, oder Voiceserver für Mumble und TeamSpeak 3 einzurichten. Wenn es allerdings um Mails geht, kann man echt daran verzweifeln. Die meisten Ressourcen behandeln nur einen kleinen Aspekt des Ganzen und/oder sind hoffnungslos veraltet.
Wenn man – vorallem nach den Skandalen der letzten Monate – von den traditionellen Mailbetreibern wegmöchte, bleibt einem als Semi-DAU wohl nur die Möglichkeit, sich alles über lange Zeit hinweg irgendwie zusammenzuwurschteln. Hierbei entstehen Probleme aller Art, einige sind vielleicht einfacher zu beheben, andere für Nicht-Gurus vielleicht sogar schier unmöglich.
Was ich damit sagen möchte: deine ISPMail-Tutorials sind allem Anschein nach die weltweit einzige, umfangreiche Anlaufstelle für einen kompletten, eigenständigen Mailserver. Ich und vermutlich sehr, sehr viele andere würden sich freuen, wenn du die Reihe weiterführen würdest.
Natürlich will ich dich keinesfalls drängen oder gar zwingen; du sollst nur wissen, dass die Arbeit keinesfalls umsonst ist und vermutlich viele Leute da draußen sehr zufrieden mit dem Resultat sind, dass sie dem Folgen deines Leitfadens verdanken.
Hach, ihr seid so lieb zu mir. 🙂 Und es hat geholfen. Ich habe gestern 6 Stunden weiter am Wheezy-Tutorial gearbeitet. Ich gehe davon aus, dass die aktuellen Teile bis Mittwoch fertig sein werden. Am Wochenende werde ich mich voraussichtlich noch um AMaViS für Malware-Scanning mit SpamAssassin und Quotas kümmern. Dann geht die neue Version online.
Es ist erschreckend, wie oft sich der Teufel im Detail versteckt. Und am liebsten würde ich auch schon wieder ganze Abschnitte neu schreiben, weil sie mir didaktisch nicht so richtig gefallen. Egal… Schluss mit dem Perfektionismus. Hauptsache die Artikel sind technisch und inhaltlich korrekt.
Hallo Christoph. Ohne deinen Tutorial wäre ich seit langem aufgeschmissen. Immer wieder komme ich zurück um mich noch mal zu überzeugen ob es wirklich so war wie ich es mir gedacht habe. Mittlerweile laufen mehrere Mail-Server basierend auf deinen Tutorial. Richte doch einen PayPal Konto für Spenden ein. Ich bin mir sicher dass es genug Leute gibt die gerne eine Spende an das ISPMail Projekt geben.
Vielen Dank für alles
Das nenne ich mal nettes Feedback. Vielen Dank. Einen Flattr-Button hatte ich bereits eingerichtet, aber der wird leider viel zu selten benutzt. Den Paypal-Spenden-Button hatte ich irgendwann mal entfernt, als ich wieder eine Phase hatte, in der mir Paypal unheimlich auf den ethischen Keks ging. Aber ich nehme den Hinweis gerne auf und werde gleich mal wieder einen Arme-Leute-Paypal-Button neben dem Flattr-Button platzieren.
Hi Christoph,
First off—thanks for writing these awesome tutorials! It’s a massive help.
Just wondering how you’re going on getting this ‘live’ and ready to go? I’ve recently got a new server running and would love to get mail going…
I’m stuck on this step from old tutorial because of this new version of Dovecot.
Thanks!
Thanks for your interest and motivation. 🙂 The Wheezy tutorial is close to release. The last page about the Roundcube/Webmail with server-side sieve filtering needs to be completed which I assume is done by tomorrow (5.11.). I know that I'm very late this time…
Hi Christoph,
as well as Lao I set up a new server recently and wait for your awesome tutorial to be finished.
But as we all know, you do it on your spare time… so please don’t worry when you just don’t hit that deadline. Even when it would take another week, if would definitely be worth it to wait for!!
So thank you so much for your great work and keep going on with it! I’m going to hit that Flattr-Button as well (although I am student and chronically out of money 😉 ) but it is of course the smallest way to thank you for!
Viele Grüße aus Dresden!
Hi Christoph,
You don’t have to apologyze. You are giving a lot to the community and if it takes longer than you expected, nobody has any right to complain. I understand you want to meet your own goal, but all things considered, your work is still first class.
Hi there!
Yes, by all means please don’t take my comment as a means to rush! I’m happy to wait until things are finished and done properly; was just wondering how things are going—which sounds great by the way!
Thanks again, for all your good work!
-Lao
i love you for this tutorial mate 🙂
its seems if you don tadd the user_query to the dovecot-sql.conf.ext then you will get
Nov 7 14:56:25 Debian-70-wheezy-64-minimal dovecot: auth-worker(13066): mysql(127.0.0.1): Connected to database mailserver
Nov 7 14:56:25 Debian-70-wheezy-64-minimal dovecot: auth-worker(13066): Warning: mysql: Query failed, retrying: Table ‘mailserver.users’ doesn’t exist
Nov 7 14:56:25 Debian-70-wheezy-64-minimal dovecot: auth-worker(13066): Error: sql(mail@mailhost.com): User query failed: Table ‘mailserver.users’ doesn’t exist (using built-in default user_query: SELECT home, uid, gid FROM users WHERE username = ‘%n’ AND domain = ‘%d’)
any ideas?
ok no. was my bad.
How a u solution this problem? i have 2.
Just comment out this part of “auth-sql.conf.ext” (add “#” in front of each line):
userdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
Was inattentive when read and missed phrase
“..Comment out (add “#” in front) the standard “userdb” section. And add:..”
did the same mistake.. maybe is better to make it clear that you shouldn’t have more than one userdb section..
Right – I have edited the text slightly and hope that now fewer readers stumble here.
In the code for dovecot-sql.conf.ext, there’s some unfortunate pagination (if you aren’t cutting and pasting) of the password substitution comment that makes it look like an additional directive in the configuration. I had a third line in the code that read:
access password instead
Perhaps there’s a better way to deal with end-user substitutions (maybe a different color or a documentation convention).
I am trying to set up the server to delete mail after 30 days old. I’ve found in the dovecot wiki the expunge command, and the search command that can be used in place while testing.
I need to use the iterate_query but it isn’t working.
When trying to run: doveadm search -A mailbox Junk savedbefore 3d I receive an error: “doveadm: Error: Failed to iterate through some users”.
Research and poking around indicates this is because the userdb is static, and not querying the sql. If I add a second userdb then this works, but the notes make it clear above we are not to do that; and I think it breaks mail when I tested it.
So, how can I get the iterate_query to work with our settings?
Thanks!
I think I found my answer while searching for something else in one of the previous tutorials. I haven’t tried it but this looks to be the solution, for anyone else wanting the same thing.
https://workaround.org/ispmail/lenny/quotas
The solution to enable quotas is to disable the static db and change it to an SQL lookup. This will enable quotas as well as (probably) fix the iterate_query.
Hello all,
My certs mailserver.pem in ../private and mailserver.pem in ../certs have key password, in dovecot file 10-ssl.conf have this option “ssl_key_password = ” and i write the password , but in postfix, I can,t do.
not find this option, and my postfix tell me:
“Dec 24 10:28:27 mercuriovm postfix/smtpd[6357]: warning: cannot get RSA private key from file /etc/ssl/private/mailserver.pem: disabling TLS support”
Workaround for this?
Thanks.
Hi,
I know this guide is for Debian but I’ve managed successfully (more or less) so far on Ubuntu 12.04
I encountered some issues when I hit the ‘Testing email delivery’ page:
Dec 26 13:54:24 testflight dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +r perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755)
I fixed this by uncommenting the fields in /etc/dovecot/conf.d/10-master.conf: ‘unix_listener auth-userdb’ and added vmail to user and group fields.
Another issue is that that ‘auth-sql.conf.ext’ file is not present after install dovecot packages on Ubuntu. I think you should mention that that file should also contain these lines:
passdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
Thanks for a fantastic guide! Definitely the best one I’ve worked with.
I ran into the same issues with my configuration. But the solution from Darren did not help me at all. I have the same error even after changing the above mentioned lines in the configuration. Are there more ideas on this subject?
I added “protocols = imap” in /etc/dovecot/dovecot.conf and changed the rights for /var/run/dovecot/auth-userdb to vmail
Had similar issues as others had with userdb but with master. My solution was the following addition to 10-master.conf
# Add new subsection just below unix_listener auth-userdb section
unix_listener auth-master {
mode = 0660
user = vmail
}
Gestern habe ich meinen neuen Server mit Debian Wheezy eingerichtet. Nach gut 1.5 Jahren ISPConfig 3 wollte ich wieder weg von Webinterfaces und mich lieber auf der Konsole austoben. Nachdem ich den Separator in Dovecot auf / gesetzt habe, werden Mailboxen/Unterordner dennoch im Format .Foo.Bar angelegt. Die Ausgabe von doveadm kann unter http://nopaste.info/623fc8c02b.html eingesehen werden.
Persönlich stört es mich nicht wirklich aber als berufstätiger Sysadmin bin ich zumeist darauf aus, dass es so zu funktionieren hat, wie man es eben einstellt. Dumme Angewohnheit von mir 😉
(I can mostly read but not write fluently in German, so I will use English for the rest of the mail.)
I also found out that dovecot uses . by default as a separator and does not use the file system directory structure, unless the LAYOUT=fs parameter is passed to mail_location, i.e.
mail_location = maildir:/var/vmail/%d/%n/Maildir:LAYOUT=fs
It is not necessary to define the separators (tested with Ubuntu 12.04), but the suffix posted by you. Thank you for commenting.
Also tested with Ubuntu 14.04. Thank you guys!
Hello, thank you for this great tutorial. I need to retreive the message from an external POP3 server with getmail and set the destionation to the local user (for example john@example.org).
I’ma ble to retreive the message fomr the external POP3 server with getmail but the destination is always /var/mail/john
Please can you tell me how configure getmail to retreive the message from my multiple POP3 accounts and delivery to the local account (fro example john@example.org) so I can read the email with IMAP from external?
Thank you
Enrico
I just know fetchmail – which is probably similar. You configure the recipient email address in fetchmail's configuration file. I assume that getmail requires the same. These tools usually fetch email and deliver them on localhost:25. So it depends on your Postfix configuration where the email is finally delivered. If you deliver to john@example.org and the email ends up in /var/mail/john then I assume that you (accidentally?) configured "example.org" to be a local domain. If you are unsure what I mean with "local" domain then please re-read this page.
I got an error on receiving mail unless I added the postmaster address in 15-lda.conf:
protocol lda {
postmaster_address = your@postmaster.address
mail_plugins = $mail_plugins sieve
}
awesome shit !!! Zu Deutsch: ” A supa Sach’ “
this tutorial is just not working according to the configuration of dovecot 2.1.17. The connection to imap server always fails, no matter if I try to telnet it or use roundcube. Iptables input for port 110 and 143 is accepted. I can’t figure out in which config file to mention these protocols since they’re not to be entered into /etc/dovecot/dovecot.conf no more. With the instruction here I always fail … any help ???
You’re the man!
it is just not working according to the configuration of dovecot v2.1.7. Can’t telnet from server or foreign network on port 143 or 110. Entries in /etc/dovecot/conf.d/10-master.conf made as instructed, firewall ports opened and all points according this,the previous and following chapters double checked … any help ???
Check if Dovecot has opened port 143 for listening:
The output should look like:
If you don't see any process listening on port 143 then check if Dovecot is running:
The output should be like:
cheers Christoph, I had some confusion going on due to mixing former versions of the tutorial with fixes from the web and not looking at your latest updates. Finally ‘apt-get -f install dovecot-mysql dovecot-pop3d dovecot-imapd dovecot-managesieved’ and another recheck of the entries in the conf files did the trick. Everything works fine now, big up !!!
I can’t get my setup to work. I have done the tutorial for Squeeze 2 times before and thought I was able to do it. Went through this tutorial about 5 times in the last 3 days. Double-checked everything, can’t find any mistakes. Still, I end up with
Apr 12 20:54:19 leaf postfix/pickup[6676]: ADDFF12005F: uid=0 from=, size=380, nrcpt=1 (queue active), relay=dovecot, delay=0.38, delays=0.21/0.01/0/0.16, dsn=5.1.1, status=bounced (user unknown), relay=local, delay=0.21, delays=0.08/0.01/0/0.12, dsn=2.0.0, status=sent (delivered to mailbox)
Apr 12 20:54:19 leaf postfix/cleanup[6718]: ADDFF12005F: message-id=<20140412185419.ADDFF12005F@leaf>
Apr 12 20:54:19 leaf postfix/qmgr[5208]: ADDFF12005F: from=
Apr 12 20:54:19 leaf dovecot: auth-worker(6724): mysql(127.0.0.1): Connected to database mailserver
Apr 12 20:54:19 leaf dovecot: auth-worker(6724): sql(mike@example.de): unknown user
Apr 12 20:54:19 leaf postfix/pipe[6722]: ADDFF12005F: to=
Apr 12 20:54:19 leaf postfix/cleanup[6718]: ED870120060: message-id=<20140412185419.ED870120060@leaf>
Apr 12 20:54:20 leaf postfix/bounce[6728]: ADDFF12005F: sender non-delivery notification: ED870120060
Apr 12 20:54:20 leaf postfix/qmgr[5208]: ED870120060: from=<>, size=1940, nrcpt=1 (queue active)
Apr 12 20:54:20 leaf postfix/qmgr[5208]: ADDFF12005F: removed
Apr 12 20:54:20 leaf postfix/local[6729]: ED870120060: to=
Apr 12 20:54:20 leaf postfix/qmgr[5208]: ED870120060: removed
I just can’t see why dovecot says the user was unknown. Any ideas are highly appreciated.
I am posting this as a reply right here because there is one thing I also can’t understand: Calling ps ax | grep dovecot/imap yields nothing. Calling ps ax | grep dovecot shows:
root@leaf /etc/dovecot # ps ax | grep dovecot
3826 ? Ss 0:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf
3844 ? S 0:00 dovecot/anvil
3845 ? S 0:00 dovecot/log
6725 ? S 0:00 dovecot/config
6842 pts/0 S+ 0:00 grep –color=auto dovecot
Like I said – I already double-checked everything I could find. If anyone has ideas on what might be wrong, please advise!
Did you try the "postmap -q mike@example.de …" trick? Has it responded that the email address is invalid?
Yes, I tried all of those tests. All worked as expected and return ‘1’ if address/alias/domain exists and return nothing if it doesn’t exist. Checked all postfix and dovecot config files for passwords, paths, database names etc… can’t find any mistakes.
Could you copy/paste the "postmap -q …" call here?
Also please copy/paste what you get when you enter "postconf virtual_mailbox_maps".
Thanks for your feedback. Here’s the postmap tests:
root@leaf /etc/postfix # postmap -q mike@example.de mysql:/etc/postfix/mysql-virtual-alias-maps.cf
mike@example.de
root@leaf /etc/postfix # postmap -q example.de mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
1
root@leaf /etc/postfix # postmap -q mike@example.de mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
1
root@leaf /etc/postfix # postconf virtual_mailbox_maps
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
Here’s a complete postconf -n call: http://pastebin.com/P847p5MV
And dovecot -n: http://pastebin.com/LCzeZcwn
Thanks for your (trying to) help!
Now at second looks this line is weird:
Apr 12 20:54:19 leaf dovecot: auth-worker(6724): sql(mike@example.de): unknown user
If Postfix would not be able to find the user account then you would get a log line like this:
… 550 5.1.1 <mike@example.de>: Recipient address rejected: User unknown in virtual mailbox table
It's apparently Dovecot which tries to lookup the user mike@example.de but fails. Could you give the /etc/dovecot/dovecot-sql.conf.ext file another look? I suspect that there is a mistake with your SQL configuration.
Unbelievable. I hadn’t previously copy/pasted from the tutorial but instead typed everything myself so I realize what I’m doing. Because you told me to re-check the file (which I actually did about 25 times already in the last couple of days) I thought I might just copy/paste from the tutorial this one time. Did that, exchanged the passwords and there – it worked. Thanks to etckeeper, I had the possibility to make a diff. Here it is:
-password_query = SELECT email as user, password FROM virtual_users WHERE email=’%s’;
+password_query = SELECT email as user, password FROM virtual_users WHERE email=’%u’;
That one letter at the end… Thank you very much for your help, and thanks a ton for the great tutorials. Keep up the good work!
Starting dovecot, I have this message regarding 10-director.conf file which is not modified anywhere in this guide… :
doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-director.conf line 29: Unknown setting: service
Nothing in in /var/log/mail.log …
Any idea ?
Thierry, I’m having that same error. It’s weird because we didn’t have to edit 10-director.conf for any reason…
I need an answer to this question as well.
Cheers!
I think I found the issue (or at least what caused it for me).
After installing postfixadmin, and making the changes suggested in comment 3152 (https://workaround.org/comment/3152#comment-3152), I forgot to the closing ‘}’ in the userdb stanza in auth-sql.conf.ext.
Closed it, and dovecot starts again without complaining about 10-director.conf
Hope this helps.
thank you for your comment. i got the same problem and solved with the closing }
Hi Christoph!!
Thank you so very much for this great tutorial!!
Do you know when you’ll have the changes for Dovecot 2 up and available? I’m really keen to move ahead with this, as your directions thusfar have been PERFECT! I greatly appreciate your effort – and the fact that you make your knowledge freely available…
Looking forward to your next writings on this tutorial!!
Cheers!
Hi Tony… Thanks for your nice feedback. The current Wheezy tutorial deals with Wheezy's Dovecot version which is 2.1.7.
You’re welcome Christoph!
The reason why I was asking is that I do not see auth-sql.conf.ext in my /etc/dovecot/conf.d/ directory…. I’m running Dovecot 2.0.19. Let me try to force an upgrade and see if that helps my issue.
If not, I’ll see what I can find about that file via Google.
Thanks!!
I had to add the line:
auth_socket_path = /var/run/dovecot/auth-master
to 10-mail.conf file, otherwise the file /var/spool/postfix/private/auth was not created and I got:
Mar 16 23:40:09 mail01 postfix/smtpd[28304]: warning: SASL: Connect to private/auth failed: No such file or directory
Mar 16 23:40:09 mail01 postfix/smtpd[28304]: fatal: no SASL authentication mechanisms
Thank you Christoph for this wonderful tutorial, which I am with since debian woody!
Are there any experiences with Perfect Forward Secrecy (PFS) for Postfix and Dovecot yet?
I am not that experienced that I could say what really happens when enabling things like
postfix:
smtpd_tls_eecdh_grade = ultra
smtpd_tls_dh1024_param_file = /etc/postfix/dh2048.pem
smtpd_tls_dh512_param_file = /etc/postfix/dh512.pem
dovecot:
ssl = yes
ssl_cipher_list = DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!CBC:!PSK:!SRP:!DSS:!SSLv2:!RC4
but if you read this http://www.heise.de/newsticker/meldung/Der-GAU-fuer-Verschluesselung-im-Web-Horror-Bug-in-OpenSSL-2165517.html, it sounds like there is a real need to get PFS to run.
typo – “default is will use” – should be “default it will use” I think.
Thanks, Tim. Fixed the typo.
Hi Christoph!
Danke schoen for the great guide! It has become my BIBLE for everything mail-related!
Could you please advise us on 20-pop3.conf? I see that you left it blank, or default. Some folks on the Internet suggest to add the following line:
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
in order to cope with some client errors, such as null characters in Outlook.
However, my mail server has been run for 5 months without issues as such. Only a customer complained that his outlook does not remove emails from ‘outbox’, sending several versions of the same email. But I think this issue is not related to dovecot.
Thank you!
Hallo Christoph, dear community!
Vorweg einen sehr herzlichen Dank für die über Jahre hinweg wiederholt gelieferten hervorragenden Tutorials! Das ist wirklich keine Selbstverständlichkeit und eine ganz tolle Sache!
My problem at the moment:
For my own convenience I set LAYOUT=fs for dovecot Maildir. Everything worked fine, but after a while I realized that there are some problems with subfolders of Inbox. It’s exactly the same as I found here:
https://www.mail-archive.com/dovecot@dovecot.org/msg53521.html
Some MUAs show the subfolders (Evolution, Thunderbird, Outlook), some don’t (K9 on Android for example). Those who show them, somehow cannot fully use them, especially subscribing/unsubscribing is not possible. Roundcube for example also shows them after creation, but I cannot see the folder in “Settings” – “Folders” then for unsubscribing or deletion.
Now one can find this fix on dovecot.org:
http://hg.dovecot.org/dovecot-2.1/rev/589d2f485a6e
But my dumb question is: How do I apply such a fix and what are the consequences regarding updates via aptitude?
Thanks for reading and everybodys help!
Martin
i got pb with dovecot i can’t start dovceot service i have an error on line 17 ok file 10-master.conf:
/etc/dovecot/conf.d/10-master.conf line 17: Unknown setting:
service
doveconf: Error: managesieve-login: dump-capability process returned 89
doveconf: Fatal: Error in configuration file
/etc/dovecot/conf.d/10-master.conf line 17: Unknown setting:
service
Hallo fils,
ok and what’s in line 17 of 10-master.conf ?
Regards, Martin
in that line i have the bigening of service imap-login {
……… but when i trying to comment (#) that line the error is going to the next one like going to line 32 and so for…
what i notice there is little tutorial talkin about that error some of talkin about upgrading to teh version 2 of dovecot any way it still dark to me
1–i just want to know there is any who follow this tutorial specifikly that step without any pb???
2– what that error means really???
3– do u think it link whith managesieve or not???
Hi,
Fantastic guide, I’ve used the previous on Squeeze for many years of happy emailing, and now I’m looking to install afresh on Wheezy. This time though, I’ve separate web, DB and email servers.
My question, therefore, what should I put in 10-ssl.conf for the ssl_cert and _key in order that it’d point toward the web server?
Regards,
Jon
Hi,
there is no “auth-ldap.conf.ext” file in etc/dovecot/conf.d .
And when i restart dovecot i get an error: “doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-auth.conf line 124: No matches
failed!”
What can i do?
In the “TLS’ifying your server” section, there’s an explanation/link for chaining the certificate generated by StartSSL in order to avoid clients complaining. I believe a similar solution is necessary for Dovecot, but their approach[1] (and, in fact, versions of Apache after 2.4.8 [2]) is to concatenate the certificate and its chaining dependencies. For example:
david@vaughn:/etc/ssl/certs$ cat mail.myserver.com.crt sub.class1.server.ca.pem | sudo tee mail.myserver.com-chained.crt
Then point courier to this chained certificate.
[1] http://wiki2.dovecot.org/SSL/DovecotConfiguration#Chained_SSL_certificates
[2] http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslcertificatechainfile – See deprecation note
P.S. Your link on this page to TLS’ifying your server is broken–using ispmail-wheezy instead of ispmail/wheezy/ (or just a relative link). But thanks for the great guide!
In order to make thunderbird happy when connecting with ssl using the free startssl certificate, dovecot and postfix need the intermediate ca certificate (www.startssl.com/certs/sub.class1.server.ca.pem):
Copy /etc/ssl/certs/mailserver.pem to mailserver-plus-ca.pem and append to this file the content of sub.class1.server.ca.pem. Then set in 10-ssl.conf: ssl_cert = /etc/ssl/certs/mailserver-plus-ca.pem.
As described elsewhere in the comments: postconf -e smtpd_tls_CAfile=/etc/ssl/certs/startssl-sub.class1.server.ca.pem
(if sub.class1.server.pem has been renamed to startssl-sub.class1.server.pem)
With these settings and imap and smtp server set to http://www.your_domain.org thunderbird accepts the startssl certificate for your_domain.org without complaining about invalid certificates.
Thanks for the tutorial!
Hi,
On the instructions is stated that “… But don’t worry. By default Dovecot sets “disable_plaintext_auth = yes” which ensures that every connection is encrypted using TLS. …”
Maybe they changed some stuff. Im using a up to date Wheezy, but have also ‘wheezy-updates’ on source.list. Fatc is that in here, Wheezy do not set “disable_plaintext_auth = yes” as default on /etc/dovecot/conf.d/10-auth.conf . This line is commented after a dovecot fresh install.
Hallo Christoph,
I’m using your tut since years, great work!
Why do you put the folder “Maildir” in the end? In my opinion this is useless. Plus you didn’t use this in former guides, at least I didn’t have it in former configs.
Kind regards
Peter
Well, at least the Squeeze maildir already used the additional "Maildir" level. I see why you consider this directory unnecessary. But Dovecot puts additional files in the "virtual home directory" (which is the directory where "Maildir" lives). For example it puts server-side based mail rules there (Sieve). If you would omit the "Maildir" level then Dovecot will mistake such control files and filter rules as mail directories and show them to the user.
You may have luck leaving the directory out if you don't plan to use Sieve filters. But even then I can imagine that further index and cache files may get in your way.
Hallo Christoph,
thanks for your replay! Did not have problems untill today but by implementing the managesieve plugin in roundcube got indeed exactly the discribed problem. :-/ Now I understand why! 😉
/etc/dovecot/conf.d/auth-sql.conf.ext line 38: Unknown setting: driver
Did dovecot syntax change again? I followed this tutorial perfectly and triple checked every line of code and I get this error as soon as I try to start dovecot. It doesn’t want to take the last part where you told us to add…
driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser password=kljkl;jasdfasdf
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_users WHERE email=’%u’;
to the end of the file. I did so and now it won’t work.
Only way I could figure out how to get this to work is to move the code into the dovecot-sql.conf.ext file inside /etc/dovecot folder. It would not work inside the conf.d folder in the auth-sql.conf.ext file.
i have problem, relay=none, delay=0.04, delays=0.03/0.01/0/0.01, dsn=4.3.0, status=deferred (address resolver failure)
ep 24 14:40:42 bilbo3 postfix/trivial-rewrite[20367]: warning: virtual_mailbox_domains lookup failure
Sep 24 14:40:42 bilbo3 postfix/error[20370]: 71010192: to=
Check whether Postfix can get information from the database properly. Use the "postmap -q …" command as described.
i reboot server and is ok
Hello!
Thanks for a splendid guide to getting mail to work.
I’m running Debian 7.7.
I can send and receive mails with postfix, but I can’t get Dovecot to work.
When I restart dovecot i get this message in /var/log/mail.log:
Dec 12 22:36:13 voyager dovecot: master: Warning: Killed with signal 15 (by pid=1214 uid=0 code=kill)
Dec 12 22:36:13 voyager dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Dovecot is running but not listening.
saturday@voyager:/etc/dovecot/conf.d$ ss -tlnp | grep :143
LISTEN 0 100 :::143 :::*
LISTEN 0 100 *:143 *:*
I’ve done the Dovecot-guide twice, uninstalled and purged Dovecot once and tried again.
I’ve checked all other files and I just can’t understand whats wrong. I’ve checked on websites, but not found any solution based on this configuration.
Database seem to work, I’ve tested the files and got “1” as answer running them.
When I try to connect to Dovecot with Roundcube I get different error messages. I first got this:
IMAP Error in /usr/share/roundcube/program/include/rcube_imap.php (205): Login failed for john@example.com from 192.168.2.1. Empty startup greeting (localhost:143)
And later this:
IMAP Error in /usr/share/roundcube/program/include/rcube_imap.php (205): Login failed for john@example.com from 192.168.2.1. LOGIN: Plaintext authentication not allowed without SSL/TLS, but your client did it anyway. If anyone was listening, the password was exposed.
Best regards,
Tobias
Hello Christoph!
First of all I have to say it is a very good tutorial, please don’t stop working on it. 🙂
After I did the dovecot configuration I do get an error in mail.log, coming up form dovecot when I try to get E-mails with Thunderbird. I configured Thunderbird to get Mails on port 993, connection security with SSL/TLS using IMAP protocol. Error message is:
dovecot: imap-login: Fatal: Can’t load ssl_cert: error:0906D066:PEM routines:PEM_read_bio:bad end line
Apache2 https is working with this cert, so problem shouldn’t come from cert. Do you have any suggestion what I do can try to fix it?
Thanks in advanced Martin.
After I copied the content of cert from StartSSL it was with blank lines included. So I removed all blank lines from cert file and this fixed the error.
Hi Christian,
I followed the complete tutorial and it seems to work pretty good so far ( mail is working, roundcube is working ) my mail is even considered normal not spam and I can send and receive mail using roundcube. However, now I want to bind my mail account to outlook.com ( adding an additional account for sending/receiving) however, it cannot connect for some reason.
I checked the logs and it says:
Mar 9 21:25:36 boreas dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=65.55.41.7, lip=149.xxx.xxx.xxx, TLS: Disconnected, session=
and it seems the user=<> is empty. I have no clue why, i`ve been searching why this is empty ( IMAP is working fine.. ) but pop3 is not working for some reason.
ANy idea why this is?
To actually get to the next chapter testing mail, I needed to delete a rule in /etc/dovecot/dovecot-sql.conf.ext
So I ended up with:
driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser password=fLxsWdf5ABLqwhZr <--- use your database default_pass_scheme = PLAIN-MD5 password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; The "access password instead" threw an error telling it misses a '=' somewhere. Could not find anything regarding this statement on Google > Dovecot.
Thanks for a concise tutorial. It saved the day when my mail server was broken after the upgrade to Debian Jessie. As it turns out, the upgrade disabled all database authentication, so vmail wouldn’t work anymore. Your step-by-step description finally allowed me to repair things. Your beer money is on the way!
I’ve got this output:
Jun 16 17:55:54 srv03 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Jun 16 17:55:54 srv03 dovecot: config: Error: /usr/lib/dovecot/managesieve: /usr/local/lib/libssl.so.1.0.0: no version information available (required by /usr/lib/dovecot/libdovecot-storage.so.0)
Jun 16 17:55:54 srv03 dovecot: config: Error: /usr/lib/dovecot/managesieve: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by /usr/lib/dovecot/libdovecot-storage.so.0)
Jun 16 17:55:54 srv03 dovecot: config: Error: /usr/lib/dovecot/managesieve: /usr/local/lib/libssl.so.1.0.0: no version information available (required by /usr/lib/dovecot/libdovecot-storage.so.0)
Jun 16 17:55:54 srv03 dovecot: config: Error: /usr/lib/dovecot/managesieve: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by /usr/lib/dovecot/libdovecot-storage.so.0)
I doubled checked I followed all the correct steps in the config files. Is there anything I am missing?
Thanks
Hi, on occasion I get a 500 website message when I arrive at this webpage. Just a heads up, best wishes
Thanks for the hint. I will check the logs.