Testing IMAP access
You have already finished the configuration of Dovecot so fetching emails using the IMAP protocol will already work. Let’s give it a try using a simple but powerful IMAP client: mutt.
mutt -f imaps://john@example.org@webmail.example.org
The connection URL may look a little confusing because of the two “@” characters. Usually mutt expects imaps://user@server. And as we use the email address as the “user” part you get this look. You should get prompted for the password which is “summersun”. If you get any certificate warnings then check if you used the correct server name to connect to and if you finished the certificate/LetsEncrypt part earlier in this guide.
After logging in you will see an empty inbox:
Very good – so IMAP works.
Enabling Roundcube
Not everyone is a hardcore user who uses an application like Thunderbird, Evolution, kmail or A**le Mail to manage their emails. It surprises how many users are happy with a web interface to their emails. Okay, if you like. So let’s provide a webmail interface. A very (if not the most) common software for that purpose is Roundcube.
When you installed the Roundcube package at the very beginning of this guide it created a /etc/apache2/conf-available/roundcube.conf file that was linked to /etc/apache2/conf-enabled/. So the general Apache configuration is already ready. All you have to do is link your virtual host for webmail to the directory that contains the Roundcube installation.
The configuration file suggests this Apache configuration line:
Alias /roundcube /var/lib/roundcube
That would make RoundCube accessible under https://webmail.example.org/roundcube. However I would rather omit the “/roundcube”. That is easily done by changing the DocumentRoot in your /etc/apache2/sites-enabled/webmail.example.org-https.conf file to:
DocumentRoot /var/lib/roundcube
Reload the Apache configuration…
service apache2 reload
…and you can now access the webmail interface at https://webmail.example.org/
You could already log in by entering “localhost” as a server. But let’s improve the configuration a litte. The “Server” is always “localhost”. So edit the /etc/roundcube/config.inc.php file and set:
$config['default_host'] = 'localhost';
Now when you reload the login form the “Server” field should be gone.
Try to login using our example user “john@example.org” with his password “summersun”. That will create an IMAP connection to “localhost” (on the server) and show you John’s emails:
The inbox is still empty but you logged in successfully. Nice.
Set up HTTP to HTTPS redirection
For those users who forget to type “https” instead of “http” let us also set up an automatic redirection so that they will be forwarded to the secure URL. Edit the /etc/apache2/sites-enabled/webmail.example.org-http.conf file and insert this anywhere within the VirtualHost section:
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/.*
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [QSA,L,R=301]
Experienced sysadmins may wonder why I didn’t just use the simpler form of a redirection like…
# Don't use this! Redirect permanent / https://webmail.example.org/
The reason is that directing all requests from HTTP to HTTPS would break LetsEncrypt’s auto-renewal. If a certificate is due for renewal the “certbot” will put a file into the .well-known/acme-challenge subdirectory of your virtual hosts’s DocumentRoot to verify your server. If you redirected those requests to the HTTPS vhost then LetsEncrypt’s verification would fail. Use the RewriteCond(ition) we can exclude that path from the redirection.
Also make sure that the redirection module is enabled in Apache:
a2enmod rewrite
To make your change work:
service apache2 reload
Now when you open http://webmail.example.org/ in your browser you should get redirected to the HTTPS URL automatically.
HTTP? HTTPS? Redirections? Confused?
Let’s reiterate how the two virtual hosts are supposed to work:
- HTTP (/etc/apache2/sites-enabled/webmail.example.org-http.conf)
/.well-known/acme-challenge/ -> serve from /var/www/webmail.example.org for verification of Let’s Encrypt challenges
anything else -> redirect to HTTPS so that users always use Roundcube in a safe manner - HTTPS (/etc/apache2/sites-enabled/webmail.example.org-https.conf)
/ -> serve Roundcube from /var/lib/roundcube
I hope that makes things a bit clearer. Pay attention to the names of the two different configuration files.
Plugins
Further down in /etc/roundcube/config.inc.php there is a list of plugins that Roundcube loads. Add at least the “managesieve” and “password” plugins so that the setting looks like this:
$config['plugins'] = array( 'managesieve', 'password', );
There are for more plugins available. Take a look at the /var/lib/roundcube/plugins/ directory. You can add any of them. Custom configuration of the plugins goes into the /etc/roundcube/plugins/* directories.
Next an optional setting. The default session lifetime in Roundcube is 10 minutes. That means if a user is not using the webmail interface for more than 10 minutes they will be logged out. I found that annoying and increased that timeout to one hour. To do that at the end of the config file add:
$config['session_lifetime'] = 60;
And if you would like to change the default logo of Roundcube that can be done by setting:
$config['skin_logo'] = './ispmail-logo.png';
You just need to copy that image file by that name to /var/lib/roundcube/ispmail-logo.png. The logo should be 177×49 pixels large. Feel free to take this nifty ISPmail logo I crafted. 🙂
If the logo appears to be broken then make sure that the permissions are correct:
chmod a+r /var/lib/roundcube/ispmail-logo.png
When you reload the login form in the browser it will then look like this instead:
Configuring the managesieve plugin
The “managesieve” plugin will allow your users to manage automatic rules to manage their email. These rules are stored on the server and will be run automatically. You need to configure this plugin though. A default configuration can be found at /usr/share/roundcube/plugins/managesieve/config.inc.php.dist on your system. Copy it to the location where Roundcube will look for it:
cp /usr/share/roundcube/plugins/managesieve/config.inc.php.dist \ /etc/roundcube/plugins/managesieve/config.inc.php
No further changes are required.
Configuring the password plugin
If you want to allow your users to change their passwords then use the password plugin. Copy the default configuration file /usr/share/roundcube/plugins/password/config.inc.php to the right place:
cp /usr/share/roundcube/plugins/password/config.inc.php.dist \ /etc/roundcube/plugins/password/config.inc.php
The configuration file at /etc/roundcube/plugins/password/config.inc.php requires a couple of changes though. We need to tell it how our database works and what to do when a user wants to change their password. The first setting deals with the minimal length of the password. I recommend to enforce at least 10 characters. In fact the complexity of the password is not that important. Consider XKCD as food for thought on password security. So set:
$config['password_minimum_length'] = 10;
We should allow the user to use the old password as the new password. It may sound stupid but as we are upgrading the password scheme from the weak unsalted MD5 to the better SHA2 algorithm we should allow that:
$config['password_force_save'] = true;
Next the password plugin needs to know how to access your database:
$config['password_db_dsn'] = 'mysql://mailadmin:ChangeMe@localhost/mailserver';
Replace “ChangeMe” by the randomly generated password you created earlier for the “mailadmin” database user.
Now tell the plugin how to actually write the new password hash into the database:
$config['password_query'] = "UPDATE virtual_users SET password=CONCAT('{SHA256-CRYPT}', ENCRYPT (%p, CONCAT('$5$', SUBSTRING(SHA(RAND()), -16)))) WHERE email=%u;";
Whoa, that looks weird, doesn’t it? That SQL query generates a password hash from these parts:
- The string “{SHA256-CRYPT}”. It tells Dovecot explicitly that this password is a salted SHA256 hash. You may have different kinds of encrypted passwords in your database so this makes it clear.
- The actual encrypted password that is generated using your operating system’s crypt() function (MariaDB calls it when you use the ENCRYPT SQL function) using…
- The new plaintext password (Roundcube replaces “%p” by it).
- “$5$” – that stands for using the SHA-256 algorithm. Check “man crypt” to see which algorithms are supported by crypt().
- And SUBSTRING(SHA(RAND()), -16) is used as a salt. A salt is just some additional randomness that makes it much much harder to reverse-engineer the actual password from the encrypted string.
- And of course we just want to change the password of the current email user. So “WHERE email=%u” makes sure we choose the right row in the database. Roundcube replaces “%u” with the user name – which is the same as the email address in our case.
(I used to recommend just using “dovecot pw -s SHA256-CRYPT” to generate passwords. You can do that, too. But in fact MySQL is capable of generating salted SHA256 hashes without calling any shell command. Thanks for the hint, Martin.)
Changing the DocumentRoot to /var/lib/roundcube results in 403 – forbidden
also
/etc/apache2/sites-enabled/webmail.workaround.org-https.conf should have been probably example.org
the 403 forbidden was caused by the apache being installed later than roundcube so some hooks weren’t activated during the installation dpkg-reconfigure fixed it but some of the configurations were re-written so the /etc/roundcube/config.inc.php needs to be revisited
Module RewriteEngine was not available by default.
Rewrite module can be enabled by entering command “a2enmod rewrite” followed by “systemctl restart apache2” to reload apache2.
also in my case there was a mod enabled by default that was conflicting with rewrite so it had to be disabled with a2dismod
Thanks for the hint. I have added “a2enmod rewrite” to the documentation.
What was the other mod that caused a conflict?
Thx. I’m not seeing any conflicting mods, so I think all is fine. Install on fresh system works flawlessly.
I don’t quite get how certbot works with changing the DocumentRoot; in https://workaround.org/ispmail/stretch/letsencrypt-certificate we pointed it to /var/www/webmail.example.org, here we use from now on /var/lib/roundcube. Does it find out by itself?
It’s a bit hidden in the details. There are two virtual hosts. One for HTTP and one for HTTPS. Basically they work like this:
HTTP:
/ -> redirect to HTTPS
/.well-known/acme-challenge/ -> serve from /var/www/webmail.example.org
HTTPS:
/ -> serve Roundcube from /var/lib/roundcube
Let me know if that doesn’t work for you. I admit that you have to follow the article very closely because sometimes I refer to /etc/apache2/sites-enabled/foo-http.conf and at other times to …/foo-https.conf
Ah, clever! I was wondering why the DocumentRoot was only changed in the https file. Thanks for clarifying.
I think a single line
RedirectMatch 301 (^!/\.well-known/acme-challenge/).* https://www.example.org
has the same result. Correct?
Sorry for misleading you. It does NOT work, I was wrong!
The “managesieve” plugin doesn’t work out of the box in roundcube (Error log says: PHP Fatal error: Uncaught Error: Class ‘Net_Sieve’ not found in /usr/share/roundcube/plugins/managesieve/lib/Roundcube/rcube_sieve.php:65).
It looks like there is an missing dependency as you can read here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798307
So you have to install the package “php-net-sieve” manual: “apt-get install php-net-sieve”
After that it worked for me. Perhaps you could add this somewhere in the tutorial.
Thanks for the hint. I’m surprised that the bug was closed in 2015 but still existed at the point of release in 2017. But I have verified it and you are right. It’s added to the “apt install roundcube…” statement on the page about installing packages.
Apparently now you could simplify the password update task, by modifying:
$config[‘password_algorithm’] = ‘sha256-crypt’;
$config[‘password_algorithm_prefix’] = ‘{SHA256-CRYPT}’;
$config[‘password_dovecotpw’] = ‘/usr/local/sbin/doveadm pw’; // for dovecot-2.x – i don’t know if really needed
$config[‘password_query’] = ‘UPDATE virtual_users SET password = %P WHERE email = %u;’;
As you can see, it simplifies a lot the query needed to update the password. As a matter of fact, in my case, your example did not work at all, it just gave an error in log:
PHP Parse error: syntax error, unexpected ”, ENCRYPT (%p, CONCAT(” (T_CONSTANT_ENCAPSED_STRING)
I ended up back here while googling the solution to that same error. I did manage to finally get it done your way, but I had to use back ticks for all the quotes; for reasons I don’t understand.
I had the same issue; what reported on tutorial works fine.
PLEASE be careful about the quotes!! On sample configuration file a single quote is present (‘) but on the tutorial a DOUBLE quotes is indicated (“) !
This was my issue! After fixing this everything worked fine.
The “managesieve” password will allow your users to manage automatic rules to manage their email.
I suppose you wanted to say “The “managesieve” plugin…” ?
I knew it was something with “p”. 🙂 Thanks for the hint.
Hi Christoph. Can you also include the rspamd webinterface in sites available? I think it is very usefull to fight spam.
Since /usr/share/roundcube/plugins/password/config.inc.php contains the password for mailadmin but everyone can read it, shouldn’t it be readable by root and postfix, too, like mysql-*.cf flies?
Should the $config[‘des_key’] be changed? The comments in /etc/roundcube/config.inc.php suggest so…
If it helps anyone, I had trouble with performing the test in Mutt, it would never get to the password prompt, it would just fail to connect spit out “GPGME: CMS protocol not available”.
Installing the gpgsm package resolved that.
Also I found that Cloudflare caused major headaches with SSL certificates, it tripped me up as I could access https traffic within the network, but when accessing https://webmail.example.org it just forwarded me to http://webmail.example.org and gave me a file listing of test.txt.
Pausing the website on Cloudflare ‘resolved’ that – for the time being at least.
Hi Christoph!
Thanks a lot for this very detailed tutorial.
I also had trouble with the mutt test, I believe I have done everything ok but the mutt test doesn’t work, I tried installing the gpgsm packeage but that didn’t solve it, on the mutt records I have this after trying the mutt command locally from the server:
[2018-12-28 21:10:43] Reading configuration file ‘/etc/Muttrc.d/compressed-folders.rc’.
[2018-12-28 21:10:43] Reading configuration file ‘/etc/Muttrc.d/gpg.rc’.
[2018-12-28 21:10:43] Reading configuration file ‘/etc/Muttrc.d/notmuch.rc’.
[2018-12-28 21:10:43] Reading configuration file ‘/etc/Muttrc.d/smime.rc’.
[2018-12-28 21:10:43] Using default IMAP port 143
[2018-12-28 21:10:43] Using default IMAPS port 993
[2018-12-28 21:10:43] Reading imaps://john@example.or@webmail.myserver.com/…
[2018-12-28 21:10:43] Looking up webmail.myserver.com…
[2018-12-28 21:10:43] Connecting to webmail.myserver.com…
[2018-12-28 21:10:43] gnutls_handshake: The TLS connection was non-properly terminated.
[2018-12-28 21:10:45] Connected to webmail.myserver.com:993 on fd=4
[2018-12-28 21:10:45] Closing connection to webmail.myserver.com…
[2018-12-28 21:10:45] 4> a0000 LOGOUT^M
[2018-12-28 21:10:45] Error: no TLS socket open
[2018-12-28 21:10:47] mutt_socket_write: error writing (No such file or directory), closing socket
[2018-12-28 21:10:47] imap_cmd_step: grew buffer to 512 bytes
[2018-12-28 21:10:47] mutt_socket_readchar: attempt to read from closed connection.
[2018-12-28 21:10:47] imap_cmd_step: Error reading server response.
[2018-12-28 21:10:47] mutt_socket_close: Attempt to close closed connection.
Where myserver is of course the server I am setting.
If I go to https://webmail.myserver.com I can see the test webpage successfully (I created one in there)
This is a great tutorial! Email servers are usually a bear to setup. Thanks for putting the time into creating it.
I’m having one issue, my inbox has INBOX.Junk and INBOX.Trash instead of nested folders.
Christoph, I believe your assumtion that a simple https-redirect will break Certbot’s http challenge is incorrect.
1. from personal experience. I have been redirecting all traffic to https for years through a slightly more complicated “redirectmatch ^/(.*) https://secure.example.org/$1” , and certbot works just fine!
2.As I understand it from certbot’s documentation for the http-challenge, Certbot actually temporarily modifies your virtualhost-configuration to perform the verification! You can even successfully request certificates for domains that your apache isn’t configured to host at all, as long as apache is running and that domain’s DNS server is set up to send incoming traffic for that domain your way.
really great tutorial. I only have a problem with logging into mutt, the password seams not to work and i am not really sure were “sumersun” comes from. i would highly appreciate if some buddy could explain what i am doing wrong. or what the way is to change the pw.
cheers
It’s at the end of the “Preparing the database” section.
dovecot pw -s SHA256-CRYPT
All thanks !
I can’t install Carddav plugin for roundcube on Debian…
If someone have an idea :
https://github.com/blind-coder/rcmcarddav/issues/260#issuecomment-491851860
I must be missing something obvious, but my redirect from http to https isn’t working at all. My https site works fine, but using http doesn’t boot me over. My rewrite module is enabled, and I’ve reloaded (and restarted!) Apache. Any ideas out there?
My etc/apache2/sites-enabled/webmail.mysite.com-http.conf file looks like this:
ServerName webmail.mysite.com
DocumentRoot /var/www/webmail.mysite.com
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/.*
RewriteRule ^https://%{HTTP_HOST}%{REQUEST_URI} [QSA,L,R=301]
and my etc/apache2/sites-enabled/webmail.mysite.com-https.conf file looks like this:
ServerName webmail.mysite.com
DocumentRoot /var/lib/roundcube
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/webmail.mysite.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/webmail.mysite.com/privkey.pem
Include /etc/phpmyadmin/apache.conf
[code]~# $config[‘password_db_dsn’] = ‘mysql://mailadmin:{mailadminpassword}@localhost/mailserver’; # Copied and pasted from the tutorial.
bash: [password_db_dsn]: command not found[/code] # Server response
Is there an alternative command to achieve this objective? Or is there a typo in your version, Christoph?
Doh. Should have been a file entry, not a command. Corrected and working now.
As I wanted to use BCRYPT instead of an insecure hash like SHA256, I used this settings for the password add-on.
$config[‘password_algorithm’] = ‘dovecot’;
$config[‘password_algorithm_prefix’] = ‘{BLF-CRYPT}’;
$config[‘password_dovecotpw’] = ‘/usr/bin/doveadm pw -r 13’;
$config[‘password_dovecotpw_method’] = ‘BLF-CRYPT’;
$config[‘password_query’] = “UPDATE virtual_users SET password=%D WHERE email=%u LIMIT 1”;
i had upgraded roundcube from version 1.2.3 (the default version from debian repo) to 1.3.10. But I was hit with the authentication code failure when i tried to login john@example.org. I found the solution is to edit roundcube config.inc.php and amend it to $config[‘smtp_server’] = ‘tls://mail.example.org’ and “maybe” adding $config[‘smtp_auth_type’] = ‘PLAIN’ may help.
Hi all, it is safe to upgrade roundcube from the original stretch repo to version 1.4.3 temporary changing the main repository from:
deb http://ftp.debian.org/debian/ stretch main non-free contrib
to
deb http://ftp.debian.org/debian/ bullseye main non-free contrib
and execute:
apt install -y roundcube roundcube-plugins roundcube-plugins-extra roundcube-mysql
and after the update, restore the repository to stretch?
Or is better to update the files one by one as explained on the official roundcube git?
Thank you