Creating a TLS encryption key and certificate

(If you are unfamiliar with the abbreviation “TLS“: it is the successor to SSL but works one the same principle.)

The internet is the best invention since sliced bread but it has become an evil place more than ever. We are dealing with script kiddies, out-of-control secret services and organized crime on a large scale. It is completely out of the question to run any unencrypted services over the internet. I even consider any password immediately wasted that went through the internet in plain text. So we need to create an encryption key and a certificate for Postfix (SMTP), Dovecot (IMAP/POP3) and Roundcube (Webmail/HTTPS).

You may argue that using encrypted connections between a users’s computer and the mail server is futile. After all the email is forwarded to other mail servers in plain text. While this may be true for some mail servers most administrators strive to enable TLS for the server-to-server communication, too. You cannot rely on it for confidential correspondence but it’s better than nothing. Fun fact: not very long ago a couple of large german freemail providers founded an “email made in Germany” alliance. Guess what cool thing they did to make the world a better place… they enabled TLS. 🙂

Caveat: do not use different certificates for Postfix and Dovecot. At least the MacOS mail client would refuse your connection with confusing error messages.

Why makes mail clients trust a certificate?

Just a few certificate authorities that your browser trusts by default. Do you?

Just like with your favorite web browser there is a list of “trusted” certificate authorities in the world. Yes, your browser trusts companies you probably have never heard of. And so does your mail client. Decades ago companies managed to convince browser developers to make their browser trust all certificates that were cryptographically signed by them. Those certificates were technically the same as self-signed certificates. They just had a signature that was created by a computer of a certain company. Most of the time there isn’t even a human checking the signing request. I consider today’s certification system broken.

In the previous tutorial for Debian Jessie I had a lengthy comparison of self-signed certificates, company PKIs, LetsEncrypt and paid certificates. Let’s cut it short – we will use LetsEncrypt. There is no reason to pay the certificate mafia money any more. Why do I consider them mafia-like? Because it is plain wrong to exchange money for trust. And the recent history of awkward failures shows that they deserve no trust. Even my preferred good old certificate authority StartSSL (because they issued certificates for free) has failed so badly that all major browsers now consider their certificates untrustworthy. Just recently they even shut down their business.

LetsEncrypt is a not-for-profit project run by a Californian corporation called ISRG (Internet Security Research Group). They managed to set up an infrastructure that finally allows everyone in the world to get trusted certificates for free. Their authority’s certificate is nowadays installed as trusted by most web browsers and other software – like mail clients. If your boss asks… their certificates are technically just as secure as a self-signed certificates. And, no, there is no reason to ever pay for a certificate again. Why do others still do that? Likely because they are victims of marketing and FUD.

The certificate will be used in three places:

  1. the webmail interface (driven by Apache)
  2. Postfix (for authentication during SMTP communication with your users)
  3. Dovecot (for authentication during IMAP communication)

Preparing the Apache web server for HTTP

Let’s start with the Apache. Your server may serve multiple name-based virtual hosts. That just means that your web server can distinguish which files it is supposed to send to the client depending on the host name used in the URL. As an example I will assume that you want to offer a host name webmail.example.org to your users. Of course your server will have another name in a domain that you control. I will use that example throughout the tutorial though and keep that name printed in bold letters to remind you that you have to use your own host name everywhere.

First you need a web root directory for that host name:

mkdir /var/www/webmail.example.org

Next you will have to create a virtual host configuration file. Apache on Debian uses a neat system to manage virtual hosts:

  • /etc/apache2/sites-available/*.conf contains two configuration files by default. “000-default.conf” is a HTTP virtual host and “default-ssl.conf” is a HTTPS virtual host configuration. You can create arbitrary files here. Apache will not consider them automatically.
  • /etc/apache2/sites-enabled/*.conf contains symbolic links (“symlinks”) pointing to configuration files in the /etc/apache2/sites-available directory. Only *.conf links in this directory will be loaded by Apache.

This system allows you to enable and disable virtual hosts without having to destroy any configuration. Debian ships with the “a2ensite” (short for “apache2 enable site”) and “a2dissite” commands. In addition to some sanity checks those commands essentially create or remove symlinks between “sites-available” and “sites-enabled”.

(Note: Configuration files must have a “.conf” suffix or else they will get ignored.)

You may remove the default symlinks in /etc/apache2/sites-enabled/* unless you use them already.

Create a new virtual host configuration file /etc/apache2/sites-available/webmail.example.org-http.conf and make it contain:

<VirtualHost *:80>
  ServerName webmail.example.org
  DocumentRoot /var/www/webmail.example.org
</VirtualHost>

The simple configuration makes Apache handle HTTP requests (on the standard TCP port 80) if a certain line in the request from the browser reads “Host: webmail.example.org”. So the browser actually tells your Apache web server which server name it is looking for. That allows for multiple web sites on a single IP address. (By the way: this works on HTTPS, too, nowadays, thanks to Server Name Indication.)

Enable the site:

a2ensite webmail.example.org-http

You will be told:

To activate the new configuration, you need to run:
 systemctl reload apache2

Do that.

Let’s check if the configuration works. Put a test file into your web root directory:

echo "Just a test" > /var/www/webmail.example.org/test

Now when you open the URL http://webmail.example.org/test in your browser you should see the text “Just a test”.

This is enough setup to make LetsEncrypt issue a certificate for you.

Getting a LetsEncrypt certificate

Debian Stretch contains the most common tool to manage LetsEncrypt certificates – certbot. Install it:

apt install certbot

To get a certificate for your domain run:

certbot certonly --webroot --webroot-path /var/www/webmail.example.org -d webmail.example.com

The first time you do that you will get asked for your email address so LetsEncrypt can send you reminders if your certificate would expire. You will also have to agree to their terms of service.

If everything worked well you should get output like:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for webmail.example.org
Using the webroot path /var/www/webmail.example.org for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0049_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0049_csr-certbot.pem

IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
  /etc/letsencrypt/live/webmail.example.org/fullchain.pem. Your
  cert will expire on 2018-04-01. To obtain a new or tweaked version
  of this certificate in the future, simply run certbot again. To
  non-interactively renew *all* of your certificates, run "certbot
  renew"
- If you like Certbot, please consider supporting our work by:

  Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
  Donating to EFF: https://eff.org/donate-le

Do you wonder what actually happened? Well, the Certbot created the private/secret key (that only your server must know). It then placed a certain file into /var/www/webmail.example.org/.well-known/… and had it verified by the LetsEncrypt service that visited your web site. That was proof enough that you own the domain (because you are in control of the DNS zone) and that you have access to the web server.

Apache and HTTPS

Unencrypted HTTP communication is not an option any more nowadays. And now that you have a certificate let’s add a virtual host that can handle HTTPS.

Create a new file /etc/apache2/sites-available/webmail.example.org-https.conf containing:

<VirtualHost *:443>
 ServerName webmail.example.org
 DocumentRoot /var/www/webmail.example.org
 SSLEngine on
 SSLCertificateFile /etc/letsencrypt/live/webmail.example.org/fullchain.pem
 SSLCertificateKeyFile /etc/letsencrypt/live/webmail.example.org/privkey.pem
</VirtualHost>

This virtual host configuration looks suspiciously similar to the HTTP virtual host above. It just listens on port 443 (standard port for HTTPS) instead of port 80. It enables the “SSLEngine” that handles encryption and gets information about the certificate for your web server (that is shown to your users) and the private key (that the web servers uses to decrypt the user’s communication).

Enable the SSL module in Apache:

a2enmod ssl

Then enable the virtual host for HTTPS:

a2ensite webmail.example.org-https

And reload the configuration again:

systemctl reload apache2

Now when you point your web browser to https://webmail.example.org your browser should tell you that it trusts the web site’s certificate:

(Yes, sorry, this is not webmail.example.org. But I do not own the example.org domain and thus cannot get a valid certificate for it. This is my own site.)

So should you keep the HTTP virtual host? Yes. Use it to redirect your users to the HTTPS site if they forget to put “https://” in front of the URL. And HTTP is always required for LetsEncrypt. So what is the solution?

Automatic renewal

Getting a certificate is only half the battle. LetsEncrypt’s certificates expire after 90 days. So renewing certificates automatically and in time is vital. The certbot package adds an automatic cron job in /etc/cron.d/certbot for that purpose. It is run twice a day and after a random delay (so that the LetsEncrypt service is not getting too many requests at the same time) checks if certificates are due for renewal. That is done by calling “certbot -q renew”. The “-q” stands for “quiet” and prevents that you get two emails per day telling you what the certbot did.

There is one puzzle piece missing though. Even when the renewal worked it will only update the certificate files. But the software components – Postfix, Dovecot and Apache – will not notice the change. So we need to add a so called post-hook to certbot that triggers a restart of all processes thereafter.

Create a simple shell script in /usr/local/bin/certbot-post-hook that contains:

#!/bin/sh
service postfix restart
service dovecot restart
service apache2 restart

Make that script executable:

chmod u+x /usr/local/bin/certbot-post-hook

Run it as a test:

/usr/local/bin/certbot-post-hook

It will not output anything. It should just run without error and restart all three services that will use the certificate.

Now edit the automatic cron job in /etc/cron.d/certbot and add the highlighted part:

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q --post-hook /usr/local/bin/certbot-post-hook renew

To be sure that the renewal will work you can simulate the process by running

certbot --dry-run --post-hook /usr/local/bin/certbot-post-hook renew

Well done. You have implemented LetsEncrypt for all your services now. Let’s go on.

24 thoughts on “Creating a TLS encryption key and certificate

  • 2018-01-21 at 19:43
    Permalink

    a2enmod webmail.example.org-https -> a2ensite webmail.example.org-https
    a2enmod ssl

    Reply
    • 2018-01-21 at 19:53
      Permalink

      Thanks, fixed.

      Reply
      • 2018-01-23 at 12:23
        Permalink

        I would probably also run a2enmod before a2ensite, as you otherwise get

        Reloading Apache httpd web server: apache2 failed!
        The apache2 configtest failed. Not doing anything. … (warning).
        Output of config test was:
        AH00526: Syntax error on line 4 of /etc/apache2/sites-enabled/webmail.example.org-https.conf:
        Invalid command ‘SSLEngine’, perhaps misspelled or defined by a module not included in the server configuration
        Action ‘configtest’ failed.

        Reply
        • 2018-02-01 at 08:24
          Permalink

          Right, thanks. Just changed the order.

          Reply
  • 2018-01-21 at 21:24
    Permalink

    Thumbs up for using the “http-01” challenge 🙂

    Reply
  • 2018-01-22 at 10:35
    Permalink

    Why not to use –post-hook option of certbot ?

    Reply
    • 2018-01-22 at 10:36
      Permalink

      I mean in the cron job.

      Reply
      • 2018-01-22 at 11:09
        Permalink

        I mentioned that. Or how do you mean?

        Reply
        • 2018-01-22 at 13:08
          Permalink

          Ah, sorry. I haven’t noticed that the new certbot creates automatic cron job.

          Reply
  • 2018-01-22 at 17:29
    Permalink

    I think you also want to request `imap.example.org` in this part of the guide, as this imap address is used later in the guide (and “required” for external mail clients to avoid SSL warnings).

    In addition `smtp.example.org` might be worth adding as well (as commonly used).

    Reply
  • 2018-01-23 at 11:22
    Permalink

    I think there’s a step I’m missing… going to example.org/test yields “this is just a test”, but the site cannot be found for “webmail.example.org/test”. Do I need to make some adjustments to my domain settings first?

    Reply
    • 2018-01-23 at 11:33
      Permalink

      I use webmail.example.org in my examples everywhere. Do you use both “webmail.example.org” and “example.org” in your configuration?

      Reply
      • 2018-01-23 at 11:49
        Permalink

        My configuration is as in the first “Preparing the Apache web server for HTTP” section:

        ServerName webmail.example.org
        DocumentRoot /var/www/webmail.example.org

        I’ve removed the other symlinks in /etc/apache2/sites-enabled as you mentioned (only 000-default.conf was symlinked there).

        example.org is replaced by my domain name. I’ve added “webmail.example.org” as a subdomain in the interface of my hoster (1&1), so now I can reach /var/www/webmail.example.org via “webmail.example.org”. However I can still reach it from example.org as well?

        Thanks for your quick reply!

        Reply
        • 2018-01-23 at 11:58
          Permalink

          Enabling 000-default.conf with “a2ensite 000-default” resulted in “example.org” displaying something else (the welcome screen). I’m a bit mystified why in the absence of this symlink it would simply show what’s on the “webmail.example.org” subdomain…

          Reply
          • 2018-01-23 at 12:44
            Permalink

            And to solve my own question: from the linked Apache document

            If no matching ServerName or ServerAlias is found in the set of virtual hosts containing the most specific matching IP address and port combination, then the first listed virtual host that matches that will be used.

  • 2018-01-23 at 12:38
    Permalink

    The certbot command seems to be stuck on running the post-hook

    aving debug log to /var/log/letsencrypt/letsencrypt.log

    ——————————————————————————-
    Processing /etc/letsencrypt/renewal/webmail.example.org.conf
    ——————————————————————————-
    Cert not due for renewal, but simulating renewal for dry run
    Renewing an existing certificate
    Performing the following challenges:
    http-01 challenge for webmail.example.org
    Waiting for verification…
    Cleaning up challenges
    Generating key (2048 bits): /etc/letsencrypt/keys/0006_key-certbot.pem
    Creating CSR: /etc/letsencrypt/csr/0006_csr-certbot.pem
    ** DRY RUN: simulating ‘certbot renew’ close to cert expiry
    ** (The test certificates below have not been saved.)

    Congratulations, all renewals succeeded. The following certs have been renewed:
    /etc/letsencrypt/live/webmail.example.org/fullchain.pem (success)
    ** DRY RUN: simulating ‘certbot renew’ close to cert expiry
    ** (The test certificates above have not been saved.)
    Running post-hook command: /usr/local/bin/certbot-post-hook

    However, running /usr/local/bin/certbot-post-hook works as expected

    /usr/local/bin# /usr/local/bin/certbot-post-hook
    Stopping Postfix Mail Transport Agent: postfix.
    Starting Postfix Mail Transport Agent: postfix.
    Restarting IMAP/POP3 mail server: dovecot.
    Restarting Apache httpd web server: apache2.

    Reply
  • 2018-01-24 at 05:12
    Permalink

    Thanks for the great guide, I’ve been using it for years, without any problems.

    Just a small hint: Since Debian 9 uses systemd, why not use “systemctl restart postfix” (and dovecot and apache2) instead of “service postfix restart”? I don’t actually like systemd, but it’s the new standard in Debian, and you already use systemctl to reload the apache2 configuration.

    Reply
    • 2018-01-24 at 12:28
      Permalink

      I believe the guide uses “systemctl”, but only in this section of the guide. Since debian still supports sysvinit, I’d rather argue to replace these instances with “service” which works with both init systems while systemctl obviously only does with systemd.

      Reply
    • 2018-02-13 at 13:04
      Permalink

      Forget that!! The validation cant find the files it is hunting for, but its not index.html.. I am now guessing that the script doesn’t have the permissions to write to the www/mydomain folder.. as its returning a 404 in the validation, and there are no files other than test in there.. yet a dns must be active for me to see http://mydomain/test

      Reply
  • 2018-02-14 at 22:31
    Permalink

    Great tutorial! I used it few times (making 4 or 5 mail servers) in past 4 years.

    I guess, I should put some feedback.
    Try replace certbot with acme.sh, it use less resources and never ask about root privilages

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close