Big picture

The mail server that you are about to set up uses several software components. Let me first explain briefly what the purpose of each software is:

• Debian “Buster” – the operating system
• Postfix received incoming emails from the internet and sends out outgoing emails to other mail servers. The component that speaks SMTP.
• rspamd runs sanity checks on an incoming email to determine how likely it is to be spam
• Dovecot stores emails on your hard disk, applies filters and lets your users fetch their emails using the POP3 and IMAP protocols
• Roundcube is a webmail interface so users can read their emails using a web browser, manage their email rules and change their password
• MariaDB (formerly known as MySQL) is a database that stores information about your domains, email aliases and email accounts

What happens when someone sends you an email?

Let us assume that you are responsible for the email domain example.org and someone on the internet sends an email to john@example.org. This is what happens step by step:

1. REMOTE: Hey, DNS server. I have an email for someone in the example.org domain. Can you tell me the name of the responsible mail server?
   DNS: According to the DNS zone of example.org it is mx.example.org.
2. REMOTE: Nice. Do you have an an IP address for mx.example.org?
   DNS: I have an IPv4 address here. It is 85.25.72.76.
3. REMOTE connects to that IP address on TCP port 25 which is by definition used for SMTP – the simple mail transport protocol.
   POSTFIX: Welcome. I am Postfix. Who is there? (“220 mx.example.org ESMTP Postfix”)
   REMOTE: Hi, I am remote server. (“EHLO remoteserver”)
   POSTFIX: Nice to meet you. I can offer a few features like pipelining and encryption by the way. (“STARTTLS, PIPELINING, SIZE 4000000, …”)
   REMOTE: Let’s switch to an encrypted connection. (“STARTTLS”)
   (The connection is now using TLS encryption.)
   REMOTE: I have an email from donald@duck.com here. (“MAIL FROM:<donald@duck.com>”)
   POSTFIX: I see. (“Ok”)
   REMOTE: The email is meant for john@example.org. (“RCPT TO:<john@example.org>”)
4. POSTFIX: Hey database. (Connects to TCP port 3306 on the local host to talk to MariaDB.) Could you check if example.org is one of our mail domains? (“SELECT … from virtual_domains …”)
   MariaDB: Yes, I have a domain like that.
   POSTFIX: Oh, good. And do you have a mailbox or forwarding for someone called john@example.org? (“SELECT … from virtual_aliases/virtual_users …”)
   MariaDB: Yes, there is a mailbox for that address.
   POSTIFX: Hey, remote server. The recipient looks good. (“Ok”)
   REMOTE: Roger. Then here’s the actual email. (“DATA”)
   (The remote server sends the email header and body.)
5. (POSTFIX connects to port 11332 on the local host to reach the rspamd.)
   POSTFIX: Hey, rspamd. I have a new email here. Could you give it a look for, you know, spam and stuff?
   RSPAMD: Sure. Well, there are a few minor issues. But generally the mail looks good. I suggest you accept it.
   POSTFIX: Hey, remote server. Your email is fine.
6. (Postfix uses a socket file at /var/spool/postfix/private/dovecot-lmtp to talk to Dovecot.)
   POSTFIX: Hey, Dovecot. Here is a new email for john@example.org.
   DOVECOT: Got it.
7. (Dovecot checks for additional Sieve rules and then stores the email on disk at /var/vmail/example.org/john/Maildir/INBOX)

Phew, that was a lot. I hope it helps to understand though how the different components are involved until an email gets delivered.

A lot of things are happening when your users get an email. The next situation I would like to explain is…

What happens if a user fetches their email using IMAP/POP3?

This process is way simpler. It looks like this:

1. The user usually has a mail client (also called a mail user agent) that can use the POP3 or IMAP protocol to fetch emails from the mail server. I prefer Thunderbird for example. That mail client connects to the POP3 (TCP 110) or IMAP (TCP 143) port, sends the STARTTLS command that initiates an encrypted connection and sends the user’s username (which equals the email address in our case) and their password. If possible users should use IMAP. POP3 is pretty much deprecated nowadays and lacks support for multiple folders on the server.
2. Dovecot sends a query to the MySQL database and verifies that the username and password belong to a known user. The password is not stored in plain text in the database. Instead it computes the password hash and compares it to the hash in the database. If the password is wrong then Dovecot will refuse the login.
3. We have a fixed scheme how emails are stored on disk. So if the user is called jane@example.org then Dovecot will look for email files in /var/vmail/example.org/jane/Maildir/… and send the user the requested emails.

Nowadays many users seem to prefer webmail to a mail client installed on their computers. As an email power user I honestly do not understand that trend. But who am I to judge. So let’s take a look how webmail works technically:

What happens if a user reads their email using web mail?

The Roundcube software that provides the web mail interface is basically a PHP software that is a gateway between HTML pages and a built-in IMAP client. So when a user uses their browser to connect to the web mail interface…

1. The users points the browser to the HTTPS URL of the webmail interface. Apache web server receives the connection and runs the PHP scripts of Roundcube. Roundcube shows a login form for the username and password. The user enters the username (that equals the email address) and the password and submits the login form.
2. Roundcube connects to Dovecot using IMAP and forwards the username and password to check if they are valid.
3. Dovecot treats this connection similar to a mail client connection. It queries the MariaDB database to verify the username (=email address) and password.
4. If the authentication was successful Dovecot fetches the mail files from disk and sends them through IMAP to Roundcube. Roundcube then renders the emails as HTML and the user can read it.

So you see that the web mail access also works through IMAP. The user does not realize that though.

Okay, the final scenario I would like to explain is…

What happens if the user wants to send an email to the internet?

Of course your users also want to send emails to other internet users. But they cannot send the email directly to the destination mail server. First their mail client does not know which destination server is responsible for the recipient (hint: DNS) – that functionality just is not built in. And second the user is likely assigned a dynamic IP address which is blocked by most mail servers because they tend to get abused by infected Windows PCs that send out spam. So the correct way to send an email to the internet is through your mail server. This is called relaying because your mail server acts as a relay. In this example your user wants to send an email to fred@example.net.

1. The user writes the email in their mail client and clicks on “Send”. The mail client establishes an SMTP connection to Postfix. To make sure that the user is allowed to send email through your system it requires a username and password along with the email. This information is sent in an encrypted way.
2. Postfix could now check the password in the database directly. But as Dovecot already knows how to handle authentication it is easier to ask Dovecot to verify the username and password. (SMTP authentication in Postfix is surprisingly ugly.)
3. Dovecot now sends a query to the MariaDB database to check if the username and password (hash) are correct and tells Postfix the result.
4. Postix knows now that it is authorized to send the email on behalf of the user. It tells the user that it successfully accepted the email. The email is put into Postfix’s mail queue for further processing. Postfix will now query a DNS (name server) to determine the responsible destination mail server. As the recipient has an “…@example.net” address it checks the MX record of the “example.net” domain and then gets the respective IP address.
5. Postfix now knows which mail server to send the email to. It opens an SMTP connection and delivers the email.