The big picture

The mail server that you are about to set up uses several software components. Let me first explain briefly what the purpose of each software is:

• Debian “Stretch” – the operating system
• Postfix receives incoming emails from the internet and forwards emails to other mail servers
• rspamd checks every incoming email for spam/malware and either rejects it or puts it into the user’s Junk folder
• Dovecot stores emails on your hard disk, applies filters and lets your users fetch their emails using the POP3 and IMAP protocols
• Roundcube is a webmail interface so users can read their emails using a web browser, manage their email rules and change their password
• MariaDB is a database that stores information about your domains, email aliases and email accounts

What happens when someone sends you an email?

Let us assume that you are responsible for the email domain example.org and someone on the internet sends an email to john@example.org. This is what happens step by step:

1. The remote mail server asks its nearest DNS server for the MX (Mail eXchanger) server of the recipient’s domain. The domain here is example.org so it asks for the “MX” record of the example.org domain. The DNS server replies that the host name of the reponsible destination mail server is mx15.example.org. In case there is no MX record it will try to use the A record as a fallback. The DNS server may even respond with multiple records having different priorities. The remote server will try all entries in order. This allows for load balancing or fallback mail exchangers in large environments.
2. The mail server asks the DNS server again to find out the IP address of the mail server mx15.example.org. The DNS server responds with the IP address.
3. Now the mail server establishes a TCP connection on port 25 (which is by definition used for SMTP – the simple mail transport protocol) to that IP address and reaches the Postfix daemon on your mail server. It sends Postfix the sender’s and receiver’s email address and the actual email content. The SMTP communication is held open while the next step occurs – the spam/malware check using rspamd. If rspamd is certain that the email is spam/malware then the still-open SMTP dialog is cancelled with an appropriate error message “5.7.1 Spam message rejected”. rspamd can also make Postfix accept the email and flag it as spam. Or it can tell Postfix to greylist the sending server (making it try again in a few minutes).
4. Postfix talks to the rspamd process and passes the email as well as the sending IP address. rspamd runs a large number of checks and computes a numeric score. A low score means that the email is probably legit. A high score means it likely is spam or even contains malware. Depending on the rspamd configuration it will tell Postfix to accept the email, delay the email (greylisting) or reject the email with an error message.
5. If rspamd has accepted the email then Postfix will run further checks – called restrictions. Is your mail server even responsible for the domain of the recipient? Does the email address exist in that domain? If the recipient is not valid then it will reject the email again – still during the SMTP dialog.
6. Now Postfix has finally accepted the email and closes the SMTP connection to the sending server. It then forwards the email to the Dovecot process on your server. That communication uses LMTP – a lightweight version of SMTP. Postfix then removes the email from its queue.
7. Dovecot checks if the recipient has defined any sieve rules. An example: “if the header contains ‘List-Id: rspamd-user’ then ‘move to folder mailinglists/rspamd-user’“. In addition to the user’s sieve rules it runs global sieve rules. We use them to move email that rspamd tagged as spam into a seperate “Junk” folder.
8. Dovecot saves the email into a file of the recipient user’s mail directory. Example: “/var/vmail/example.com/john/Maildir/…”

Phew, that was a lot of information. Don’t try to remember all that. Just come back when you seem to get lost between software components later. The further you get the more sense it will make.

A lot of things are happening when your users get an email. The next situation I would like to explain is…

What happens when a user fetches their email using IMAP/POP3?

This process is way simpler. It looks like this:

1. The user usually has a mail client (also called a mail user agent) that can speak the POP3 or IMAP protocol to fetch emails from the mail server. There are tons of mail cilents out there. I prefer Thunderbird or Evolution on Linux for example. The mail client connects to the POP3 (TCP 110) or IMAP (TCP 143) port, sends the STARTTLS command to initiate an encrypted connection and sends the user’s username (which equals the email address in our case) and the password. If possible users should use IMAP which always leaves the emails on the server and which also supports multiple folders. POP3 is very limited and is barely used any more.
2. Dovecot does a query to the MariaDB database and verifies that the username and password belong to a known user. The password is not stored in plain text in the database. Dovecot uses salted hashes which can be used to verify a password but not make it readable. That is a modern and safe way to store passwords. If the password is wrong then Dovecot will refuse the login.
3. We use a simple scheme how emails are stored on disk. The username for logging in is the same as the email address. So if the user is called jane@example.org then Dovecot will look for email files in /var/vmail/example.org/jane/Maildir/… and send the user the requested emails.

Surprisingly many users (especially non-nerds) do not like to install and configure a mail client. They prefer to read email in their browser. No problem. That’s what we will offer a webmail interface for.

What happens if a user reads their email using web mail?

The Roundcube software that provides the web mail interface is basically a PHP software that is a gateway between HTML pages and a built-in IMAP client. It speaks HTML to the user so it works in a browser. And it speaks IMAP towards Dovecot to fetch emails. That way it mimics a mail client.

1. The users points the browser to the HTTPS URL of the webmail interface. The Apache web server receives the connection and runs the PHP scripts of Roundcube. Roundcube shows a login form. The user enters the username (=email address) and password and submits the login form.
2. Roundcube connects to Dovecot using IMAP and forwards the username and password to check if they are valid.
3. Dovecot treats this connection similar to a mail client connection. It queries the MariaDB database to verify the username (=email address) and password.
4. If the authentication was successful Dovecot fetches the mail files from disk and sends them through IMAP to Roundcube. Roundcube then renders the emails as HTML and the user can read it.

So you see that the web mail access also works through IMAP. The user does not realize that though.

And the last scenario I would like to explain is…

What happens if the user wants to send an email to the internet?

Of course your users want to be able to send email to other internet users. But they cannot send the email directly to the destination mail server. First their mail client does not know which destination server is responsible for the recipient – that functionality just is not built in. And second the user is likely assigned a dynamic IP address which is blocked by most mail servers because they tend to abused by infected Wind*ws PCs that send out spam. So the correct way to send an email to the internet is through your mail servers. This is called relaying because your mail server acts as a relay. In this example your user wants to send an email to fred@example.net.

 

 

 

1. The user writes the email in their mail client and clicks on “Send”. The mail client establishes an SMTP connection to your Postfix. To make sure that the user is allowed to send email through your system it requires a username and password along with the email. This information is sent in an encrypted way.
2. Before accepting the email Postfix checks the username and password first. Postfix has means to talk to a SQL database but its features are surprisingly limited and complicated. To simplify the process it asks Dovecot as it already knows how to handle authentication.
3. Dovecot sends a query to the MariaDB database to check if the username and password (hash) are correct and tells Postfix the result.
4. Postix knows now that it is authorized to send the email on behalf of the user. It tells the user that it successfully accepted the email. The email is put into Postfix’s mail queue for further processing. Postfix will now query its nearest DNS (name server) to determine the responsible destination mail server. As the recipient has an “…@example.com” address it checks the MX record of the “example.com” domain and then gets the respective IP address.
5. Postfix now knows which mail server to send the email to. It opens an SMTP connection and delivers the email.