The big picture

The mail server that you are about to set up uses several software components. Let me first explain briefly what the purpose of each software is:

• Debian “Jessie” – the operating system
• Postfix received incoming emails from the internet and sends out outgoing emails to other mail servers
• SpamAssassin runs sanity checks on an incoming email to determine how likely it is to be spam
• Dovecot stores emails on your hard disk, applies filters and lets your users fetch their emails using the POP3 and IMAP protocols
• Roundcube is a webmail interface so users can read their emails using a web browser, manage their email rules and change their password
• MySQL is a database that stores information about your domains, email aliases and email accounts

What happens when someone sends you an email?

Let us assume that you are responsible for the email domain example.org and someone on the internet sends an email to john@example.org. This is what happens step by step:

1. The remote mail server asks its nearest DNS server for the MX (Mail eXchanger) of the recipient’s domain. The domain is example.org so it asks for the “MX” record of the example.org domain. The DNS server replies that the host name of the reponsible destintaion mail server is mx.example.org. (In case there is no MX record it will try to use the A record as a fallback.)
2. The mail server asks the DNS server again to find out the IP address of the mail server mx.example.org. The DNS server responds with the IP address (or multiple IP addresses).
3. Now the mail server establishes a TCP connection on port 25 (which is by definition used for SMTP – the simple mail transport protocol) to that IP address. It tells the receiving server about the sender’s and receiver’s email address.
4. The Postfix software on the receiving server does a query to the MySQL database to find out if it is responsible for that recipient. If it would not find any information about the recipient in the database it would now respond with an error message and terminate the SMTP connection. The sending server would generate a non-delivery receipt (also called bounce) so that the sender knows that the recipient’s email address was not deliverable.
5. The recipient seems to be valid. So Postfix accepts the email, puts it into its incoming queue and sends a copy to the SpamAssassin software using the milter protocol. SpamAssassin will run various checks to compute the probability that the incoming email is spam. It adds an entry to the mail header (that contains control information) telling the score (=spam probability).
6. Postfix forwards the email to the Dovecot software which is responsible for delivering the email to disk.
7. Dovecot knows that the user directory of john@example.com is /var/vmail/example.com/john and checks if there are filter rules defines. It also has a global filter rule that stores spam emails into a folder called “Junk” if it reaches a spam score of at least 5. Finally it saves the email as a file to disk at /var/vmail/example.com/john/Maildir/…

A lot of things are happening when your users get an email. The next situation I would like to explain is…

What happens if a user fetches their email using IMAP/POP3?

This process is way simpler. It looks like this:

1. The user usually has a mail client (also called a mail user agent) that can use the POP3 or IMAP protocol to fetch emails from the mail server. I prefer Thunderbird for example. That mail client connects to the POP3 (TCP 110) or IMAP (TCP 143) port, sends the STARTTLS command that initiates an encrypted connection and sends the user’s username (which equals the email address in our case) and their password. If possible users should use IMAP which always leaves the emails on the server and which also supports multiple folders. POP3 is very limited and pretty much deprecated nowadays.
2. Dovecot does a query to the MySQL database and verifies that the username and password belong to a known user. The password is not stored in plain text in the database. Instead it computes the password hash and compares it to the hash in the database. If the password is wrong then Dovecot will refuse the login.
3. We have a fixed scheme how emails are stored on disk. So if the user is called jane@example.org then Dovecot will look for email files in /var/vmail/example.org/jane/Maildir/… and send the user the requested emails.

Some users are do not like to install and configure a mail client. They prefer to read email in their browser. So…

What happens if a user reads their email using web mail?

The Roundcube software that provides the web mail interface is basically a PHP software that is a gateway between HTML pages and a built-in IMAP client. So when a user uses their browser to connect to the web mail interface…

1. The users points the browser to the HTTPS URL of the webmail interface. Apache web server receives the connection and runs the PHP scripts of Roundcube. Roundcube shows a login form for the username and password. The user enters the username (that equals the email address) and the password and submits the login form.
2. Roundcube connects to Dovecot using IMAP and forwards the username and password to check if they are valid.
3. Dovecot treats this connection similar to a mail client connection. It queries the MySQL database to verify the username (=email address) and password.
4. If the authentication was successful Dovecot fetches the mail files from disk and sends them through IMAP to Roundcube. Roundcube then renders the emails as HTML and the user can read it.

So you see that the web mail access also works through IMAP. The user does not realize that though.

Okay, the final scenario I would like to explain is…

What happens if the user wants to send an email to the internet?

Of course your users want to be able to send email to other internet users. But they cannot send the email directly to the destination mail server. First their mail client does not know which destination server is responsible for the recipient – that functionality just is not built in. And second the user is likely assigned a dynamic IP address which is blocked by most mail servers because they tend to abused by infected Windows PCs that send out spam. So the correct way to send an email to the internet is through your mail servers. This is called relaying because your mail server acts as a relay. In this example your user wants to send an email to fred@example.net.

1. The user writes the email in their mail client and clicks on “Send”. The mail client establishes an SMTP connection to your Postfix. To make sure that the user is allowed to send email through your system it requires a username and password along with the email. This information is sent in an encrypted way.
2. Postfix could now check the password in the database directly. But as Dovecot already knows how to handle authentication it is easier to ask Dovecot to verify the username and password. (SMTP authentication in Postfix is surprisingly ugly otherwise.)
3. Dovecot now sends a query to the MySQL database to check if the username and password (hash) are correct and tells Postfix the result.
4. Postix knows now that it is authorized to send the email on behalf of the user. It tells the user that it successfully accepted the email. The email is put into Postfix’s mail queue for further processing. Postfix will now query its nearest DNS (name server) to determine the responsible destination mail server. As the recipient has an “…@example.net” address it checks the MX record of the “example.net” domain and then gets the respective IP address.
5. Postfix now knows which mail server to send the email to. It opens an SMTP connection and delivers the email.