MX entries
So now you have your working mail server. But how do other mail servers get to know you? The answer lies in the most important service on the internet: DNS. Assume that a mail server somewhere on the other end of the internet wants to send an email to john@example.org. The server must find out the name (and IP address) of the mail server responsible for the example.org domain. This is how that works:
The remote server queries its DNS server for the MX (Mail eXchanger) record of the “example.org” domain. (If no MX record was found it tries again and asks for A (address) records. That’s a fallback solution.) Let’s run a query for a real domain to get an idea. Let’s ask which servers receive email for …@gmail.com.
$> host -t MX gmail.com
gmail.com mail is handled by 10 alt1.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 20 alt2.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 30 alt3.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 40 alt4.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 5 gmail-smtp-in.l.google.com.
So as a result we get 5 different MX records. Each of them consists of a priority and the host name of the mail server. A mail server would pick the entry with the highest priority (=the lowest number) and establish an SMTP connection to that host. In this example that would be the priority 5 server gmail-smtp-in.l.google.com. If that server could not be reached then the next best server with priority 10 would be used and so on. So all you have to do in your own DNS zone is add an MX entry pointing to your mail server. If you want to run a backup mail server (which is outside of the scope of this tutorial) then you can add a second entry with a an equal or lower priority.
A mistake some people make is using an IP address in MX records. That is not allowed. An MX record always points to a host name. You will have to add an A record for your MX record to point to the actual IP address of your mail server.
For science
Fallback to A entries
It is advised to explicitly name mail servers for your domain in the MX records. If you can’t do that for whatever reason then the remote mail server will just do an A record lookup for the IP address and then send email there. If you just run one server for both the web service and the email service then you can do that. But if the web server for your domain is located at another IP address than your mail server then this won’t work.
Im not sure if it’s discussed but adding reverse dns to my domain got me through gmails spam filtering.
It’s in fact imperative to have a forward-confirmed reverse dns AND a matching helo. Your mail servers outgoing IP addresses (IPv4 & IPv6) resolve to a name. The very same name must resolve back to the those IP addresses AND match the helo. Otherwise your mail might get rejected.
You can, however, have a very different name in your mx record, as far as I know.
You are correct! And rejecting the mismatched FCRDNS and mismatched HELO/EHLO connections gets rid of A LOT of spam, viruses/malware and phishing messages.
Many large email providers have separate inbound and outbound servers, so the MX records can point to completely different hosts.
Another comment on the backup MX.
If you do use a backup mailserver, be sure that it knows about the domains and the users on the primary. If you don’t, you end up with the backscatter problem mentioned earlier in the guide.
Do note, that the backup server does NOT need the password hashes, nor does it need the actual mapping up the aliases.
It does need the list of domains, and a combined list of valid aliases and users.
I used to run a pair of mail filters for the ISPs that I worked for. Every 15 minutes, the filters would call the mail server(s) for a list of valid users. There are a number of ways this can be done, via ssh, via a webpage (use a password protected location, or a token for authentication), or even via replication from the MySQL/MariaDB. You could even have the main mail server check for updates to the databases, and push those changes to the mail filters or backup server(s.)