Relaying
Your mail server is almost ready for use. But one puzzle piece is missing. Your users can already fetch emails but they need to be able to send emails, too. Mail server know how to send an email to another mail server by looking up MX records. Mail clients however are expected to send email to their mail server first which then forwards email to the internet. When Postfix receives an email from a user forwards it to another server this is called relaying. Even if the mail client would know how to deal with MX records it would not be able to send the email. Users are often sitting a home at a DSL line. And such IP networks are usually blocked off from sending mail by so called real-time blacklists.
Am I going too fast? Okay, let’s walk through the different scenarios one by one.
Incoming email
When someone on the internet sends an email to john@example.org some other mail server will deliver the email using SMTP to your mail server. Postfix will determine that it’s responsible for email addresses in the example.org domain and accept the email. John can then use POP3 or IMAP to fetch the email from your server.
Outgoing email (without authentication)
John is on the internet somewhere and wants to send an email to lisa@example.com. Your mail server is not responsible for the “example.com” domain so it receives John’s email and forwards (relays) it to the mail server that is responsible for …@example.com email addresses. This may seem like a harmless scenario but your mail server must deny that:
Why? Because anyone can claim to be John and make your mail server forward mail. If an attacker (like a spammer) would send millions of spam emails in John’s name through your server then other organisations will accuse you of spamming. Your mail server would be what people call an “open relay”. This is not what you want because your mail server would get blacklisted and you will not be able to send out mail to most other servers. So without any proof that John is actually John your server will reject the email.
Outgoing email (with authentication)
So how does John prove his identity? He needs to use authenticated SMTP. This is similar to the previous case but John’s email program will also send his username and password.
Of course we are making sure that his authentication happens over an encrypted connection.
mynetworks
In addition to using SMTP authentication you can tell Postfix to always relay email for certain IP addresses. The mynetworks setting contains the list of IP networks or IP addresses that you trust. Usually you define your own local network here. The reason John had to authenticate in the above example is because he is not sending the email from your local network.
If your users are using the webmail interface running on your mail server then they will not need to authenticate. Roundcube sends email to localhost which is trusted according to the mynetworks setting.
Make Postfix use Dovecot for authentication
Authenticated SMTP with Postfix has been a hassle in the past. It was done through the SASL (Simple Authentication and Security Layer) library that was once part of the Cyrus mail server. And it wasn’t “simple” at all. It was nearly impossible to debug and threw error messages that were gibberish and misleading. Fortunately nowadays we can make Postfix ask the Dovecot server to verify the username and password. And as you already configured Dovecot’s authentication part this you just need to make Postfix talk to Dovecot. Postfix just needs some extra configuration:
postconf smtpd_sasl_type=dovecot postconf smtpd_sasl_path=private/auth postconf smtpd_sasl_auth_enable=yes
This enables SMTP authentication and tells Postfix that it can talk to Dovecot through a socket file located at /var/spool/postfix/private/auth. Do you remember that Postfix runs in a sandboxed chroot directory? That’s at /var/spool/postfix. So we configured Dovecot to put a socket into that directory to allow communication with Postfix.
Enable encryption
The following settings enable encryption, define the key and certificate and enforce that authentication must only occur over encrypted connections:
postconf smtpd_tls_security_level=may postconf smtpd_tls_auth_only=yes postconf smtpd_tls_cert_file=/etc/ssl/certs/mailserver.pem postconf smtpd_tls_key_file=/etc/ssl/private/mailserver.pem
In previous versions of the ISPmail guide you also needed to set the smtpd_recipient_restrictions to restrict relaying to authenticated users. Fortunately the new version of Postfix in Jessie (2.11) has better defaults. Postfix offers a new setting called “smtpd_relay_restrictions” that deals with relaying requests in the “RCPT TO” phase of the SMTP dialog. So essentially it works like the “smtpd_recipient_restrictions” but is checked first. And as the smtpd_relay_restrictions have a reasonable default we don’t have to add anything to make authenticated relaying work.
By the way: Postfix 2.10 has changed the behavior of the smtpd_*_restrictions altogether. If a restriction makes Postfix reject an email it will wait until after the “RCPT TO” line. Read the Postfix documentation for a description of the reasons for that change.
The smtpd_tls_security_level is set to “may” to allow encrypted communication when Postfix receives an email. This not only applies to users sending emails but also remote mail servers that send email to you. It may sound tempting to change that to “encrypt” which would enforce encryption and reject any attempts to create an unencrypted SMTP connection. But unfortunately there are still mail servers in the world that can’t encrypt and would not be able to deliver email to you. So “may” is the proper setting.
Although emails may still get delivered unencrypted from other servers we should make sure that users don’t accidentally (or out of ignorance) send their passwords unencrypted over the internet. That’s what we enforce using “smtpd_tls_auth_only=yes”. Now we should be safe.
How SMTP authentication works
Are you curious how SMTP authentication looks on a protocol level? Let’s go through that. In previous versions of this guide we used “telnet” to connect to TCP port 25 and speak SMTP. But now we enforce encryption and can’t do SMTP authentication unencrypted. Let’s try the unencrypted part:
telnet localhost smtp
The server will let you in:
Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mailtest ESMTP Postfix (Debian/GNU)
Say hello:
ehlo example.com
Postfix will present a list of features that are available during the SMTP dialog:
250-mailtest 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN
Let me briefly explain what these lines mean:
- PIPELINING
This is a feature to speed up SMTP communication. Usually the remote system has to wait for a response to every command it sends. Pipelining allows the remote server to send bulks of commands without waiting for a response. Postfix will just store these commands and execute them one by one. If you told Postfix to forbid pipelining it would disconnect the remote server when it tries to send bulks of commands without waiting for the proper reply. It is mainly a feature against spamming programs that don’t behave. - SIZE 10240000
The remote server is allowed to send emails up to 10 MB large. This has long been a common maximum size for emails. However nowadays 40 MB or even more are more common sizes because emails have grown larger. - VRFY
Allows remote servers to verify a given name or email address. For example the remote server could send “VRFY john” and your server might respond “250 John Doe <john@example.org>”. It can be used to verify that a certain recipient email address is deliverable - ETRN
A command that a remote system can send to flush the Postfix queue of mails for a certain domain. It can be used if the remote system had technical problems and failed to receive email for a while. Then it could send an ETRN command to make your server start sending outstanding emails for that domain. It is rarely used. - STARTTLS
This tells the remote system that it might start switching from this unencrypted to an encrypted connection by sending the “STARTTLS” command. It will then start negotiating a TLS-encrypted connection. You could compare it to an HTTP connection that suddenly switches over to an encrypted HTTPS connection. The advantage is that you can start talking SMTP on TCP port 25 and don’t have to open up a second TCP port like 465 which is the “SSMTP” (secure SMTP) port and only accepts encrypted connections. - ENHANCEDSTATUSCODES
This enables more three-digit return codes for various conditions. See the RFC2034 if you are curious. - 8BITMIME
In ancient times SMTP only processed 7-bit characters. You couldn’t transfer special characters like “Ä” or “ß” without special encoding. 8BITMIME allows a transmission of emails using 8-bit characters. Still many emails are specially encoded using ISO8859-1 or UTF-8. - DSN
It enables DSNs (delivery status notofications) that allows the sender to control the messages that Postfix creates when an email could not be delivered as intended
However one important line is missing here that would allow us to send our username and password:
250-AUTH PLAIN LOGIN
We told Postfix to only allow authentication when the connection is encrypted. So we are not offered authentication over this plaintext connection.
Are you still connected? Okay, good. So we need an encrypted connection using TLS. You could enter STARTTLS:
STARTTLS
And the server would reply:
220 2.0.0 Ready to start TLS
However now it’s getting complicated because you would have to speak TLS encryption which is not a language that humans speak. But we can use OpenSSL to help us with the decryption. Terminate the SMTP connection. Now run:
openssl s_client -connect localhost:25 -starttls smtp
You will see a lot of output. OpenSSL has connected to TCP port 25 and issued a STARTTLS command to switch to an encrypted connection. So whatever you type now will get encrypted. Enter:
ehlo example.com
And Postfix will send a list of capabilities that will look like this:
250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-AUTH PLAIN LOGIN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN
And now that we are using an encrypted connection Postfix offers us to authenticate. So let us send the authentication string with a Base64-encoded password:
AUTH PLAIN am9obkBleGFtcGxlLm9yZwBqb2huQGV4YW1wbGUub3JnAHN1bW1lcnN1bg==
The server should accept that authentication:
235 2.7.0 Authentication successful
Disconnect from Postfix:
QUIT
Goodbye:
221 2.0.0 Bye
Authentication works. Well done.
Base64-encoded passwords
You may wonder how I got to the long cryptic string that apparently contained John’s email address and his password. It is just a Base64-encoded (not encrypted!) version of the string “john@example.org<NULL-BYTE>john@example.org<NULL-BYTE>summersun”.
One way to create that string is using Perl:
perl -MMIME::Base64 -e \
'print encode_base64("john\@example.org\0john\@example.org\0summersun")';
The submission port
Although I have been talking about SMTP on port 25 to relay mails there is actually a better way: using the submission port on TCP port 587 (as described in RFC 4409). The idea is to use port 25 for transporting email (MTA) from server to server and port 587 for submitting (MSA) email from a user to a mail server. As many users still use SMTP by default you will likely have to run both ports.
Edit your /etc/postfix/master.cf file and add a service for the submission port like this (mind the indentation from the second line on). Don’t use the suggested and commented-out submission service in the master.cf – just ignore it.
submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth -o smtpd_sasl_security_options=noanonymous -o smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf -o smtpd_sender_restrictions=reject_sender_login_mismatch -o smtpd_sasl_local_domain=$myhostname -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
Basically this new service uses the “smtpd” daemon which is the piece of software that responds if you open an SMTP connection on TCP port 25. But it gets a few extra options set…
- in the /var/log/mail.log mark the connections to the submission port as “postfix/submission/smtpd” (syslog)
- enforce encryption on this port (security level)
- enforce user authentication (sasl)
- make sure that an authenticated user can only send emails in their own name (sender_login_maps)
Restart the Postfix server:
service postfix restart
Your users can now use the submission port to send email.
Protecting against forged sender addresses
Wait a minute. We can make sure that a user can only send emails using their own sender address? Cool. How does that work? Take a look at this line we just used above when we added the submission service:
smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf
The smtpd_sender_login_maps is a Postfix mapping with two columns:
- the email address the user wants to use as a sender address
- the username that was used when authenticating at the mail server
The reason I am just using the /etc/postfix/mysql-email2email.cf mapping is that it fits perfectly here. That mapping maps the email address to itself if it exists. Confused? Look at the SQL query that the “email2email” mapping runs:
SELECT email FROM virtual_users WHERE email='%s'
So when a user wants to relay an email Postfix checks the virtual_users table looking for the sender address. Say that John wants to send out an email from his email address “john@example.org”. Postfix will check the virtual_users table if there are any rows with an email field like that. It will find one and return the email field – which is just again the email address. Postfix expects to get back the username that was used to login. We are using the email address as a login username – so that mapping works. Only if John would try to send an email with a forged sender address then Postfix will see that it does not match his account and reject the email. Your mail.log would read something like:
NOQUEUE: reject: RCPT from foo.bar[…]: 553 5.7.1 <forged@email.address>: Sender address rejected: not owned by user john@example.org; from=<forged@email.address> to=<…>
It is your own decision whether you want to use the smtpd_sender_restrictions like that.
Hi Christoph,
it seems you’ve got a typo at the Enable encryption section.
You’ve got mailserver.pem twice.
postconf smtpd_tls_cert_file=/etc/ssl/certs/mailserver.pem
postconf smtpd_tls_key_file=/etc/ssl/private/mailserver.pem
Shouldn’t it read
postconf smtpd_tls_cert_file=/etc/ssl/certs/mailserver.pem
postconf smtpd_tls_key_file=/etc/ssl/private/mailserver.key
???
Totzdem vielen Dank für das ausführliche und gut erklärte Tutorial!
Grüße, Frank.
Frank, thanks for the comment. However at https://workaround.org/ispmail/jessie/create-certificate I write that both files have a “.pem” suffix. Have I accidentally used “.key” anywhere?
Actually, Christoph, I think you may have, as I struggled for a couple of hours yesterday trying to figure out why a service wouldn’t start. I spent so much time on it, I’m not even sure which service it was, but looking back at everything, I think you might have fixed it already.
Hi Christoph,
thanks for the awesome tutorial. It helps a lot.
You write that postfix does not allow VRFY without authentication. On my machine I can just telnet and ask “VRFY ” and it gives a valid answer without authentication.
Oct 31 23:31:10 ovpn postfix/smtpd[5833]: connect from unknown[5.28.118.115]
Oct 31 23:32:42 ovpn postfix/smtpd[5833]: NOQUEUE: reject: VRFY from unknown[5.28.118.115]: 550 5.1.1 : Recipient address rejected: User unknown in virtual mailbox table
_domain_>
Has the default setting changed?
Cheers,
Hannes
Hi Hannes… thanks for the feedback. I’m a bit unsure if the behavior changed or if I have been wrong all the time. After all hardly any remote system would be able to authenticate prior to verifying a recipient with VRFY. Indeed you are right that VRFY does not have the restriction. So I corrected that paragraph. Thanks.
Hi Christoph, thanks for an excellent guide for setting up an email server.
I do wonder if you have some thought on a situation: Most of the ISP’s my clients are connected through blocks any traffic on port 25. Solution would be to have the clients connect through port 587 instead.
There are (at least) two solutions to this problem:
1: Uncomment the #submission line in postfix’ master.cf
2: Redirect all incoming traffic on port 587 to port 25 locally on the mail server.
Which approach would you recommend? Intuitively I would just redirect all traffic, but maybe there is a better way to do this?
Thanks,
Erik
Hi Erik. Port 587 should be open anyway – I will add that to the guide. So enabling the “submission” service in the master.cf is the right way. However if your provider is blocking port 25 then other mail servers won’t be able to reach you. Sounds like you better operate your server at another ISP.
Hi Christoph.
My server’s ISP is not blocking anything, but many of my clients ISP’s are (probably because they’re expected to use the ISP’s smtp servers for outgoing mail, but that doesn’t work well with portable devices which are connected to many different ISP’s throughout the day). So the problem only arise when the clients want to use my email server as a relay for their outgoing email (which they are expected to do), and their ISP is blocking their traffic over port 25. Their client software will then try port 587 instead, and that’s why I need my server to listen to port 587, either directly or via redirects.
I see – that’s totally fine. Mail clients are supposed to use port 587 anyway. Port 25 is mainly used by other mail servers.
I also struggled on this Point and uncommenting the Submission was the solution for me.
Something of note I have found while sending test emails, emails are returned when sending to a gmail address with an error about not being within IPv6 guidelines. A quick fix that seems to have worked was adding “inet_protocols=ipv4” to the postfix main.cf. Still getting strange return to senders from outlook/hotmail accounts with error “550 SC-001”, have not found any real solution, though from my research so far it seems to be a hotmail server issue rather than Postfix/Dovecot.
mserv:# openssl s_client -connect localhost:25 -starttls smtp
CONNECTED(00000003)
didn’t found starttls in server response, try anyway…
140283132397200:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:795:
Is there any reason for this? I’ve searched for so long but could not find anything. SMTP works flawlessly when on localhost but it timeouts when you try to reach it from outside.
I forgot to add a very important thing – server was upgraded to Jessie (from Wheezy). I’m trying to understand what do I need to change in Postfix config file, but I’m afraid to simply reinstall the package – mail server is used by many people and it will probably break many things. Before upgrade to Wheezy you could send mail from any client, now it’s only possible via web interface (thus localhost).
Hello people who migrate from Wheezy to Jessie using this tutorial!
1) Install dovecot-lmtpd package (yes you don’t have it).
2) Backup Postfix configuration file:
mv main.cf main2.cf
3) Reinstall Postfix and receive a new config file:
apt-get install –reinstall postfix
4) Open main.cf, find inet_interfaces, change it to “all”.
5) Also comment out these directives from main.cf:
#default_transport = error
#relay_transport = error
4) Go through this tutorial, skip database part (you have it already). Pay attention to new LMTPD thing. Follow this tutorial, change configs as needed.
5) You have finished tutorial, now comment out this directive in main.cf if you don’t need forced TLS. Or leave it if you’re okay with forcing TLS.
#smtpd_tls_auth_only = yes
6) Uncomment this in master.cf:
submission inet n – n – – smtpd
This will allow you to use port 587 (with TLS and whistles).
6) ???
7) PROFIT!
Hope this helps someone who is migrating from previous tutorial.
Hi Christoph,
Postfix/Dovecot won’t authenticate my test users when trying smtp plain auth via openssl s_client. Plain auth + base64 encoded emailaddress0emailaddress0password gives 535 5.7.8 Error: authentication failed: (nothing after that)
In /var/log/mail.log I see that the base64 encoded string is passed through to Dovecot, and Dovecot replies dovecot: auth: Debug: client passdb out: FAIL#0111
If I use mutt via imap, Dovecot accepts the username and password: dovecot: auth: Debug: client passdb out: OK#0111#011user=john@example.org
The seperate tests using postmap (postmap -q john@example.com pgsql:/etc/postfix/postgres-virtual_**) return valid results.
“And as you already configured Dovecot’s authentication part this you just need to make Postfix talk to Dovecot. Postfix just needs some extra configuration:”
I got the configuration, and the socket exists:
0 srw-rw—- 1 postfix postfix 0 Jan 19 10:53 /var/spool/postfix/private/auth
Is there an obvious reason for Dovecot not to authenticate a user via Postfix that is valid directly via IMAP?
Best regards,
Boudewijn
Hi everyone!
Thanks to Chistoph for his excellent tutorial
Boudewijn,
I also had problems to authenticate with Postfix / Dovecot via openssl s_client but I solved it in another way as explained by Christoph. But the principle remains the same.
Instead of generating the base64 encoding of the username and password once, I did it twice as follows ( example john@example.org, summersun):
– First generate the username Base64 encoding
perl -MMIME::Base64 -e ‘print encode_base64(“john\@example.org”);’
– 2nd, generate password encoding:
perl -MMIME::Base64 -e ‘print encode_base64(“summersun”);’
After that, execute the following commands
1) openssl s_client -connect localhost:25 -starttls smtp
2) ehlo example.org
3) Enter AUTH LOGIN
4) Enter the username encoding
5) And enter the password encoding
Try this procedure hoping this help you to authenficate with Postfix/Dovecot
Best regards !
Hi YaMahoma,
Thank you for your suggestion. Login still fails, but finally I got the idea I found the cause: besides SQL-authentication, there is also authentication for system users active in /etc/dovecot/conf.d/auth-system.conf.ext:userdb {….
The result was that for every login to dovecot there were two groups of lines in /var/log/mail.log: first one for local users (that failed, even with local users, because PAM is not configured for dovecot), then it succeeds via SQL.
I commented out all userdb’s in the system.conf.ext-file, and now there is only one group of lines per login to dovecot.
I imagine the first (unsuccessful) login attempt broke the system for Postfix.
In the mean time I seem to have broken my certificate chain (first only cacert server certificate with error on the chain, then concatenated with the cacert class3 certificate) , and now I can not use openssl s_client anymore (there are no options after connecting to port 25 with smtp).
I’ll post an update once I found a solution; for the record and search results: Verify return code: 20 (unable to get local issuer certificate).
Thanks again!
PS: does anyone have an idea where to find a description of those codes in mail.log, such as AUTH#0111
CONT#0111
AUTH#0112
FAIL#0111
Are they numbered sessions? Searching the net gave lots of logs, but no explanation.
Hi all,
Sorry for filling this excellent tutorial with noise.
The varying certificate errors that openssl s_client threw on connecting to postfix could be overcome by pointing openssl to my chained certificate explicitly:
openssl s_client -connect mail.example.com:25 -starttls smtp -CApath /etc/ssl/certs/mailserver_chain.pem
In my case, the not-chained certificate gave
Verify return code: 21 (unable to verify the first certificate)
—
250 DSN
while only the server certificate gave return code 20:
Verify return code: 20 (unable to get local issuer certificate)
—
250 DSN
I got confused, because openssl would in itself give green light for the certificate chain:
openssl verify /etc/ssl/certs/mailserver_chain.pem
/etc/ssl/certs/mailserver_chain.pem: OK
I won’t be able to test login in tonight anymore, but it is a step closer again 🙂
Best regards,
Boudewijn
There is a small mistake in your openssl s_client command. The -CApath should point to the /etc/ssl/certs directory, not to the mailserver_chain.pem file. Try the following command instead:
openssl s_client -CApath /etc/ssl/certs/ -connect localhost:25 -starttls smtp
If you have the smtpd_tls_cert_file in main.cf pointed to your chained certificate the result of the above command should be:
Verify return code: 0 (ok)
Been following your Tutorial for 3 version now, thanks so much!
Ran into a new problem at this point, and can’t figure it out: I connect via Telnet, follow instructions until STARTTLS, but when I enter the openssl line the connection closes:
“Connection closed by foreign host.”
/var/log/mail.log says:
warning: TLS library problem: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol:s23_srvr.c:650
Any ideas what’s going wrong here??
It’s a terrific tutorial, but:
How to prevent misuse of the server ?, How to avoid using the server to generate SPAM ?, How do I limit the number of emails that can be sent by an account?
You forgot to put the content of /etc/postfix/mysql-email2email.cf:
user = mailuser
password = fLxsWdf5ABLqwhZr <– use your own database password here
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM virtual_users WHERE email=’%s’
Also Roundcube allows forged sender addresses: such bad behaviour must be fixed.
You can force roundcube to use the submission port (and thus to follow its rules) by changing the port that roundcube uses to connet to the smtp server in defaults.inc.php. Change it to port 587 as follows:
// SMTP port (default is 25; use 587 for STARTTLS or 465 for the
// deprecated SSL over SMTP (aka SMTPS))
$config[‘smtp_port’] = 587;
Did you resolve this problem? I followed the instructions and can forge sender address with RC. I believe the problem is that Roundcube default is set to send all email messages with the PHP mail() function unless $config[‘smtp_server’] is set. However, even after setting RC smtp_server to localhost, I am still able to forge sender address. Anyone have any ideas on solving this?
additional info:
With RC $config[‘smtp_server’] set to localhost and $config[‘smtp_port’] = 587, I am getting an error on sending from RC stating that “failed to add recipient … client host rejected”. I have searched for a solution and tried (it seems) endless combinations for settings in both postfix and roundcube. if anyone has any thoughts, it would be greatly appreciated.
Your smtpd_sender_login_maps is wrong because it doesn’t take into account aliases. It should be:
query = SELECT email FROM virtual_users WHERE email=’%s’ UNION SELECT destination FROM virtual_aliases WHERE source=’%s’
Christoph,
I’m a huge fan of your work here as you now know. I had gone through the whole tutorial and I’ve been delighted with the result. There was one final little bit that I’ve been struggling with – sending from postfix to the destination using TLS.
I found the solution here:
(Is it the subtle difference between smtp_ and the smtpd_ I *think* but I’m not sure – one is for receiving the other is for relaying??)
http://stackoverflow.com/questions/17407541/postfix-relay-from-clear-text-to-tls
[code]
smtp_use_tls = yes
smtp_tls_security_level = may
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file = /etc/pki/tls/private/site.key
smtpd_tls_cert_file = /etc/pki/tls/certs/site.crt
smtpd_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt
smtpd_tls_loglevel = 9
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom
smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
[/code]
I believe I have experienced a similar issue … When a user logs-in to the server via Roundcube and sends off an email to a Gmail user, when the Gmail user reads the email in Gmail it is flagged as not having been encrypted. Is that correct in that the email is sent out in plain text?
Kevin,
That’s correct it’s sent in plain text if the red broken padlock is shown to the gmail user.
You’ll probably find that it’s sent in plaintext even if they send it from their client email program (Outlook, Thunderbird etc) rather than just from RoundCube. The only difference is that the connection between the client and dovecot is TLS encrypted, but not from postfix to the destination mailbox.
[Client] %%%ENCRYPTED%%%%>>> [Dovecot]==>[Postfix]
then
[Postfix]—–IN CLEAR—>>>[Destination]
I found the site http://checktls.com/ very helpful. You can diagnose incoming and outgoing emails to see if they are sent using TLS or not.
Once I applied the modifications above to my postfix configuration, restarted postifx then they were shown as being sent with TLS.
Trystan
Thanks for your explanation. After reading the Postfix docs in a little more detail, I’ve simply added:
smtp_tls_security_level = may
and now have encrypted email going to Gmail. Thanks!
Kevin
Kevin,
Great to hear you have it working too. Let’s hope Christoph adds it to the config at some point too.
Trystan
In fact I have it set on my test server. I’m confused why I didn’t mention that in the guide. Thanks.
Thank you for your contribution, it works well and I have now encrypted email going to Gmail.
However in the same configuration when a user logs-in to the server via Roundcube and sends off an email to a Gmail user, when the Gmail user reads the email in Gmail it is flagged :
“Gmail couldn’t verify that ‘mydomain.com ‘Actually feels messages (and not a spammer) ”
with this link: https://support.google.com/mail/answer/180707?hl=en
how to avoid this?
thanks for you help.
please read :
“Gmail couldn’t verify that ‘mydomain.com’ actually SENT message (and not a spammer) “
First, check if you have a proper SPF record:
https://en.wikipedia.org/wiki/Sender_Policy_Framework
IIUC you only have this problem when using roundcube. I guess, roundcube runs on the same host like postfix and does not authenticate (processes on localhost can use postfix to send mails without authentication). Try to change the configuration (SMTP section) of roundcube so that it authenticates before sending mail.
Cheers,
Hannes
Hi Hannes,
Thanks for you time.
Indeed roundcube runs on the same host like postfix, but no, the problem occurs also if I send email from SMTP without roundcube.
I checked SPF Record: there was no SPF record in my DNS zone.
I added a TXT entry like this:
example.com. IN TXT “v = spf1 ip4 : 192.0.2.0/24 ip4 : 198.51.100.123 has -all ”
And now the email in Gmail isn’t flagged anymore with ““Gmail couldn’t verify that ‘mydomain.com’ …..”
My problem solved :o)
Thanks!
Jelo
Hi all.
Just to confirm that adding
smtp_tls_security_level = may
make sure that the message is sent encrypted (and gmail doesn’t complain about this).
Thanks
Christoph,
I’m not sure if it helps, or if it’s relevant but the email I got from you wasn’t sent using TLS.
Trystan
Hi Christoph,
From my side everything work fine but the problem i face is that my mail server cannot receive from another provider (eg: gmail, my professionnal email, etc.).
The /var/log/mail.log doesn’t display anything when I try to send email from that provider above. Help me please 🙁
I head the same problem. My postfix server was only listening on localhost. Try
sudo netstat -plntu
If you see
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 5807/master
check that the value of inet_interfaces in /etc/postfix/main.cf
It should be set to all not 127.0.0.1.
Hi Christoph,
we are configuring our mail server to serve different domains. Each domain must use a different ssl certificate; how can we configure postfix to deal with this in the smtpd parameters?
Dear Christoph,
some days ago I have got a couple of mails rejected by gmail with the following message:
Our system has detected that 550-5.7.1 this message does not meet IPv6 sending guidelines regarding PTR 550-5.7.1 records and authentication. Please review 550-5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for more 550 5.7.1 information. jy5si4998590wjc.55 – gsmtp
The strange thing is that if I try to send the same e-mails through Roundcube, all it goes fine, while sending with Apple Mail fails.
By looking further at the postfix log, I have noticed that when I use Apple Mail to send e-mails, postfix sends over ipv6, while when employing Roundcube, it sends over ipv4.
According to gmail, then, I have some issues with ipv6; again, this is quite strange, since my ipv6 server address has a valid reverse hostname!
As a workaround, I have explicitly set the inet_protocols parameter to ipv4 in main.cf:
inet_protocols = ipv4
(default was ‘all’, I guess) and now all it works fine.
However, how could it be possible that postfix selects inet_protocols depending on the mail client?
Thanks,
Luigi
Hi Christoph,
I can still forge the the sender address when using port 25. Is this expected or did I miss anything?
Thanks again for the tutorial!
Hannes
Some of the rules for the submission port are overridden and are thus different from the default rules that apply to smtpd on port 25. So indeed this is to be expected. If I understand correctly you could add the smtpd_sender_login_maps and smtpd_sender_restrictions to the first smtp entry in master.cf or just add them to main.cf (please correct me if I’m wrong).
I took a different approach: I only allow any form of relaying on the submission port and use port 25 only for receiving e-mail from other mailservers. To this end I have the following in main.cf:
smtpd_relay_restrictions = permit_mynetworks reject_unauth_destination
(localhost is still allowed to relay)
And in the submission port entry in master.cf I’ve added:
-o smtpd_relay_restrictions=permit_mynetworks,permit_sasl_authenticated,defer_unauth_destination
Without this line, the smtpd process on the submission port also uses the smtpd_relay_restrictions as defined in main.cf. Instead we want to allow relaying for authenticated clients, so we have to override this in master.cf
Hi Christoph,
as Niccolò wrote, you have to change your sql-query to look up aliases. Otherwise no one is able to send from an alias address and gets rejected.
I’m studying the ISPMail tutorial for my new (Jessie) mail server to replace my old Sqeeuze based mail server. In my old situation I use the Smarthost from my domain registrar to send email of my users out to the Internet.
In postfix main.cf I have “relayhost = smarthost.registrar-domain.tld” and the files:
sasl_passwd
sasl_passwd.db
With the logon credentials to the smarthost.
I’m not used to Dovecot but Courier is this still the way to go for using a Smarthost for sending mails?
I am using this guide to setup an ISPmail server just to test this setup locally before I set up a personal email server on a VPS, and when I do “telnet localhost smtp” I /AM/ getting the line “250-AUTH PLAIN LOGIN” without having to “openssl s_client -connect localhost:25 -starttls smtp”. Is this because I set “auth_mechanisms = plain login” in order to support Outlook clients? I am following this guide almost exactly, except I am omitting the RoundCube webclient because I intend to only access this via an IMAP client such as Outlook or iOS mail app, perhaps this might have something to do with it also?
Thanks for the help, great guide!
Nevermind, I had apparently typo’d the smtp_tls_* lines as smtp_tl_*, this issue has been resolved.
Thanks again, great guide!
Will having “smtpd_tls_security_level=encrypt” in the submission section of the master.cf snippet above conflict with the setting “smtpd_tls_security_level=may” in main.cf?
I added this to /etc/postfix/main.cf
smtp_tls_security_level = may
(smtp – no d)
I was testing my email delivery on those “check TLS email” websites and this setting allowed me to get the green check mark, which is the gold standard of running a DIY email server 😉
This will allow postfix to opportunistically use TLS/SSL on outgoing mail, not just incoming mail. You could also set it to require encryption, but this might affect interoperability with other mailservers.
Thanks for the awesome guide!
Hello all.
My mail.log file shows lines like this one :
Jun 15 08:11:00 machine postfix/submission/smtpd[7534]: fatal: /etc/postfix/mysql-email2email.cf, line 1: missing ‘=’ after attribute name: “SELECT email FROM virtual_users WHERE email=’%s'”
I typed, re-typed, copied, pasted this “=”. The lines are still there.
Any help would be valued.
Kind Regards, Panos
Could you paste your entire mysql-email2email.cf file? Perhaps the problem is elsewhere. (Don’t show us your password though…)
Indeed, i did something wrong. Fixed now. (Note: please delete my next comment, it is useless).
Sincere thanks!!
The contents of : /etc/postfix/mysql-email2email.cf
SELECT email FROM virtual_users WHERE email=’%s’;
Just this one line. I’ve done something wrong, yes?
Panos
It should contain lines starting with: user, password, hosts, dbname, query
Hi,
I noticed a case that some other may have. I’m not allowed to use my username at for instance usera@domaina.com to send mail from my alias userb@domainb.com.
Is there a tweak I can apply to postfix to authorise something like that ?
Cheers.
Hi,
I also run into the “lost connection after STARTTLS” after typing starttls. I am a little stuck up at the moment. Can you steer me in the right direction?
First of all: Thanks for this great tutorial and all the explanations. It helped me a lot to set up my mailserver.
One thing I noticed is that it was possible on my machine that a sender from a configured virtual domain was able to send mails to a user from another virtual without authentication on port 25 from a client.
Example:
Assuming I host mails for the domains a.com and b.com:
When a client (like Thunderbird in my case) is configured to use SMTP on port 25 with starttls and NO authentication from the adress user@a.com then he can send mails to user@b.com (when user@b.com exists).
This is imho only desired for mails from localhost but not for clients. This would be desired if my server is not hosting mails for a.com.
Maybe I missed something from your tutorial and this case is already covered.
Finally I found the solution in http://serverfault.com/a/689940
I took the second method Zulakis describes and modified the mysql…cf to match the local database.
Additionally I had to add this proxy table to proxy_read_maps
Do you see any problems with this solution?
Hi, Christoph,
I used your tutorial to install my server – I had an operating server based on Ubuntu and the howto at flurdy.com, but a Ubuntu upgrade broke it and I was unable to rec over.
Everything works fine for receive. I cannot send email from my Windows machines on the same local network, though.
When I run the test using “openssl s_client -connect localhost:25 -starttls smtp” and the base64 string generated from my email address, I get “535 5.7.8 Error: authentication failed:”.
mail.log shows “postfix/smtpd[15086]: warning: localhost[127.0.0.1]: SASL plain authentication failed:”
How do I troubleshoot this? Nothing appears in /var/log/mysql/error.log.
The email I entered below is from the server in question – it receives fine, but cannot send.
Did some searching – found saslfinger, which tells me “There is no smtpd.conf that defines what SASL should do for Postfix.
SMTP AUTH can’t work!” when I used the -c option, and “There is no smtpd.conf that defines what SASL should do for Postfix.
SMTP AUTH can’t work!” when I use -s.
Tried postconf smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd – get “postconf: warning: =: unknown parameter
postconf: warning: hash:/etc/postfix/sasl_passwd: unknown parameter
smtp_sasl_password_maps =”
I also see comments in the Postfix SASL Howto about Dovecot not handling client authorization.
Whatr am I missing?
I was having trouble with “fatal: no SASL authentication mechanism” error.
I searched and searched but finally found this:
https://wiki.debian.org/PostfixAndSASL
Like you, I didn’t have a smtpd.conf file and I followed this “guide ” to get my email working.
Thanks for the guide it’s working like a charm.
But when I want to enable sender_login_maps, postfix fails to send mails:
Dec 12 11:54:21 mail postfix/smtpd[4501]: NOQUEUE: reject: RCPT from unknown[192.168.1.100]: 554 5.7.1 : Relay access denied; from= to= proto=ESMTP helo=
The config is 1:1 from your guide, but tcpdump never shows any query for the sender_login_maps option. What did I miss?
Note: I’ve changed the SQL query to allow alias sending too, but as described, postfix NEVER asks mysql about hist sender_login_maps.
Any ideas?
Thanks!
Stephan,
did you solve this issue? Does this happen with only roundcube or does it happen with both roundcube and other email clients?
can anybody tell me why i cannot connect with thunderbird? the web interface is working great but my localhost cannot telnet to myserver:25 and also thunderbird cannot find a connection, i have correct servername and username and password. not possible to get my email server and account from my own linux machine into thunderbird
The most common reason is that your ISP is firewalling TCP port 25. From your client try “telnet myserver 25” and see if you get a connection.
It would be nice to have a link or a litte description how to change the maximal email sizes: https://www.electrictoolbox.com/postfix-email-size-limit/
Thanks a lot for this nice Tutorial!
Thanks for that one !
Hi Christoph!
I’ve set up my own first email server, six month ago, following your guide and all seem works very fine!
Only a doubt:
on the client seem impossible to set the authentication method for the smtp and the imap to “password encrypted”, although I obtain connection error.
On your tutorial, if I’m not mistaken was contemplated. Could you suggest me where I should look into?
many thanks!
hi 🙂
i wanted to know how i am allow /prevent use my mail server (smtp service ) as a smart host in other mail systems ?
how can i configuring it in postfix and control it?
thanks a lot
Hey, nice guide!
I just wanted to share a SQL select query that (although pretty slow) matches allowed relays against aliases and catch-alls. That is something I really wanted to have, as I want to reply to catch-alls via RoundCube. I could not get it to work properly without this :/ If there is a better way, please let me know!
Just create
/etc/postfix/mysql-findtarget.cf
with the following content (replacing the password)
user = mailuser
password = changeMe
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM virtual_users WHERE email=’%s’ UNION ALL SELECT destination as email FROM virtual_aliases WHERE source=’%s’ UNION ALL SELECT destination as email FROM virtual_aliases WHERE source=CONCAT(‘@’, SUBSTRING_INDEX(‘%s’, ‘@’, -1));
Then run
postmap -q something@yourcathalldomain.com mysql:/etc/postfix/mysql-findtarget.cf
and you will see your target, which is allowed to relay the message.
If that worked you can set it via
smtpd_sender_login_maps=mysql:/etc/postfix/mysql-findtarget.cf
(replace the one from this guide) in your /etc/postfix/master.cf and you are golden. You are now able to relay messages that you aliased to the sending account.
I followed your tutorial to set up a mail server.
But I have a problem: I am able to send massage with telnet port 25 and the sending does not require any client authentication.
I would like port 25 to be closed for outgoing messages. Clients will use port 587 instead of 25.
How can I configure postfix to have this working
Hi Christoph!
Cant send any email (( Error 550 5.1.1 Recipient address rejected
What i did wrong?
Thank you
Something wrong with postfix/submission/smtpd
a cant send to gmail (((
I keep getting the following error when trying to send email, this is using the Mail app on my iPhone from my home network, I have followed the guide on this page and executed all specified commands, yet it still gives the below errors:
May 16 00:28:51 tequila postfix/submission/smtpd[13937]: connect from my.external.hostname[12.34.56.78]
May 16 00:28:51 tequila postfix/submission/smtpd[13937]: NOQUEUE: reject: RCPT from my.external.hostname[12.34.56.78]: 554 5.7.1 : Client host rejected: Access denied; from= to= proto=ESMTP helo=
Christof!
I would like to first thank you for creating such an informative write up! I have learned so much following this guide, I feel like I have taken a course. I am pretty much up and running after following your guidance, however; a problem still remains with TLS. Authentication seems to work fine – I can connect to my mail server securely via IMAP port 993 and SMTP port 587 and can send and receive emails. The problem is, when a test email shows up on my gmail account, it is reported with the red lock as unencrypted.
There are a couple of errors that may point to the problem. For one, in my mail.log I keep getting this:
dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=, rip=x.x.x.x, lip=x.x.x.x, TLS handshaking: SSL_accept() failed: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown: SSL alert number 46
This is confusing because I also get this:
dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=19808, secured
Also, when I use:
openssl s_client -connect localhost:25 -starttls smtp
ehlo mydomain.com
Instead of the expected output, I get a bunch of SSL info like my letsencrypt certificate (the actual PGP hash), a TLS session ticket, and some other ID, protocol, and cipher information. Then the only capability listed is ‘250 DSN’ – nothing else.
I have rechecked all of my configuration and everything seems to be fine. Can you possibly point me in the right direction?
Okay, I found the fix. First of all, disregard the errors I mentioned because they either a) were user error on my part or b) have nothing to do with this. Also, it is entirely possible that I skimmed over this configuration setting, I looked for it again but still couldn’t find it.
When setting up Postfix to enable encryption ‘postconf smtpd_tls_security_level=may’ this only takes care of incoming emails. In order to allow for outgoing tls encryption, one must also include:
postconf smtp_tls_security_level=may
Notice the subtle difference? smpd is for inbound, smtp is for outbound
As soon as I added this line to my main.cf, google showed TLS encryption is being used on my outbound messages. I hope that this is correct and that others find it useful. Thanks again for making this resource available. And apologies for misspelling your name, Christoph!