Relaying
Your mail server is almost ready for use. But one puzzle piece is missing. Your users can already fetch emails but they need to be able to send emails, too. Mail server know how to send an email to another mail server by looking up MX records. And the receiving mail server will happily accept any email that is destined for a valid recipient. Mail clients however are expected to send email to their mail server first which then forwards email to the internet. When Postfix receives an email from a user and forwards it to another server this is called relaying. Postfix acts as a relay.
So why do the mail client not work the same way? Couldn’t they also just look up the MX record and send the email to the destination server? Technically, yes. But users are often sitting at home using a DSL line. And such IP networks are usually blocked off from sending mail. Some ISPs block the SMTP port on their routers. And most receiving mail servers check real-time blacklists to block DSL IP addresses. The reason is that home users are more likely to have infected Wind*ws PCs that send out spam.
Am I going too fast? Okay, let’s walk through the different scenarios one by one.
Incoming email
When someone on the internet sends an email to john@example.org, some other mail server will deliver the email using SMTP to your mail server. Postfix will determine that it’s responsible for email addresses in the example.org domain and accept the email. John can then use POP3 or IMAP to fetch the email from your server.
Outgoing email (without authentication)
John is on the internet somewhere and wants to send an email to lisa@example.com. Your mail server is not responsible for the “example.com” domain so it receives John’s email and would have to forward (relay) it to the mail server that is responsible for …@example.com email addresses. This may seem like a harmless scenario but your mail server must deny that:
Why? Because anyone can claim to be John and make your mail server forward mail. If an attacker (like a spammer) would send millions of spam emails in John’s name through your server then other organisations will accuse you as the operator of the mail server of spamming. Your mail server would be what people call an open relay. This is not what you want because your mail server would get blacklisted and you will not be able to send out mail to most other servers. So without any proof that John is actually John your server must reject the email.
Outgoing email (with authentication)
So how does John prove his identity? He needs to use authenticated SMTP. This is similar to the previous case but John’s email program will also send his username and password.
Of course we are making sure that his authentication happens over an encrypted connection so John’s password is blared out.
Postfix setting “mynetworks”
In addition to using SMTP authentication you can tell Postfix to always relay email for certain IP addresses. The mynetworks setting contains the list of IP networks or IP addresses that you trust. Usually you define your own local network here. The reason John had to authenticate in the above example is because he is not sending the email from your local network.
If your users are using the webmail interface running on your mail server then they will not need to authenticate. Roundcube sends email to localhost which is trusted according to the mynetworks setting.
Make Postfix use Dovecot for authentication
Enabling SMTP authentication in Postfix is surprisingly easy. You already configured Dovecot’s regarding user authentication. So let’s just make Postfix utilize that by telling it to ask the Dovecot server to verify the username and password. Postfix just needs some extra configuration. Run these commands on the shell:
postconf smtpd_sasl_type=dovecot postconf smtpd_sasl_path=private/auth postconf smtpd_sasl_auth_enable=yes
This enables SMTP authentication and tells Postfix that it can talk to Dovecot through a socket file located at /var/spool/postfix/private/auth. Do you remember that Postfix runs in a sandboxed chroot directory? That’s at /var/spool/postfix. It cannot access any files outside of that directory. But fortunately in a previous section you edited the /etc/dovecot/conf.d/10-master.conf file and made Dovecot place a socket file into /var/spool/postfix/private/auth to allow communication from Postfix.
Enable encryption
The following settings enable encryption, define the key and certificate and enforce that authentication must only occur over encrypted connections:
postconf smtpd_tls_security_level=may postconf smtpd_tls_auth_only=yes postconf smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.example.org/fullchain.pem postconf smtpd_tls_key_file=/etc/letsencrypt/live/webmail.example.org/privkey.pem postconf smtp_tls_security_level=may
Note #1: In the past you may have configured your smtpd_recipient_restrictions to restrict relaying to authenticated users. Postfix nowadays has setting called “smtpd_relay_restrictions” that deals with relaying requests in the “RCPT TO” phase of the SMTP dialog. So essentially it works like the good old “smtpd_recipient_restrictions” but is checked first. smtpd_relay_restrictions has a reasonable default so authenticated relaying works automatically.
Note #2: Postfix 2.10 has changed the behavior of the smtpd_*_restrictions altogether. If a restriction makes Postfix reject an email it will wait until after the “RCPT TO” line. Read the Postfix documentation for a description of the reasons for that change.
The smtpd_tls_security_level is set to “may” to allow encrypted communication when Postfix receives an email. This not only applies to users sending emails but also remote mail servers that send email to you. It may sound tempting to change that to “encrypt” which would enforce encryption and reject any attempts to create an unencrypted SMTP connection. But unfortunately there are still mail servers in the world that can’t encrypt and would not be able to deliver email to you. So “may” is the proper setting. Same goes for smtp_tls_security_level for outgoing email sent to other mail servers.
Although emails may still get delivered unencrypted from other servers we should make sure that users don’t accidentally (or out of ignorance) send their passwords unencrypted over the internet. That’s what we enforce “smtpd_tls_auth_only=yes”. Now our users should be safe.
How SMTP authentication works
Are you curious how SMTP authentication looks on a protocol level? Let’s go through that. In previous versions of this guide we used “telnet” to connect to TCP port 25 and speak SMTP. But now we enforce encryption and can’t do SMTP authentication unencrypted. Let’s try the unencrypted part:
telnet localhost smtp
The server will let you in:
Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail ESMTP Postfix (Debian/GNU)
Say hello:
ehlo example.com
Postfix will present a list of features that are available during the SMTP dialog:
250-mailtest 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN
Let me briefly explain what these lines mean:
- PIPELINING
This is a feature to speed up SMTP communication. Usually the remote system has to wait for a response to every command it sends. Pipelining allows the remote server to send bulks of commands without waiting for a response. Postfix will just store these commands and execute them one by one. If you told Postfix to forbid pipelining it would disconnect the remote server when it tries to send bulks of commands without waiting for the proper reply. It is mainly a feature against spamming programs that don’t behave. - SIZE 10240000
The remote server is allowed to send emails up to 10 MB large. This has long been a common maximum size for emails. However nowadays 40 MB or even more are more common sizes because emails have grown larger. - VRFY
Allows remote servers to verify a given name or email address. For example the remote server could send “VRFY john” and your server might respond “250 John Doe <john@example.org>”. It can be used to verify that a certain recipient email address is deliverable - ETRN
A command that a remote system can send to flush the Postfix queue of mails for a certain domain. It can be used if the remote system had technical problems and failed to receive email for a while. Then it could send an ETRN command to make your server start sending outstanding emails for that domain. It is rarely used. - STARTTLS
This tells the remote system that it might start switching from this unencrypted to an encrypted connection by sending the “STARTTLS” command. It will then start negotiating a TLS-encrypted connection. You could compare it to an HTTP connection that suddenly switches over to an encrypted HTTPS connection. The advantage is that you can start talking SMTP on TCP port 25 and don’t have to open up a second TCP port like 465 which is the “SSMTP” (secure SMTP) port and only accepts encrypted connections. - ENHANCEDSTATUSCODES
This enables more three-digit return codes for various conditions. See the RFC2034 if you are curious. - 8BITMIME
In ancient times SMTP only processed 7-bit characters. You couldn’t transfer special characters like “Ä” or “ß” without special encoding. 8BITMIME allows a transmission of emails using 8-bit characters. Still many emails are specially encoded using ISO8859-1 or UTF-8. - DSN
It enables DSNs (delivery status notofications) that allows the sender to control the messages that Postfix creates when an email could not be delivered as intended
However one important line is missing here that would allow us to send our username and password:
250-AUTH PLAIN LOGIN
We told Postfix to only allow authentication when the connection is encrypted. So we are not offered authentication over this plaintext connection.
Are you still connected? Okay, good. So we need an encrypted connection using TLS. You could enter STARTTLS:
STARTTLS
And the server would reply:
220 2.0.0 Ready to start TLS
However now it’s getting complicated because you would have to speak TLS encryption which is not a language that humans speak. So let’s quit this using the “QUIT” command.
But we can use OpenSSL to help us with the decryption. Now run:
openssl s_client -connect localhost:25 -starttls smtp
You will see a lot of output. OpenSSL has connected to TCP port 25 and issued a STARTTLS command to switch to an encrypted connection. So whatever you type now will get encrypted. Enter:
ehlo example.com
And Postfix will send a list of capabilities that will look like this:
250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-AUTH PLAIN LOGIN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN
And now that we are using an encrypted connection Postfix offers us to authenticate. So let us send the authentication string with a Base64-encoded password:
AUTH PLAIN am9obkBleGFtcGxlLm9yZwBqb2huQGV4YW1wbGUub3JnAHN1bW1lcnN1bg==
The server should accept that authentication:
235 2.7.0 Authentication successful
Excellent. You are logged in through SMTP. Disconnect from Postfix:
QUIT
Goodbye:
221 2.0.0 Bye
Authentication works. Well done.
Base64-encoded passwords
You may wonder how I got to the long cryptic string that apparently contained John’s email address and his password. It is just a Base64-encoded (not encrypted!) version of the string “john@example.org<NULL-BYTE>john@example.org<NULL-BYTE>summersun”.
One way to create that string is using Perl:
perl -MMIME::Base64 -e \
'print encode_base64("john\@example.org\0john\@example.org\0summersun")';The submission port (optional)
Although I have been talking about SMTP on port 25 to relay mails there is actually a better way: using the submission port on TCP port 587 (as described in RFC 4409). The idea is to use port 25 for transporting email (MTA) from server to server and port 587 for submitting (MSA) email from a user to a mail server. As many users still use SMTP by default you will likely have to run both ports.
Edit your /etc/postfix/master.cf file and add a service for the submission port like this (mind the indentation from the second line on). Don’t use the suggested and commented-out submission service in the master.cf – just ignore it.
submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth -o smtpd_sasl_security_options=noanonymous -o smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf -o smtpd_sender_restrictions=reject_sender_login_mismatch -o smtpd_sasl_local_domain=$myhostname -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
Basically this new service uses the “smtpd” daemon which is the piece of software that responds if you open an SMTP connection on TCP port 25. But it gets a few extra options set…
- in the /var/log/mail.log mark the connections to the submission port as “postfix/submission/smtpd” (syslog)
- enforce encryption on this port (security level)
- enforce user authentication (sasl)
- make sure that an authenticated user can only send emails in their own name (sender_login_maps)
Restart the Postfix server:
service postfix restart
Your users can now use the submission port to send email.
Protecting against forged sender addresses
Wait a minute. We can make sure that a user can only send emails using their own sender address? Cool. How does that work? Take a look at this line we just used above when we added the submission service:
smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf
The smtpd_sender_login_maps is a Postfix mapping with two columns:
- the email address the user wants to use as a sender address
- the username that was used when authenticating at the mail server
The reason I am just using the /etc/postfix/mysql-email2email.cf mapping is that it fits perfectly here. That mapping maps the email address to itself if it exists. Confused? Look at the SQL query that the “email2email” mapping runs:
SELECT email FROM virtual_users WHERE email='%s'
So when a user wants to relay an email Postfix checks the virtual_users table looking for the sender address. Say that John wants to send out an email from his email address “john@example.org”. Postfix will check the virtual_users table if there are any rows with an email field like that. It will find one and return the email field – which is just again the email address. Postfix expects to get back the username that was used to login. We are using the email address as a login username – so that mapping works. Only if John would try to send an email with a forged sender address then Postfix will see that it does not match his account and reject the email. Your mail.log would read something like:
NOQUEUE: reject: RCPT from foo.bar[…]: 553 5.7.1 <forged@email.address>: Sender address rejected: not owned by user john@example.org; from=<forged@email.address> to=<…>
It is your own decision whether you want to use the smtpd_sender_restrictions like that.
Doesn’t `smtpd_sender_login_maps` also require `mysql:/etc/postfix/mysql-virtual-alias-maps.cf` in case you want to send an e-mail from one of your aliases?
That’s right. I haven’t had that case yet. My users usually use their main address and just have aliases pointing to it. Your suggestion totally makes sense. Have you tried that?
Yes I did and it works fine in most cases:
In all scenarios I will assume that john.doe@example.org is the mailbox/account.
Expected results (using jane.doe@example.org as identity):
1) [OK] Alias: source (jane.doe@example.org) to account (john.doe@example.org)
2) [NOK] Alias: source (jane.doe@example.org) to account (non-existing@example.org)
3) [OK] Catch-All: source (@example.org) to account (john.doe@example.org)
4) [NOK] Catch-All: source (@example.org) to account (non-existing@example.org)
“Unexpected”/not-working result (using hello.world@example.org as identity):
source (hello.world@example.org) to alias (jane.doe@example.org)
source (jane.doe@example.org) to account (john.doe@example.org)
The above scenario will fail because it’s 2 levels deep and will thus result in a falsy result. However that’s in my opinion an edge-case. For performance and simplicity I would recommend users to use 1 level-deep aliases anyway.
I have several aliases for my main email address – some of which are there to in case I am registering somewhere I am in doubt whether or not they will spam me.
Some of them are for my gamer IDs – so I was very surprised when I got an 553 error trying to respond to an email to one of those IDs (because in the previous version of the guide it worked).
Please update main text to be the mysql-virtual-alias-maps.cf
Holy crap thank you for pointing this out. It was driving me crazy, couldn’t send email from my alias.
Hi , till this page i been following all steps ,
hostname:domain.com
MX mail.domain.com email handeled by domain.com priority 0
A record mail.domain.com xxxxxx
A record domain.com same xxxxxx
domain.com/webmail :80
#1 roundcub able to send email gmail but wont receive a reply on it , even tried sending from gmail to it its out from gmail but never received ?
#2 i tried to add an email using Thunderbird client i was able to imap mai.domain.com:143 normal pass ,smtp mail.domain:587 ,only with this combination everything else fail ,it will received emails as well never send with errtor “Sending of the message failed.
The message could not be sent because connecting to Outgoing server (SMTP) mail.domain.com failed. The server may be unavailable or is refusing SMTP connections. Please verify that your Outgoing server (SMTP) settings are correct and try again.”?
sorry typo its the opposite :
MX domain.com email handeled by mail.domain.com priority 0
#1 If you dare mention the actual real domain we might be able to check if it works and what error we get when we try to deliver email there.
#2 Please try with port 143, STARTTLS and “password, normal” in Thunderbird. That should work.
Hi Christoph,
I have followed this section and have 2 questions/observations.
1) only one of the certificates for one domain is defined here in smtpd_tls_cert_file and smtpd_tls_cert_key so it will be used for encrypting all the smpt traffic disregarding the domain it will go out from? Would it be possible to have this for each domain separately somehow ?
2) It seem that with this config gmail always receives un-authenticated messages (and is complaining about that fact a lot – actually moving mail straight to spam). Not sure about the cause with my shallow knowledge of postfix.
Any insight would be appreciated
#1 As far as I know SMTP clients do not support SNI (server name indication) that allow them to tell which host name they want to talk to. It’s just important that you present a certificate. “smtpd_*” by the way means the receiving side of Postfix. Any configuration option with “smtpd_” means incoming connections and anything with “smtp_*” is for outgoing.
#2 I believe that “unauthenticated” means that it’s not DKIM signed. Although DKIM signatures will help keep mail out of spam I wonder if that’s really the only reason. Could it be that your IP address is on an RBL (blacklist)? Have you verified that?
Regarding DKIM: I’m working on the section about that and expect it to be online in 1-2 weeks. That may help with the “unauthenticated” issue.
Hi Christoph,
thanks for the reply. and clarification about #1
as for the #2 the IP seems to be listed in one spamlist in sweden (?) but nowhere else. The only thing I can think of is that the domain was moved between countries and maybe in combination with DKIM google just doesn’t like that?
Anyway thanks again and I am looking forward to the DKIM part.
Typo?
postconf smtpd_tls_cert_file=/etc/letsencrypt/live/mail.example.org/fullchain.pem
postconf smtpd_tls_key_file=/etc/letsencrypt/live/mail.example.org/privkey.pem
(since across the guide, we’ve used “webmail.example.org”)
Right, thanks
Hi Christoph,
for me, the VRFY command only works with the full e-mail: VRFY john@example.org – Maybe you can verify this?
Regards,
Andy
Note: If you have any special characters such as @ or ‘ or ! you must put \ in front of it to escape the character.
(i was getting fails until i tried that)
Thanks for the new guide!
I am having trouble with STARTTLS on the 587/submission. I can send mail to the server on 25/smtp and STARTTLS works. But if I send mail via port 587, I am offered STARTTLS, but get a try again later message – 454 4.3.0 Try again later
I did not install rspamd. I am using amavis and spamassassin.
Any hints on how to get this to work? It is working on Jessie.
root@mailtest:/etc/postfix# telnet mail.example.com 587
Trying 12.34.56.78…
Connected to mail.example.com.
Escape character is ‘^]’.
220 mail.example.com ESMTP Postfix (Debian/GNU)
ehlo mailtest.example.com
250-mail.example.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
STARTTLS
454 4.3.0 Try again later
Found it!
submission inet n – – – – smtpd should be submission inet n – y – – smtpd
postfix is running in a jail!
Thanks again! These guides are great and make it easy to set up our own email servers.
OK, last comment in this conversation with myself… maybe
It seems that between Jessie and Stretch the default for chroot has changed from yes to no.
From Stretch:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) ***(no)*** (never) (100)
# ==========================================================================
So any services that are pulled forward from Jessie need to be adjusted accordingly.
Thanks a lot for this guide, just brilliant 🙂
I had no issues so far, ev. worked really great, but now I am not able to authenticate
AUTH PLAIN Base64-encodedPassword
535 5.7.8 Error: authentication failed:
I am able to log on to Roundcube, but if I try it to authenticate here it doesn’t work.
So far I made ev. according to the manual and every check I did was successful.
Any idea what I maybe missed ?
Regards
Hampe
Tests with a client worked well over port 587, seems that I had issues with the Base64-encoded password.
Try this:
perl -MMIME::Base64 -e \
‘print encode_base64(“john\@example.org\0john\@example.org\0summersun”)’;
I copied it from the jessie guide and if it works for me.
Looks like I had a copy paste problem in my CMS. The perl example on the page works now.
Just perfect, it worked like this 🙂
Thanks a lot for your help 🙂
Thanks for the new guide!!!
I want to make sure that an authenticated user can only send emails in their own name, but I have a problem when implementing reject_sender_login_mismatch in the submission port.
I test from 2 different email clients (Evolution and Thunderbird) and the behavior is not the same when changing the identity of the sender.
The following happens with the Evolution client:
– The mail is rejected correctly.
(5.7.1 : Sender address rejected: not owned by user john@example.org)
– In the postfix logs, identiti2@example.org appears in the from.
(Jan 31 09:47:12 mail postfix / qmgr [19885]: 5E4B61C08EC: from = , size = 530, nrcpt = 1 (queue active))
With the Thunderbird client the following happens:
– The mail is not rejected.
– In the postfix logs john@example.org appears in the from.
I found that the difference between the two is in how the headers are made up:
Evoution: Return-Path: identiti2@example.org and From: identiti2@example.org
Thunderbird: Return-Path: john@example.org and From: identiti2@example.org
¿Any idea how to solve the problem?
—-
From roundcube solve it with the following configuration in /etc/roundcube/config.inc.php:
// Set identities access level:
// 0 – many identities with possibility to edit all params
// 1 – many identities with possibility to edit all params but not email address
// 2 – one identity with possibility to edit all params
// 3 – one identity with possibility to edit all params but not email address
// 4 – one identity with possibility to edit only signature
$ config [‘identities_level’] = 3;
What does the mail.log show when you send authenticated from Thunderbird? Are you sure that you set the username for authentication correctly and not accidentally as the identity?
I set up clients again and now it works fine on both. I’m sorry, it was my mistake.
Hi !
First of all, thanks for your tutorial, it’s really useful and complete!
But, I have two remaining problems… I made everything like you, but I still have an SSL issue, and my mails aren’t crypted (which is a problem…). I have to say that I don’t understand where’s my error, because it should be errors in my conf. (I used LetsEncrypt, and write the path in the dovecot ssl file and postfix conf).
I also can’t use the AUTH PLAIN, I have a “authentication failed” each time I try… (I try the perl command line). Maybe a smtp error? I don’t see anything in the log file linked to that.
Hope you may help, and thanks for your tutorial!
(I’m sorry for my English, I’m a French student 😉 )
Hi,
A simplest way to have a base64 encoded string is to use the base64 command in the shell :
echo -n ‘john@example.org john@example.org summersun’ | base64
the result is :
am9obkBleGFtcGxlLm9yZyBqb2huQGV4YW1wbGUub3JnIHN1bW1lcnN1bg==
Thanks for the hint. However I get slightly different strings. Even your string is not exactly the same as what Perl outputs. I tried escaping the @ character as well as adding null bytes. Not sure what I’m missing.
in other examples, you have put the strings to change in bold ( postconf smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.example.org/fullchain.pem ) above and others in that section..
Shouldn’t we protect SMTP on port 25 with “reject_sender_login_mismatch” in the same way we do for submission port?
Sorry for the late response… isn’t that what smtpd_sender_login_maps= is supposed to do?
I believe we should also opportunistically use TLS on outgoing mail, not just incoming:
postconf smtp_tls_security_level=may
Thanks… I don’t know how I missed that.
Hi Christoph
I think this should be mentioned in the submission port text itself, that you have to use
…
-o smtpd_sender_login_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf
…
if you want to send emails with you aliases, since that in most cases is what users want!
Thanks.
Great job again!
I’m having trouble getting Roundcube to send mail via SMTP instead of the PHP mail() function.
Authentication works fine via the steps listed on this page (via telnet and openssl) on both port 25 and 587.
But I am getting a 250 Authentication failed error when trying to send a message through Roundcube.
In /etc/roundcube/config.inc.php I have:
$config[‘smtp_server’] = ‘localhost’ (webserver serving Roundcube is on the same server as Postfix/Dovecot)
$config[‘smtp_port’] = 587; (tried 25 as well, did not work)
$config[‘smtp_user] = ‘%u’; (substitutes the Roundcube username for login, which is the full email address)
$config[‘smtp_pass’] = ‘%p’; (substitutes the password used to log in to Roundcube)
I also tried hardcoding smtp_user and smtp_pass to those of a valid account with no success.
Submitting through the PHP mail() function works just fine, however.
Justin, try to set:
$config[‘smtp_server’] = ‘tls://fqdn.of.your.server’;
After some digging, I got the following to work when trying to authenticate using the “openssl s_client -connect localhost:25 -starttls smtp” command:
perl -MMIME::Base64 -e ‘print encode_base64(“\000john\@example.org\000summersun”)’;
A base64 encode using only one zero in between doesn’t work. Also, I used the username only once.
See http://www.postfix.org/SASL_README.html#server_test.
I am having issues when sending emails from a client using port 587. I can send using port 25 with no problem.
Using Thunderbird I get a message that STARTTLS command is needed but the server does not show it as being available.
I tried using telnet on both ports 25 and 587. I can connect to both ports. When I run the command STARTLS on port 587 I get an error “454 4.3.0 Try again later”. Then EHLO command returns STARTLS as being available.
Anyone have any ideas as what to check to resolve this?
What happens if you “telnet yourmailserver 25” and enter a dummy greeting like “ehlo foobar”? You should get a list like this:
250 jen.workaround.org
ehlo foobar
250-jen.workaround.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
Is STARTTLS shown?
Yes, on both ports. When I run STARTTLS on port 25 the command works, when I do it on port 587 I get the error.
On Port 25:
220 dandderwen.uk ESMTP Postfix (Raspbian)
EHLO dandderwen.uk
250-dandderwen.uk
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
STARTTLS
220 2.0.0 Ready to start TLS
On port 587
220 dandderwen.uk ESMTP Postfix (Raspbian)
EHLO dandderwen.uk
250-dandderwen.uk
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
STARTTLS
454 4.3.0 Try again later
I have found what is causing the problem. I was using SpamAssassin because rspamd is not available for armhf unless I build it from the sources.
I just disabled SpamAssassin and I can send mail via port 587. I found this error in the logs “warning: connect to Milter service unix:/spamass/spamass.sock: No such file or directory”.
Why this only happens on port 587 I don’t know. I will need to investigate further.
Hello,
Thanks for this guide it’s excellent.
But I have a problem. My server doesn’t protect for against forged sender addresses.
This options:
“-o smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf
-o smtpd_sender_restrictions=reject_sender_login_mismatch”
in master.cf file not working.
Any ideas?
Regards
Hello Christoph ,
Thank you for the guide , it is an excelent guide to install and configure an email sistem, very good explanations and very good examples.
I have seen taht the email that are sent by the server are not encrypted , so I suggest to add:
smtp_tls_security_level = may
in the configuration to have the outbound connection encrypted to ( in the config only the inbound connections are encrypted if the inbound server can do it ).
If you want to enable smtp via TLS on port 465 you can use the below config.
Notice if your clients are using SSL/TLS as protocol and not STARTTLS the option
smtpd_tls_wrappermode=yes
Is important.
I struggled 2-3 hours with Thunderbird.
With postfix in double verbose mode I found the reason.
At first I had just copied the submission configuration and changed the submission to smtps.
A lesson learned.
#port 465
smtps inet n – – – – smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_security_level=encrypt
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_sasl_type=dovecot
-o smtpd_sasl_path=private/auth
-o smtpd_sasl_security_options=noanonymous
-o smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf
-o smtpd_sender_restrictions=reject_sender_login_mismatch
-o smtpd_sasl_local_domain=$myhostname
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
Weird problem. Set up and able to log in OK through roundcube, also telnet tests all work OK. If I send an email from my gmail account it doesn’t get there, but even more annoying, when I send an email from roundcube to my gmail account it doesnt leave the server. In the logs I get the error message that my gmail address cannot be found in the virtual domains table. It seems it is treating the outgoing mail as incoming!
That was the virtual alias maps it was looking in for my gmail address! Sorry typo in original, still not sending emails out and all sent emails sitting in postfix queue. Being a fairly old man, I cant afford to pull out any more hair
RESOLVED
After a night’s sleep, I came back to the problem and saw what the error was. The clues were there last night but I was a bit too tired to see them properly. Using SWALK, I got a 450 error temporary look up error. Logs showed look up error on virtual aliases table. But phpmyadmin and roundcube logged in OK. Showing it had to be an error with mailuser. I double checked the .cf files in postfix and could see no problems. Minutely close examination showed that copying over mailuser’s password from a temporary file I had used to transfer it during set up, some how an extraneous character had crept in at the end. So eidt the .cf files, recompile them, restart Postfix and all is working again.
Christoph, Thanks for your posts on how to configure and run a mail server. – Thanks to you, I’ve been running my own mail server for over 1 year.
I have an issue with receiving forward emails. Specifically, if I send an email as bob@mydomain.com to lisa@example.com , and lisa has her email set to be forwarded to sam@mydomain.com . Postfix will reject the email:
NOQUEUE: reject: RCPT from mx1.examle.com[111.222.333.444]: 553 5.7.1 :
Sender address rejected: not logged in; from= to=
This seems to be tied to:
smtpd_sender_restrictions = reject_sender_login_mismatch
An internet search brought me to this page:
http://postfix.1071664.n5.nabble.com/553-5-7-1-Sender-address-rejected-not-logged-in-td15073.html
With the relevant text stating:
This is a feature that forces every address that can be looked up in
$smtpd_sender_login_maps (apparently all of your valid local
recipients?) to authenticate in order to be used as a sender. This
feature is not usually suitable on a server that handles anything
other than initial mail submission.
And no resolution is listed in the thread.
Any ideas on how to allow the forwarded email to be accepted?
Thanks
I wonder why the log reads “Sender address rejected: not logged in”. Did you use authenticated SMTP?
smtpd_sender_login is pretty restrictive. But most webmail providers I came across seem to enforce that you send only as yourself and not anyone else in the domain. If that causes trouble with virtual_aliases please see the first comment on this page to reflect forwarding and this restriction.
Christoph,
Thanks for the reply. – I’m not sure what your mean “Did I use authenticated SMTP?” – My mail client, using the submission port connected to postfix, postfix received the mail, postfix sent the mail to the remote server, the remote server accepted the mail, the remote server then forwarded the mail back to my server, my server rejected the mail because the from was listed as bob@mydomain.com (one of my users).
As I am a novice at mail server administration, is it common/acceptable for a mail server to keep the original “from” user on a forwarded email? – (It appears that the website stripped the users from my 1st post. The postfix log entry showed from=bob@mydomain.com). I just forwarded an email from my gmail account to one of my users and the mail was of course accepted, and the log entry showed from=myaccount@gmail.com .
It seems to me that the remote mail server is not changing the from.
I’m a bit confused. No, mail server are not supposed to change the sender address. But as long as the email is forwarded to a valid recipient that should still work. But “Sender address rejected” normally happens if you relay emails through your own mail server but you authenticated as the wrong user or (as shown in your logs) do not authenticate at all.
Many thanks for this blog series — the best one I have ever seen about setting up an e-mail server.
However, I hope you can shed some light on this:
I’m struggling with From: address spoofing — in Thunderbird, I can Edit my From address from xyz@my.tld to fake.address@my.tld and it will still be delivered, even though fake.address@my.tld is not an alias of xyz@my.tld.
Relevant lines in master.cf:
-o smtpd_sender_login_maps=sqlite:/etc/postfix/oli-vmail/virtual_alias_maps.cf,sqlite:/etc/postfix/oli-vmail/email2email.cf
-o smtpd_sender_restrictions=reject_sender_login_mismatch
virtual_alias_maps.cf has the query:
SELECT destination FROM virtual_aliases WHERE source = ‘%s’
email2email.cf has the query:
SELECT email FROM virtual_users WHERE email = ‘%s’
Oct 11 21:00:07 mail.info postfix/submission/smtpd[9107]: E622620231A: client=*.*.*[*.*.*.*], sasl_method=PLAIN, sasl_username=xyz@my.tld
Oct 11 21:00:07 mail.info postfix/cleanup[9110]: E622620231A: message-id=
Oct 11 21:00:08 mail.info postfix/qmgr[103]: E622620231A: from=, size=3908, nrcpt=1 (queue active)
I should note than in the received e-mail, the headers read:
Return-Path:
From:
Many thanks,
Olivier
Hi Christoph,
I’ve been using Postfix/Dovecot setup for years now thanks also to your tutorial. I found out by experience some comments/annotations for your tutorial:
– When users enable the submission port, they want to use it as the only way for a user to send mails, so you should tell in the “The submission port (optional)” section to disable authentication on the SMTP 25 standard port (smtpd_sasl_auth_enable=no in the main.cf). In this way users will not be allowed to use 25 to send mails.
– I also noticed that in a mail server setup with submission port enabled, others mail servers can send emails that have “From” addresses belongings to our domains and they are delivered correctly. Assuming the setup is done to force users (who have the permission to send email from ours domains) to use the submission port, we can ban receiving these emails on the 25 port. I come to this simple solution:
In the main.cf file we add:
smtpd_sender_restrictions = check_sender_access mysql:/etc/postfix/mysql-virtual-block-internal-domains-outside-maps.cf
smtpd_restriction_classes = block_internal_domain_outside
block_internal_domain_outside = reject
In the file mysql-virtual-block-internal-domains-outside-maps.cf:
… authentication and db selection …
query = SELECT ‘block_internal_domain_outside’ FROM virtual_domains WHERE name=’%d’ AND enabled=1
In this way when some other mail server tries to send us an email that has the sender address containing a domain we own, this mail gets rejected.
Thanks anyway,
Misha
Hi Christoph,
Thanks to you I’ve got the mail server up and running successfully. I just have one issue, I would like my mail server mail.example.com to allow my website example.com to send notifications to subscribers via smtp. I’ve been searching on your site and the internet for several days and I’ve not found anything that I understand to make it work. I’m able to send and receive mail via IMAP with the Thunderbird client. The firewall ports are open for 25, 110, 143, 465, 587, 993, 995. Also, I can send notifications to subscribers from example.com using Amazon and Gmail smtp relay without any problems. I’ve looked at the mail.log and I don’t seen anything regarding any error that give me any clue as to would I need to address. I’m sure it’s something simple that I’m missing. Please advise.
If your mail.log doesn’t show anything then perhaps your web site is not sending out notifications. Or it is sending elsewhere but not your mail server. I would expect at least a “relaying denied” error.
Your mail server needs to allow relaying. That can happen either by adding your web server’s IP address to Postfix’ “mynetworks” setting. Or by creating a mail account and using authenticated SMTP to send emails from your web server.
Christoph,
I’ve changed the main.cf per your instructions with no success. NOTE: I’ve replace my server hostname/domain name with mail.example.com or example.com
I’ve tried both port 465 and 587 using the same login credentials on my Joomla site that worked successfully in Thunderbird.
When I use Amazon’s relay I immediately get the success message and the test mail is delivered to the appropriate email user.
Here’s the Message I get immediately upon a successful smtp connection using Amazon’s relay: “The email was sent to support@example.com using SMTP. You should check that you’ve received the test email.”
The mail log is still not showing any error when I try to smtp from my site. This may be due to the Joomla log error, “unable to connect to ssl://ecom.example.com:465”
This is the first mail server I’ve successfully got to work from scratch, so all this information is new to me and I’m still trying to sort things out (I’ve learned much from your instructions). Please look over the below for me and give me some clue as to what needs to be done to make this work via smtp from my website. What am I missing here?
——– Postfix main.cf settings
#myorigin = /etc/mailname
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no
# appending .domain is the MUA’s job.
append_dot_mydomain = no
# Uncomment the next line to generate “delayed mail” warnings
#delay_warning_time = 4h
readme_directory = no
# See http://www.postfix.org/COMPATIBILITY_README.html — default to 2 on
# fresh installs.
compatibility_level = 2
# TLS parameters
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = mail.example.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, mail.example.com, localhost.example.com, , localhost
relayhost =
#mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 xx.xxx.xx.xx/24
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
## All of the below added to configure per Workaround instructions
virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp
## Relaying with SMTP authentication via Dovecot
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
## Enable encryption
smtpd_tls_security_level = may
smtpd_tls_auth_only = yes
smtp_tls_security_level = may
## Make Postfix use rspamd
smtpd_milters = inet:127.0.0.1:11332
non_smtpd_milters = inet:127.0.0.1:11332
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
———————————-
Mail log as follows
—– Successful authentication from Thunderbird Port on 465
Nov 7 21:51:35 ecom postfix/smtps/smtpd[8514]: connect from 107-208-xxx-xxx.lightspeed.okcbok.sbcglobal.net[107.208.xxx.xxx]
Nov 7 21:51:35 ecom postfix/smtps/smtpd[8514]: 88D68A2AAA: client=107-208-xxx-xxx.lightspeed.okcbok.sbcglobal.net[107.208.xxx.xxx], sasl_method=PLAIN, sasl_username=support@example.com
Nov 7 21:51:35 ecom postfix/cleanup[8516]: 88D68A2AAA: message-id=
Nov 7 21:51:36 ecom postfix/qmgr[6114]: 88D68A2AAA: from=, size=733, nrcpt=1 (queue active)
Nov 7 21:51:36 ecom postfix/smtp[8518]: connect to gmail-smtp-in.l.google.com[2607:f8b0:400e:c09::1b]:25: Cannot assign requested address
Nov 7 21:51:36 ecom postfix/smtps/smtpd[8514]: disconnect from 107-208-xxx-xxx.lightspeed.okcbok.sbcglobal.net[107.208.xxx.xxx] ehlo=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=6
Nov 7 21:51:36 ecom postfix/smtp[8518]: 88D68A2AAA: to=, relay=gmail-smtp-in.l.google.com[74.125.195.26]:25, delay=1.1, delays=0.62/0.01/0.19/0.27, dsn=2.0.0, status=sent (250 2.0.0 OK 1541649096 q22-v6si2555357pgb.368 – gsmtp)
Nov 7 21:51:36 ecom postfix/qmgr[6114]: 88D68A2AAA: removed
—– Successful authentication from Thunderbird Port on 587
Nov 7 21:49:10 ecom postfix/submission/smtpd[8496]: connect from 107-208-xxx-xxx.lightspeed.okcbok.sbcglobal.net[107.208.xxx.xxx]
Nov 7 21:49:10 ecom postfix/submission/smtpd[8496]: B306AA2AAA: client=107-208-xxx-xxx.lightspeed.okcbok.sbcglobal.net[107.208.xxx.xxx], sasl_method=PLAIN, sasl_username=support@example.com
Nov 7 21:49:10 ecom postfix/cleanup[8499]: B306AA2AAA: message-id=
Nov 7 21:49:11 ecom postfix/qmgr[6114]: B306AA2AAA: from=, size=733, nrcpt=1 (queue active)
Nov 7 21:49:11 ecom postfix/smtp[8501]: connect to gmail-smtp-in.l.google.com[2607:f8b0:400e:c09::1b]:25: Cannot assign requested address
Nov 7 21:49:11 ecom postfix/submission/smtpd[8496]: disconnect from 107-208-xxx-xxx.lightspeed.okcbok.sbcglobal.net[107.208.xxx.xxx] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Nov 7 21:49:12 ecom postfix/smtp[8501]: B306AA2AAA: to=, relay=gmail-smtp-in.l.google.com[74.125.195.26]:25, delay=1.8, delays=0.68/0.01/0.26/0.84, dsn=2.0.0, status=sent (250 2.0.0 OK 1541648952 w2si2277291pgs.264 – gsmtp)
Nov 7 21:49:12 ecom postfix/qmgr[6114]: B306AA2AAA: removed
———————————-
—– Joomla Error log for test email notification from my site
2018-11-08T02:55:22+00:00 ERROR 107.208.xxx.xxx mail Error in JMail API: Connection: opening to ssl://ecom.example.com:465, timeout=300, options=array (
)
2018-11-08T02:56:25+00:00 ERROR 107.208.xxx.xxx mail Error in JMail API: Connection failed. Error #2: stream_socket_client(): unable to connect to ssl://ecom.example.com:465 (Connection timed out) [/var/www/html/example.com/libraries/vendor/phpmailer/phpmailer/class.smtp.php line 294]
2018-11-08T02:56:25+00:00 ERROR 107.208.xxx.xxx mail Error in JMail API: SMTP ERROR: Failed to connect to server: Connection timed out (110)
Looks like Joomla cannt reach your mail server on port 465 at all. What if you run “telnet econ.example.com 465” from the Joomla server? I would assume you do not get a connection.
Does your (or the ISP’s) firewall block port 465?
Christoph,
I agree Joomla is not reaching my mail server on port 465. What’s weird is Joomla works just fine with Amazon SMTP relay on port 465 with no issue at all. I’ve never run telnet from Joomla, so I have to research how to do that. The server is not on a home ISP, it’s a VPS with a static IP that I alone manage, so port 465 is not blocked because I have it open in the iptables. Here again, when I use Thunderbird SMTP via port 465 or 587 no problem connecting. I suppose it could be a bug with Joomla. Scratching my head on this one. Any other suggestions other than to telnet from Joomla?
My server receives email. I can send using roundcube, however using a mail client to send does not work. I openssl connect as per the example to both 25 and 587, but I always get authentication failed when using the base64 encoded password. I have used echo -n ‘john@example.org john@example.org summersun’ | base64 to get my password and copied and pasted and get the same result.
Christoph,
I got it working now.
I traced down the issue to one iptable rule on the web-server not the mail-server that was blocking the connection. Still, weird because the rule doesn’t interfere with the Amazon or Gmail relay from working with the web-server.
Nice guide.
ETRN is intended for servers which are not always online, very commen before the time of web browsers. If the sending server is only retrying now and then, boths severs may never meet. ETRN is the method to trigger retrying for the domains destined for the conntected server.
Hello Christoph,
I write a post only to notify you that there are some pictures that don’t load correctly on this page.
Thanks 🙂
I wonder how those links got broken. Thanks for the note. Should be fixed now.
Same problem also on Jessie tutorial:
https://workaround.org/ispmail/jessie/relaying-smtp-authentication
Thanks 🙂
Hi Christoph,
I have me too the “retry again” message when I try to send an email with STARTTLS over port 587.
All work well if I send using port 25 but not on 587.
I notice this error on the mail.err log when I try to use the port 587.
postfix/submission/smtpd[15138]: error: open /etc/postfix/mysql-email2email.cf: No such file or directory
How can I solve this problem?
Thank you
Never mind… I miss a part of the tutorial… sorry
Thank you 🙂
I think we also set as below.
postconf myhostname=example.org
postconf myorigin=example.org
postconf masquerade_domains=example.org
some server reject my mail relayed by my SMTP with the massage, “Helo command rejected: need fully-qualified hostname.”
Hi. Thanks for the guide. If I want to relay my mail through my isp, what should I include? I am pretty sure I need to fill in relayhost and port (587) and authentication. Should this be done with password authentication or certificates? Can this be done by adding the following?
postconf smtp_tls_cert_file=/etc/letsencrypt/live/webmail.example.org/fullchain.pem
postconf smtp_tls_key_file=/etc/letsencrypt/live/webmail.example.org/privkey.pem
postconf smtp_tls_security_level=may
Do i have to add something else?
Thanks in advance.
Hello.
There is a page describing relaying through ISP – https://workaround.org/ispmail/wheezy/sending-from-dynamic-ip-address
Also, make sure that your SPF and TXT records in DNS are set to reflect your usage of ISP.
Hello.
There is another page describing relay using ISP – https://workaround.org/ispmail/wheezy/sending-from-dynamic-ip-address
Ratko
Thank you for your reply. This is what I was looking for. I did something similar.
Another question (this is not the place to ask this, i know): Do you know how to make it so that the display name becomes the full name of the user? For example John Doe , and it will show as John Doe in the inbox.
If you have mail setup like one described in a ISPmail guide, once you login to Roundcube you can configure mail user identity, among other things which full name will be associated with specific mailbox.
Yes, I have. Just wondered if it could be done by adding another column in the database. Thank you for the reply.
I added a check so I can send mail with alias emails that are related to my account.
You have to use unionmap for postfix or UNION for mysql.
smtpd_sender_login_maps=unionmap:{mysql:/etc/postfix/mysql-email2email.cf,mysql:/etc/postfix/mysql-virtual-alias-maps.cf}
smtpd_sender_restrictions = reject_unknown_sender_domain,reject_sender_login_mismatch
Or use UNION.
smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email-and-alias-maps-combined.cf
mysql-email2email-and-alias-maps-combined.cf
user = mailuser
password = password
hosts = localhost
dbname = mailserver
query = SELECT destination FROM virtual_aliases WHERE source=’%s’ UNION SELECT email FROM virtual_users WHERE email=’%s’
Hello All from Paris
Very good job, my server is running perfect thank you
I have different emails (gmail/yahoo) and i would like to centralize all with Roundcube and my email server
I read different articles about fetchmail, is it a good solution? Can you help me with
Thanks
Fred