Authenticated SMTP

Relaying

Before we dive into SMTP authentication I want you to understand what relaying actually means. When Postfix receives an email and needs to forward it to another server this is called relaying.

Incoming email

When someone on the internet sends an email to john@example.org some other mail server will deliver the email using SMTP to your mail server. Postfix will determine that it's responsible for email addresses in the example.org domain and accept the email. John can then use POP3 or IMAP to fetch the email from your server.

Outgoing email (without authentication)

John is on the internet somewhere and wants to send an email to lisa@example.com. As your mail server is not responsible for the "example.com" domain it would have to forward the email to the responsible mail server. So your server receives John's email and forwards (relays) it to the mail server that is responsible for ...@example.com email addresses. This may seem like a harmless case your mail server will deny that:

Why? Because anyone can claim to be John and make your mail server forward mail. If an attacker (like a spammer) would send millions of spam emails through your server then other organisations will accuse you of spamming. Your mail server would be what people call an "open relay". This is not what you want because your mail server's IP address would get blacklisted and you will have serious trouble ever sending out mail again. So without any proof that John is actually John your server will reject the email.

Outgoing email (with authentication)

So how does John send his email? He needs to use authenticated SMTP. This is similar to the previous case but John's email program will send his username and password first.

mynetworks

In addition to using SMTP authentication you can tell Postfix to always relay email for certain IP addresses. The mynetworks setting contains the list of IP networks or IP addresses that you trust. Usually you define your own local network here. The reason John had to authenticate in the above example is because he is not sending the email from your local network (usually).

Enabling SMTP authentication in Postfix

Authenticated SMTP with Postfix has been a hassle in the past. It was done through the SASL (Simple Authentication and Security Layer) library that was once part of the Cyrus mail server. It was nearly impossible to debug and threw error messages that were gibberish and misleading. Fortunately nowadays we can make Postfix ask the Dovecot server to verify the username and password. And as you already configured Dovecot's authentication part this is really easy now. Postfix just needs some extra configuration.

postconf -e smtpd_sasl_type=dovecot
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_sasl_auth_enable=yes
postconf -e smtpd_recipient_restrictions=" \
  permit_mynetworks \
  permit_sasl_authenticated \
  reject_unauth_destination"

smtpd_sasl_auth_enable enables SMTP authentication altogether. And the smtpd_recipient_restrictions define rules that are checked after the remote user sends the RCPT TO: line during the SMTP dialog. In this case relaying is allowed if:

  • permit_mynetworks: the user is in the local network (mynetworks) or
  • permit_sasl_authenticated: if the user is authenticated or
  • reject_unauth_destination: the mail is destined to a user of a domain that is a local or virtual domain on this system (mydestination, virtual_alias_domains or virtual_mailbox_domains).

There are further restrictions (smtpd_client_restrictions, smtpd_helo_restrictions, smtpd_sender_restrictions) that get checked during the different states of the SMTP dialog (IP connection, HELO/EHLO command, MAIL FROM command) but for now you should put all restrictions into the smtpd_recipient_restrictions.

If you are curious to see how an authenticated SMTP session works on a TCP level then the Lenny tutorial has information on that.

40 Comments

I have some strange problems

I have some strange problems with sending emails to every remote domian. In logs I get something like:

May 4 19:56:03 psgroove postfix/smtpd[1233]: E664B1ABC025: client=unknown[127.0.0.1], sasl_method=plain, sasl_username=user@localdomian.net
May 4 19:56:39 psgroove postfix/cleanup[1256]: E664B1ABC025: message-id=<20110504175603.E664B1ABC025@localdomian.net>
May 4 19:56:39 psgroove postfix/qmgr[1044]: E664B1ABC025: from=<user@localdomian.net>, size=345, nrcpt=1 (queue active)
May 4 19:56:39 psgroove postfix/smtp[1257]: fatal: unknown service: smtp/tcp
May 4 19:56:40 psgroove postfix/qmgr[1044]: warning: private/smtp socket: malformed response
May 4 19:56:40 psgroove postfix/qmgr[1044]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description
May 4 19:56:40 psgroove postfix/master[1039]: warning: process /usr/lib/postfix/smtp pid 1257 exit status 1
May 4 19:56:40 psgroove postfix/master[1039]: warning: /usr/lib/postfix/smtp: bad command startup -- throttling
May 4 19:56:41 psgroove postfix/error[1258]: E664B1ABC025: to=<user@gmail.com>, relay=none, delay=54, delays=52/1.1/0/0.81, dsn=4.3.0, status=deferred (unknown mail transport error)
May 4 17:56:46 psgroove postfix/smtpd[1233]: disconnect from unknown[127.0.0.1]

Sending emails between a few local domains works perfect, also receiving emails from remote domains works.

I found some tips that this error gets because of /etc/services, but:

# grep smtp /etc/services
smtp 25/tcp mail
ssmtp 465/tcp smtps # SMTP over SSL
# grep smtp /var/spool/postfix/etc/services
smtp 25/tcp mail
ssmtp 465/tcp smtps # SMTP over SSL

So there must be some other reason.

Question please?

Should;

postconf -e smtpd_recipient_restrictions=permit_mynetworks

and

postconf -e smtpd_recipient_restrictions=permit_sasl_authenticated

write to main.cf?

Everything else has written to there, but for some reason, those two lines are not appearing.

If so, is there any reason why they can't be written in manually?

I'm getting an authentication error and it's reasonably clear to me from /var/log/mail.log, it's because my local network is not being permitted.

The relevant part of the log says:

reject: RCPT from unknown [192.168.0.27] 554 5.7.1 <private email address> Relay access denied from=<john@example.org> to=<private email address> etc. (I've used <private email address> in place of the actual address in the log).

The last thing I want to do however, is end up with an open relay, so I'm treading very carefully.

Thanks for your reply.

Please understand Christoph, I would not wish to be rude, but for me, using the Lenny tutorial for this part was a little clearer and I can now send emails from my server; YES!!!

When I try and receive emails however, I get this error message:

Sending of password did not succeed. Mail server 192.168.0.17 responded: Authentication failed.

This is using john@example.org with a password of summersun. In other words, exactly as your tutorial suggests.

The error log shows:

May 25 01:21:25 mail dovecot: pop3-login: Disconnected (auth failed, 3 attempts): user=<john>, method=PLAIN, rip=192.168.0.11, lip=192.168.0.17

Any guidance would be gratefully received - thank you.

OK

Please keep in mind, this is written from a total newbie to email servers, who therefore needs to be spoon fed. Also please keep in mind, with my set-up, copy and paste is not an option. Everything I do has to be hand typed from reading it on a screen, using a KVM switch on to the server and typing the command in.

I was having a problem with getting a local network machine to see the test emails (john@example.org) off of the email server. On examining the error log, it was reasonably apparent (even to me!) the issue was about the local machine not being able to authenticate with the server and at first, I couldn't understand why.

I had already completed your previous tests successfully using mutt on the server. Further examination of the main.cf file, indicated there was no reference to permit_mynetworks and permit_sasl_authenticated. You kindly found the "\" errors in the tutorial (see posts above), but from what I was reading, I couldn't work out what the result in the main.cf file was supposed to look like. Was it a single line separated with \ or two lines? Should there be spaces between the text and the \ and did that matter?  I really wasn't sure. So I tried multiple variations with no improvement.

I then referred to the Lenny tutorial to see if there was anything different and sure enough there was.

Specifically, you referred to this:

$> postconf -e mynetworks=192.168.50.0/24

That one command, was a game changer, because now I could see where mynetworks was getting it's information from.

I also found the command:

$> postconf -e smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination

which for me, was much easier to relate to. Comma separated values have long been a common approach to command separation and made sense to me. I guess, the "\" does the same thing, though I've personally, never come across it before.

So in conclusion, this is written with all the best intentions, not as some sort of flame was against your excellent tutorial and I thank you once again, for the effort you put into this, to help people just like me. I hope this helps.

By the way, Christoph don't

By the way, Christoph don't mind but my opinion the old one is much better. It explains things like doing an SSL auth through telnet which is definitely a convenient way to test it and lots of other useful tips.

I suggest everyone to have a look at the old one.

:(

Sadly i had the same problem but didn't read/find the commetaries. Also found the comma problem checking a prior instal·lation I did with lenny.

smtp auth faild

Unable to access postfix smtp via Thunderbird

Everything seems to be working fine. I can send and receive emails from the from server and access them remotely via IMAP. But Thunderbird doesn't recognize the SMTP. When I try to create an account in it, it recognizes the IMAP only, cycling through the SMTP possibilities before giving up.

any sugestions ?

Regards.

Same here!

Have you tried to telnet from remote location to your email server? Do you get anything to your mail.log? it works with telnet just fine but thunderbird is no go and I do not get anything smtp related to my mail.log. It would be much appreciated if anyone could point us in the right direction. I'm sure we are not the only ones facing this problem. 

relay access denied

I have done your tutorials in the past, and for some reason I can't get this one to allow outside relaying.

NOQUEUE: reject: RCPT from c-71-233-250-111.hsd1.ma.comcast.net[71.233.250.111]: 554 5.7.1 <blah@blah.net>: Relay access denied; from=<john@example.com> to=<blah@blah.net> proto=ESMTP helo=<VOID>

I have no clue where to go from here.  Everything else works except relaying

I just tested authenticated

I just tested authenticated SMTP via telnet. I found out that if i authenticate as john@mydomain.net I can send mail from carl@mydomain.net. This seems strange to me. (However, if I don't authenticate I can't send mails to other mail servers as intended.)

Yes, I know that.

Yes, I know that. Authentication is only required when sending mails to other mail servers. I think that a user of my mail server shouldn't be able to send mails as another user on my server. But after having authenticated as, for example, john@mydomain.net, I can specify "mail from:<jack@mydomain.net>" in SMTP and Postfix won't complain.

I'm sorry, it seems I didn't

I'm sorry, it seems I didn't understand your question. Yes, if you followed this howto any user of your server will be able to send mails as anyone else. If you find how to avoid it (I want users being able to send mails only with their account + their aliases), please let me know.

Hey there. After reading a

Hey there. After reading a bit of Postfix's documentation, I found out about the setting smtpd_sender_login_maps. In combination with reject_sender_login_mismatch in smtpd_sender_restrictions, this seems to be the solution to this problem.

I created a file /etc/postfix/mysql-sender-login-maps.cf:

user = mailuser
password = mailuser2011
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM virtual_users WHERE email = '%s' UNION SELECT destination FROM virtual_aliases WHERE source = '%s'

As you can probably see from the query, Postfix passes the given envelope sender and expects the SASL username that is authorized to send mails from this address to be returned.

Then I changed /etc/postfix/main.cf:

smtpd_sender_login_maps = mysql:/etc/postfix/mysql-sender-login-maps.cf
[...]
smtpd_sender_restrictions = reject_sender_login_mismatch

And after a postfix reload, it seems to work! Postfix allowed me to send mails from my real email address and from all of my aliases, and rejected me if I tried to send mails as another user. (This has the nice side effect of blocking unauthenticated mails from user@mydomain.net to user@mydomain.net which wouldn't be blocked otherwise.)

Struggling with SMTP

I have a working postfix and dovecot that talk to the database and receive mail. Unfortunately, I am struggling hard to connect over SMTP. The weird thing is that it works if I create a base64 encoded string using the java file doanloadable here:

http://chiralsoftware.com/linux-system-administration/source/base64password.jar

 

No other applications or scripting engines are able to produce a base64 encoded string that works! None of my mail applications can connect to the server but I am able to connect using telnet and "auth plain" when using the base64 encoded username and password generated by the java file downloaded.

The base64 encoded string that works is shorter and does not end in any equal "=" signs like the other generated.

What am I doing wrong? I'm pulling my hair out over here. Been working on this for weeks.

Matthias R. Wiora

Hello,

please be aware - even if it's hard to understand... - with the configuration of the tutorial e.g. outlook 2003 clients will be denied by sending via SMTP due to problem regarding the authentication.

You have to replace in /etc/dovecot/dovecot.conf the following

 

auth default {
  # Space separated list of wanted authentication mechanisms:
  #   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
  #   gss-spnego
  # NOTE: See also disable_plaintext_auth setting.
  mechanisms = plain
}

though:

auth default {
  # Space separated list of wanted authentication mechanisms:
  #   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
  #   gss-spnego
  # NOTE: See also disable_plaintext_auth setting.
  mechanisms = plain login
}

to get authenticated SMTP working.

µatthias

log files from sending an receiving mails

Hello together,

here are the log files from sending an

/var/log/mail.log

Sep 30 20:15:03 mopple postfix/master[28969]: warning: process /usr/lib/postfix/smtpd pid 30742 exit status 1
Sep 30 20:15:03 mopple postfix/master[28969]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling
Sep 30 20:15:33 mopple postfix/pickup[30623]: 49C3FD22224: uid=0 from=<root>
Sep 30 20:15:33 mopple postfix/cleanup[30752]: 49C3FD22224: message-id=<20110930181533.GA30738@s5y.de>
Sep 30 20:15:33 mopple postfix/qmgr[28972]: 49C3FD22224: from=<root@s5y.de>, size=395, nrcpt=1 (queue active)
Sep 30 20:15:33 mopple postfix/smtp[30754]: 49C3FD22224: to=<private Mail>, relay=gmail-smtp-in.l.google.com[74.125.39.26]:25, delay=0.67, delays=0.04/0.01/0.19/0.44, dsn=2.0.0, status=sent (250 2.0.0 OK 1317406533 r26si3071311fam.53)
Sep 30 20:15:33 mopple postfix/qmgr[28972]: 49C3FD22224: removed
 

the answer mail

Sep 30 20:18:05 mopple postfix/smtpd[30764]: fatal: parameter "smtpd_recipient_restrictions": specify at least one working instance of: check_relay_domains, reject_unauth_destination, reject, defer or defer_if_permit
Sep 30 20:18:06 mopple postfix/master[28969]: warning: process /usr/lib/postfix/smtpd pid 30764 exit status 1
Sep 30 20:18:06 mopple postfix/master[28969]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling
Sep 30 20:19:06 mopple postfix/smtpd[30765]: fatal: parameter "smtpd_recipient_restrictions": specify at least one working instance of: check_relay_domains, reject_unauth_destination, reject, defer or defer_if_permit

Order of smtpd_recipient_restrictions matters!

Hi there,

just busted 2 hours at getting the smtp send to work without success. Sending from the localhost via webmail (roundcube) was working just fine but I couldn't get it to work from thunderbird. Authentication etc all worked fine, but the log files showed a reject for relaying any mail from an external sender such as Thunderbird. Turns out that I had the order of parameters the wrong way around as:

smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, permit_sasl_authenticated

postfix will stop after the reject_unauth_destination line and doesn't consider the next parameter at all.

The correct way around is

smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

 

possible trap: postfix reload vs. restart

hi there,

a small note for whom it may concern:

Took me days to figure out a strange behaviour of my freshly installed mailserver. It would not take any SMTP connections from outside mynetworks. Not via telnet, Thunderbird nor others. No logfile entries, just "connection rejected".

"netstat -an |grep LISTEN" showed

...

tcp        0      0  127.0.0.1:25              0.0.0.0:*               LISTEN

...

After lots of swearing and reconfiguring i finally did reSTART, not reLOAD the postfix server, from then on: everything as it should be. Stupid me!

Hope that helps at least one of you!

greets,

Oliver

 

p.s. still anybody developing this tutorial? The mailinglist is suspiciously quiet these days...

Test it with swaks

apt-get cache show swaks

apt-get install swaks

swaks --to test1@example.com --from john@example.org -tls --server 127.0.0.1 --auth LOGIN --auth-user john@example.org --auth-password summersun

 

Have fun :-)

solved 554 error

after some other research i found in my master.cf file to followind line in submission and smtps section :
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
just changed reject to permit and all is working like a charm

my networks

Hi,
I have this set up and running beautifly on my one vps server. my question is this: how do i add my other vps to the my networks in postfix to allow that server to use it as a relay for form mail? or should i do it a different way? i have two vps's one running ubuntu which is a lamp for web hosting and my debian which is my mx.

TLS Enforce

If you want to enforce a client to use TLS just add that to your main.cf
smtpd_tls_security_level = may
smtpd_tls_auth_only = yes

Postfix keeps listening on port 25 but before you can use AUTH a TLS connection is needed.

Or just use this from terminal:
postconf -e smtpd_tls_security_level=may
postconf -e smtpd_tls_auth_only=yes

not working in centos 6.3

Dude, I tried everything here, the config files are now split up into 1000s (dovecot) and it took a little hunting but smtp still allows mail to be sent from whatever email address to whatever email address. It doesn't ask for login, it doesn't give errors, just a useless box.

Assistance

Hey Christoph,
If i can be of assistance with the write-up, i'll be more than happy to. Actually, I've started it, and based it around using Postfixadmin. I've gotten as far as the socket listen (master/client) in Dovecot but ran into some trouble, so i'm trying to figure out how the new system works. If you like, I can send what i have to the mailing list, or directly to you, or none at all. I'm easy like that :)

article

I found what looked to be a good article based on Cent 6.3 here http://linuxservconf.blogspot.com/2012/02. Mind you I haven't tested it, so I can't speak for it's accuracy, but having learned a lot of what's goning on here, and the why's to go along with the how's, it looks reasonably accurate, except Cent doesn't have all the apps in it's repos (though the author of the article does provide all needed files.)

Cheers