If you are tired of faked emails from popular domains like eBay or Paypal then there is hope. And you can even protect your own domain from spammers sending in your name. It is about SPF which is short for sender policy framework. Basically the owner of a domain defines which IP addresses are allowed to send email. Let's take an example. If you have the dig tool around then run…
$> dig +short workaround.org txt
and you should see something like
"v=spf1 ip4:85.214.93.191 ip4:85.214.149.150 -all"
This entry means that if someone sends you an email with a sender email address of …@workaround.org then you should only accept it if it was sent from one of these two IP addresses. The "-all" at the end tells you that no other IP addresses should be accepted (FAIL). Another definition like "~all" means a SOFTFAIL - you should at least be suspicious and perhaps increase the spam score. (Actually "~all" is widely used but pretty useless. Either you know what you are doing and use "-all" or leave it off entirely. If you want to test if SPF workd then you could use "?all".) So if someone sent you an email from …@workaround.org from another IP address then you should drop it - it's spam.
Many organisations already have SPF records that you can use to reduce the amount of spam you receive. So your duties as a mail server administrator are:
- set up a TXT entry in your DNS zone defining which IP addresses are allowed to send email in the name of your domain
- check SPF entries of sending email servers and reject email coming from unauthorized IP addresses
Set up an SPF entry
Obviously you need to have full control of your DNS zone to add a TXT record. If you don't then speak to your ISP and let them add the TXT entry. The SPF record needs to be properly machine readable. I suggest you go to the OpenSPF web site and use their wizard to create a proper SPF entry. Add this string as a TXT record for your domain. Just like above you should get the SPF entry back if you run:
$> dig +short mydomain.com txt
You need to be aware of one caveat though which is also SPFs biggest problem: if your users forward the email somewhere else then it might get rejected. By all means you need to be sure that your users always send email just through your mail server.
Check other mail servers' SPF entries
Fortunately there is a Debian package for the tumgreyspf software which makes checking SPF entries easy. Just install it:
$> apt-get install tumgreyspf
tumgreyspf is a policy daemon written in Python that does both greylisting and SPF checking of incoming emails. Using it is detailed in the /usr/share/doc/tumgreyspf/README.Debian file. Basically it boils down to adding one line to your smtpd_sender_restrictions (or smtpd_recipient_restrictions if you put everything in there) making use of it. Example:
smtpd_sender_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
[ ... ]
check_policy_service unix:private/tumgreyspf
[ ... ]
permit
And to define the program that is called when using this policy service you need to add two lines to your /etc/postfix/master.cf:
tumgreyspf unix - n n - - spawn
user=tumgreyspf argv=/usr/bin/tumgreyspf
Now reload your Postfix and that's it:
$> postfix reload
Case: SPF okay
Watch your /var/log/mail.log logfile. Every incoming email should now log an additional line from tumgreyspf. If the SPF check was positive then you will get:
tumgreyspf[24672]: sender SPF authorized: QUEUE_ID=""; identity=mailfrom; client-ip=26.21.244.31; helo=squedge2.squ.edu.om; envelope-from=…@squ.edu.om; receiver=…@workaround.org;
This means that the sender …@squ.edu.om was allowed in after checking the SPF entry.
Case: SPF fail
If the SPF check fails then you will see something like:
tumgreyspf[24672]: SPF fail - not authorized: QUEUE_ID=""; identity=mailfrom; client-ip=41.234.18.141; helo=gmx.de; envelope-from=…gmx.de; receiver=…christoph-haas.de;
The email got rejected. One spam mail less in the world. :)
Case: SPF softfail
The third case is when the SPF entry does not enforce a FAIL (-all) but just uses SOFTFAIL (~all). In your log file it will look like:
tumgreyspf[20408]: domain owner discourages use of this host: QUEUE_ID=""; identity=mailfrom; client-ip=220.245.2.67; helo=220-245-2-67.static.tpgi.com.au; envelope-from=…@rollouts.com; receiver=…@workaround.org
Unfortunately the SOFTFAIL does not actually reject the email. But the sender still believes that this IP address should not be sending email in their name. Luckily tumgreyspf adds this information to a "Received-SPF" header into the email. Just like:
Received-SPF: Softfail (domain owner discourages use of this host) identity=mailfrom; client-ip=61.146.93.243; helo=mail.163gd.com; envelope-from=…@cantv.net; receiver=…@christoph-haas.de;
So you still have a chance to mark such emails in spam in your email client by filtering out emails having a "Received-SPF: Softfail" header.
(Unfortunately tumgreyspf does not have a configuration option to treat SOFTFAIL as FAIL.)
Case: No SPF information
If the remote domain is ignorant and stupid and does not have any SPF entries yet then your log file will read:
Received-SPF: Neutral (access neither permitted nor denied) identity=mailfrom; client-ip=80.65.65.222; helo=mail.unze.ba; envelope-from=…@gmail.com; receiver=…@christoph-haas.de;
16 Comments
great job Chris!
Submitted by Anonymous (not verified) on
Just one thing:
You can decide of using greylisting with tumgreyspf or using postgrey.
If you want to continue to use postgrey, edit the file /etc/tumgreyspf/default.conf and change:
#CHECKERS = spf,greylist
in
CHECKERS = spf
(as Emmanuel Revah said in ML)
Otherwise, if you want to use tumgreyspf for greylist also don't change nothing in /etc/tumgreyspf/default.conf but comment out the line:
check_policy_service inet:127.0.0.1:10023 (or 60000)
in /etc/postfix/main.cf and remove the package postgrey with
dpkg -P postgrey
Regards,
Fabrizio
IPV6?
Submitted by GieltjE (not verified) on
Tumgreyspf doesn't really like ipv6, it works with it, but the cleanup doesn't appreciate it.
I can't figure out how to reach the author though, atleast not a proper way.
SPF checks and backup MX
Submitted by Anonymous (not verified) on
(p.s. i sent this through the ML which doesn't seem to respond, is it still working ?)
Hi, Manu! Above all you
Submitted by Christoph Haas on
Hi, Manu!
Above all you need to make sure that the backup MX server always has the same security checks as the main MX server. Spammers often send to the relay with the least priority hoping that the security measures are lower and that it helps them get their crap through.
SPF generally suffers from the problem where emails are forwarded somewhere else. So you need to make sure your backup MX server is listed as "mynetworks" at your main server so that your Postfix trusts it even before doing the SPF checks.
Christoph
Hey Christoph
Submitted by Anonymous (not verified) on
It seems that is what I will have to do, however I see this as an inconvenient in the case where it's "friends" that are backup MX for each other. Everyone has their own little ways of doing things, for example one server will use RBL (I don't anymore).
Of course there is no real way around that unless there was a mechanism where:
Of course in the meantime the best way for me is to ask my friends to not filter any of my emails (as much as possible) and use SPF because SPF record is there for a reason. For now it's safe to say that with SPF fake positives only exist if there is a config error, not just people talking about debt consolidation, viagra, canadian meds, and people who saw your profile. : ]]]
manu
Selective greylisting guide
Submitted by Anonymous (not verified) on
I found this other guide to a mailserver setup.
Their section about selective greylisting is good.
http://www200.pair.com/mecham/spam/virtual2p2.html#grey
/Thomas
Mail rejection on SOFTFAIL
Submitted by Anonymous (not verified) on
tumgreyspf is written in python, so you can reject mails on SOFTFAIL. This link describes the modification: http://blog.desgrange.net/tag/tumgreyspf/
Long SPF entries
Submitted by Anonymous (not verified) on
Hello everyone!
It's a few months ago since I began using SPF. Firstly, I was glad with this system, but after a while, I recognized that I don't get all the mails I needed to get. To specify my problem, I don't get mails from Domains whose SPF entry is too long. E.g. from amazon.com.
After a short manual lookup for the SPF-Entry, I get the following answer:
mail:~# dig +short amazon.com txt
;; Warning: Message parser reports malformed message packet.
;; Truncated, retrying in TCP mode.
"v=spf1 ip4:207.171.160.0/19 ip4:87.238.80.0/21 ip4:72.21.193.0/24 ip4:72.21.196.0/22 ip4:72.21.208.0/24 ip4:72.21.205.0/24 ip4:72.21.209.0/24 ip4:194.154.193.200/28 ip4:194.7.41.152/28 ip4:212.123.28.40/32 ip4:203.81.17.0/24 ~all"
"spf2.0/pra ip4:207.171.160.0/19 ip4:87.238.80.0/21 ip4:72.21.193.0/24 ip4:72.21.196.0/22 ip4:72.21.208.0/24 ip4:72.21.205.0/24 ip4:72.21.209.0/24 ip4:194.154.193.200/28 ip4:194.7.41.152/28 ip4:212.123.28.40/32 ip4:203.81.17.0/24 ~all"
How do I have to modifie my settings of tumgreyspf to solve this problem?
Thank you for your answers :)
tumgreyspf is sticking to the
Submitted by Christoph Haas on
tumgreyspf is sticking to the respective RFC 4408. You can't change its behavior to support broken SPF entries. The SPF entry you quotes looks way too large to be valid. I recomment using kitterman.com to determine whether an entry is valid. Many organisations don't get SPF right but I insist that they fix it. Nobody urges them to use SPF but if they do for christ's sake they need to do that right. There is only one domain I'm whitelisting here due to boss pressure. :) In other cases I let the users put the pressure on the responsible people to fix their SPF entry.
But Amazon.com SPF is valid...
Submitted by Anonymous (not verified) on
The SPF entries ARE valid but too long to fit into a single UDP packet so any tool doing SPF needs a TCP fallback or they end up being truncated - and with IPv6 growing in use, SPF entries are going to get longer. I ended up not using tumgreyspf because of this issue and instead used:
https://help.ubuntu.com/community/Postfix/SPF
Which is the official SPF solution for Postfix that the Postfix source code itself points to (README.SPF points to http://www.openspf.org/Software). I followed the Python route after I confirmed that the Python version did not have issues with really long SPF records that exceed the size of a UDP packet. It is also being maintained with regular updates and has a more versatile configuration "with sane defaults" and seems to work well out of the box.
SRS rewriting Headers of forwarded mails
Submitted by Anonymous (not verified) on
Christoph, thank you very much for your excellent tutorial. Even me got it running ...
One question arised when users forwarded their mails via the alias table to their email service at gmx.de:
host mx1.gmx.net[213.165.67.98] said: 550 5.1.0 {mimp002} Sender rejected on SPF fail - (http://portal.gmx.net/serverrules) (in reply to MAIL FROM command)These are their serverrules:
How can this be done using the configuration created with your tutorial?
Thank your very much,
christoph
Email forwarding is a general
Submitted by Christoph Haas on
Email forwarding is a general problem with SPF. See also: http://en.wikipedia.org/wiki/Sender_Policy_Framework#FAIL_and_forwarding
As you won't be able to change GMX's SPF entry you either need to send email through the GMX mail servers or use your own sender domain where you have control over the SPF record. I would suggest doing the former and send all email through the GMX relay. Things you will want to read on:
=> sender_dependent_relayhost_maps
=> http://www.postfix.org/SASL_README.html#client_sasl
I'm not sure if I understand
Submitted by Anonymous (not verified) on
I'm not sure if I understand that: So I've to create an GMX account for sending forwarded mail from GMX senders to GMX recipients through this account? Won't GMX also check, if the sending MAIL FROM correspondents to the account?
If doing so, I would have to create such an account for every email service that uses such a policy - sure, this is the way to go?
What about that SRS thing - as far as I have googled there are two ways to do that with postfix (besides patching postfix):
These two way are looking more elegant to me than using these external mail servers. Do you believe there's a chance that there'll be an how-to-like explanation about how to achieve this?
Regards,
Christoph
Your original request was to
Submitted by Christoph Haas on
Your original request was to send emails to the internet with a sender address of @gmx.net. So you already have a gmx.net account, right? And if you configured your email client (e.g. Thunderbird/Iceweasel) to send emails out you would send them through the GMX SMTP server and you would authenticate with your GMX user account.
What you would have to do is configure Postfix the same way you would configure your mail client. Send outgoing emails through the GMX SMTP server and authenticate with your GMX account (username/password).
There is nothing you can do at your side to work around SPF. Emails from domains using SPF records will only be accepted by other mail servers if they originate from the IP addresses mentioned in the SPF entry. In the case of GMX:
Thank you for you answer,
Submitted by Anonymous (not verified) on
Thank you for you answer, that whole SPF thing is quiet clear to me.
Obviously, I was not clear enough about my problem:
I'm running a mailserver using the setup of your tutorial (BTW: by now without the sieve part).
Several users are using it.
Some users forward their mail to third party email services, e.g. GMX.
What now happens is the following:
How can this be done using the setup of your excellent tutorial? There has to be some rewriting thing at the forwarding part of message sending. As far as I've read, the creator of postfix is not a friend of SPF, which doesn't seem to make things easier. But there should be a solution involving things like 'milter', but I don't have an idea where to start and how to handle it without breaking what I have (a working system).
Please forgive my clueless questions - was that laid out now in a reasonable way?
Do you have a hint - or better: a solution?
Thank you,
Christoph
Ah, okay, thanks for
Submitted by Christoph Haas on
Ah, okay, thanks for the clarification. The short answer is: never use any forwarding if you are using an email address where the domain has an SPF entry.
If you still like to forward emails then you would have to rewrite the "envelope from" and that is probably hard and error prone.
Or if you know and trust the system that is doing the forwarding then you could use "check_client_access" before you do the SPF checks in your main.cf and explicitly allow incoming emails from that server regardless of the SPF entries.
Sorry, I don't have any brighter ideas.