SPF to avoid spoofing

If you are tired of faked emails from popular domains like eBay or Paypal then there is hope. And you can even protect your own domain from spammers sending in your name. It is about SPF which is short for sender policy framework. Basically  the owner of a domain defines which IP addresses are allowed to send email. Let's take an example. If you have the dig tool around then run…

$> dig +short workaround.org txt

and you should see something like

"v=spf1 ip4:85.214.93.191 ip4:85.214.149.150 -all"

This entry means that if someone sends you an email with a sender email address of …@workaround.org then you should only accept it if it was sent from one of these two IP addresses. The "-all" at the end tells you that no other IP addresses should be accepted (FAIL). Another definition like "~all" means a SOFTFAIL - you should at least be suspicious and perhaps increase the spam score. (Actually "~all" is widely used but pretty useless. Either you know what you are doing and use "-all" or leave it off entirely. If you want to test if SPF workd then you could use "?all".) So if someone sent you an email from …@workaround.org from another IP address then you should drop it - it's spam.

Many organisations already have SPF records that you can use to reduce the amount of spam you receive. So your duties as a mail server administrator are:

  • set up a TXT entry in your DNS zone defining which IP addresses are allowed to send email in the name of your domain
  • check SPF entries of sending email servers and reject email coming from unauthorized IP addresses

Set up an SPF entry

Obviously you need to have full control of your DNS zone to add a TXT record. If you don't then speak to your ISP and let them add the TXT entry. The SPF record needs to be properly machine readable. I suggest you go to the OpenSPF web site and use their wizard to create a proper SPF entry. Add this string as a TXT record for your domain. Just like above you should get the SPF entry back if you run:

$> dig +short mydomain.com txt

You need to be aware of one caveat though which is also SPFs biggest problem: if your users forward the email somewhere else then it might get rejected. By all means you need to be sure that your users always send email just through your mail server.

Check other mail servers' SPF entries

Fortunately there is a Debian package for the tumgreyspf software which makes checking SPF entries easy. Just install it:

$> apt-get install tumgreyspf

tumgreyspf is a policy daemon written in Python that does both greylisting and SPF checking of incoming emails. Using it is detailed in the /usr/share/doc/tumgreyspf/README.Debian file. Basically it boils down to adding one line to your smtpd_sender_restrictions (or smtpd_recipient_restrictions if you put everything in there) making use of it. Example:

smtpd_sender_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    [ ... ]
    check_policy_service unix:private/tumgreyspf
    [ ... ]
    permit

And to define the program that is called when using this policy service you need to add two lines to your /etc/postfix/master.cf:

tumgreyspf unix  -      n       n       -       -       spawn
    user=tumgreyspf argv=/usr/bin/tumgreyspf

Now reload your Postfix and that's it:

$> postfix reload

Case: SPF okay

Watch your /var/log/mail.log logfile. Every incoming email should now log an additional line from tumgreyspf. If the SPF check was positive then you will get:

tumgreyspf[24672]: sender SPF authorized: QUEUE_ID=""; identity=mailfrom;
   client-ip=26.21.244.31; helo=squedge2.squ.edu.om;
   envelope-from=…@squ.edu.om;
   receiver=…@workaround.org;

This means that the sender …@squ.edu.om was allowed in after checking the SPF entry.

Case: SPF fail

If the SPF check fails then you will see something like:

tumgreyspf[24672]: SPF fail - not authorized: QUEUE_ID=""; identity=mailfrom;
   client-ip=41.234.18.141; helo=gmx.de;
   envelope-from=…gmx.de;
   receiver=…christoph-haas.de;

The email got rejected. One spam mail less in the world. :)

Case: SPF softfail

The third case is when the SPF entry does not enforce a FAIL (-all) but just uses SOFTFAIL (~all). In your log file it will look like:

tumgreyspf[20408]: domain owner discourages use of this host: QUEUE_ID="";
   identity=mailfrom; client-ip=220.245.2.67; helo=220-245-2-67.static.tpgi.com.au;
   envelope-from=…@rollouts.com; receiver=…@workaround.org

Unfortunately the SOFTFAIL does not actually reject the email. But the sender still believes that this IP address should not be sending email in their name. Luckily tumgreyspf adds this information to a "Received-SPF" header into the email. Just like:

Received-SPF: Softfail (domain owner discourages use of this host) identity=mailfrom;
   client-ip=61.146.93.243; helo=mail.163gd.com;
   envelope-from=…@cantv.net; receiver=…@christoph-haas.de;

So you still have a chance to mark such emails in spam in your email client by filtering out emails having a "Received-SPF: Softfail" header.

(Unfortunately tumgreyspf does not have a configuration option to treat SOFTFAIL as FAIL.)

Case: No SPF information

If the remote domain is ignorant and stupid and does not have any SPF entries yet then your log file will read:

Received-SPF: Neutral (access neither permitted nor denied) identity=mailfrom;
   client-ip=80.65.65.222; helo=mail.unze.ba;
   envelope-from=…@gmail.com; receiver=…@christoph-haas.de; 

16 Comments

great job Chris!

Just one thing:

You can decide of using greylisting with tumgreyspf or using postgrey.

If you want to continue to use postgrey, edit the file /etc/tumgreyspf/default.conf and change:

#CHECKERS = spf,greylist

in
 

CHECKERS = spf

(as Emmanuel Revah said in ML)

Otherwise, if you want to use tumgreyspf for greylist also don't change nothing in /etc/tumgreyspf/default.conf  but comment out the line:

check_policy_service inet:127.0.0.1:10023   (or 60000)

in /etc/postfix/main.cf and remove the package postgrey with

dpkg -P postgrey

Regards,

Fabrizio

IPV6?

Tumgreyspf doesn't really like ipv6, it works with it, but the cleanup doesn't appreciate it.

I can't figure out how to reach the author though, atleast not a proper way.

SPF checks and backup MX

Hello,

Something was bothering me so I confirmed by testing.

Example: If a legitimate domain using SPF sends an email to me, but for
some reason uses a backup MX of mine (say a friend's server), then my
server will check against backup MX and not the actual SMTP of the client
who sent the email.  This results in "good" emails being rejected.

I guess the solution would be to:

1. put the backup MX in a "permit" setting somewhere
2. have the backup MX use the same SPF checks

I might be the only one to have just figured this out, but maybe this
could be included in the tutorial ?   Also if there are other ideas i'm
open.

manu
----
http://manurevah.com

 

(p.s. i sent this through the ML which doesn't seem to respond, is it still working ?)

Hey Christoph

 

It seems that is what I will have to do, however I see this as an inconvenient in the case where it's "friends" that are backup MX for each other. Everyone has their own little ways of doing things, for example one server will use RBL (I don't anymore).

 

Of course there is no real way around that unless there was a mechanism where:

  • a backup MX adds a header like "Real SMTP: IP"
  • we could have a group called "backup mx" (as opposed to mynetworks)
  • Emails received get checked agains SPF via the usual way UNLESS it comes from "backup mx" in which case it should use "Real SMTP: IP"

 

Of course in the meantime the best way for me is to ask my friends to not filter any of my emails (as much as possible) and use SPF because SPF record is there for a reason. For now it's safe to say that with SPF fake positives only exist if there is a config error, not just people talking about debt consolidation, viagra, canadian meds, and people who saw your profile.  : ]]]

 

manu

Long SPF entries

Hello everyone!

It's a few months ago since I began using SPF. Firstly, I was glad with this system, but after a while, I recognized that I don't get all the mails I needed to get.  To specify my problem, I don't get mails from Domains whose SPF entry is too long. E.g. from amazon.com.

After a short manual lookup for the SPF-Entry, I get the following answer:

 

mail:~# dig +short amazon.com txt
;; Warning: Message parser reports malformed message packet.
;; Truncated, retrying in TCP mode.
"v=spf1 ip4:207.171.160.0/19 ip4:87.238.80.0/21 ip4:72.21.193.0/24 ip4:72.21.196.0/22 ip4:72.21.208.0/24 ip4:72.21.205.0/24 ip4:72.21.209.0/24 ip4:194.154.193.200/28 ip4:194.7.41.152/28 ip4:212.123.28.40/32 ip4:203.81.17.0/24 ~all"
"spf2.0/pra ip4:207.171.160.0/19 ip4:87.238.80.0/21 ip4:72.21.193.0/24 ip4:72.21.196.0/22 ip4:72.21.208.0/24 ip4:72.21.205.0/24 ip4:72.21.209.0/24 ip4:194.154.193.200/28 ip4:194.7.41.152/28 ip4:212.123.28.40/32 ip4:203.81.17.0/24 ~all"

 

How do I have to modifie my settings of tumgreyspf to solve this problem?

Thank you for your answers :)
 

But Amazon.com SPF is valid...

The SPF entries ARE valid but too long to fit into a single UDP packet so any tool doing SPF needs a TCP fallback or they end up being truncated - and with IPv6 growing in use, SPF entries are going to get longer.  I ended up not using tumgreyspf because of this issue and instead used:

https://help.ubuntu.com/community/Postfix/SPF

Which is the official SPF solution for Postfix that the Postfix source code itself points to (README.SPF points to http://www.openspf.org/Software).  I followed the Python route after I confirmed that the Python version did not have issues with really long SPF records that exceed the size of a UDP packet.  It is also being maintained with regular updates and has a more versatile configuration "with sane defaults" and seems to work well out of the box.

SRS rewriting Headers of forwarded mails

Christoph, thank you very much for your excellent tutorial. Even me got it running ...

One question arised when users forwarded their mails via the alias table to their email service at gmx.de:

 host mx1.gmx.net[213.165.67.98] said: 550 5.1.0 {mimp002} Sender rejected on SPF     fail - (http://portal.gmx.net/serverrules) (in reply to MAIL FROM
command)

 

These are their serverrules:

"We check the sending server for SPF compliance if the user hasn't disabled this setting. The addresses of forwarded e-mail therefore MUST be rewritten using SRS standards."

How can this be done using the configuration created with your tutorial?

Thank your very much,

christoph

 

 

    

I'm not sure if I understand

I'm not sure if I understand that: So I've to create an GMX account for sending forwarded mail from GMX senders to GMX recipients through this account? Won't GMX also check, if the sending MAIL FROM correspondents to the account?

If doing so, I would have to create such an account for every email service that uses such a policy - sure, this is the way to go?

What about that SRS thing - as far as I have googled there are two ways to do that with postfix (besides patching postfix):

  • using policyd with an extra server for doing these things (not sure if I understood that right)
  • using filters (???)

These two way are looking more elegant to me than using these external mail servers. Do you believe there's a chance that there'll be an how-to-like explanation about how to achieve this?

Regards,
Christoph

Thank you for you answer,

Thank you for you answer, that whole SPF thing is quiet clear to me.
Obviously, I was not clear enough about my problem:

I'm running a mailserver using the setup of your tutorial (BTW: by now without the sieve part).
Several users are using it.
Some users forward their mail to third party email services, e.g. GMX.

What now happens is the following:

  • User a@example.com receives an email from b@gmx.com
  • the mail gets forwarded to a@gmx.com (MAIL FROM is still b@gmx.com)
  • GMX complains, mails from GMX cannot be delivered by my mail server, because GMX uses SPF. The SPF entry for gmx.com says that only servers of GMX are trusted for sending mails from gmx.com.
  • All that is true and a logical result of SPF. So, my mail server has to rewrite a specific part (return path?) of the forwarded mail in a way that GMX can clearly see, that this mail is a forwarded one.

How can this be done using the setup of your excellent tutorial? There has to be some rewriting thing at the forwarding part of message sending. As far as I've read, the creator of postfix is not a friend of SPF, which doesn't seem to make things easier. But there should be a solution involving things like 'milter', but I don't have an idea where to start and how to handle it without breaking what I have (a working system).

Please forgive my clueless questions - was that laid out now in a reasonable way?

Do you have a hint - or better: a solution?

Thank you,
Christoph