So many bad guys everywhere. Paranoia you say? Well, take a look at your mail log. While I was preparing this guide I could barely follow it. The mail server I used for testing was by far not ready and I already got visits by a bunch of chinese IP addresses trying to guess accounts of email users. Scoundrels constantly scan the internet for mail servers and do not tire to find a way to abuse them. Let’s do something about that before they actually guess email accounts successfully.
There was one pattern that caught my eye:
Jan 24 23:59:34 jen postfix/smtpd: lost connection after AUTH from unknown[22.214.171.124]
Jan 24 23:59:35 jen postfix/smtpd: lost connection after AUTH from unknown[126.96.36.199]
Jan 24 23:59:36 jen postfix/smtpd: lost connection after AUTH from unknown[188.8.131.52]
Jan 24 23:59:37 jen postfix/smtpd: lost connection after AUTH from unknown[184.108.40.206]
Jan 24 23:59:38 jen postfix/smtpd: lost connection after AUTH from unknown[220.127.116.11]
Some chinese system is trying to authenticate against my Postfix time and again. And believe it or not – it was doing that for days. My log file grew up to 500 MB per day just from those dorks. Let’s put an end to that.
A standard piece of software I install on servers for that purpose is fail2ban. Go put it on your mail server:
apt install fail2ban
It knows about a lot of patterns in your log files that report something harmful. The software runs in the background and follows the various log files. Each new line in the log is compared against regular expressions. If a match is found it blocks the IP address that was mentioned in the log.
Adding a filter
As fail2ban does not know that “lost connection after AUTH” is a failure we need to add that pattern. Create a new file /etc/fail2ban/filter.d/postfix-ispmail.conf with this content:
before = common.conf
_daemon = postfix(-\w+)?/(?:submission/|smtps/)?smtp[ds]
failregex = ^%(__prefix_line)slost connection after AUTH from \S+\[<HOST>\]$
journalmatch = _SYSTEMD_UNIT=postfix.service
This defines a new filter called “postfix-ispmail”. The name is derived from the filename of the filter. The important line is the failregex. It described a regular expression – a machine-readable description of what are you searching for. %(…)s expands a variable. And __prefix_line is a pattern describing the left part of each log line: “Jan 25 00:32:01 jen postfix/smtpd: “. The “postfix/smtpd” part is defined by the _daemon variable. “\S+” means that at this place you expect one or more character that are not a space. The brackets need to be escaped by a backslash because they have a special meaning in regular expressions. And <HOST> is a built-in pattern of fail2ban that stands for an IP address. You should get yourself acquainted with regular expressions even if they may scare you at first. They are more powerful than scary actually. 🙂
Adding a jail
fail2ban can now understand the pattern we are looking for. But where do we look? And what do we do if we find a match? That’s our next task. Create another file /etc/fail2ban/jail.d/postfix-ispmail.conf with this content:
[postfix-ispmail] logpath = %(syslog_mail)s enabled = true port = smtp,submission
The first line refers to the name of the filter. We called the filter file “postfix-ispmail.conf” so this is the filter name (the “.conf” is ignored). The log file we want to follow is defined in the syslog_mail variable which gets defined elsewhere (in the paths-debian.conf file). It points to /var/log/mail.log. We want to have this jail enabled because the default in /etc/fail2ban/jail.conf is having it disabled. And finally we define which network ports we want to block in order to block an attacker.
Make it so
Time for action. Restart fail2ban to load your configuration changes:
service fail2ban restart
To see what fail2ban did take a look at /var/log/fail2ban.log. In my example it logged:
2018-01-24 23:59:34,443 fail2ban.filter : INFO [postfix-ispmail] Found 18.104.22.168
2018-01-24 23:59:35,430 fail2ban.filter : INFO [postfix-ispmail] Found 22.214.171.124
2018-01-24 23:59:36,434 fail2ban.filter : INFO [postfix-ispmail] Found 126.96.36.199
2018-01-24 23:59:37,427 fail2ban.filter : INFO [postfix-ispmail] Found 188.8.131.52
2018-01-24 23:59:38,413 fail2ban.filter : INFO [postfix-ispmail] Found 184.108.40.206
2018-01-24 23:59:38,781 fail2ban.actions : NOTICE [postfix-ispmail] Ban 220.127.116.11
It found the “lost connection after AUTH” line mentioning that chinese IP address five times and decided to ban it. The reason is part of the configuration in /etc/fail2ban/jail.conf:
# “bantime” is the number of seconds that a host is banned.
bantime = 600
# A host is banned if it has generated “maxretry” during the last “findtime”
findtime = 600
# “maxretry” is the number of failures before a host get banned.
maxretry = 5
So by default if such a rogue pattern occurs at least maxretry times within findtime seconds the IP will be banned for bantime seconds. You can customize these values but please do not edit the jail.conf. Instead create a jail.local file as described in “man 5 jail.conf”.
Do you wonder how that banning actually works? fail2ban adds new rules to your host’s firewall using iptables. The list of current iptables rules can be viewed using…
It will show your firewall rules. fail2ban creates a new chain for each jail you configure. I call my jail “ispmail-postfix” so my rules look like:
Chain f2b-postfix-ispmail (1 references) pkts bytes target prot opt in out source destination 3 152 REJECT all -- * * 18.104.22.168 … 22 972 REJECT all -- * * 22.214.171.124 … 3 152 REJECT all -- * * 126.96.36.199 … 837 39050 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
But there is a nicer way to see the current status of a jail:
fail2ban-client status postfix-ispmail
It will show something like:
Status for the jail: postfix-ispmail |- Filter | |- Currently failed: 1 | |- Total failed: 168 | `- File list: /var/log/mail.log `- Actions |- Currently banned: 14 |- Total banned: 36 `- Banned IP list: 188.8.131.52 184.108.40.206 220.127.116.11 …
You may want to browse through the pre-defined filters in either /etc/fail2ban/jail.conf or by looking at the files in /etc/fail2ban/filter.d/. By default only the “sshd” filter is enabled by the /etc/fail2ban/jail.d/defaults-debian.conf file. But feel free to add further files enabling…
That will put an end to brute force attacks once and for all. Fun fact… two days after writing this I had already banned 1645 IP addresses.
(If we come up with more patterns that fail2ban doesn’t know yet I will add them. Did you find any?)
Many ISPs support IPv6 nowadays. Unfortunately the version of fail2ban in Debian Stretch does not. IPv6 support has been added in version 0.10 but you get version 0.9.6 by default. There is no official backport of the new version yet. But I can at least offer you a quick backport so that you can use version 0.10 on Debian Stretch.
Install it using…
dpkg -i fail2ban_0.10.2-1_all.deb
…and make sure any missing dependencies are installed…
apt -f install
Now fail2ban will be able to block IPv6 addresses.