Fighting brute force attacks

So many bad guys everywhere. Paranoia you say? Well, take a look at your mail log. While I was preparing this guide I could barely follow it. The mail server I used for testing was by far not ready and I already got visits by a bunch of chinese IP addresses trying to guess accounts of email users. Scoundrels constantly scan the internet for mail servers and do not tire to find a way to abuse them. Let’s do something about that before they actually guess email accounts successfully.

Attack patterns

There was one pattern that caught my eye:

Jan 24 23:59:34 jen postfix/smtpd[31879]: lost connection after AUTH from unknown[122.231.43.242]
Jan 24 23:59:35 jen postfix/smtpd[31879]: lost connection after AUTH from unknown[122.231.43.242]
Jan 24 23:59:36 jen postfix/smtpd[31879]: lost connection after AUTH from unknown[122.231.43.242]
Jan 24 23:59:37 jen postfix/smtpd[31879]: lost connection after AUTH from unknown[122.231.43.242]
Jan 24 23:59:38 jen postfix/smtpd[31879]: lost connection after AUTH from unknown[122.231.43.242]

Some chinese system is trying to authenticate against my Postfix time and again. And believe it or not – it was doing that for days. My log file grew up to 500 MB per day just from those dorks. Let’s put an end to that.

Installation

A standard piece of software I install on servers for that purpose is fail2ban. Go put it on your mail server:

apt install fail2ban

It knows about a lot of patterns in your log files that report something harmful. The software runs in the background and follows the various log files. Each new line in the log is compared against regular expressions. If a match is found it blocks the IP address that was mentioned in the log.

Adding a filter

As fail2ban does not know that “lost connection after AUTH” is a failure we need to add that pattern. Create a new file /etc/fail2ban/filter.d/postfix-ispmail.conf with this content:

[INCLUDES]
before = common.conf

[Definition]
_daemon = postfix(-\w+)?/(?:submission/|smtps/)?smtp[ds]
failregex = ^%(__prefix_line)slost connection after AUTH from \S+\[<HOST>\]$
ignoreregex =

[Init]
journalmatch = _SYSTEMD_UNIT=postfix.service

This defines a new filter called “postfix-ispmail”. The name is derived from the filename of the filter. The important line is the failregex. It described a regular expression – a machine-readable description of what are you searching for. %(…)s expands a variable. And __prefix_line is a pattern describing the left part of each log line: “Jan 25 00:32:01 jen postfix/smtpd[32600]: “. The “postfix/smtpd” part is defined by the _daemon variable. “\S+” means that at this place you expect one or more character that are not a space. The brackets need to be escaped by a backslash because they have a special meaning in regular expressions. And <HOST> is a built-in pattern of fail2ban that stands for an IP address. You should get yourself acquainted with regular expressions even if they may scare you at first. They are more powerful than scary actually. 🙂

Adding a jail

fail2ban can now understand the pattern we are looking for. But where do we look? And what do we do if we find a match? That’s our next task. Create another file /etc/fail2ban/jail.d/postfix-ispmail.conf with this content:

[postfix-ispmail]
logpath = %(syslog_mail)s
enabled = true
port = smtp,submission

The first line refers to the name of the filter. We called the filter file “postfix-ispmail.conf” so this is the filter name (the “.conf” is ignored). The log file we want to follow is defined in the syslog_mail variable which gets defined elsewhere (in the paths-debian.conf file). It points to /var/log/mail.log. We want to have this jail enabled because the default in /etc/fail2ban/jail.conf is having it disabled. And finally we define which network ports we want to block in order to block an attacker.

Make it so

Time for action. Restart fail2ban to load your configuration changes:

service fail2ban restart

To see what fail2ban did take a look at /var/log/fail2ban.log. In my example it logged:

2018-01-24 23:59:34,443 fail2ban.filter [32003]: INFO [postfix-ispmail] Found 122.231.43.242
2018-01-24 23:59:35,430 fail2ban.filter [32003]: INFO [postfix-ispmail] Found 122.231.43.242
2018-01-24 23:59:36,434 fail2ban.filter [32003]: INFO [postfix-ispmail] Found 122.231.43.242
2018-01-24 23:59:37,427 fail2ban.filter [32003]: INFO [postfix-ispmail] Found 122.231.43.242
2018-01-24 23:59:38,413 fail2ban.filter [32003]: INFO [postfix-ispmail] Found 122.231.43.242
2018-01-24 23:59:38,781 fail2ban.actions [32003]: NOTICE [postfix-ispmail] Ban 122.231.43.242

It found the “lost connection after AUTH” line mentioning that chinese IP address five times and decided to ban it. The reason is part of the configuration in /etc/fail2ban/jail.conf:

# “bantime” is the number of seconds that a host is banned.
bantime = 600

# A host is banned if it has generated “maxretry” during the last “findtime”
# seconds.
findtime = 600

# “maxretry” is the number of failures before a host get banned.
maxretry = 5

So by default if such a rogue pattern occurs at least maxretry times within findtime seconds the IP will be banned for bantime seconds. You can customize these values but please do not edit the jail.conf. Instead create a jail.local file as described in “man 5 jail.conf”.

Banning magic

Do you wonder how that banning actually works? fail2ban adds new rules to your host’s firewall using iptables. The list of current iptables rules can be viewed using…

iptables -nvL

It will show your firewall rules. fail2ban creates a new chain for each jail you configure. I call my jail “ispmail-postfix” so my rules look like:

Chain f2b-postfix-ispmail (1 references)
 pkts bytes target prot opt in out source destination 
    3 152   REJECT all  --  *  *   125.106.22.171 …
   22 972   REJECT all  --  *  *   115.213.146.75 …
    3 152   REJECT all  --  *  *   1.195.252.154 …
  837 39050 RETURN all  --  *  *   0.0.0.0/0 0.0.0.0/0

But there is a nicer way to see the current status of a jail:

fail2ban-client status postfix-ispmail

It will show something like:

Status for the jail: postfix-ispmail
|- Filter
| |- Currently failed: 1
| |- Total failed: 168
| `- File list: /var/log/mail.log
`- Actions
 |- Currently banned: 14
 |- Total banned: 36
 `- Banned IP list: 1.199.191.44 125.106.250.51 115.213.233.184 …

More jails

You may want to browse through the pre-defined filters in either /etc/fail2ban/jail.conf or by looking at the files in /etc/fail2ban/filter.d/. By default only the “sshd” filter is enabled by the /etc/fail2ban/jail.d/defaults-debian.conf file. But feel free to add further files enabling…

  • dovecot
  • postfix
  • postfix-sasl
  • roundcube-auth

That will put an end to brute force attacks once and for all. Fun fact… two days after writing this I had already banned 1645 IP addresses.

(If we come up with more patterns that fail2ban doesn’t know yet I will add them. Did you find any?)

6 thoughts on “Fighting brute force attacks

  • 2018-01-25 at 13:56
    Permalink

    Hi,

    Thanks for you great tutorial, it helped me a lot!

    Maybe it would be an idea to add the detected spam to the postfix fail2ban jail?

    I am not fit enough in fail2ban configuration, but I think with matching for the line

    Jan 25 07:14:08 srv5 postfix/cleanup[4939]: 5877618800AB: milter-reject: END-OF-MESSAGE from q212.mkzz.xyz[103.203.148.212]: 4.7.1 Spam message rejected; from= to= proto=ESMTP helo=

    This should work, shouldn’t it?

    Tanks and best regards
    Christof

    Reply
  • 2018-01-26 at 13:48
    Permalink

    Note: the filter is named roundcube-auth for me on Stretch instead of roundcube.

    Reply
    • 2018-01-26 at 13:54
      Permalink

      Thanks. Fixed.

      Reply
  • 2018-02-04 at 12:27
    Permalink

    Hi

    Any ideas why my fail2ban is missing the “paths-debian.conf”?
    I mean I fixed it, but why/how? What am I missing?

    Reply
      • 2018-02-10 at 11:58
        Permalink

        Okay… just to close this, it was my mistake. I screwed the config files up while migrating from an old system. Everything is like it should now….

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close