Firewalling and brute force mitigation

So many bad guys everywhere. Paranoia you say? Well, take a look at your mail log. While I was preparing this guide I could barely follow it. The mail server I used for testing was by far not ready and I already got visits by a bunch of foreign IP addresses trying to guess accounts of email users. Scoundrels constantly scan the internet for mail servers and do not tire to find a way to abuse them. Let’s do something about that before they actually guess email accounts successfully.

First install the nftables package:

apt install nftables

Basic firewall rules with nftables

Let’s begin by blocking all incoming network ports except those that are needed to run your mail server. Older Debian releases used the iptables tool to help you add firewall rules. Debian Bullseye comes with a newer version of the Linux kernel that is instead defaulting to nftables. The good news is that you can still use iptables commands. The bad news is that most tools do not yet support nftables. However one distant future day the old syntax will not work any more so let’s dive straight into the new syntax and use nftables.

First let’s make the iptables command use nftables. Run:

update-alternatives --config iptables

You will be able to choose between iptables-nft (the new stuff) and iptables-legacy (the old stuff). Go with iptables-nft. Now the iptables command will manage nftables rules. However you can explicitly use the iptables-legacy command if you want to access the old-school iptables rules.

So where would you store the firewall rules? The recommended way is to create a file /etc/nftables.conf and add rules there. Although the new syntax takes some getting used to, it is at least pretty readable. This is my default file for new mail servers:

#!/usr/sbin/nft -f
flush ruleset
table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;

    iifname lo accept
    ct state established,related accept
    tcp dport { ssh, http, https, imap2, imaps, pop3, pop3s, submission, smtp } ct state new accept

    # ICMP: errors, pings
    ip protocol icmp icmp type { echo-request, echo-reply, destination-unreachable, time-exceeded, parameter-problem, router-solicitation, router-advertisement } accept
    # ICMPv6: errors, pings, routing
    ip6 nexthdr icmpv6 counter accept comment "accept all ICMP types"

    # Reject other packets
    ip protocol tcp reject with tcp reset
  }
}

Bear with me that I will not explain this file in-depth. Feel free to read more about nftables in the Debian wiki and the wiki of the nftables project.

To active these firewall rules use systemctl:

systemctl enable nftables
systemctl start nftables

If all worked as expected you can see that systemd loaded your rules:

systemctl status nftables

You should see “Active: active (exited)” as the status. It is normal that its status is “exited” because there is no permanent process running. The rules are applied just once and that’s it.

Although nftables has been there for quite a while many projects still do not support it. Fortunately an important tool to block off attackers does: fail2ban.

Mitigate brute-force attacks

Let’s install fail2ban:

apt install fail2ban

This is a nice little tool. It knows about a lot of patterns in your log files that reveal unwelcome things happening. The software runs in the background and follows the various log files. Each new line in the log is compared against regular expressions (search patterns). If a match is found multiple times then it blocks that IP address for a while to stop the rogue activity.

Restart nftables…

systemctl restart nftables

Watch your /var/log/fail2ban.log. If you don’t want to wait for something to happen then you can do a couple of SSH or IMAP logins from another system. Just be aware that you will be locked out from that IP address for a few minutes. The log file will show your login attempts:

2021-11-07 18:27:11,799 [7661]: INFO    [sshd] Found 2a03:4000:2:c0b::1 - 2021-11-07 18:27:11
2021-11-07 18:27:13,603 [7661]: INFO    [sshd] Found 2a03:4000:2:c0b::1 - 2021-11-07 18:27:13
2021-11-07 18:27:14,204 [7661]: INFO    [sshd] Found 2a03:4000:2:c0b::1 - 2021-11-07 18:27:13
2021-11-07 18:27:14,205 [7661]: INFO    [sshd] Found 2a03:4000:2:c0b::1 - 2021-11-07 18:27:13
2021-11-07 18:27:17,409 [7661]: INFO    [sshd] Found 2a03:4000:2:c0b::1 - 2021-11-07 18:27:16
2021-11-07 18:27:18,011 [7661]: NOTICE  [sshd] Ban 2a03:4000:2:c0b::1

fail2ban has now banned an IP by adding firewall rules. Check your firewall tables:

nft list tables

…and you should see new tables called “ip” and “ip6”:

table inet filter
table ip filter
table ip6 filter

To see the current nftables rules:

nft list ruleset

…which will show…

table ip6 filter {
	chain INPUT {
		type filter hook input priority filter; policy accept;
		meta l4proto tcp tcp dport 22 counter packets 50 bytes 10414 jump f2b-sshd
	}

	chain f2b-sshd {
		ip6 saddr 2a03:4000:2:c0b::1 counter packets 13 bytes 840 reject
		counter packets 37 bytes 9574 return
	}
}

The nft syntax takes a bit of getting used to. But the “f2b-sshd” chains should be there that blocks incoming connections to TCP port 22. And in the eighth line you see the IPv6 address from above getting blocked.

A more readable way is using the “fail2ban-client” tool that is already installed:

fail2ban-client status

…which shows the different jails in operation…

Status
|- Number of jail:	1
`- Jail list:	sshd

A jail is where fail2ban adds rogue IP addresses to block them. fail2ban also distinguishes the different ports. So if someone is attacking your SSH service he will only get blocked for the SSH port. Other ports like SMTP or IMAP will still be accessible.

By default only the jail for SSH connections is enabled. But fail2ban also understands the log files used by Postfix and Dovecot. See the /etc/fail2ban/jail.conf for the list of available jails. So you don’t have to invent something new but just turn the jails on. That is best done by creating a new file /etc/fail2ban/jail.local that contains:

[apache-auth]
enabled = true

[dovecot]
enabled = true
port    = pop3,pop3s,imap2,imaps,submission,465,sieve

[postfix]
enabled = true

[sieve]
enabled = true

The reason that the port is redefined in the [dovecot] section is that fail2ban called the IMAP port “imap”. But on Debian servers the service is called “imap2” in the /etc/services file.

After all these changes it is a good idea to check if your fail2ban configuration is still valid:

fail2ban-server -t

Is fail2ban happy? Great. Restart fail2ban after your last change:

systemctl restart fail2ban

To see what fail2ban did take a look at /var/log/fail2ban.log. This is what it looked like after a few minutes:

fail2ban.filter  : INFO    [sshd] Found 177.139.194.62 - 2019-12-27 20:50:43
fail2ban.filter  : INFO    [sshd] Found 177.139.194.62 - 2019-12-27 20:50:43
fail2ban.filter  : INFO    [sshd] Found 177.139.194.62 - 2019-12-27 20:50:44
fail2ban.filter  : INFO    [sshd] Found 49.88.112.112 - 2019-12-27 20:50:46
fail2ban.filter  : INFO    [sshd] Found 134.209.152.176 - 2019-12-27 20:50:46
fail2ban.filter  : INFO    [sshd] Found 49.88.112.112 - 2019-12-27 20:50:48
fail2ban.filter  : INFO    [sshd] Found 134.209.152.176 - 2019-12-27 20:50:49
fail2ban.filter  : INFO    [sshd] Found 49.88.112.112 - 2019-12-27 20:50:50
fail2ban.filter  : INFO    [sshd] Found 49.88.112.112 - 2019-12-27 20:50:54
fail2ban.filter  : INFO    [dovecot] Found 85.25.72.76 - 2019-12-27 20:51:09
fail2ban.filter  : INFO    [dovecot] Found 85.25.72.76 - 2019-12-27 20:51:29
fail2ban.filter  : INFO    [sshd] Found 61.72.255.26 - 2019-12-27 20:51:33
fail2ban.filter  : INFO    [sshd] Found 49.88.112.112 - 2019-12-27 20:51:35
fail2ban.filter  : INFO    [sshd] Found 61.72.255.26 - 2019-12-27 20:51:35
fail2ban.actions : NOTICE  [sshd] Ban 49.88.112.112
fail2ban.filter  : INFO    [sshd] Found 49.88.112.112 - 2019-12-27 20:51:37
fail2ban.filter  : INFO    [dovecot] Found 85.25.72.76 - 2019-12-27 20:51:42
fail2ban.filter  : INFO    [dovecot] Found 85.25.72.76 - 2019-12-27 20:52:03
fail2ban.filter  : INFO    [dovecot] Found 85.25.72.76 - 2019-12-27 20:52:18
fail2ban.actions : NOTICE  [dovecot] Ban 85.25.72.76

Status please

fail2ban comes with a handy shell command called fail2ban-client. To see which jails (=where to look for attackers) are active:

$> fail2ban-client status
Status
|- Number of jail:    4
`- Jail list:    apache-auth, dovecot, postfix, sshd

So all four expected jails are active. Do you want to know which IP addresses are banned for failed SSH login attempts?

$> fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed:    3
|  |- Total failed:    13
|  - File list:    /var/log/auth.log - Actions
   |- Currently banned:    4
   |- Total banned:    5
   `- Banned IP list:    222.186.175.148 222.186.190.2 49.88.112.116 2a03:4000:1c:3f0::1

Grab a bag of popcorn and enjoy how your mail server grows its list of blocked idiots. 🙂