Protect your VPS against automated attacks with Fail2ban

Get a BitLaunch VPS today and protect it with Fail2ban on Ubuntu.

Introduction

Security is paramount importance for system admins. There are so many tools for security such as firewalls and network sniffers and so on. However sometimes we need to expose a service to the public and still we need to make sure all our users are legitimate.

A simple example of this is an FTP server, we don't know where the client will connect to the server -so we can not use a simple IP firewall- however we still need to protect from password guessing bots.

What is Fail2ban?

There are so many applications similar to this, but Fail2ban shines as a security tool in these situations with its flexible configuration and default filters.

Fail2ban works with the daemons/services log files and matches the unwanted entries with regex. Once unwanted requests made it follows and -depending on the configuration- bans the IP that made the request.

This tutorial is going to cover how to install Fail2ban on Ubuntu, and will then move on to how to use Fail2ban to protect your VPS server. However, it's worth noting that Fail2ban shouldn't be used as a replacement for traditional VPN security practices. Rather, it should be used in combination with them to produce the best result.

Prerequisites

Any recent Ubuntu Server would work with all the commands provided. In this article, we're using one of BitLaunch's Ubuntu 20.04 LTS VPS servers.

Preparing Our Server

Let's prepare our server for our setup. We will first update the local package index and upgrade any packages that are outdated.

$ apt-get update
Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists... Done                        


$ apt-get -y upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
....
....

How to Install Fail2ban on Ubuntu

Ubuntu default repositories includes Fail2ban so we do not need to add any external repositories.

Let's install Fail2ban on Ubuntu package using the apt package manager.

$ apt install -y fail2ban
Reading package lists... Done
Building dependency tree       
Reading state information... Done
....
....

Allow Fail2ban to persist on boot:

$ systemctl enable fail2ban
Synchronizing state of fail2ban.service with SysV service script with /lib/systemd/systemd-sysv-install.

Start the Fail2ban service:

$ systemctl start fail2ban

Let's check the Fail2ban service's status:

$ systemctl status fail2ban
● fail2ban.service - Fail2Ban Service
     Loaded: loaded (/lib/systemd/system/fail2ban.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2020-06-03 12:43:39 UTC; 3s ago
       Docs: man:fail2ban(1)
    Process: 1395 ExecStartPre=/bin/mkdir -p /run/fail2ban (code=exited, status=0/SUCCESS)
   Main PID: 1396 (f2b/server)
      Tasks: 7 (limit: 614)
     Memory: 12.2M
     CGroup: /system.slice/fail2ban.service
             └─1396 /usr/bin/python3 /usr/bin/fail2ban-server -xf start

Jun 03 12:43:39 5eca7132aa8cc30001a9eb8b systemd[1]: Starting Fail2Ban Service...
Jun 03 12:43:39 5eca7132aa8cc30001a9eb8b systemd[1]: Started Fail2Ban Service.
Jun 03 12:43:40 5eca7132aa8cc30001a9eb8b fail2ban-server[1396]: Server ready

How to Configure Fail2ban on Ubuntu

There are several Fail2ban configuration files, but here is the directory structure at the time of writing:

$ /etc/fail2ban$ ll
total 72
drwxr-xr-x   6 root root  4096 Jun  3 10:19 ./
drwxr-xr-x 112 root root  4096 Jun  3 12:20 ../
drwxr-xr-x   2 root root  4096 Jun  3 10:19 action.d/
-rw-r--r--   1 root root  2817 Jan 11 10:01 fail2ban.conf
drwxr-xr-x   2 root root  4096 Mar  2 13:20 fail2ban.d/
drwxr-xr-x   3 root root  4096 Jun  3 10:19 filter.d/
-rw-r--r--   1 root root 25740 Jan 11 10:01 jail.conf
drwxr-xr-x   2 root root  4096 Jun  3 10:19 jail.d/
-rw-r--r--   1 root root   645 Jan 11 10:01 paths-arch.conf
-rw-r--r--   1 root root  2827 Jan 11 10:01 paths-common.conf
-rw-r--r--   1 root root   573 Jan 11 10:01 paths-debian.conf
-rw-r--r--   1 root root   738 Jan 11 10:01 paths-opensuse.conf

How to test Fail2ban

Let's run a few Fail2ban test. First, we can check our default enabled ssd jail:

$ fail2ban-client status
Status
|- Number of jail:      1
`- Jail list:   sshd

Let's view our jail's status:

$ fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     40
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     8
   `- Banned IP list:   z.z.z.z

We can see that sshd jail is using filter monitoring /var/log./auth.log file. We can also see that fail2ban already got on with it and banned an IP trying to guess our password.

To see the process better let's try to trigger the ssh filter by trying the wrong password multiple times. You will see that at the end the "connection refused" message, following commands that are entered from a different server/client:

$ ssh x.x.x.x
root@x.x.x.x's password: 

Permission denied, please try again.
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 
root@x.x.x.x: Permission denied (publickey,password).
$ ssh x.x.x.x
root@x.x.x.x's password: 

Permission denied, please try again.
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 
root@x.x.x.x: Permission denied (publickey,password).
$ 
$ ssh x.x.x.x
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 
root@x.x.x.x: Permission denied (publickey,password).
$ ssh x.x.x.x
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 
root@x.x.x.x: Permission denied (publickey,password).
$ ssh x.x.x.x
root@x.x.x.x's password: 
Permission denied, please try again.
root@x.x.x.x's password: 


^C
$ ssh x.x.x.x
ssh: connect to host x.x.x.x port 22: Connection refused
$

You can see that we are blocked.

Now let's have a look to our hosts' iptables, as we can see y.y.y.y (our ssh client machine) is banned and will be refused to connect.

$ iptables -L -v -n
.....
Chain f2b-sshd (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   24  2028 REJECT     all  --  *      *       z.z.z.z              0.0.0.0/0            reject-with icmp-port-unreachable
   26  2028 REJECT     all  --  *      *       y.y.y.y              0.0.0.0/0            reject-with icmp-port-unreachable
 3352  268K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 .....

As you can see, Fail2ban creates chains in iptables to manage the reject actions, we are seeing the one for ssh here.

Fail2ban Whitelist

This is a good place to talk about whitelisting, as sometimes we can try multiple times legitimately and can block ourself from the machine. To avoid this, we can whitelist our own IP addresses in the jail.conf as follows:

....
[DEFAULT]

ignoreip = 127.0.0.0/8 10.0.0.0/8 y.y.y.y
....

We have whitelisted following:

127.0.0.0/8 is an internal block
10.0.0.0/8 is our own VPN block
y.y.y.y is our external management server

You can also remove an existing ban with the Fail2ban unban command

$ fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 7
|  |- Total failed:     581
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     29
   `- Banned IP list:   z.z.z.z

$ fail2ban-client set sshd unbanip z.z.z.z
1

$ fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 7
|  |- Total failed:     582
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 0
   |- Total banned:     29
   `- Banned IP list:

Demonstration with Nginx

Here we apply the required configurations from scratch to ban clients that request non-existing pages repeatedly.

First we install Nginx:

$ apt install nginx
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  nginx
0 upgraded, 1 newly installed, 0 to remove and 10 not upgraded.
Need to get 3616 B of archives.
After this operation, 45.1 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 nginx all 1.17.10-0ubuntu1 [3616 B]
Fetched 3616 B in 0s (161 kB/s)
Selecting previously unselected package nginx.
(Reading database ... 78873 files and directories currently installed.)
Preparing to unpack .../nginx_1.17.10-0ubuntu1_all.deb ...
Unpacking nginx (1.17.10-0ubuntu1) ...
Setting up nginx (1.17.10-0ubuntu1) ...

Create a file under /etc/fail2ban/filter.d/ called nginx-404.conf, copy the contents of the text below, and save it.

[Definition]

# x.x.x.x - - [03/Jun/2020:13:34:35 +0000] "GET /admin HTTP/1.1" 404 163 "-" "curl/7.68.0"
failregex = ^<HOST> -.*"(GET|POST|HEAD).*" 404 .*$

ignoreregex =

Here we tell Fail2ban what to look for, this is a regex expression for Fail2ban log lines to match

Now create a file under /etc/fail2ban/jail.d/ called nginx-404.conf, and copy the contents of the below and save

[nginx-404]

enabled = true
filter  = nginx-404
logpath = /var/log/nginx/access.log
port    = http,https
maxretry= 5
findtime= 60

Reload fail2ban so it reads the new configuration files

$ systemctl reload fail2ban

Let's check our new jail is there

$ fail2ban-client status
Status
|- Number of jail:      2
`- Jail list:   nginx-404, sshd

On an another machine run the following 6 times:

$ curl http://x.x.x.x/admin
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.17.10 (Ubuntu)</center>
</body>
</html>

At the end you will be banned:

$ curl http://x.x.x.x/admin
curl: (7) Failed to connect to x.x.x.x port 80: Connection refused

$ fail2ban-client status nginx-404
Status for the jail: nginx-404
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     5
|  `- File list:        /var/log/nginx/access.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   y.y.y.y

In this demonstration we have configured maxretry a small number, for the real use cases this number must be increased.

You can again remove the ban with the unbanip command

Conclusion

Fail2ban is a great security tool which is highly configurable and already includes many default configuration you can use. Please explore filter.d directory to see what filters can be applied. Big thanks to authors of Fail2ban.