Securing Webservers - FirewallD Deployment on Ubuntu 20.04
Installation
apt-get remove ufw
apt-get install firewall-applet
Setup
Before enabling your firewall you need to check what ports are in use on your server:
netstat -lntup
This will list all processes that are actively listening on ports on your system:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address PID/Program name
tcp 0 0 127.0.0.53:53 621/systemd-resolve
tcp 0 0 0.0.0.0:22 689/sshd: /usr/sbin
tcp6 0 0 :::22 689/sshd: /usr/sbin
tcp6 0 0 :::443 1045/docker-proxy
tcp6 0 0 :::10050 4130/docker-proxy
tcp6 0 0 :::2222 1078/docker-proxy
tcp6 0 0 :::8080 1099/docker-proxy
tcp6 0 0 :::80 1065/docker-proxy
You can also run netstat -ltup to get a list where known ports are named by the service that usually occupies them - in case you don't know what a certain port is doing in this list:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address PID/Program name
tcp 0 0 localhost:domain 621/systemd-resolve
tcp 0 0 0.0.0.0:ssh 689/sshd: /usr/sbin
tcp6 0 0 [::]:ssh 689/sshd: /usr/sbin
tcp6 0 0 [::]:https 1045/docker-proxy
tcp6 0 0 [::]:zabbix-agent 4130/docker-proxy
tcp6 0 0 [::]:2222 1078/docker-proxy
tcp6 0 0 [::]:http-alt 1099/docker-proxy
tcp6 0 0 [::]:http 1065/docker-proxy
Services
FirewallD offers service presets that you can use to allow all necessary traffic for those services to work correctly:
firewall-cmd --get-services
RH-Satellite-6 amanda-client amanda-k5-client amqp amqps apcupsd audit bacula bacula-client bb bgp bitcoin bitcoin-rpc bitcoin-testnet bitcoin-testnet-rpc bittorrent-lsd ceph ceph-mon cfengine cockpit condor-collector ctdb dhcp dhcpv6 dhcpv6-client distcc dns dns-over-tls docker-registry docker-swarm dropbox-lansync elasticsearch etcd-client etcd-server finger freeipa-4 freeipa-ldap freeipa-ldaps freeipa-replication freeipa-trust ftp ganglia-client ganglia-master git grafana gre high-availability http https imap imaps ipp ipp-client ipsec irc ircs iscsi-target isns jenkins kadmin kdeconnect kerberos kibana klogin kpasswd kprop kshell kube-apiserver ldap ldaps libvirt libvirt-tls lightning-network llmnr managesieve matrix mdns memcache minidlna mongodb mosh mountd mqtt mqtt-tls ms-wbt mssql murmur mysql nfs nfs3 nmea-0183 nrpe ntp nut openvpn ovirt-imageio ovirt-storageconsole ovirt-vmconsole plex pmcd pmproxy pmwebapi pmwebapis pop3 pop3s postgresql privoxy prometheus proxy-dhcp ptp pulseaudio puppetmaster quassel radius rdp redis redis-sentinel rpc-bind rsh rsyncd rtsp salt-master samba samba-client samba-dc sane sip sips slp smtp smtp-submission smtps snmp snmptrap spideroak-lansync spotify-sync squid ssdp ssh steam-streaming svdrp svn syncthing syncthing-gui synergy syslog syslog-tls telnet tentacle tftp tftp-client tile38 tinc tor-socks transmission-client upnp-client vdsm vnc-server wbem-http wbem-https wsman wsmans xdmcp xmpp-bosh xmpp-client xmpp-local xmpp-server zabbix-agent zabbix-server
You can find out what ports are opened by a service inside the definition files in /usr/lib/firewalld/services/ - e.g.:
cat /usr/lib/firewalld/services/zabbix-server.xml
<?xml version="1.0" encoding="utf-8"?>
<service>
<short>Zabbix Server</short>
<description>Zabbix is a mature and effortless enterprise-class open source monitoring solution for network monitoring and application monitoring of millions of metrics.</description>
<port protocol="tcp" port="10051"/>
</service>
We can add all releavant service presets with:
firewall-cmd --permanent --zone=public --add-service=ssh
firewall-cmd --permanent --zone=public --add-service=git
firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-service=https
Ports
To add specialized ports run:
firewall-cmd --permanent --zone=public --add-port=2222/tcp
firewall-cmd --permanent --zone=public --add-port=8080/tcp
firewall-cmd --permanent --zone=public --add-port=4505-4506/tcp
firewall-cmd --reload
Docker
To actively reject traffic to a specific port:
firewall-cmd --permanent --add-rich-rule='rule family=ipv4 port port="9200" protocol="tcp" reject'
firewall-cmd --reload
BUT Docker will override this! Make sure to bind all ports that you don't want to be open to the internet to localhost!
Example docker-compose.yml:
ports:
- '127.0.0.1:9200:9200'
- '127.0.0.1:9300:9300'
Enable FirewallD
systemctl enable --now firewalld
systemctl status firewalld
Verify that everything is set up correctly:
firewall-cmd --zone=public --list-all
public
target: default
icmp-block-inversion: no
interfaces:
sources:
services: dhcpv6-client git http https salt-master ssh zabbix-agent zabbix-server
ports: 2222/tcp 8080/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
Prevent Bruteforce SSH attacks
Reject new incoming ipv4 connections when more than 2 attempts per minute are made. It will also log a message about this:
firewall-cmd --add-rich-rule='rule family="ipv4" service name="ssh" log prefix="SSH Bruteforce:" level="warning" limit value="2/m" accept limit value="2/m"' --permanent
If you have both ipv4 and ipv6 configured you’ll probably want the more generic version:
firewall-cmd --add-rich-rule='rule service name="ssh" log prefix="SSH Bruteforce:" level="warning" limit value="2/m" accept limit value="2/m"' --permanent
Create a Blacklist
firewall-cmd --permanent --new-ipset=blacklist --type=hash:net --option=family=inet --option=hashsize=4096 --option=maxelem=200000
–permanent= use to make changes to the permanent configuration–new-ipset= name of the new IP/net blacklist–type= storage hash type, "net" is for subnets, while "ip" for individual ip addresses–option=family= IPv4 or IPv6 network, inet is for IPv4–option=hashsize= the initial hash size of the list–option=maxelem= max number of elements
wget https://www.ipdeny.com/ipblocks/data/aggregated/ru-aggregated.zone
Populate the blacklist:
firewall-cmd --permanent --ipset=blacklist --add-entries-from-file=./ru-aggregated.zone
To add individual IP addresses or net blocks by yourself:
firewall-cmd --permanent --ipset=blacklist --add-entry=4.46.116.112
firewall-cmd --ipset=blacklist --add-entry=4.46.116.112
Redirect the blacklist to the drop zone
firewall-cmd --permanent --zone=drop --add-source=ipset:blacklist
firewall-cmd --reload
Block and Enable ICMP
firewall-cmd --zone=public --query-icmp-block=echo-reply
If you get "no", that means there isn’t any icmp block applied, let’s enable (block) icmp.
firewall-cmd --zone=public --add-icmp-block=echo-reply
Lockdown Rules
It’s possible to change the firewalld rules by any local applications, which have the root privileges. To avoid making changes to firewalld rules, we have to put a lock-down in ‘firewalld.conf‘ file. This mostly used to protect the firewalld from any unwanted rules changes by any applications.
nano /etc/firewalld/firewalld.conf
Lockdown=yes
firewall-cmd --reload
firewall-cmd --query-lockdown
To On/Off lockdown mode, use the following combination.
firewall-cmd --lockdown-on
firewall-cmd --lockdown-off
How to Reset when things go Wrong
Delete your Zone Settings:
rm -rf /etc/firewalld/zones
Using the below set of commands you will set accept rule for all types of connections.
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
This will confirm, iptables gonna accept all requests for all types of connections.
Using below set of commands, delete your currently configured rules from iptables.
iptables -F INPUT
iptables -F OUTPUT
iptables -F FORWARD
Or you can do it in single command:
iptables -F
That’s it! Your iptables are reset to default settings i.e. accept all!
fail2ban-firewalld
Configure fail2ban (see below) to block hosts via firewalld.
apt-get update && apt-get upgrade -y
apt-get install fail2ban
fail2ban
fail2ban is a daemon to ban hosts that cause multiple authentication errors.
fail2ban will monitor the SystemD journal to look for failed authentication attempts for whichever jails have been enabled. After the number of failed attempts specified it will add a firewall rule to block that specific IP address for an amount of time configured.
Start by installing the package on your system - Debian, Ubuntu or on Centos through EPEL.
The jail.conf file will enable Fail2ban for SSH by default for Debian and Ubuntu, but not CentOS. All other protocols and configurations (HTTP, FTP, etc.) are commented out. If you want to change this, create a jail.local for editing:
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Once installed the next step is to configure a jail (a service you want to monitor and ban at whatever thresholds you’ve set). By default IPs are banned for 1 hour. The best practice is to override the system defaults using _.local files instead of directly modifying the _.config files:
# cat /etc/fail2ban/jail.local
[DEFAULT]
# "bantime" is the number of seconds that a host is banned.
bantime = 1d
# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime = 1h
# "maxretry" is the number of failures before a host get banned.
maxretry = 5
After 5 attempts within the last hour the IP will be blocked for 1 day.
The next step is to configure a jail. In this tutorial sshd is shown but the steps are more or less the same for other services. Create a configuration file inside /etc/fail2ban/jail.d:
# cat /etc/fail2ban/jail.d/sshd.local
[sshd]
enabled = true
Next enable and start the fail2ban service.
systemctl enable --now fail2ban
systemctl status fail2ban
to check the status of fail2ban and make sure the jail is enabled enter:
fail2ban-client status
Status
|- Number of jail: 1
`- Jail list: sshd
fail2ban-client status sshd
Status for the jail: sshd
|- Filter
| |- Currently failed: 8
| |- Total failed: 4399
| `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
|- Currently banned: 101
|- Total banned: 684
`- Banned IP list: ...
tail -f /var/log/fail2ban.log
Check IP address geo location and add country ban lists where necessary whois ip-addrss | grep -i country.
Unbanning an IP Address
In order to remove an IP address from the banned list, parameter IPADDRESS is set to appropriate IP which needs unbanning. The name "sshd" is the name of the jail, in this case the "sshd" jail that we configured above. The following command does the job.
fail2ban-client set sshd unbanip IPADDRESS
Blacklisting Script and Configuration
The following script to automate the process as much as possible:
#!/bin/bash
# Based on the below article
# https://www.linode.com/community/questions/11143/top-tip-firewalld-and-ipset-country-blacklist
# Source the blacklisted countries from the configuration file
. /etc/blacklist-by-country
# Create a temporary working directory
ipdeny_tmp_dir=$(mktemp -d -t blacklist-XXXXXXXXXX)
pushd $ipdeny_tmp_dir
# Download the latest network addresses by country file
curl -LO http://www.ipdeny.com/ipblocks/data/countries/all-zones.tar.gz
tar xf all-zones.tar.gz
# For updates, remove the ipset blacklist and recreate
if firewall-cmd -q --zone=drop --query-source=ipset:blacklist; then
firewall-cmd -q --permanent --delete-ipset=blacklist
fi
# Create the ipset blacklist which accepts both IP addresses and networks
firewall-cmd -q --permanent --new-ipset=blacklist --type=hash:net \
--option=family=inet --option=hashsize=4096 --option=maxelem=200000 \
--set-description="An ipset list of networks or ips to be dropped."
# Add the address ranges by country per ipdeny.com to the blacklist
for country in $countries; do
firewall-cmd -q --permanent --ipset=blacklist \
--add-entries-from-file=./$country.zone && \
echo "Added $country to blacklist ipset."
done
# Block individual IPs if the configuration file exists and is not empty
if [ -s "/etc/blacklist-by-ip" ]; then
echo "Adding IPs blacklists."
firewall-cmd -q --permanent --ipset=blacklist \
--add-entries-from-file=/etc/blacklist-by-ip && \
echo "Added IPs to blacklist ipset."
fi
# Add the blacklist ipset to the drop zone if not already setup
if firewall-cmd -q --zone=drop --query-source=ipset:blacklist; then
echo "Blacklist already in firewalld drop zone."
else
echo "Adding ipset blacklist to firewalld drop zone."
firewall-cmd --permanent --zone=drop --add-source=ipset:blacklist
fi
firewall-cmd -q --reload
popd
rm -rf $ipdeny_tmp_dir
This should be installed to /usr/local/sbin and don’t forget to make it executable!
chmod +x /usr/local/sbin/firewalld-blacklist
Then create a configure file: /etc/blacklist-by-country:
# Which countries should be blocked?
# Use the two letter designation separated by a space.
countries=""
And another configuration file /etc/blacklist-by-ip, which is just one IP per line without any additional formatting.
For this example 10 random countries were selected from the ipdeny zones:
# ls | shuf -n 10 | sed "s/\.zone//g" | tr '\n' ' '
nl ee ie pk is sv na om gp bn
Now as long as at least one country has been added to the config file it’s ready to run!
firewalld-blacklist
firewall-cmd --info-zone=drop
firewall-cmd --info-ipset=blacklist | less
See also - auto blacklist updates: https://pagure.io/firewalld-blacklist/tree/master
download the service file and timer to /etc/systemd/system/ and enable the timer:
systemctl daemon-reload
systemctl enable --now firewalld-blacklist.timer