Running a server on the internet is quite an adventure but it's not always easy to keep track of log files, security and potential threads. This article describes some actions carried out on this server to mitigate script kiddies.
Among the good practices, I have
- a subscription to the Debian security mailing list,
- automatic security updates (enabled by default with the Scaleway Debian images)
- A monitoring tool (Munin) which provides useful graphs to watch the activity of the server.
- Fail2Ban
- Backups
The following paragraphs describe how I analyze the country IP banned by Fail2Ban.
As explained on their website, Fail2Ban scans log files and bans IPs that show the malicious signs: too many password failures, seeking for exploits, etc. Generally Fail2Ban is then used to update firewall rules to reject the IP addresses for a specified amount of time, although any arbitrary other action (e.g. sending an email) could also be configured. Out of the box Fail2Ban comes with filters for various services (apache, mail, ssh, etc).
Recently, I had the need to check if Belgian IP were blacklisted. Most of my users are Belgian and one of my Fail2Ban rules was too strict. I decided to log the IP in a file to perform a geolocalisation analysis to detect and prevent false positives.
The fail2ban-blacklist script was used to log blacklisted IP into a CSV file. The analysis is performed on another computer.
Scripts
The following script is called generate_report.py. It read a CSV file that has several information about bans: the date, time and IP. The country IP are discovered with the whois information thanks to a script. Finally, a barplot is generated to visualize the amount of hits per country. The whole process is launched with the report.sh script.
generate_report.py
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import sys
import matplotlib
import os
import subprocess
import re
import logging
logging.basicConfig(
level=logging.DEBUG,
format="[%(asctime)s] %(levelname)s %(message)s",
datefmt="%H:%M:%S",
stream=sys.stdout)
def get_country(ip):
whois = subprocess.Popen(['whois', ip], stdout=subprocess.PIPE)
whois.wait()
# Prevent problems if output is not utf8
str_whois = whois.communicate()[0].decode("utf-8", "replace")
find_country = re.search(r'country:(.*)', str_whois)
if find_country:
return find_country.group(1).strip()
else:
return "NONE"
def plot_graph(df=None):
plt.figure(figsize=(16, 10), )
df.country.value_counts().plot.bar(figsize=(16, 10))
plt.xlabel('Country', fontsize=16)
plt.ylabel('Counts', fontsize=16)
plt.title('Counts of blacklisted countries', fontsize=20)
plt.savefig("fail2ban_report.png", dpi=150, facecolor='w', edgecolor='w',
orientation='portrait', papertype=None, format=None,
transparent=False, bbox_inches=None, pad_inches=0.1,
frameon=None, metadata=None)
def report():
filename = 'blacklist.pkl'
df_backup = None
df_csv = None
df = None
df_concat = None
if os.path.exists(filename):
df_backup = pd.read_pickle(filename)
filename = r'blacklist.csv'
df_csv = pd.read_csv(filename, encoding="UTF-8", sep=',', engine='python', )
if len(df_backup) < len(df_csv):
df_tmp = df_backup[['DATE', 'TIME', 'IP_ADDRESS']]
df_concat = pd.concat([df_csv, df_tmp]).drop_duplicates(keep=False)
df_concat['country'] = ""
# df = df_backup.append(df_concat,sort=False)
logging.info("Append {} lines".format(len(df_concat)))
else:
logging.info("No difference between backup and CSV")
df = df_backup.copy()
friend_list = []
# df_friends = pd.DataFrame(columns=list(df_backup.columns.values))
df_friends = pd.read_pickle("blacklist_friends.pkl")
if df_concat is not None:
for idx, row in df_concat.iterrows():
# Do not process the dataframe multiples times.
if not row['country']:
logging.info("Process IP : {}".format(str(row['IP_ADDRESS'])))
country = get_country(str(row['IP_ADDRESS']))
df_concat.loc[idx, 'country'] = country
else:
country = row['country']
# country == 'Be' do not work
country = str(country)
if country.lower() in ['be', 'other_friendly_country']:
logging.info(row)
row['country'] = country
friend_list.append(row)
df = df_backup.append(df_concat, sort=False)
try:
logging.info("Append friends")
df_friends = df_friends.append(friend_list)
df_friends.to_pickle("./blacklist_friends.pkl")
except IndexError:
logging.info("No friend to append")
df.to_pickle("./blacklist.pkl")
plot_graph(df)
if __name__ == "__main__":
report()
report.sh
This script copy the Fail2Ban CSV file from the server (whois requests are forbidden on my VPS), generate the data and display the bar plot with the help of typop, a built-in function of terminology, a great terminal emulator for Linux/BSD/UNIX systems.
#!/bin/sh
scp user@agayon.be:/etc/fail2ban/report.py .
python generate_report.py
typop fail2ban_report.png
exit 0
Graphs
This graph was generated on 5 of december 2018. Some countries are more represented but the threat is global.
The following graphs is generated by Munin. It display the number of ban per jail.
