Introduction
Servers are the core for your business operations as they store data, running applications and more. They are important – but not impenetrable, and hence, can fall victim to hackers. This means it is no longer under your control, and this spells trouble on so many levels like performance, data security, downtime, and others. Being a fully protected cloud server means having multiple levels of security in place. With each level addressing a different type of threat – and combining to form an impenetrable barrier. This becomes a difficult task because just uncovering and blocking individual threats isn’t enough. It’s also important to defend against complex threats and take preventative action all the time. To effectively manage hackers, botnets, attackers, and malicious activities using BitNinja modular security system. Here we’re going to explain how each module can help you enforce them to stay secure.
What is BitNinja?
BitNinja is an easy-to-use server security tool mixing the most powerful defence mechanisms. Every BitNinja protected server learns from every attack and the system applies this information automatically on all BitNinja enabled servers. This way the shield is getting more and more powerful with every single attack. BitNinja has different modules for different aspects of cyberattacks. It is super easy to install, requires virtually no maintenance and able to protect any server by providing immediate protection against a wide range of cyberattacks. Anyone who wants to secure the server from different kinds of attacks can use it. Mostly, Website Developers, hosting providers, Digital Agency and anyone who owns servers and wants to keep them safe on the Internet can use it as it is a maintenance-free security solution.
Modules
There are 3 module categories. There are general modules for system functionalities, detection modules for detecting different threats your server faces, and captcha modules for providing self-removal functionality for users if their IPs are greylisted for any reason.
Detection Modules
Honeypots
Port Honeypot
This is a general honeypot module. This module will set up to 100 honeypots on your server on random ports chosen from the 1.000 most popular ports. This module will detect if someone does a deep port scan on your server (except syn stealth scan and some others). The module will also capture any traffic on these honeypots and reply to the requests, so when the attacker tries to exploit one of these fake services, it will generate incidents. This is a very effective way to catch early on both direct attacks and botnet activities.
The module does not bind on actual ports, but binds on a port above 60.000 and uses iptables rules to forward from the actual ports. We use this to avoid any port from being blocked from real services. If a daemon starts listening a honeypot port, the module will automatically stop using that port as honeypot. When the module starts, it also lists all the open sockets in listening mode and won’t start honeypot on active ports. This way the module will automatically avoid any collision with real services. If you want, you can set ports to always use for honeypot purposes and you can set up ports that you never want to be used as a honeypot.
Note
If you have cPanel or Plesk installed on your server, the module will automatically detect that, and will add the ports used by these services to the exclude list. Port Honeypot won’t try to start a honeypot on these ports.
For some protocols we have implemented chat scripts. Chat scripts are used for faking services even more realistically. For example, in case of the telnet protocol (an old protocol, usually listens on port 23), if you try to open the honeypot created by BitNinja, you will see a prompt and a default message like this:
This is an unrestricted telnet server.
Please do not use for production purposes
#
The attacker will think that this is an open telnet session for the server and will probably try to download malware and do other malicious activities. All the information will be collected by BitNinja and sent to the BitNinja center. The offending IP will be greylisted. All greylisted IPs can still connect to honeypots, but all other real services are blocked (except the CAPTCHA services).
We keep improving the chat scripts for the different protocols so we can get more information from the attackers and use this information to further tweak the BitNinja protection system. Currently we have chat scripts for these protocols:
FTP (21)
telnet (23)
SMTP (25)
IMAP (143)
POP3 (110)
The chatscripts are stored at /opt/bitninja/modules/PortHoneypot/lib/ChatScriptFactory.php Do you feel like you could contribute by implementing further protocols? Feel free to send us your chat script file so we can include it in the next BitNinja version. In the case of ports without a chatscript, the system chooses a chatscript randomly, so don’t be surprised if you find ftp, telnet, smtp, imap, and pop3-like services everywhere on your server. They are not real services, just honeypots :-)
Configuration
Here is how to set up a list of ports to be used as honeypots and the list of ports you never want to be used for honeypot purposes.
Config file location: /etc/bitninja/PortHoneypot/config.ini
;
; List of ports that will always be honeypotted
;
[ports_always_use]
ports[]=21
ports[]=23
ports[]=53
;
; List of ports the module will never use for honeypot purposes
;
[ports_never_use]
ports[]=80
ports[]=25
ports[]=22
;
; Starting port for Porthoneypot.
; If you experience ftp issues when using Porthoneypot, it could be because ftp is trying
; to create data connection port on one of the ports that Porthoneypot is using.
; To solve this issue, you can restart ftp service to recognize used ports or set
; ftp passive port range and PortHoneypot startport to not intercept each other.
;
[startport]
startport=60500
;
; You can control how many additional random honey ports should be started. Default value is 100.
;
[random_mines]
random_mines=100
Web Honeypot
Honeypotting is a security technique where you set up a system or subsystem to pretend that there is a vulnerable service available. The attacker, hacker, or bot will easily see that there is a vulnerability and will try to abuse it. As the honeypot is not providing the service it advertises in reality, it will rather collect the attack data and block the attack. This technique is similar for setting up traps against your enemies. This is very effective against both automatized and targeted attacks.
This module is responsible for setting up such honeypot traps on your PHP based websites.
How can you implement a web honeypot?
It is quite easy and straightforward. Create a PHP file with this content:
<?php
/*
* BitNinja Server Security
* All rights reserved.
* https://bitninja.io
*
* @author Zsolt Egri <[email protected]>
* @copyright C 2014 Web-Server Kft
* @package BitNinja
* @subpackage HoneypotHttp
* @version 1.0
*/
/*
* Function to send request data to the SenseWebHoneypot module of BitNinja.
*/
function sendData()
{
$socket = stream_socket_client("tcp://127.0.0.1:60099", $errno, $errorMessage);
stream_set_timeout($socket, 1, 0);
socket_set_blocking($socket, 1);
if ($socket === false) {
return FALSE;
}
$dataToSend = json_encode(array(
'server' => $_SERVER,
'post' => $_POST,
'get' => $_GET,
'file' => __FILE__,
'pid' => getmypid(),
'uid' => getmyuid()
));
while (strlen($dataToSend)!==0)
{
$bytesWritten = fwrite($socket, $dataToSend);
$dataToSend = substr($dataToSend, $bytesWritten);
}
fclose($socket);
return TRUE;
}
?>
<!-- Your content should go here... -->
<html>
<head>
<title>BitNinja Honeypot</title>
</head>
<body>
This is a honeypot file. Please leave it.
</body>
</html>
<?php
/*
* Finaly, we flush the output - send the content to the client - and
* call the sendData() function to send the request to BitNinja.
*/
flush();
sendData();
?>
Of course you can combine this code as you like. You are able to set up your own logic regarding what is considered malicious. These incidents will be sent to the web honeypot module, greylisting the offending IP on all your servers. This example code can be found at /opt/bitninja/modules/SenseWebHoneypot/examples/example_honeypot_file.php. To use it:
copy it where the attacker can reach it.
rename it to something interesting like admin_login.php
change the owner to the proper web user
done :)
Note
Set up some honey on your web applications for hackers! You can set up web traps by hiding some virtual bugs in your application. What do you think about a cookie like this: admin=false You can then verify the value of the admin cookie and if it’s anything other than false, you can send the incident to the web honeypot module. Believe me, no hacker can resist his/her curiosity to try what happens when they change this value to true. ;-)
Note
Are there web bots crawling your forum all the time or trying to send some spam? Just add a link to your honeypot file and don’t forget to deny all the good bots using a robots.txt entry. Bad bots will follow the link and be trapped, whereas good bots will avoid the trap.
Note
Ever heard about Google Dorks? Put some honeypots in URLs which can be found in the Google Hack Database and watch who tries to access it.
Note
Attackers try to Directory brute force your site? Make your default 404 page go to a honeypot.
Note
Found an infected file or backdoor on your server? You can replace it with a honeypot. MalwareDetection has an option to auto honeypotify an infected script. You can enable it in /etc/bitninja/MalwareDetection/config.ini
[core]
honeypotify = 'enabled'
AntiFlood
The duty of this module is to receive incidents from other modules and prevent attackers from flooding your system with incidents.
For example, the CaptchaHttp module shows a CAPTCHA screen for visitors if they are greylisted. But what if an attack is ongoing against the CAPTCHA module? The AntiFlood module will find this flood attempt and temporarily blacklist the IP, preventing the overload of the HTTP CAPTCHA module.
The basic idea of BitNinja is to integrate different security tools into a single modular system. This way the modules can intercommunicate and use each other’s services. The AntiFlood module is also an often-used module for other BitNinja processes. Anything that can send incidents will send them to the AntiFlood module as well. This way if there are some trials the log analyzer, HTTP CAPTCHA, or SMTP CAPTCHA catch, this information will all be available locally for the AntiFlood module. This way it can detect the attack and prevent further overuse and scans.
Configuration
Threshold limits can be set in the config file (/etc/bitninja/AntiFlood/config.ini).
;
; Limits for AntiFlood module
;
[limits]
;limit before getting on blacklist
blacklist=15
;limit before getting on greylist
greylist=10
;1 hour
expiredtime=3600
;5 days
agedtime=432000
DosDetection
BitNinja has a built-in module for monitoring current active connections and interfering in case of a potential denial of service attack. If there are more than 80 connections from a single IP, BitNinja detects it as an attack. The 80 threshold can be configured on a per port basis.
Configuration
To change the default thresholds create a config file at /etc/bitninja/DosDetection/config.ini
An example content for the config.ini file:
;
; Thresholds set to DoS Detection
;
[tresholds]
general = 80
; Threshold for remote SMTP servers.
remote[25] = 200
remote[53] = 200
; Threshold for local ports
local[22] = 40
;You can set restrictions for remote and local ports. For example to change
;the number of connections allowed to the default IMAP4 port (143) you can do this:
local[143] = 150
When BitNinja detects a DoS attack, it will block the IP for 1 minute and then place it to the greylist allowing the user to delist his/her IP. To change the default threshold, you can modify /etc/bitninja/IpFilter/config.ini, or add this section with the proper value:
[times]
;
; Temporary blacklist time in DoS suspicious requests; default: 60
;
tmp_bl_sec = 60
You can read more about the greylist on the CaptchaHttp - Http Captcha module page.
Don’t forget to restart BitNinja after creating the custom restrictions.
service bitninja restart
Web Application Firewall
Web Application Firewall (WAF) constantly scans and analyzes the incoming traffic flow to your server, looking for malicious content based on different factors. Used in conjunction with Log Analysis, WAF guarantees an extremely low false-positive rate while stopping attacks against the applications running on your server:
BitNinja provides automatic updates and firewall rules. We constantly patch new vulnerabilities for you.
For custom needs, you can easily set up a list of whitelisted domains or URLs.
In case of a suspicious web application, you can switch the WAF into a strict mode with tighter rules to avoid any further infection.
BitNinja’s Web Application Firewall is a zero-configuration service, so you don’t need to waste time setting up your WAF and configuring rules.
Log Analysis
This module is responsible for analyzing log files on your server and responding with the appropriate actions upon any malicious activity found. New log files are added constantly for auto-detecting. We first apply new log analyzer rules in test mode. All test incidents generated this way will be analyzed for false positives, so we can keep the rate low. These incidents will not show up on the dashboard.
BitNinja can find log files in the following paths at the moment:
Apache access logs
Debian / Ubuntu default path - /var/log/apache2/access.log
RHEL / Red Hat / CentOS / Fedora Linux default - /var/log/httpd/access_log
FreeBSD default - /var/log/httpd-access.log
/home/accesslog/access.log
Plesk vhosts at - /var/www/vhosts/[domain]/statistics/log/access_log
/var/www/vhosts/[domain]/system/log/access_log
Virtualmin vhost logs at - /var/log/virtualmin/*access_log
cPanel logs at /usr/local/apache/logs/access_log
cPanel per user logs at /usr/local/apache/domlogs/
DirectAdmin vhost logs
/var/log/httpd/domains/
Apache error logs
Debian /var/log/apache2/error.log
CentOS /var/log/httpd/error_log
FreeBSD /var/log/httpd-error.log
Plesk vhosts error logs - /var/www/vhosts/[domain]/statistics/log/error_log
/var/www/vhosts/[domain]/system/log/error_log
Virtualmin vhosts error logs
cPanel error logs at /usr/local/apache/logs/error_log
NginX access logs
/var/log/nginx/access.log
NginX error logs
/var/log/nginx/error.log
System-wide logs
Debian /var/log/auth.log
CentOS /var/log/secure
/var/log/messages
cPanel logs * /usr/local/cpanel/logs/login_log
Exim4 logs
Debian /var/log/exim4/mainlog
CentOS /var/log/exim/main.log
cPanel, Plesk /var/log/exim_mainlog
Postfix logs
/var/log/maillog
/var/log/mail.log
ProFTP logs
/var/log/proftpd/proftpd.log
VsFtp logs
/var/log/vsftpd.log
MySQL logs
Based on the files opened by the MySQL processes; it’s name should end with .err
Configuration
You can find the configuration file for this module at /etc/bitninja/SenseLog/config.ini .
Disabling rules
BitNinja has several supervisors, each of these are responsible to recognise a specific attack and ban the origin IP
On the code snippet below, you can see how to disable supervisors.
;
; Example: Enable/Disable Supervisors
;
[supervisor]
disabled[] = 'ApacheNoscript'
disabled[] = 'FtpVsftpd'
The list of available supervisors
ApacheAbdul protects against attacks with abdullkarem in the request string. For further information, see our blog post .
ApacheE107 finds directory traversal attacks for vulnerable E107 installations.
ApacheGooglemap2 prevents reflective DDoS attacks via plugin_googlemap2.
ApacheJoomla recognises and prevents Autoshell upload attempts.
ApacheJoomlaLogin prevents brute-force attacks against Joomla sites.
ApacheMagento protects against Magento vulnerabilities including directory traversal using magmi.php.
ApacheParentDenied gives protection agains a broad range of directory traversal attacks.
ApacheRecommendedSite prevents spamming attempts of a known botnet.
ApacheShellshockFilter protects against code injection over Shellshock.
ApacheSuspiciousUserAgent finds requests from well-known tools based on the user agent.
ApacheWpConfig is used against directory traversal attacks targeting wp-config.php.
ApacheWpEnumeration prevents Wordpress user enumeration attacks.
ApacheWpLogin prevents brute-force attacks against Wordpress-based sites.
ApacheWpXmlrpc prevents reflective DDoS attacks via xmlrpc.php.
ApacheWpXmlrpcSuspiciousUserAgent quickly prevents brute-force attacks from a known bot.
CpanelLogin protects cPanel accounts from brute-force attacks.
FtpProftpd prevents brute-force attacks against ProFTP servers.
FtpPureftpd prevents brute-force attacks against PureFTP servers.
FtpVsftpd prevents brute-force attacks against VsFTP servers.
MysqlLogin prevents brute-force attacks against MySQL servers.
NginxHttpAuth protects sites with basic HTTP authentication against brute-force attacks.
PostfixLogin prevents brute-force attacks against Postfix servers.
SshdLoginfail prevents brute-force attacks against OpenSSH servers.
Specifying custom log paths
Custom log paths can be added to every available log detector via these lines. Please don’t add rotated log files to paths, because it will duplicate incidents, which causes anomalies in your statistics. Please also note that SenseLog is only able to handle 3000 - 4000 log files for the time being. Above this number SenseLog might crash and retry to detect log files. This issue and limitation will be solved shortly.
;
; Custom log paths for LogDetectors
; The ini array 'logpathes' can be path to a file or a pattern, like: /var/log/vhost_*/*_access.log .
; This pattern will load all matching logfiles to the monitored list.
;
[ApacheAccess]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[ApacheError]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[Auth]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[CpanelLogin]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[Exim]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[FtpProftpd]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[FtpPureftpd]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[FtpVsftpd]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[MysqlError]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[NginxError]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
[PostfixLogin]
;logpathes[] = '/path/to/your/log'
;logpathes[] = '/path/to/your/log2'
SenseLog configurations can be reloaded via BitNinja CLI, without restarting BitNinja. This will make your changes applied instantly.
bitninjacli --module=SenseLog --reload
MalwareDetection
BitNinja has an excellent module for file-based malware detection. If attackers can break through the defense line of honeypots and the web application firewall (see our other detection modules here: detection-modules), malware detection is the next line of defense to stop them infecting your server. It is also very useful to scan your server if you just started to use BitNinja for the first time, as there could be different malwares and backdoor programs on your server. Cleaning your files is vital.
We ship new malware definition files automatically so it is always up to date without any manual intervention.
Prerequirements
The module uses the inotify-tools package to monitor the filesystem for changes. You need this package to be available before you can use the module. We ship our own inotify-tools package with BitNinja to ensure the installation. If you need to install inotify-tools manually for some reason, please follow the steps stated here: Optional: manual installation of inotify-tools
Before you enable the Malware Detection module, there are some steps you have to follow on your server to prepare it for the launch of the module.
On most servers, default number of user watches have to be expanded from the default to a much higher number. BitNinja Linux Client increases it automatically up to 30000000.
If it is not large enough, you can increase it manually, e.g.:
echo 35000000 > /proc/sys/fs/inotify/max_user_watches
Now check if everything is ready to start the malware detection module:
cat /proc/sys/fs/inotify/max_user_watches
// This should be the result:
// 30000000
whereis inotifywait
// Something similar should be the result
// inotifywait: /usr/bin/inotifywait /usr/bin/X11/inotifywait /usr/share/man/man1/inotifywait.1.gz
// check if inotify-tools is on the latest version
// on deb-based systems
dpkg-query --showformat='${Version}' --show inotify-tools
// on rpm-based systems
rpm -q --qf "%{VERSION}\n" inotify-tools
// result: 3.14-2
You are ready to start the malware detection module now! Before lauching the module please read through the configuration section to prepare the module.
Configuration
To change the default configurations of the malware detection module you have to edit the module config file at /etc/bitninja/MalwareDetection/config.ini. We always try to set up reasonable defaults for our config files so you don’t have to mess with configs, but in this case there are some configurations you are probably willing to do before starting the module.
Example content for the config.ini file:
; Core settings
;
[core]
; do not check files over scan_max_file_size bytes
;scan_max_file_size = 786432 ; 768 KB 1048576 ; 1 MB
; scan process CPU scheduling priority
;scan_cpunice = 19
; scan process IO scheduling priority
;scan_ionice = 6
; move malicious file to quarantine; enabled|disabled
quarantine = 'enabled'
; replace malware to honeypot; enabled|disabled
honeypotify = 'enabled'
; if replace is enabled, this file will be copied instead of malware
;honeypot_file = '/opt/bitninja/modules/SenseWebHoneypot/examples/example_honeypot_file.php'
;
; File whitelist settings
;
[whitelist]
; do not scan files match these extensions
;extensions[] =
; do not scan files match these path definition expressions
;paths[] = '/var/lib/bitninja'
;paths[] = '/var/log/bitninja'
;paths[] = '/var/cache/awstats'
;paths[] = '/opt/bitninja'
;paths[] = '/bin'
;paths[] = '/sbin'
;paths[] = '/lib'
;paths[] = '/lib32'
;paths[] = '/usr/bin'
;paths[] = '/usr/sbin'
;paths[] = '/usr/lib'
;paths[] = '/usr/lib32'
;paths[] = '/home/accesslog'
;paths[] = '/home/virtfs'
;paths[] = '/home/cagefs-skeleton'
;paths[] = '/usr/share/cagefs-skeleton'
;paths[] = '/home/cpeasyapache/src'
;paths[] = '/proc'
;paths[] = '/dev'
;paths[] = '/sys'
; do not scan files match these types based on file command
;types[] =
;
; Inotify settings
;
[inotify]
; ignore files match these path definition expressions ; e.g: .log
;exclude[] = '^/var/tmp/mysql.sock$'
;exclude[] = '^/tmp/mysql.sock$'
;exclude[] = '^/var/cache/buagent/md0.cache.data$'
;exclude[] = '^/var/tmp/#sql_.*\.MYD$'
;exclude[] = '^/tmp/#sql_.*\.MYD$'
;exclude[] = '^/tmp/#sql_.*\.MYI$'
;exclude[] = '^/tmp/#sql_.*\.MAD$'
;exclude[] = '^/tmp/#sql_.*\.MAI$'
;exclude[] = '^/tmp/lshttpd/*\.sock*'
;exclude[] = '^/tmp/lshttpd/\.rtreport\.*'
;exclude[] = '^/var/tmp/clamav-.*'
;exclude[] = '^/tmp/clamav-.*'
;exclude[] = '^/var/lib/bitninja'
;exclude[] = '^/var/log/bitninja'
;exclude[] = '^/var/cache/awstats'
;exclude[] = '^/usr/local/maldetect/quarantine'
;exclude[] = '\.sock$'
;exclude[] = '\.log$'
;exclude[] = '^.*_log$'
;exclude[] = '^.*_log\.processed$'
;exclude[] = '^.*_ssl_log\.webstat$'
;exclude[] = '^/home/accesslog'
;exclude[] = '^/home/virtfs'
;exclude[] = '^/home/cagefs-skeleton/'
;exclude[] = '^/usr/share/cagefs-skeleton/'
;exclude[] = '\.sock$'
;exclude[] = '^/home/.*?/mail/'
;exclude[] = '^/home/cpeasyapache/src/'
;exclude[] = '^/var/tmp/clamav-.*'
;exclude[] = '^/var/lib/bitninja/quarantine$'
; inotify monitoring array
; This is were you can define what to be monitored by the
; malware detection module.
;file_path[] = '/tmp'
;file_path[] = '/home'
;file_path[] = '/var/www'
Quarantine behaviour
The quarantine behaviour depends on a setting in the module’s config.ini file. If the ‘quarantine’ option is enabled, BitNinja moves the infected file to quarantine.
You can find the quarantine directory here: /var/lib/bitninja/quarantine so you can restore any files if you need. This module is also integrated with another BitNinja module, the WebHoneypot module (you can read more about the web honeypot module here: web-honeypot). When a botnet attacks your system it will try to infect your files. If you only quarantine the file, the botnet will try to upload other malware and other variations of the infection and will eventually succeed and infect your system. If the ‘honeypotify’ config option is enabled, the module will automatically copy a honeypot file in place of the original malware. This way when the botnet comes back to check the operation of the malware infection, the honeypot will collect the IP address of the attacking bot and add it to the greylist, immediately protecting all your servers against this attacker.
Example quarantine directory structure:
├── var
│ ├── lib
│ │ ├── quarantine
│ │ │ ├── 2016
│ │ │ │ ├── 12
│ │ │ │ │ ├── 10
│ │ │ │ │ ├── 17
│ │ │ ├── 2017
│ │ │ │ ├── 01
│ │ │ │ ├── 02
│ │ │ │ │ ├── 01
│ │ │ │ │ ├── 05
│ │ │ │ │ ├── 10
│ │ │ │ │ ├── 17
│ │ │ │ ├── 03
│ │ │ │ │ ├── 12
│ │ │ │ │ ├── 13
Example quarantinized malware information…
7650_jquery94.php 7650_jquery94.php.info
… where file with .info extension contains information about the malicious file (original owner, group, file permissions, file path, malware name, file size, create time), while the other is the malicious file itself. Both files are prefixed with a unique identifier.
Disabling the quarantine functionality
If you’d like to disable the quarantine behavior of the Malware Detection module, you have to change a setting in the module’s config.ini file:
Locate the config file at /etc/bitninja/MalwareDetection/config.ini
Look for the Core settings section
Insert this line after the last [core] setting (e.g. the honeypot_file setting)
quarantine = 'disabled'
The example config.ini file’s core section should look something like this:
; Core settings
;
[core]
; do not check files over scan_max_file_size bytes
;scan_max_file_size = 786432 ; 768 KB 1048576 ; 1 MB
; scan process CPU scheduling priority
;scan_cpunice = 19
; scan process IO scheduling priority
;scan_ionice = 6
; replace malware to honeypot; enabled|disabled
honeypotify = 'enabled'
; if replace is enabled, this file will be copied instead of malware
;honeypot_file = '/opt/bitninja/modules/SenseWebHoneypot/examples/example_honeypot_file.php'
quarantine = 'disabled'
Whenever you’d like to enable the quarantine function again, you should simply change the quarantine setting to ‘enabled’.
Note
If you edit the content of the /etc/bitninja/MalwareDetection/config.ini file, you have to reload the Malware Detection config for the changes to take effect. You can do this by using the following command:
bitninjacli --module=MalwareDetection --reload
Scan path
The most important config change you will probably do is defining what directories the module should monitor for file changes. The defaults are the /tmp, /home and /var/www directories, but you can add any other directories by defining new paths. For example:
file_path[] = '/var'
Exclude patterns
You can define patterns to be excluded from being scanned. This is useful to exclude files like log files.
exclude[] = '/var/www'
Don’t forget to reload the configuration of Malware Detection after altering it for the settings to take effect.
Module initialization
When the Malware Detection module starts at the first time (or any time the inotifywait process stops or is killed) the system have to reinitialize the inotify process. This is a very CPU and disc intensive process and can take hours to finish. (It should traverse all directories and set a flag on them to be monitored. Sorry, this is how the Linux kernel handles the inotify functionality.)
You can find the module logs at /var/log/bitninja/mod.malware_detection.log
Note
We’d like to help you by sharing our own experiences regarding the module’s resource usage. We have 30+ production servers at our shared hosting company right now. In one of the bigger ones, the number of files is 11 714 930. In this case the initial step takes about 5 hours and it doesn’t cause any load issues after that. The average load on this server is 3.
Manual scanning
BitNinja is continuously listening for file changes, but you are also able to run manual scanning with the CLI tool. E.g. in order to look after malware in the /var/www folder, use this command:
bitninjacli --module=MalwareScanner --scan=/var/www
If you’d like to cancel a malware scan that’s already been started, you can do it using the following command:
bitninjacli --module=MalwareScanner --cancel
Manual scanning works fine without installing inotify-tools.
Restoring files from quarantine
You can manually restore files from the quarantine using the following command:
bitninjacli --restore=/path/to/file
Post-detection, post-quarantine and post-honeypotify scripts
You can implement your own business logic to run after each detected malware. You should write your own scripts in an .sh file and place it in the /etc/bitninja/MalwareDetection/detection.conf.d folder.
For post-detection scripts, name your file using the pd_ prefix, e.g. like this: pd_my_script.sh. These scripts will run immediately after the module detects an infected file.
For post-quarantine scripts, the prefix should be pq_, e.g. pq_my_script.sh. These scripts will be executed after the module moves the malware file to quarantine.
For post-honeypotify scripts, the prefix should be ph_, e.g. ph_my_script.sh. Scripts that start with ph_ will run after the module replaces the infected file with a honeypot file.
Note
For the post-quarantine scripts to run, the quarantine function must be enabled in the module’s config.ini file. For the post-honeypotify scripts to work, the honeypotify function must be enabled in /etc/bitninja/MalwareDetection/config.ini.
Example for a post-detection script: pd_example.sh
#!/bin/bash
# This post detection script just echoes out the path to the detected malware
echo "Malware found at path [$1] Malware name is [$2]" > /path_you_wish_to_save_outcome/malware.txt
Optional: manual installation of inotify-tools
Install inotify-tools on .DEB based systems
apt-get update && apt-get install inotify-tools
Install inotify-tools on .RPM based systems
yum --enablerepo=epel install inotify-tools
Note
On some systems the inotify-tools package is not available by default, so you have to enable the EPEL repository.
Package inotify-tools has to be on the latest version (v3.14).
This will install the inotify-tools package on your server so the module can use it.
DefenseRobot
If you want to use the DefenseRobot module, first you have to activate it with the following command:
bitninjacli --module=DefenseRobot --enabled
After enabling, the module will wait for events coming from the MalwareDetection module, e.g. when a malware has been found. With this information in hand, the module tries to find log lines related to the malware upload within the configured time window, which is 30 seconds before the malware is changed (ctime).
The default watched logs for this module is found in SenseLog’s LogDetector categories, which are collected in the following order:
xferlog
PureFtpd logs
vsftpd logs
Apache access logs
Nginx access logs
cPanel access logs
Plesk access logs
WAF access logs
These categories hold various techniques for detecting log files. E.g.: XferLog tries to detect and find log files at paths:
/var/log/proftpd/xferlog
/var/log/xferlog
/var/log/plesk/xferlog
If relevant information has been found in these log files, various actions can be performed using the correlation information.
These actions are the following (for now):
SendToShogun: Creates a BL_BN_LOG type incident with log only type. It will not greylist IPs, but we’ll have the information about the attacker. It’s enabled by default.
LogToFile: Save the correlation information about the incident to /var/log/bitninja/correlations/YYYY/MM/DD/hh_mm_uniqid folder. It’s enabled by default.
GreyList: Changes the SendToShogun action’s incident level to a valid incident. The IPs will be greylisted and the incident level will not be log only. It’ll be enabled by default after the test period.
SaveUnFilteredLoglines: Saves every log line in the time window. E.g: HTTP access logs will contain GET, HEAD, PUT request lines too. IPs in these lines will not trigger any incidents, but it can help to find out what happened on the system during that time. It’s disabled by default.
CollectUnWatchedLogs: SenseLog has other LogDetectors, like Auth, Exim, PostfixLogin. With this action these log lines will be collected too, but IPs found in them will not trigger any incidents. It’s disabled by default.
These options can be changed in the module’s configuration file which is located at /etc/bitninja/DefenseRobot/config.ini
Configuration options
[core]
time_window = 30
[actionManager]
actions[] = 'SendToShogun'
actions[] = 'LogToFile'
;actions[] = 'GreyList'
;actions[] = 'SaveUnFilteredLoglines'
;actions[] = 'CollectUnWatchedLogs'
;
; Change Control Panel/FTP user password
; Not Implemented yet
;
;actions[] = 'ChangePassword'
;
; Automatically WAF Honeypotify abused domain/uri
; Not Implemented yet
;
;actions[] = 'WAFHoneypotify'
Outbound Web Application Firewall
BitNinja’s OutboundWAF module scans only outgoing connections real-time, with the built-in http proxy solution. Upon activation BitNinja injects an iptables rule into your nat table to redirect all outgoing traffic not made by root to ports 80 to 127.0.0.1:60100, a port dedicated for BitNinja OutboundWAF. The OutboundWAF process analyzes the outgoing traffic and makes a connection from your server to a remote server and fetch the actual content.
If BitNinja finds any suspicious connections, it will send the incident to BitNinja analyzer central and try to find which process and script is responsible for it.
Limitations
- BitNinja OutboundWAF currently has a limit of 1000 simultaneous connections.
If your server manages more than this amount, please see the configuration section on how to raise this limitation.
- BitNinja OutboundWAF forks a new process for every request to spray the load
between multiple CPUs. Every process currently has about a 1.7 MB memory footprint, so it is designed to be resource friendly, but be aware of resource usage in large scale installations.
Requests made by root are not watched.
- BitNinja OutboundWAF uses ss to find which process is responsible for the malicious
request, and on some server is can cause higher load.
Activating and Deactivating the OutboundWAF module
Here is how to activate the OutboundWAF module:
Log in to https://admin.bitninja.io/
Navigate to https://admin.bitninja.io/modules
Select the server on which you want to activate the OutboundWAF module.
Switch OutboundWAF on.
Or you can use the command line
bitninjacli --modeule=OutboundWAF --enabled
BitNinja’s OutboundWAF will be activated within 15 seconds.
To deactivate the OutboundWAF module you have 3 options:
You can use the dashboard and set the OutboundWAF module to disabled and save it.
You can use the command line
bitninjacli --module=OutboundWAF --disabled
You can shut down BitNinja (this is the most radical solution, but it will also disable the OutboundWAF appropriately)
service bitninja stop
Configuration
The OutboundWAF module uses the config file located at /etc/bitninja/OutboundWAF/config.ini
In some cases you don’t want the OutboundWAF to monitor your outgoing http connections. Eg.: using an HAProxy load balancer or an Nginx reverse proxy. You can disable the OutboundWAF monitoring for these users by adding their names in the configuration like:
[usercontrole]
ignoredUsers[] = 'bitninja-waf'
ignoredUsers[] = 'bitninja-ssl-termination'
Captcha modules
The CaptchaHttp module is used to identify false-positives on the BitNinja greylist and allow human web users to remove themselves from the greylist easily. In other words, if an IP is placed onto the BitNinja greylist and tries to access a web page on a server protected by BitNinja, the request will be forwarded to an alternative web server (created by BitNinja) and a Browser Integrity Check (BIC) page will be shown to the user. If the browser integrity test is successful then the BIC page will redirect them to the original url after 5 seconds. You won’t have to fill or type anything because the process is automatic. If your server generates an incident locally, then BitNinja will show the CAPTCHA page instead of the BIC one. If the user is a human visitor, he/she can easily solve the CAPTCHA and the IP automatically gets delisted. The visitor will be redirected to the original page and he/she can go on surfing the pages. The submission of the CAPTCHA form is delayed by 2 seconds.
The web server created to serve the CAPTCHA page is specially tuned to sustain a high traffic load and serve every request from memory. It is a one-threaded, event-driven web server so it cannot overload a multi CPU system. The memory consumption of the server is also limited, so it cannot eat up all the system memory. In case of an overload of this server, it will crash and restart automatically after 5 minutes. This tiny web server was designed especially for serving CAPTCHA pages, receiving any malicious traffic, and sending it to BitNinja central for further analysis, virtually creating a huge honey farm out of the servers using BitNinja.
Relations to other modules
CaptchaHttp sends events to the Shogun module in order to send it to BitNinja Central, as well as to the AntiFlood module to detect flood attempts and block them.
Configuration
You can find the configuration file for this module at /etc/bitninja/CaptchaHttp/config.ini
[CaptchaHttpSettings]
; The backlog is the limit for the queue of incoming connections.
socketBacklogSize = 50
; How many times BIC page will be shown for new visitors
maxBICRetries = 1
; Add HoneyConent meta tags to captcha page
loadHoneyContent = 0;
; Add Captcha service for greylist HTTP connections on the following ports
http_captcha_ports[]=8080
http_captcha_ports[]=8888
; Add Captcha service for greylist HTTPS connections on the following ports
https_captcha_ports[]=8443
https_captcha_ports[]=8888
Note
The above mentioned HTTP/HTTPS configurations are not enabled by default.
Customizing the BIC page
You can easily customize the texts and styles of the BIC page on your server. It works on a per server basis and supports multilingual BIC sites.
The BIC page is translated to the following languages so far:
English
Hungarian
Japanese
German
Spanish
French
Italian
Dutch
Polish
Russian
Turkish
Danish
BitNinja has its default templates in the
/opt/bitninja/modules/CaptchaHttp/lib/www/browser_integrity_check.html
file.
You shouldn’t change this file as the package manager will overwrite these files with every new BitNinja version, and sometimes we make changes on these files.
If you would like to use your own templates, you can use the
/etc/bitninja/CaptchaHttp/www
directory. Any files in this directory
take precedence over the /opt/bitninja/modules/CaptchaHttp/lib/www
counterparts.
You can choose what to modify, or you can even overwrite all the templates
and place them here.
BitNinja will serve the file based on the accepted language header info sent by
the browser. You can have, for example, a Greek version for people using Greek
browsers and an English version for the rest of the world. In this case you
should copy the original browser_integrity_check.html file and create a
browser_integrity_check_gr.html
file for Greek users with the two-character
country code. (If you have made a translation and haven’t changed anything else,
like the personalized logo, feel free to send us your translation so we can include
it in the official BitNinja package ;-) )
Note
Please remember, the language displayed depends on the browser language settings. If there is a Greek user with an English browser, it will display the English BIC page.
For the changes to take place, you have to restart BitNinja as we cache every file to memory in order to speed up file serving and save system resources.
Note
Please don’t remove any html tags or javascript from the BIC page (except the loading icon), because it can cause errors. You can customize the css style inside the style tag.
Customizing the CAPTCHA page
You can customize the look and feel of the CAPTCHA page on your server. Currently it works on a per server basis and we support multilingual CAPTCHA pages.
The CAPTCHA page is translated to the following languages:
English
Hungarian
German
Spanish
French
Greek
Italian
Indonesian
Dutch
Polish
Portugese
Russian
Turkish
Vietnamese
Japanese
Danish
BitNinja has its default templates in the /opt/bitninja/modules/CaptchaHttp/lib/www
directory. There you can find the default English page, as well as the JavaScript
files that are responsible for changing the language based on the visitor’s
browser’s language setting.
The v2/js/translations.js
file contains the translated strings and the tags that are used in the template HTML file.
The v2/js/detect-browser-language.js
is responsible for getting the browser’s language and setting the template page accordingly.
If no translation is available in the v2/js/translations.js
in the selected language, the CAPTCHA page will be shown in English by default.
You shouldn’t change these files as with every new BitNinja version the package manager will overwrite these files, and sometimes we make changes on these files.
If you would like to use your own templates, you can use the
/etc/bitninja/CaptchaHttp/www directory.
Any files in this directory
take precedence over the /opt/bitninja/modules/CaptchaHttp/lib/www
counterparts.
You can choose what to modify, or you can even overwrite all the templates
and place them here.
BitNinja will serve the file based on the accepted language header info sent by the browser. You can have, for example, a Greek version for people using Greek browsers and an English version for the rest of the world. (If you have made a translation and haven’t changed anything else like the personalized logo, feel free to send us your translation so we can include it in the official BitNinja package ;-) )
Note
Please remember, the language displayed depends on the browser language settings. If there is a Greek user with an English browser, it will display the English CAPTCHA page.
For the changes to take place, you have to restart BitNinja as we cache every file to memory in order to speed up file serving and save system resources.
The CaptchaSmtp module is used to identify false positives on the BitNinja greylist and allow human users to remove themselves from the greylist easily.
In other words, if an IP is placed onto the BitNinja greylist and the IP tries to access your server through SMTP protocol, the request will be forwarded to an alternative fake mail server (created by BitNinja). We log the commands sent by the remote server and create and incident out of them. At the same time we reply to the server with a message
550 5.7.1 Your IP ({{client_ip}}) is on the BitNinja server security greylist If you are not a spammer bot, you can delist this IP on (http://{{server_ip}}/spam_delist) You can only delist this IP a few times! It is good idea to warn your service provider about this incident.
If it was a false positive, the users can delist their IPs (or really the server’s IP) but in case of automatic spam activities, we log the incidents.
The provided link will be valid for 10 days or until BitNinja restarts.
The CaptchaSmtp CAPTCHA page will create newMistake level incidents which allow the visitor 10 tries to solve the CAPTCHA. This threshold can be configured in the AntiFlood modules config file.
Relations to other modules
CaptchaSmtp sends events through the Shogun module in order to send it to BitNinja Central, and to the AntiFlood module to detect flood attempts and block them.
Customizing the CAPTCHA page
You can customize the look and feel of the CAPTCHA page on your server. Currently it works on a per-server basis and we support multilingual CAPTCHA pages.
Customizing port number of CaptchaSmtp
You can customize the default port number in the config file (/etc/bitninja/CaptchaSmtp/config.ini) by defining a new section called [userPorts] and underneath adding the line defaultPort. You can define additional ports too. Eg.:
[userPorts]
defaultPort = 25
additional[] = 26
Enable StartTLS
CaptchaSmtp now supports StartTLS with the help of Robert Campbell, one of our most helpful customers. StartTLS is not enabled by default, if you want to enable it you must add the following lines to the end of CaptchaSmtp config (/etc/bitninja/CaptchaSmtp/config.ini).
[ssl]
certPath="path to cert"
;should we use encryption by default?
defaultTls=false
Customizing the IP address for generating the delist link
The CaptchaSmtp module needs a public IP address for generating the delist link. You can customize this IP, or set a domain for this purpose by updating the config file at /etc/bitninja/CaptchaSmtp/config.ini:
;
; What address is used by CaptchaSmtp delist link.
; By default CaptchaSmtp will use the server\'s first public IP.
; You can add domain as well with out protocol. Like: example.com
;
[delisturl]
mail_captcha_ip=1.2.3.3
;mail_captcha_ip=example.com
The CaptchFtp module is an FTP server capable of both active and passive mode, integrated into the BitNinja agent. This module will check the IP addresses that try to connect to it, and detect if the IP is on the BitNinja greylist. If the IP is greylisted, it will simulate the FTP connection and won’t allow any real FTP operations for the malicious IP. Using the simulated FTP connection, the owner of the greylisted IP will find a file named: your_IP_is_greylisted_README.txt
This file contains instructions on how to delist the IP address. The instructions explain that the owner of the greylisted IP address should visit the IP of the server in a browser and resolve the CAPTCHA.
If the attacker with the greylisted IP address will try to upload a file using the simulated FTP connection, the file will be saved only to the BitNinja quarantine folder. This folder is located in /var/lib/bitninja/CaptchaFtp. For example if the attacker is trying to upload a file named backdoor.php on 2nd January 2019, the file will be saved to this location: /var/lib/bitninja/CaptchaFtp/2019/01/02/ and will be renamed by the following pattern: <hour_of_upload>_<minute_of_upload>_<unique_id>
Note
For even more information, you can read our blog article about the CaptchaFtp module on our website: https://bitninja.io/blog/2019/01/07/new-feature-available-ftp-captcha
The CaptchaFtp module is disabled by default. You can enable it using the following command:
bitninjacli --module=CaptchaFtp --enabled
Relations to other modules
CaptchaFtp sends events to the Shogun module in order to send it to BitNinja Central, as well as to the AntiFlood module to detect flood attempts and block them.
Customizing the port of CaptchaFtp
You can customize the default port of the CaptchaFtp module in the /etc/bitninja/CaptchaFtp/config.ini file by modifying the port numbers in the configuration.
The default configurations are the following:
[FtpServer]
listen_addr = "0.0.0.0"
listen_port = 60210
low_port = 60211
high_port = 60250
max_conn = 10
max_conn_per_ip = 3
server_name = "BitNinja FTP CAPTCHA server"
If you want to change the default port from 60210 to something else, you should modify the value of listen_port in the config.ini.
If you want to change the default range for FTP passive mode from 60211-60250, you should change the values of low_port and high_port in the config.ini.
After you’ve changed the values in the config file and saved it, you should reload the CaptchaFtp module with the following command:
bitninjacli --module=CaptchaFtp --reload
General modules
This section contains general purpose modules and modules written for providing base services for BitNinja.
CLI
This module is not an active module. It contains source code for bitninjacli. You can read more about it in the Command-Line Interface chapter.
DataProvider
This module is responsible for sending out regular updates about the server to the BitNinja analysis central. Data like system resource usage, system load, memory Usage, and BitNinja protection efficiency is sent for statistical purposes.
This module is also responsible for sending the server’s IPs to our cloud, where we automatically add them to your whitelist so all your BitNinja servers can communicate with each other without restrictions.
IpFilter
This module is responsible for the IP filtering functionality. BitNinja uses the ipset Linux kernel module to filter IP lists effectively, using minimal system resources at a stunning speed. You can find more details about this Linux module here: http://ipset.netfilter.org/
BitNinja manages many different ipsets for you automatically. Here are the different ipsets:
heimdall-greylist
heimdall-blacklist
heimdall-esentiallist
heimdall-user-blacklist
heimdall-user-whitelist
When the IpFilter module boots up on your server, it will try to detect the available ipset version. There are 3 options:
ipset 6.x
ipset 4.x
no ipset support, simulating the functionality with iptables
ipset 6.x
In recent Linux distributions, you can enjoy the efficiency and speed of ipset v 6.x. Ipset 6.x is a complete rewrite of the previous major version, 4.x. Ipset 6.x. is the best option you can have for BitNinja.
ipset 4.x
The older 4.x version has some limitations like having a maximum size of 65500 IP/ipsets, having a different set of cli options from the new version, and having a maximum of 255 ipsets. The BitNinja greylist has a size of millions of IPs. We have to split the greylist into smaller lists to use it with ipset 4.x. BitNinja splits the greylist into 100 smaller lists.
SimulatedIpset
If there is no ipset available, BitNinja will simulate this functionality with iptables rules. This is much less efficient than using ipset so you should avoid it whenever possible. In case of container based virtualization, when ipset is not available, we fall back to simulated ipset functionality. This is the case with Virtuozzo/OpenVZ based virtual servers and any case where ipset is not available.
In case of simulated ipsets, the efficiency is so poor that we cannot use the whole greylist with millions of IPs so your server will start with an empty greylist and only use IP information gathered since the last restart.
You should try to get your server to use ipset v 6.x to enjoy the best performance. You can find more info about how to install ipset in the Installation section.
Note
If you’d like to use the country blocking feature with simulated ipset, please be aware that this function will be limited. This is because of the limitations of simulated ipset. For example, if you want to add a country having lot of IP ranges to the blacklist or whitelist, it is possible that not all of the ranges will be put on the list. At the moment, we’re limiting the number of country IP ranges on user blacklist and user whitelist for simulated ipset to 8000.
Extending IpFilter
You can define custom scripts to run whenever BitNinja finishes initializing its sets and iptables rules. This makes it possible to add your own IPs and rules to BitNinja’s ruleset or run other programs when the IpFilter module finishes its startup process.
The following steps should be made in order to set up a post-up script
You should place a file into the /etc/bitninja/IpFilter/postup-scripts.d/ directory.
The name of the file should begin with “al_”. For example, the file /etc/bitninja/IpFilter/postup-scripts.d/al_myscript.sh is a valid path for a post-up script.
Make sure that the script is owned by the root user and is runnable by the owner.
The post-up script should have the same syntax as the scripts you normally invoke from the shell. For example, its first line should be #! <the path of the inerpreter>, as usual.
After you’ve created your script, you should restart BitNinja. You can check if there were any errors in the /var/log/bitninja/mod.ipfilter.log file.
Useful post-up extension scripts
BitNinja normally block both inbound and outbound connections from and towards greyilsted or blacklisted IPs. However, you may need to allow outbound traffic towards every IP, even the blocked ones. This script should help you.
#! /bin/bash
echo "Enabling outbound traffic..."
iptables -t nat -I HEIMDALL-GREY -m state --state ESTABLISHED,RELATED -j RETURN &&
iptables -t mangle -I HEIMDALL-IN -m state --state ESTABLISHED,RELATED -j RETURN &&
echo "Done."
Sometimes, you may need to exculde specific ports from being filtered. The following script can be used in such cases. You should replace the ports defined in the PORTS variable with the ports you need to enable.
#! /bin/bash
PORTS="25,110,143" # A list of TCP ports to be allowed, separated by comma
echo "Enabling all traffic on ports [$PORTS]..."
iptables -t mangle -I HEIMDALL-IN -p tcp -m multiport --dports "$PORTS" -j RETURN &&
iptables -t nat -I HEIMDALL-GREY -p tcp -m multiport --dports "$PORTS" -j RETURN &&
echo "Done."
You can enable the WAF module to work with private IP addresses too, using the following post-up script:
#!/bin/bash
echo "Enabling WAF manually..."
IPS="127.0.0.1 1.2.3.3 1.2.3.4" # A list of IP addresses to use with WAF, change it to your preference!
iptables -t nat -N BN_WAF_REDIR
iptables -t nat -A HEIMDALL-GREY -p tcp -m tcp -j BN_WAF_REDIR
for IP in $IPS; do
echo "Creating DNAT HTTP and HTTPS redirection for ip [$IP]..."
iptables -t nat -A BN_WAF_REDIR -d $IP/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination $IP:60300
iptables -t nat -A BN_WAF_REDIR -d $IP/32 -p tcp -m tcp --dport 443 -j DNAT --to-destination $IP:60414
done
This script won’t enable the WAF module - you should do that manually on the Dashboard or using bitninjacli -, but it will create the necessary redirections.
Shogun
The Shogun is the local governor of security on a BitNinja enabled system. The duty of this module is to accept and forward security incidents accordingly. For example, when there is an incident raised by the HttpCaptcha module, it will be sent directly to the Shogun module and it will route it (after proper filtering) to other modules like AntiFlood, IpFilter as well as BitNinja central.
Filters
- flood
if the IP is already blacklisted but for some reason (deleted iptables rules for example) incidents are still coming, this filter will protect against flooding central with bogus incidents
- private network
BitNinja does nothing with sources from private IP ranges.
- user whitelist
we do nothing with incidents generated by user whitelisted IPs
- whitelist
there are some built in whitelisted IPs and reverse DNS endings the client auto whitelist. (like Google bots, Cloudflare, w3c.org, Facebook bot, etc.)
System
The duty of this module is to manage licensing, restart and update the BitNinja client, and accept commands to enable or disable the WAF module. In the future this module will manage remote configuration as well.
SslTerminating
The duty of this module is to offload HTTPS connections, and help our CaptchaHttp and WAF module to work properly with HTTPS requests. If you enable SslTerminating it will try to download and install bitninja-ssl-termination package to /opt/bitninja-ssl-termination, which is basically a HAProxy 1.8.9. We decided to build and publish our self-built HAProxy, because it will not depend on official repositories, and it will not interfere with other HAProxy installations and configurations. After installation succeeds, it will try to find currently active virtual hosts using HTTPS and extract any information which can be used to generate a valid haproxy.cfg. The module is responsible for collecting and generating pem files every 5 minutes, which can be used by bitninja-ssl-termination to offload HTTPS requests.
The module works with CaptchaHttp and WAF 2.0 modules. When a greylisted IP tries to connect to your server on 443 it will be redirected to bitninja-ssl-termination’s frontend. The CAPTCHA will appear, and the innocent visitor can solve it.
Certificate miners
To gather certificate information, SSL Terminating module currently uses three certificate miner implementations. These miners are: ApacheCertMiner, NginxCertMiner, LiteSpeedCertMiner.
- A certificate miner implementation should consist of the following steps:
Collect the public key, the private key and the chain file for the SSL Certificate and put them in a pem file.
Put the newly created pem file location in /opt/bitninja-ssl-termination/etc/haproxy/cert-list-lst file, with every domain that belongs to this certificate, separated with spaces. Like this: /opt/bitninja-ssl-termintion/etc/haproxy/certs/example.com-ssl.pem example.com www.example.com *.example.com
An example bash implementation for DirectAdmin:
echo > /opt/bitninja-ssl-termination/etc/haproxy/cert-list.lst
for i in `ls /usr/local/directadmin/data/users/*/domains/* | grep ".key" | awk -F "/" {'print $9'} | sed 's/\.[^.]*$//'` ;
do
cat /usr/local/directadmin/data/users/*/domains/"$i".key /usr/local/directadmin/data/users/*/domains/"$i".cert > /opt/bitninja-ssl-termination/etc/haproxy/certs/"$i".pem
echo "/opt/bitninja-ssl-termination/etc/haproxy/certs/$i.pem $i www.$i *.$i" >> /opt/bitninja-ssl-termination/etc/haproxy/cert-list.lst
done
The Apache certificate miner tries to retrieve VirtualHost information using apache2ctl/apachectl/httpd -S (depending on the current OS). If this miner fails to find the apachectl binary, you can add it’s path to the config.ini webserver section with: apachectlBinaryLocation = ‘path/to/binary’
The Nginx certificate miner tries to retrive the main Nginx configuration file using nginx -V. It will follow includes and collect every virtual host using SSL certificates. If this miner fails to find your Nginx binary, you can add its path to the config.ini webserver section with: nginxBinaryLocation = ‘path/to/binary’ If you are using a different main configuration which was configured at Nginx build time, you can add your main configuration file to the config.ini webserver section with: nginxMainConfigFileLocation = ‘path/to/nginx.conf’
LiteSpeed certificate miner currently tries to parse the Apache main configuration file, follow includes and collect every virtual host using SSL certificates. If you are using a different main configuration file, you can add it to the config.ini webserver section with: apacheMainConfigFileLocation = ‘path/to/apache.conf’
Pattern certificate will be coming soon and it will be able to collect certificates with file patterns like: keyfiles= ‘path/to//domains/.key’ pemfiles= ‘path/to//domains/.pem’
CLI options
You can use CLI to enable and disable SslTerminating module with:
bitninjacli --module=SslTerminating --enabled
bitninjacli --module=SslTerminating --disabled
HAProxy configuration can be reloaded, if they were manually edited with:
bitninjacli --module=SslTerminating --reload
HAProxy configuration can be regenerated, if module configurations were edited:
bitninjacli --module=SslTerminating --regenerate
Configuration
The module can be configured in /etc/bitninja/SslTerminating/config.ini. This config file contains 9 config sections. The first two are for HAProxy and Apache binary settings. The other seven sections are for configuring different sections of the haproxy.cfg, if needed.
;
; SslTerminating module uses HAProxy 1.5.8. This section defines basic configurations.
;
[haproxy]
;; where is the haproxy binary
;binaryLocation='/opt/bitninja-ssl-termination/sbin/haproxy'
;; where should we look for haproxy config file
;configLocation='/opt/bitninja-ssl-termination/etc/haproxy/haproxy.cfg'
;; where could haproxy work
;workingDirectory='/var/lib/bitninja/SslTerminating'
;; where could SslTerminating put collected cert files
;certDir = '/opt/bitninja-ssl-termination/etc/haproxy/certs'
;; where is the cert list file
;certListFile = '/opt/bitninja-ssl-termination/etc/haproxy/cert-list.lst'
;; user for haproxy
;haproxyUser='bitninja-ssl-termination'
;; Https Captcha backend port
;haproxyPort = 60413
;; front end port
;httpsPort = 443
;; Captcha front and backend settings
;CaptchaFrontEndSettings[name]= 'Captcha-https'
;CaptchaFrontEndSettings[iface]= '*'
;CaptchaFrontEndSettings[port]= 60413
;CaptchaBackEndSettings[name]= 'Captcha-https-backend'
;; WAF front and backend setting
;; not used for now
;WafFrontEndSettings[name]= 'waf-https'
;WafFrontEndSettings[iface]= '*'
;WafFrontEndSettings[port]= '60414'
;WafBackEndSettings[name]= 'waf-https-backend'
;
; Web server settings
;
[webserver]
;; if binary location not set SslTerminating module tries to find where is apache.
;apachectlBinaryLocation = '/usr/sbin/apache2ctl'
;apachectlBinaryLocation = '/usr/sbin/httpd'
;apachectlBinaryLocation = '/opt/sp/apache/bin/apachectl'
;listVirtualHostParameter= '-S'
;nginxBinaryLocation = 'usr/local/nginx/sbin/nginx'
;nginxMainConfigFileLocation = '/etc/nginx/nginx.conf'
;apacheMainConfigFileLocation = '/etc/apache2/apache2'
;
; Haproxy global settings. Every ini key value will be converted to a haproxy configuration.
; E.g.: chroot = '/var/lib/bitninja/SslTerminating'
; Will be converted to:
; chroot = /var/lib/bitninja/SslTerminating
; If you want to add multiple values, use ini arrays.
; E.g.: log[] = '/dev/log local0'
; log[] = '/dev/log local1 notice'
; This will be converted to:
; log = /dev/log local0
; log = /dev/log local1 notice
; See more about haproxy configuration at: https://cbonte.github.io/haproxy-dconv/1.5/configuration.html
;
[haproxyGlobalSettings]
;log[] = '/dev/log local0'
;log[] = '/dev/log local1 notice'
;chroot = '/var/lib/bitninja/SslTerminating'
;stats[] = 'socket /var/lib/bitninja/SslTerminating/admin.sock mode 660 level admin'
;stats[] = 'timeout 30s'
;user = 'bitninja-ssl-termination'
;group = 'bitninja-ssl-termination'
;; Default SSL material locations
;ca-base = '/etc/ssl/certs'
;crt-base = '/etc/ssl/private'
;; Default ciphers to use on SSL-enabled listening sockets.
;; For more information, see ciphers(1SSL). This list is from:
;; https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
;; An alternative list with additional directives can be obtained from
;; https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
;ssl-default-bind-ciphers = 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'
;ssl-default-server-ciphers = 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'
;ssl-default-bind-options = 'no-sslv3 no-tlsv10'
;tune.ssl.default-dh-param = 2048
;
; Haproxy default Settings
;
[haproxyDefaultSettings]
;log = 'global'
;mode = 'http'
;option[] = 'httplog'
;option[] = 'dontlognull'
;option[] = 'forwardfor'
;timeout[] = 'connect 5000'
;timeout[] = 'client 50000'
;timeout[] = 'server 50000'
;errorfile[] = '400 /opt/bitninja-ssl-termination/etc/haproxy/errors/400.http'
;errorfile[] = '403 /opt/bitninja-ssl-termination/etc/haproxy/errors/403.http'
;errorfile[] = '408 /opt/bitninja-ssl-termination/etc/haproxy/errors/408.http'
;errorfile[] = '500 /opt/bitninja-ssl-termination/etc/haproxy/errors/500.http'
;errorfile[] = '502 /opt/bitninja-ssl-termination/etc/haproxy/errors/502.http'
;errorfile[] = '503 /opt/bitninja-ssl-termination/etc/haproxy/errors/503.http'
;errorfile[] = '504 /opt/bitninja-ssl-termination/etc/haproxy/errors/504.http'
;
; Stat page setting. For security reason please Change uri, auth
;
[statPageSettings]
; Haproxy bind setting. Change it before enabling stat page.
;bind = '*:1936'
;mode = 'http'
;log = 'global'
;maxconn = 10
;timeout queue = '100s'
;stats[] = 'enable'
;stats[] = 'hide-version'
;stats[] = 'refresh 30s'
;stats[] = 'show-node'
; Haproxy stat page auth setting. Change it before enabling stat page.
;stats[] = 'auth user:password'
; Haproxy stat page uri setting. Change it before enabling stat page.
;stats[] = 'uri /haproxy?stats'
;
; Haproxy Captcha frontend settings.
; Don't use Haproxy bind configuration in thes section.
;
[CaptchaFrontEndSettings]
;option[]= "httpclose"
;reqadd = 'X-Forwarded-Proto:\ https'
;
; Haproxy Captcha backend settings
;
[CaptchaBackEndSettings]
;redirect = 'scheme https if !{ ssl_fc }'
;server= 'captcha-1 *:60412 check'
;
; Haproxy WAF frontend settings.
; Not used yet.
;
[WafFrontEndSettings]
;reqadd = 'X-Forwarded-Proto:\ https'
;default_backend = 'waf-https-backend'
;
; Haproxy WAF backend settings.
; Not used yet.
;
[WafBackEndSettings]
;redirect = 'scheme https if !{ ssl_fc }'
;server 'waf-1 *:60045 check'
Regenerating certificate information
If the SslTerminating is using expired certificate information on your server for some reason, you can regenerate the certificate information by running the following commands:
chmod +x /opt/bitninja/modules/Cli/scripts/force_recollect_ssl_certs.sh
/opt/bitninja/modules/Cli/scripts/force_recollect_ssl_certs.sh
This script contains the following commands, which you can also run manually, one after the other, if needed:
rm -f /opt/bitninja-ssl-termination/etc/haproxy/certs/*
rm -f /opt/bitninja-ssl-termination/etc/haproxy/cert-list.lst
rm -f /opt/bitninja-ssl-termination/etc/haproxy/haproxy.cfg
bitninjacli --module=SslTerminating --regenerate
bitninjacli --module=SslTerminating --reload
Disabling modules
You can disable any BitNinja module by editing the config file /etc/bitninja/config.php Here you can search for the section disabledModules and add the module you want to disable to the list.
'disabledModules'=>array(
// 'System',
// 'DataProvider',
// 'IpFilter',
// 'CaptchaHttp',
// 'CaptchaSmtp',
// 'PortHoneypot',
// 'AntiFlood',
// 'Shogun',
// 'DosDetection',
// 'SenseLog',
// 'SenseWebHoneypot',
// 'WAF'
You must restart BitNinja in order for the changes to take effect!
IP reputation
BitNinja is a disruptive technology so there are some concepts that are important to understand in order to comprehend the way BitNinja works.
IP reputation is a very effective way of securing your server. It’s a database with information about various IPs in the world. BitNinja clients use IP reputation information automatically on their servers to make security decisions and to find out more about an IP address.
Every server with BitNinja can detect and defend a wide range of attacks. The server can send gathered incident information to our central database. Based on the type, timing, and amount of incidents an IP has in the database, it is categorized into one of the following lists:
Not listed
If there is no information about an IP address, or based on the latest behavior the IP is not listed.
User Greylist
In traditional IP reputation terminology, we differentiate black and white lists. An IP can be trusted (whitelisted) or absolutely denied (blacklisted). This concept is very inflexible and this is the cause of the bad reputation that IP reputation lists have. If an IP is false-positively blacklisted, users will get angry at you because they can’t access the system they want to and they will be frustrated as they are unable to do anything about it.
That’s how the concept of greylisting was born. We wanted to represent a list of IPs we think may be malicious but we are not completely sure of it yet.
The greylist contains suspicious IPs that the BitNinja client handles with special care. BitNinja has different CAPTCHA modules for different protocols. The duty of a CAPTCHA module is as follows:
Decide if the user is human or not
Inform the user about the fact that his/her IP has been greylisted
Provide a safe way for the user to delist his/her IP
Save any requests made by non-human parties, growing the knowledgebase about the IP and the sin list.
Honeypotting by pretending to be a vulnerable system so bots will try to connect
Note
Traffic from an IP can be malicious and innocent at the same time. Think of an infected computer. The requests the virus will make are malicious but the requests the user makes by using his/her browser are innocent. If we completely block the traffic, the user will not know about the infection. Greylisting solves this by informing the user of the greylisting and helps the innocent users resolve their IPs.
If there are suspicious incidents about an IP address, the IP can be greylisted by some users. If an IP is user-greylisted, it means it is only greylisted by some users, not all BitNinja users. When we have enough information about an IP that is sending malicious requests, we move it to the global greylist. If an IP is globally greylisted, it is greylisted by all BitNinja servers.
You can check if an IP is listed by BitNinja using the web front end https://admin.bitninja.io or using our CLI front end that is called bitninjacli. You can check, add, and delete IPs from the greylist using both tools.
Note
If one of your servers catches an attack and the IP is not yet greylisted, the IP will be user greylisted and this information will be distributed across all of your servers instantly.
Global greylist
If there is enough evidence that an IP is suspicious, we move it to the global greylist and distribute this information to every BitNinja protected server.
Global blacklists
When an IP is globally greylisted and is still sending malicious requests, we identify it as dangerous. Such IPs are moved to the global blacklist we maintain. BitNinja clients will drop packets for IPs on the global blacklist. The false-positive rate of the global blacklist is very low, as there are many steps before we decide to blacklist an IP. Blacklisted IPs are moved back to the greylist from time-to-time to check if the traffic is still malicious or the system has been disinfected.
User level black/white lists
You can use BitNinja to maintain blacklists and whitelists across your servers. The BitNinja Dashboard or our command-line interface, BitNinja CLI, can be used to add/remove/check IPs against a user’s blacklist and whitelist. If you blacklist an IP, all of your servers will DROP any packets from that IP. You can add CIDRs to your blacklist or your whitelist via BitNinja Dashboard. When you whitelist an IP it guarantees BitNinja won’t interfere with that IP.
Note
Whitelisting an IP will only bypass BitNinja iptables chains. There can be other iptables rules blocking the traffic. Whitelisting is not equal to universal ACCEPT, only to bypass BitNinja.
Note
Whitelisted addresses are processed prior to blacklisted ones starting from BitNinja version 1.16.16.
Essential list
Essential list provides protection against the most dangerous IPs. These IPs are often used by the most agressive hackers all around the world. When an IP generates more than 5000 malicious requests, BitNinja places it on this list. The essential list is part of our basic IP reputation package, so it’s available for every BitNinja user.
Country blocking
You can add whole countries to your blacklist or whitelist. The country blocking (or whitelisting) feature of BitNinja uses publicly available zone files to add the selected countries’ IP ranges to the user blacklist or user whitelist. The zone files can be found at http://www.ipdeny.com/ipblocks.
You can add countries on the Dashboard in the IP manager menu.
If you have ipsetv6 or ipsetv4 and add a country or remove it to the blacklist or whitelist, the ipsets will be reloaded on your server(s).
If you don’t have an ipset, BitNinja uses iptables rules and a simulated ipset. When you add or remove a country, the iptables rules will be reloaded.
For now, if you have more than one server and add a country to your blacklist (or whitelist), the selected IP ranges will be blacklisted (or whitelisted) on all your servers.
How is it different from other security solutions?
BitNinja is a hybrid of an on-premise and cloud-based solution. You can install it on your server and it gives the protection of a defense network using different modules to the different aspects of server security, which grant the 360 ° defense solution by communicating with the cloud. BitNinja provides all modules that can intercommunicate with each other to prevent, detect malicious attacks instead of dealing with security threats separately. Moreover, while cloud-based security solutions require your traffic to be redirected through their servers, with BitNinja, you don’t have to do any configuration. Also, any failures of the cloud-based party can make your server unavailable, which is a huge risk to take. BitNinja security service runs on your own existing cloud server.
Enable Bitninja For Your Nodes
Activate BitNinja During Node Creation
Please go to MyAccount portal and log in using your credentials set up at the time of creating and activating the E2E Networks My Account.
After you log in to the E2E Networks My Account, On the left-hand side of the MyAccount dashboard, click on the ‘Nodes’ sub-menu available under the ‘Products’ section.
You will be routed to the ‘Manage Nodes’ page. Now, you have to Click on the ‘Add New Node’ button to create a node that takes you to the ‘Create Compute Node’page.
Please select the Node image.
After selecting the node image and plan, you need to click on the create button, It will take you to the final stage of the ‘Create Compute Node’ page.
Here, you can specify the node name and additional options for the node you’re creating.
In the Node security section, you can enable BitNinja security tool which is used to protect your node against a wide range of cyber-attacks. You will be charged ₹ 760.00 per calendar month for the BitNinja license.
Click on the ‘Create’ button to create a node.
It will take a few minutes to set up the node and you will take to the manage nodes page. Bitninja License activation respective information will be updated in the Node detail tab.
Also, you will receive a notification email along with bitNinja license activation key and details from BitNinja.
Activate BitNinja For Your Running Nodes
After the launch of an E2E node without enabling the bitNinja or for existing running E2E nodes, you still have the option to activate bitNinja for your node. For this, you need to follow the below steps.
Please go to the Manage Nodes page.
Now, select the node for which you wish to enable BitNinja.
Click on the BitNinja License is not activated. Click here to buy available in the node details tab.
The below confirmation window will open. click-on Buy button after reading the information on the pop-up box.
The below window will appear after BitNinja license activation is successful. You need to manually install the BitNinja agent in the respective node by executing the command shown in the message. The BitNinja license activation key and details will be shared via mail from BitNinja.
In the node details tab, bitNinja license information is updated.
Enable Bitninja For Your Load Balancer
Activate BitNinja During Load Balancer Creation
Please go to MyAccount portal and log in using your credentials set up at the time of creating and activating the E2E Networks My Account.
After you log in to the E2E Networks My Account, On the left-hand side of the MyAccount dashboard, click on the ‘Load Balancers’ sub-menu available under the ‘Products’ section.
You will be routed to the ‘Manage Load Balancers’ page. Now, you have to Click on the ‘Add New LB’ button to create a load balancer that takes you to the ‘Create Load Balancer’ page.
Please select the load balancer image and plan.
After selecting the load balancer image and plan, you need to click on the create button, It will take you to the final stage of the ‘Create load balancer’ page.
Here, you can specify the load balancer name and additional options for the load balancer you’re creating.
In the security section, you can enable bitNinja security tool which is used to protect your load balancer against a wide range of cyber-attacks. You will be charged ₹ 760.00 per calendar month for the BitNinja license.
Click on deploy button to create a load balancer.
It will take a few minutes to set up the load balancer and you will take to the manage load balancers page. Bitninja License activation respective information will be updated in the load balancer detail tab.
Also, you will receive a notification email along with BitNinja license activation key and details from BitNinja.
Activate BitNinja For Your Running Load Balancer
After the launch of an E2E load balancer without enabling the BitNinja or for existing running E2E load balancer, you still have the option to activate bitNinja for your load balancer. For this, you need to follow the below steps.
Please go to the Manage Load Balancers page.
Now, select the load balancer for which you wish to enable BitNinja.
Click on the BitNinja License is not activated. Click here to buy available in the load balancer details tab.
The below confirmation window will open. click-on Buy button after reading the information on the pop-up box.
The below window will appear after BitNinja license activation is successful. You need to manually install the BitNinja agent in the respective load balancer by executing the command shown in the message. The BitNinja license activation key and details will also be shared via mail from BitNinja.
In the load balancer details tab, BitNinja information is updated.
Terminate BitNinja License For Node
Please go to the Manage Nodes page.
Now, select the node for which you wish to disable BitNinja.
Click on the BitNinja License is activated. Click here to terminate available in the node details tab.
A confirmation window will open. click-on ‘Terminate’ button after reading the information on the pop-up box.
Terminate BitNinja License For Load Balancer
Please go to the Manage Load Balancers page.
Now, select the load balancer for which you wish to disable BitNinja.
Click on the BitNinja License is activated. Click here to terminate available in the load balancer details tab.
A confirmation window will open. click-on ‘Terminate’ button after reading the information on the pop-up box.
In the load balancer details tab, BitNinja information is updated after successful termination of BitNinja license.
Stopping and Uninstalling
If you have to stop BitNinja you can use the following command: service bitninja stop
To uninstall BitNinja packages from your node you need to run commands mentioned in the below
Debian based distribution:
apt-get purge 'bitninja*'
Rpm based distribution:
yum remove 'bitninja*'
Removing kmod-ipset on Centos5
yum remove kmod-ipset
Removing ipset on Centos5
yum remove ipset
Debian based distribution:
apt-get purge 'bitninja*'
Removing BitNinja from WHM
wget -qO-
https://get.bitninja.io/bitninja-whm.tar.gz | tar -zx && ./bitninja-whm/uninstall
Removing BitNinja from ISPmanager
wget -qO- https://get.bitninja.io/ispmgr-plugin.tar.gz | tar -zx && ./ispmgr-plugin/uninstall
Use the following command to remove SSL certificates:
rm -R /opt/bitninja-ssl-termination
Use the following command to remove the BitNinja logs folder:
rm -R /var/log/bitninja