Month: March 2019

Fail2Ban Custom filters and Testing Regex’s against existing Logs

Fail2ban is a tool that can automatically ban malicious bots trying to get into your server. Provided you set up filters and the ip address (or fqhn) is logged you can use fail2ban with any application.

Create a filter, using a regular expression (fail2ban is built with python):

In /etc/fail2ban/filter.d/my-custom-filter.conf:


[Definition]

failregex = ^www.example.com  -.* "POST \/user\/register HTTP\/1.0" 200

ignoreregex =

Now you want to test this for matches against a log file. Ensure that the log file has existing matches.

Make use of the command line tool fail2ban-regex:

fail2ban-regex /var/log/apache2/example-access.log /etc/fail2ban/filter.d/my-custom-filter.conf

You will get summary data like this:


Running tests
=============

Use   failregex file : /etc/fail2ban/filter.d/my-custom-filter.conf
Use         log file : /var/log/apache2/example-access.log


Results
=======

Failregex: 1 total
|-  #) [# of hits] regular expression
|   1) [1] ^www.example.com  -.* "POST \/user\/register HTTP\/1.0" 200
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [23636] Day/MONTH/Year:Hour:Minute:Second
`-

Lines: 23636 lines, 0 ignored, 1 matched, 23635 missed
Missed line(s): too many to print.  Use --print-all-missed to print all 23635 lines

Which lets you debug your filter regular expression to ensure it is matching the malicious log entries.

 

Finally, add the new filter to your jail.local by appending the following:


[my-custom-filter]
enabled  = true
filter   = my-custom-filter
action   = iptables-multiport[name=NoAuthFailures, port="http,https"]
logpath  = /var/log/apache2/example-access.log
banTime  = 864000
findtime = 1800
maxRetry = 3

You can read more about the configuration of jails in the manual

Getting Jenkins to deploy with ansible using SSH Agent Forwarding

Your CI/CD tool needs access to code and server, for linting, testing and deploying.
Setup up access on the various devices in a secure manner can be very time consuming. It is important to make use of available technology to make our lives easier.

Jenkins needs access

You will have created credentials for Jenkins - by creating a SSH key pair for your jenkins user. Ensure that that public key have access to the code on your version control platform (as a deploy key).

Now jenkins will be able to get your code and run various tests on it. The problem now is deployment.

jenkins-credentials

Use Jenkins's SSH credentials to Deploy

We are using ansible to deploy (from the jenkins box). So now jenkins needs access to wherever you are deploying the code to. You would do an ssh-copy-id to get it there.

But there is another problem, when ansible runs the git module to clone the repo you will get an error that the user does not have access.

Correct, the user on the box you are deploying to does not have access to the code. Now you could add that box as another deploy key but now when scaling out to many boxes you will have a hell of alot of ssh credentials to manage.

The best thing to do is use the jenkins user's cerentials that log into your target box to get the code. This is done with SSH Agent forwarding.

The first thing we will need is theĀ  jenkins SSH agent plugin.

Then enable the SSH agent for your job:

enable-jenkins-ssh-agent

 

Then install the Jenkins ansible plugin and configure it.

Finally you need to tell ansible to use SSH Agent forwarding, otherwise it just won't do it:


mkdir /etc/ansible
vi /etc/ansiible/ansible.cfg

Add the following config there:


defaults]
host_key_checking = False

[ssh_connection]
ssh_args = -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s

Of course, it is better to ensure host_key_checking is done.

Now everything should work.

Source: SSH Agent Forwarding with Ansible