block ssh brute force

Block SSH Brute Force Attacks with IPTables

Detecting a SSH Brute Force Attack

If you are under a SSH brute force attack, you will likely see something like this in your logs.

 Jan 26 03:46:02 host sshd[22731]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=125.39.22.154
 Jan 26 03:46:02 host sshd[22731]: pam_succeed_if(sshd:auth): error retrieving information about user seymour
 Jan 26 03:46:02 host sshd[22734]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.147.103.185 user=root
 Jan 26 03:46:02 host sshd[22722]: Failed password for root from 61.147.103.185 port 16563 ssh2
 Jan 26 03:46:02 host sshd[22723]: Received disconnect from 61.147.103.185: 11: Normal Shutdown, Thank you for playing
 Jan 26 03:46:03 host sshd[22705]: Received disconnect from 61.147.103.185: 11: Normal Shutdown, Thank you for playing
 Jan 26 03:46:03 host sshd[22726]: Failed password for invalid user madonna from 125.39.22.154 port 51706 ssh2
 Jan 26 03:46:03 host sshd[22917]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.147.103.185 user=root
 Jan 26 03:46:03 host sshd[22727]: Failed password for root from 61.147.103.185 port 16723 ssh2
 Jan 26 03:46:03 host sshd[22729]: Failed password for invalid user florin from 125.39.22.154 port 54958 ssh2

 

This is a bot scanning your server trying to guess passwords.

 Methods to Stop SSH Brute Force Attacks

There are basically four approaches to dealing with SSH brute force attacks:

  • Restrict SSH access by IP address
  • Change SSH to another Port
  • Use intrusion prevention tools to dynamically block access
  • Rates limit SSH sessions using IPTables

All of these approaches have theirs benefits and drawbacks.

While restricting SSH access by IP address is the most secure method, such restrictions are often not possible when dealing with web hosting services as you have multiple users with constantly changing IP addresses.

Changing the SSH port may defeat bot scans but does little against targeted attacks.  Also, this usually just frustrates your users.

Intrusion prevention tools like fail2ban and denyhosts have their place but they are subject to log based attacks.  These tools essential analyze logs using regular expressions.  Hackers have found ways around both of these tools in the past.

Lastly, you have a great tool to block ssh brute force attacks right on your server: IPtables.

Using IPtables to Stop SSH Brute Force Attacks

I like to think of this approach similar to flow rates with pipes.  Bigger pipes allow more water to flow.  Smaller pipes can handle less water.

control ssh access with iptables

To block a SSH brute force attack, we just need to slow down the flow of requests. We can do this by rate-limiting requests to SSH with iptables.

Essentially, we create a smaller pipe for new SSH sessions.  This slows brute force attacks to a point where they become ineffective.

The iptables rules are relatively simple.

/usr/sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set
/usr/sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent  --update --seconds 60 --hitcount 4 -j DROP

This rule will block an IP if it attempts more than 3 connections per minute to SSH. Notice that the state is set to NEW. This means only new connections not established ones are impacted. Established connections are the result of a successful SSH authentication, so users who authenticate properly will not be blocked.

If you need to see what’s being done, you may want to log these drops. You can do so by setting up a log rule and then using these rules instead.

/sbin/iptables -N LOGDROP
/sbin/iptables -A LOGDROP -j LOG
/sbin/iptables -A LOGDROP -j DROP
iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set
iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent  --update --seconds 60 --hitcount 4 -j LOGDROP

Notice that I’ve changed the rule from DROP to LOGDROP. This way your drops will get logged and you can see the results in your logs:

Jan 27 08:22:29 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=208.43.148.64 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=16406 DF PROTO=TCP SPT=31003 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0
Jan 27 08:22:29 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=192.168.1.1 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=5076 DF PROTO=TCP SPT=45354 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0
Jan 27 08:22:35 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=208.43.148.66 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=37295 DF PROTO=TCP SPT=21077 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0
Jan 27 08:22:35 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=208.43.148.64 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=16408 DF PROTO=TCP SPT=31003 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0

Effectively Stopping SSH Brute Force Attacks

I always try to get a sense of effectiveness of any tool or configuration we deploy. I find many “security” tools that are popular among the web hosting crowd provide little to no value. In many cases, appropriate configuration of your server or web application could achieve similar results without the hassle of maintaining a third party product.

Are the IPTables rules effective? In short yes.

During a recent attack on a server, the SSH service remained fully accessible with no service interruption.

Previously such aggressive attack would have caused service interruptions. So on the service side, this approach works. When I dug into the logs, I found three failed user attempts against SSH prior to the rate-limiting kicked in. The attack then sent 67 more attempts before it gave up.

Benefits of Using IPtables to Block SSH Attacks

The benefit of this approach is you don’t need any added software. IPtables is likely sitting on your server already, so you can easily and quickly deploy this solution.

Also, there are no “ban lists” to maintain.  People forget passwords or incorrectly setup their SSH/SFTP programs.   As a result, they trigger a block and get locked out.  You then have to manually edit some ban list to remove them or whitelist IPs.   Over time or with multiple servers, this is a time-consuming server management tasks.  By using iptables, there’s no list to maintain — leaving you time to work on more important things.

One of the drawbacks is that this approach does not lock accounts. A slow, distributed attack could fall under the radar. If it was a directed attack against a specific user account, the attacker could churn away for days or weeks without detection. For that, you would need something that can lock user accounts after failures. PAM includes a module called pam_tally that does just this. If you fail too many times, an account is locked.

Hardening SSH

In addition to these IPTables settings, there are some things you can do within the SSH configuration to harden SSH from attacks. We commonly apply these settings as part of our server support services.

The following two tabs change content below.
Jeff is the CEO and founder of rackAID. He has been working in hosting industry since the days of Rackshack's famed customer appreciation parties back in the early 2000's. Jeff is an avid cyclist, wine enthusiast, and admitted Kraftwerk listener (please don't hold that against him). You may spot him at any number of hosting, startup and small business conferences

23 Comments

  • Jason March 7, 2010 at 1:49 am

    Great article. Thank you for posting this. The other methods I had seen for blocking ssh attacks were all hackish bash or perl scripts. This is much more elegant.

  • Juli Zurovec March 15, 2010 at 10:20 am

    I find this method works very well.  We also use it on cPanel and Plesk to protect their admin areas. Just change port 22 to the port you need to block.

  • Jeffrey Huckaby April 26, 2010 at 8:34 am

    There was a typo in the first rule.  “-set” should have been written “—set”.

    /usr/sbin/iptables -I INPUT -p tcp —dport 22 -i eth0 -m state —state NEW -m recent —set

  • Chip April 26, 2010 at 4:16 am

    Just tried this guide out but can’t unfortunaley get this to work properly. I’m trying to use the one where you collect the logs but at the last command:
    iptables -I INPUT -p tcp—dport 22 -i eth0 -m state—state NEW -m recent —update—seconds 60—hitcount 4 -j LOGDROP
    I’ll get this error:
    iptables: Too many levels of symbolic links.
    Any thoughts on how too solve this? Haven’t done the first where you just drop the logs since it’s important that I have the logs

  • Alex Wu May 9, 2010 at 5:53 am

    Hi jeff, i’m new to ssh and I have the same problem as chip.
    The setup without logs works fine but the setup with the logs fails at the last step:
    iptables -I INPUT -p tcp—dport 22 -i eth0 -m state—state NEW -m recent —update—seconds 60—hitcount 4 -j LOGDROP
    Too many levels of symbolic links.

    You were quick to point out that there was a typo, but I don’t think so as I’ve tried with the new command and it doesn’t recognize it.
    I’ve noticed that the only difference from the last command and the one without the logs is the change from DROP to LOGDROP. Otherwise it works. Could it be a problem in the way “LOGDROP” is setup?
    Thank you very much for the article.

  • chrissy June 5, 2010 at 5:02 am

    Chip and Alex, I just did a quick search in the manpage for iptables, and it recommends using the rules as described in the article—except “LOGDROP” should simply be “DROP”.  The exact quote in the manpage is “So if you want to LOG the packets you refuse, use two separate rules with the same matching criteria, first using target LOG then DROP (or REJECT).”  So the first rule should jump to the “LOG” chain, and then the second rule should jump to the “DROP” chain.  As both rules check for packets matching the same criteria, those matching packets would effectively be logged and then dropped.  I may be wrong or this may be entirely different from the error you’ve been experiencing, but this is my understanding of the method’s correct usage.

  • Jeff H. June 7, 2010 at 2:17 am

    The LOGDROP is just a custom rule. It is not anything special.  In iptables you can create custom rules.  I just called it LOGDROP so I know it will log the request and then drop it.  I will double check it when I get a chance to be sure I did not make an error.

    I like using custom rule names so I can then easily drop packets into them from the main rule change.

    This approach makes iptables more modular.

  • Alex Wu June 7, 2010 at 5:07 am

    Thanks chrissy and jeff. I actually found a similar solution thanks to ufw (uncomplicated firewall), a cli frontend for iptables easy configuration. It has far less flexibility I guess but it’s perfect for my needs.
    It was developed by Canonical (Ubuntu), but being opensource I guess should be available in other distributions as well.
    Well if you end up installing it the only commands you need to use are these:
    ufw default deny incoming
    ufw limit in log 22/tcp
    ufw enable
    1st command this will set deny incoming connections by default.
    2nd command will allow traffic through the 22/tcp port and keep a log of it. It will also block any ip address that has attempted to initiate 6 or more connections in the last 30seconds.
    The last command enables the firewall.

    A thing to note is that ufw will write back the configuration to iptables everytime the system is booted, so there’s no need to fiddling around with startup scripts to get the firewall rules automatically set again when rebooted.
    And that’s all, my needs for a firewall are really basic and this little thing does what I need.

    I hope it was helpful, thanks.

  • African Movies May 20, 2011 at 10:17 am

    I’ve been looking for a way to do this using IPtables for a long time. Thanks for the tutorial

  • Rick June 8, 2011 at 8:13 am

    I had the same problem Chip and Alex reported, and I’m fairly sure the problem is in the line:
    /sbin/iptables -A LOGDROP -j LOGDROP

    which probably should read:
    /sbin/iptables -A LOGDROP -j DROP

  • Jeffrey Huckaby June 8, 2011 at 8:51 am

    @rick
    Thanks for the correction. I typically use a LOGDROP custom chain to manage logging.  If I want to log drops, I can pass it to this chain instead of the drop rule.  Will update the post.

  • Robby April 5, 2012 at 1:27 am

    For anyone looking for a good basic firewalling script, check out https://gist.github.com/2313051

    I use this script on all of my linode servers and it works great!

    I had to change the order of the commands a bit to work with how the rules were being set up in the script.

  • Niraj April 22, 2014 at 1:00 am

    Thanks for valuable inputs, Jeff.

    iptables -I INPUT -p tcp –dport 22 -i eth0 -m state –state NEW -m recent –update –seconds 60 –hitcount 4 -j DROP –> I guess you meants LOGDROP (see your following comments)

    Notice that I’ve changed the rule from DROP to LOGDROP. This way your drops

    • Jeff Huckaby April 22, 2014 at 9:41 am

      Thanks. I’ve updated the post (and fixed the formatting for the code snippets).

  • KT May 29, 2014 at 5:01 am

    I don’t know if this is a valid question: I want a good user to be able to make 3 *successful* connections on and off within a minute. The problem is that the ‘–state NEW’ in the last rule is agnostic to good/bad users: anyone attempting 3 new connections within a minute will be blocked, including my good user. Is there a way to ‘reduce’ –hitcount by 1 whenever a good user succeeds in arriving at an ESTABLISHED state (effectively –hitcount is thereby sharpened to mean ‘failure hitcount’) ? I tried to play with an OUTPUT rule to no avail. Thanks.

    • Jeff Huckaby May 29, 2014 at 9:36 am

      I don’t think IPtables can handle what you are doing. The system would have to somehow scan or record the presence of the established state and then use this information to dynamically adjust the rules.

      I am not sure why someone is connecting so frequently from random IPs? If they have a known IP, then place an allow rule before the rate limiting rules. Alternatively, you could simply raise the limit such that it will still be effective but allow your desired usage. If too many attacks are still getting through, consider tools like pam_tally or fail2ban which would work on the authentication level and can drop deny rules back into iptables.

  • KT May 29, 2014 at 9:56 pm

    Thanks Jeff. Useful advice!

  • Beau July 10, 2014 at 9:56 pm

    Curious on your suggestion for limiting SSH down to prevent goons and using cheapo SSH tools (like FileZilla) that create new SSH connections for each transfer (if a user copy/pastes the contents of a folder versus the folder by itself).

    Thank you in advance friend!

    • Jeff Huckaby July 14, 2014 at 10:37 am

      I doubt it would impact many SFTP transfer tools unless the limits are low or the user is transferring a lot of files. Interesting use case though …

  • James November 4, 2014 at 2:14 am

    Good post… Could you tell me what’s the name of log files?
    Thank Jeff!

    • Jeff Huckaby November 4, 2014 at 3:20 pm

      Logging is usually to /var/log/messages but may vary depending on your OS.

  • John November 7, 2014 at 7:57 am

    Established state in iptables does not mean SSH auth has succeeded. The state tracking extensions of iptables are related to the flow of connections. Established means the firewall has seen packets flow in both directions for that connection.

    This suggested method is not an effective means of mitigating an attack. An attacker spoofing his IP address would be able to cause a Denial of Service preventing legitimate users from accessing your server on any port. If you are going to do something like this, you should add a rule prior to this to whitelist which IP addresses are allowed to use SSH. But really, you’re much better off limiting auth attempts, which users can logon via SSH, disabling password authentication and using public keys, and taking advantage of fail2ban (or similar software).

    • Jeff Huckaby November 7, 2014 at 9:20 am

      John
      The method works very well for the issue described. In fact, we block millions of attempts everyday using this method.

      You mention that I would be “better off” limiting authentication attempts. I think you will find these rules do precisely that.

      By controlling the flow of SSH packets, you limit authentication attempts.