Protect Linux Server From Hackers

TL;DR

There are many "best practices" out on the web when it comes to protecting your Linux server from getting hacked. These include disabling SSH password logins, remove root login, changing ports, disabling IPv6, configuring firewalls, and auto-updating. Let's discuss their merits and shortcomings and whether you should use them in this article.

The Video

Introduction

So you want to make your own server running Linux to host your website? That's great! You might want to think about keeping it protected from the nefarious activities of hackers that are out there! There are steps that you can take to mitigate the risk of getting your server compromised. Many of these are listed on the internet, in guides, forums, or discussed in chat rooms. Cool, just implement them and you're golden, right?

Not quite. The main issue with these "best practices" is that they tend to be unquestioned and in fact blindly followed. That's a big red flag. We cannot stress enough the importance of knowing why security works the way it does. Understanding the steps that you are taking to make your Linux server safe from hacking is essential in avoiding a false sense of security which could leave you open to attack. We want you to be informed about these things so you can host your own Linux server with good security principles in mind and properly applied to your setup.

We looked into some of the common tips and solutions promoted by people on the web just by searching for tips to secure a linux server. After comparing the tips offered left and right, we compiled a large spreadsheet with our results, which you can view here. We've also linked the sources for the tips in the spreadsheet, if you want to see where we got the information from.

Let's talk about...

... the myths and truths of the "best practices". A lot of the tips are centered around SSH - the Secure Shell - which is a way of accessing your server from another machine on the network. We also discuss firewalls and upgrading packages on Linux and their impact on server security.

SSH is the de facto standard to remotely access another machine. To use it, you'll typically type in something like

ssh root@139.177.180.76

where root is the username and 139.177.180.76 is the IP of the machine you are accessing. You then punch in your password, and you're in, free to carry out whatever task you need on the remote machine.

Disabling SSH Password Login

The first tip that's given out is to disable the SSH password login, which is the method we just described, and instead use SSH keys. To do so, you have to update the sshd_config text file with your text editor of choice (vim, nano, ...):

vim /etc/ssh/sshd_config

where you can then disable password authentication:

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no

The assumption here is that passwords are insecure. The comment above the PasswordAuthentication parameter even says, verbatim, clear text passwords! Is it really true that passwords are insecure? No better place to find the answer than the actual Secure Shell (SSH) Protocol Architecture standard.

If you scroll all the way down to section 9.4.5. Password Authentication, there is a snippet that clearly states the weakness of passwords:

If the server has been compromised, [...]

And that's all we needed. At this point, the server is already hacked.

If the server has been compromised, using password authentication will reveal a valid username/password combination to the attacker, which may lead to further compromises.

Yes, there are some caveats. So what about SSH keys? Maybe they are more secure? If you scroll up to the section right above, 9.4.4. Public Key Authentication, you can read all about it. As you might imagine, they come with their own issues and limitations.

The use of public key authentication assumes that the client host has not been compromised. It also assumes that the private key of the server host has not been compromised.

So you see both of these are not perfect. But we also glossed over a word in the comment above about disabling of the PasswordAuthentication in the modification of the /etc/ssh/sshd_config file. That word is "tunnel". That's right, the SSH protocol establishes an encrypted tunnel between the client and server, and the cleartext password is sent there. So, the password is cleartext within the tunnel, but you can't see that because you are outside of the tunnel. So, in the end, it's not really cleartext.

This is why the ssh server has its own private key that you should absolutely verify before connecting to the remote server. It's also important to note that your machine remembers the public key of the servers you've connected to before, and it checks it every time that you connect.

So, if you mistype your hostname or IP address, or someone tries to attack you by using a man-in-the-middle strategy, your ssh client will notify you of the mismatch between the key you have and the key the real server should have.

If you're still confused, let's talk about https for a second (so TLS or SSL), as it's a great analogy. When you log into Twitter, you're also technically sending a cleartext password in the http request, but it's inside of the encrypted TLS tunnel, thanks to https. That way, if someone tries to man-in-the-middle you, like they did above with ssh, your browser will warn you and refuse to send your password... just like your ssh client would.

Right, so using passwords isn't that bad... but standard password recommendations still apply! You already know them, and we hope that you've implemented these password policies.  In case if you haven't or if you need a refresher, here they are:

• don't reuse passwords,
• avoid bruteforceable passwords, (symbols, letters, numbers, those are all good).

In fact, we recommend using password managers. There are many solutions out there that do a terrific job of implementing these recommendations. A good password manager should generate unique, random, and long passwords.

All of this isn't to say "don't use SSH keys". They have their value, as they are really convenient. If - like us - you're lazy too, you can use them so that you don't have to copy a password from your password manager, or worse, type it in manually.

The bottom line here is that we recommend you use SSH keys, but not because of security, but rather for convenience. Disabling password login doesn't magically make your server more secure.

Disabling Direct Root SSH Login

Another tip we saw come up a lot in our research suggested to disable the direct root login via the ssh protocol. Instead, we should create an unprivileged user without root permissions. Generally-speaking, it's always a good method to use the least amount of privileges required to accomplish a task.

For instance, the nginx webserver does not run as root, but instead it runs as an unprivileged user called www-data. In fact, even when it is started as root, it drops its own privileges to www-data. This means that if the webserver is successfully hacked, the hacker will not have root privileges, instead only having the www-data ones.

Sounds like a good idea for us to do on our server right? Yeah, on the surface. But if we dig a little deeper, it doesn't make as much sense. First, the use case of a server largely differs from that of a machine you (and many others) use on a daily basis. For the latter, you don't need the root privileges for many things; effectively, you use the machine to answer e-mails, play some games, go on YouTube to watch our videos, or come here to read the blog posts... you get the idea. So, you don't need root privileges on the machine you use every day. However, you typically work on the server as root because you want to install services and webservers!

And a typical recommendation that goes hand-in-hand with disabling the root login is to add the unprivileged user to the sudo group, allowing the user to execute commands with root privileges just by adding sudo at the beginning of the command. Surely user + sudo is different from root so we're safe using this approach, right? This is not correct. From a security standpoint, these are (almost) the same. By giving a user sudo capabilities, you basically elevate that user to root. Sure, it's not a direct elevation to root, but when it comes to security, it's all the same.

While some might use the argument "but you require a password to use sudo", it isn't an actual protection. The easiest way I found was simply using .bashrc, where we can make some malicious code get executed every time the user attempts to use sudo privileges:

alias sudo='sudo id; sudo'

That way, the actual user won't know they just ran some malicious command as sudo. Yikes!

Of course, by using the sudo method, some logging and accountability is introduced due to the nature of adding users to the sudo group, which is helpful in team environments and might be more desirable than giving everybody the root password.

But in the end, these aspects still are more about convenience; but disabling the root SSH login does not protect you from hackers.

Changing the Default SSH Port

In the sshd config, you can easily change the default port where ssh is listening. This is classic security through obscurity: "if you can't see or find which ports we use, you can't hack us". We'll go right ahead and spoil this one: this is demonstrably incorrect.

The reasons for this recommendation is either "everything can be hacked so hide your ssh port!" or "people will try to bruteforce if they know what port your ssh is on!". And though "everything can be hacked" is probably true(?) in some sense(?), it's typically used in this context to fearmonger.

The bottom line is, if someone has access to a ssh 0day, us commoners are probably not the target. But more importantly, changing ports does nothing to help you against such a powerful attacker. Where it might be useful is against script kiddies and scanners that only check for default ports and weak ssh passwords. But if the hacking method is through bruteforcing the password, then having a really strong and unique password, or using SSH keys, is how you actually protect yourself! Hiding your insecure entry door might just delay the inevitable "hack"...

Disabling IPv6 for SSH

IPv4 is secure, and IPv6 is not... according to some resources. So let's disable ssh listeing on IPv6!

IPv6 is better than IPv4, but you probably aren't getting much out of it - because neither is anyone else. Hackers get something from it though - because they use it to send malicious traffic. So shutting down IPv6 will close the door in their faces.

This doesn't even make sense: what kind of traffic only happens in IPv6 and not IPv4? But we concede that there are some caveats:

• The address possibilities with IPv4 are fewer than in IPv6; this means that blocking IPv4 is more expensive for the attacker, because banning IPv6 is a bit less useful since there are more addresses to choose from. Though it's probably not really a good economical take because the difference in costs involved is not that large.
• A misconfigured firewall that only covers IPv4 addresses could allow an attacker to cruise through via IPv6. But then the root issue is the misconfigured firewall, not the version of the internet protocol you're using.

IPv6 also has some concerns for larger networks. The IPv4 NAT (Network Address Translation) is probably the best type of "firewall" we can have for our home networks. They allow users to open up specific ports in the local network without having to worry about half the internet hacking into the network. This might change with IPv6. But frankly, all of this is moot if you're not hosting the server yourself.

Oh, also, we tweeted about this (tweet + replies), to which we got a great reply by @notafile:

[...] For now, making sshd listen on v6 only stops automated login attempts more effectively than fail2ban.

This is basically a better version of the "changing your default ssh port" recommendation; script kiddies don't try IPv6 as much as they do IPv4 (for now, at least), so you might be in a better position disabling IPv4 than changing the ssh ports - in terms of effectiveness! Take that, script kiddies!

Jokes aside, you actually do reduce the attack surface by disabling IPv6 on the entire network interface (not just ssh) - operating systems might still have more IPv6 related bugs (Windows example). So this idea has maybe some merit. After all, attack surface reduction is a very real and important strategy in reducing your chace of exposting a vulnerability to hackers. But it's important to know though that as a recommendation, disabling IPv6 doesn't make you a whole lot more safer (especially only disabling IPv6 for ssh). Your single hosted Linux server will not be magically more secure, so feel free to keep using IPv6.

Setting Up a Basic Firewall

We briefly brought up firewalls previously, and we wanted to discuss them in a bit more detail. Typical tips include using iptables and ufw (the Uncomplicated Firewall) to block ports to your server.

The way the firewalls works generally is to block ports, and then you just unblock the ones you need, e.g. ssh on port 22 or the webserver on ports 80 and 443. This is no different from simply having ssh listening on port 22 and the webserver on ports 80 and 443. You can double-check that that's what is going on using ss or netstat commands.

ss -tl
netstat -tlpn

Now think about it for a second. You can open ports 22, 80, and 443. And if you set up your firewall to allow communication with ports 22, 80, and 443, guess what... you've accomplished exactly nothing. That's right, they both do the same. Implementing a firewall in this instance serves no purpose from a security standpoint.

Of course, there are some nuances to consider. Firewalls, in general, are not useless, but you do need the right use case and their correct application so that they are effective in blocking unwanted connections. For instance, consider a frontend server and an SQL database that - for some obscure reason - you haven't placed into their own isolated private network (like a VPN). If the database server is listening on a port (e.g. port 3303), it'll accept any traffic that comes there, not just the traffic from the frontend server. If however a firewall is added on the database server to only accept incoming connections from the frontend server's IP, then the firewall is effective in blocking out all non-frontend traffic.

Firewalls, in essence, can help thwart attacks using ports if you configure the firewall properly. However, just setting it on its own to block all ports except the select few that you need will not do anything to boost your security. But maybe you felt cool you did something with fire.

Unattended Server Auto Upgrading

Some of the first terminal commands you've probably learned about when you started using Linux are for updating packages or the distribution itself.

sudo apt-get upgrade
sudo apt-get dist-upgrade

Some resources online suggest enabling automatic unattended upgrades of the system, using the following command.

dpkg-reconfigure unattended-upgrade

Once again, there are instance where automatic updating is good; your phone's OS is a great example of this. Contrast this with a server, where you're delivering a service, a completely different use case! The issue with automatically updating packages or distributions is that on occasion, it may break things and you'll have to go in and manually apply fixes. Also, not all of the updates are security updates. The job of a system administrator is to decide what updates are necessary, as well as how and when to update the systems to minimize disruptions. Consider a sales environment where your server is the backbone of your inventory, order processing, or other critical service; you cannot afford to have the server just drop offline for hours because automatic updates broke something and now you have to spend time fixing it. Though you might say, security should always come before sales, but the reality of it is more nuanced.

If you're using only ssh and nginx, the odds of a critical vulnerability that can seriously harm your server are pretty low, especially against a default installation.

And there are two major cases where automatic updating won't help you.

1. If a new, serious vulnerability is found, OS packages that would be automatically updated might not have the patch ready. You might have to - as the person responsible for the server - go and apply the patch yourself, or otherwise implement mitigating solutions, which defeats the purpose of auto-updating.
2. The second case is that the webapp you use for your server is a much easier target to attack than ssh or nginx. In fact, maybe your own code has vulnerabilities, or the one you cloned from GitHub. These are not covered by automatic updating, so you need to update those manually, too.

Though in some cases, the webapp might have its own security updates, as is the case for WordPress for example. They push their own updates, which helps maintain security for all their customers who use WordPress to host their content.

But in essence, the advantages of having the automatic updates are largely countered by the risk of having to fix things in case of disruption caused by package updating, but more importantly, you probably will have to patch software manually anyways.

Final words

We've given you our opinions and reasoning above concerning secure shell (SSH) password logins, public keys, root logins, changing ports, disabling IPv6, setting up firewalls, and even automatic updating of packages and distributions and their role in enhancing or harming your Linux webserver's security. It's worth mentioning that these are our opinions. Yours may differ, and in fact you might have your own reason for choosing to do otherwise... and that's great! It's a testament that you have done your research, and that you understand the ramifications of your decision. The purpose of this article was to address some common "best practices" and discuss the non-impact that they have on your server's security if they are blindly implemented.

Building your own server and understanding its inner workings and interactions with the outside world is an incredible learning opportunity, and we certainly encourage you to embark on this endeavour. However, it is worth noting that sometimes, it might be more beneficial to use Platform-as-a-Service (PaaS) providers to host your server so that you don't have to deal with the potential headaches of running your own server. That way, you can have a server providing a service on your behalf, without requiring you to act as sysadmin all the time. This is a personal decision and it's truly up to you to decide whether to host the webserver yourself, or using PaaS providers. It depends entirely on your situation and the involvement you wish to have. If you decide to host it yourself, just remember to keep this article in mind when deciding about what best practices you want to implement in terms of Linux server security.