If you have even a small hunch that your server has been compromized, it bears further investigation. In fact, it's good practice to periodically review your server even if you suspect nothing. But what should you be looking for? How can you tell what files have been modified, whether you have malicious software on your server, and which processes are problematic? This guide is designed to answer all of those questions.

Checking your recently modified files

Looking at the recently modified files list in Linux can be an excellent way of spotting suspicious changes to your server – provided you know what you're looking for. You can find a list of files using the find command, which we have an extensive guide on here. For this use case, though, the most useful syntax would be something like this:

find -type f -mtime -15 -ls

This will find all files that were modified in the last 15 days. You can naturally change this number to reflect a period that is more realistic for your use case. You can also use +15 for all files that were modified 15 days ago (for example, on March 5 if you run the command on March 20). This is useful if you suspect a compromize on a specific day.

Checking a specific folder is often useful too – for example, your log folder, to see if any have been modified or updated:

find /var/log/ -type f -mtime -15 -ls

To investigate further, you can then check the timestamp of a specific file using stat. For example:

stat /var/log/malware.sh

  File: malware.sh
  Size: 6               Blocks: 8          IO Block: 4096   regular file
Device: 801h/2049d      Inode: 1871        Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-03-10 08:13:10.903053851 +0000
Modify: 2023-03-10 08:13:10.903053851 +0000
Change: 2023-03-10 08:13:10.903053851 +0000
 Birth: 2023-02-10 08:13:10.903053851 +0000

You should be looking for anything that does not make sense – unusual permissions, dates of files that don't add up, and files (such as logs) that should be there but aren't.

Checking your log files

Most actions on your Linux server will be logged, which makes checking your log files one of the best ways to spot suspicious activity. However, you must be aware that a clever attacker will modify your log files to remove their activity. You can use the stat command to tell when a file was last modified, but be aware that this can be falsified too. Checking log files is not a foolproof way of spotting malicious activity, then, but it will help to catch low-hanging fruit – malware or actors who are not proficient enough to hide their activity.

Some of the logs you should check include:

  • /var/log/auth.log: Shows all authentication logs. This allows you to investigate failed login attempts password changes, etc. On RedHat or CentOS, this will instead be var/log/secure.
  • /var/log/faillog: A list of failed logins, useful for investigating brute force/login credential hacks.
  • /var/log/wtmp: A record of every login/out. Check whether someone was accessing the server at a time they shouldn't have been.
  • /var/log/apt: Logs when an application is installed using apt on Ubuntu machines. Useful for checking if any unsanctioned applications have been installed. On CenOS this will be /log/yum.
  • /var/log/syslog: Shows a log of activity data throughout the system, providing an overview of the general goings on. On CentOS/RHEL this will be /var/log/messages.

Provided the attacker has not covered their tracks, combing these logs should give you a good idea of whether something suspicious has occurred.

Checking for suspicious processes

Most servers are hacked for their resources, not their information. If your server is compromized, it's most commonly so that an attacker can use it in a botnet or for brute force attacks on other servers. To do this, the attacker (most likely via an automated tool) will establish several outbound connections and create an associated process.

By analysing our processes, we may be able to figure out if anything suspicious is going on. But, be aware – it's not uncommon for hackers to replace the tools you'll be using to investigate with hacked versions that won't give you accurate information. Remain skeptical if other signals are pointing to a server compromize.

First things first, check your active connections using netstat -an. This will show you all connections and their port numbers in a list.

Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 193.149.189.216:38038   91.189.91.38:80         TIME_WAIT
tcp        0      0 193.149.189.216:22      81.229.38.48:60860      ESTABLISHED
tcp        0   1112 193.149.189.216:22      83.121.2.254:50537      ESTABLISHED
tcp        0      0 193.149.189.216:22      83.123.9.169:60272      ESTABLISHED
tcp        0      0 193.149.189.216:22      2.187.234.37:22363      ESTABLISHED
tcp        0     33 193.149.189.216:22      83.121.2.254:49256      LAST_ACK
tcp        0     33 193.149.189.216:22      5.216.59.94:51280       LAST_ACK

The entries above are quite normal looking. TCP traffic on ports commonly used by genuine processes. However, let's say we want to investigate what processes are making connections on a particular port.  We'll go with port 50537, as that's on our list:

lsof -i tcp:50537 -P -R

The lsof command (list running files) should return a list of which processes that are using this port, as well as their parent process.

COMMAND   PID PPID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    49728 3453 root    4u  IPv4 334132      0t0  TCP 64117a5178d79d0001fa80a7:22->81-229-38-48-no2744.tbcn.telia.com:50537 (ESTABLISHED)

We can find out which files the process is using by filtering by its PID:

lsof -p 49728

COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
sshd    49728 root  cwd    DIR                8,1     4096      2 /
sshd    49728 root  rtd    DIR                8,1     4096      2 /
sshd    49728 root  txt    REG                8,1   917192   2757 /usr/sbin/sshd
sshd    49728 root  mem    REG                8,1   309600  23258 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2
sshd    49728 root  mem    REG                8,1    18424   3109 /usr/lib/x86_64-linux-gnu/security/pam_env.so
sshd    49728 root  mem    REG                8,1    26696   3120 /usr/lib/x86_64-linux-gnu/security/pam_limits.so
sshd    49728 root  mem    REG                8,1    18424   3124 /usr/lib/x86_64-linux-

This will be a long list, but it will tell you where the program is located. We can see here that ours started in /usr/sbin/sshd .

Dealing with the malicious program

If you believe it is malicious, you can kill it with kill 49728 (replace this with the right PID). You can then remove the file's execution rights with chmod -x /usr/sbin/httpd.

That should stop it running for now, but it's worth making sure that it, or another start-up command that will recreate it, isn't set to run whenever you boot your server. For this, we can use the usual crontab -l and remove entries coming from your suspect folder by using crontab -e and deleting its entry.

That (hopefully) deals with this particular malicious program. But there could easily be more, which is why it's usually safest to just rebuild your server entirely. Check common folders such as /etc and /tmp/ and try to root out other suspicious processes using the steps above.

Once you're done, update your software and server to ensure they're patched against the latest vulnerabilities.

Scanning for malicious software with RKHunter

You should never 100% rely on automated tools for security, but RKHunter or other anti-virus software can be a good supplement to manual checks. RKHunter helps you to identify whether your server has a rootkit by checking for common variants in common locations. It will also check for malware by checking running processes for suspicious files, identifying login backdoors, searching for suspicious directories and apache behavior, checking for suspicious network port usage and system start-up files, etc.

It's important to understand that passing the test doesn't necessarily mean that you don't have a rootkit or malware just that RKHunter couldn't find any. There is a possibility that a malicious process is interfering with RKHunter's output or that it simply does not have the capability to detect the malware you have. Still, it is a useful time-saving tool for identifying common security issues and can be used to run regular precautionary scans.

First, install it:

sudo apt update
sudo apt install rkhunter -y

Now let's perform some configuration. Open the config file using nano:

sudo nano /etc/rkhunter.conf

Look for WEB_CMD="/bin/false" manually or by using the find function via Ctrl + W. Comment out the line by adding a # in front of it.

#WEB_CMD="/bin/false"

Find UPDATE_MIRRORS and set it to option 1:

UPDATE_MIRRORS=1

This will make RKHunter check the mirrors file for updates when we use the --update option later.

Also set MIRRORS_MODE to 0. This will allow RKHunter to use any mirror when looking for updates, rather than just local ones.

MIRRORS_MODE=0

Press Ctrl + O followed by Ctrl + X to save and exit. Double-check that you haven't made any typos by running sudo rkhunter -C. No output means your config file has passed the test.

Now you can update RKHunter's data files and run a check:

sudo rkhunter --update
sudo rkhunter --check

RKHunter will provide a helpful list of what it has checked for and whether it has found any issues. Once it has finished, you can view the full log by typing:

sudo nano /var/log/rkhunter.log

Should RKHunter find any suspicious files, use the process outlined above to remove its execution rights and perform other necessary actions, such as using rm to delete it.

Closing words

That about covers the basics of checking for malicious server activity in Linux, but there is obviously a lot more to it. If you are in doubt and using your VPS for important or sensitive tasks, pay a Linux security professional.

For additional help, BitLaunch customers can reach out to our support via live chat. Our support agents cannot diagnose the security issues of every customer's server, but they may be able to point you in the right direction.