10 Essential Steps to Secure and Harden Your Linux Server

Why Linux servers stay on attackers' radar

Public Linux servers are scanned almost immediately after they become reachable. SSH password attempts, web application probes and leaked keys tested against your address range are routine background noise. If you run production workloads, server security cannot be an afterthought.

There is no magic hardening script. A safer host comes from a baseline that you can apply, verify and maintain across every machine. I have learned this the unglamorous way: a system can look quiet while an important control is missing.

These are ten practical areas I check on internet-facing Linux servers. They will not make a host invulnerable, but they remove many easy paths and make the remaining ones easier to detect.

Step 1: Start with a minimal, fully patched system

If the operating system is outdated, later configuration work has a weak foundation. On a fresh VPS or dedicated server, patch it first, then remove services and packages you do not need.

Use a minimal base image

Choose a maintained, minimal distribution image rather than one carrying a collection of services you may never use. With providers like VPS.TC VPS or dedicated servers, start with the leanest suitable Linux template and add software deliberately.

Apply security updates immediately

After confirming that you can log in through the provider console, update the system before installing your application stack. On Debian and Ubuntu:

sudo apt update && sudo apt full-upgrade -y

On RHEL, Rocky or AlmaLinux:

sudo dnf upgrade --refresh -y

Reboot when a kernel, systemd, glibc or another component requiring a restart was upgraded. Check before and after instead of treating every package update as proof that a reboot is necessary:

sudo needs-restarting -r

That command is available on RHEL-family systems with dnf-utils or the current dnf-plugins-core package. On Debian and Ubuntu, review /var/run/reboot-required when it exists.

Automate future security updates

For Debian and Ubuntu, install and configure unattended-upgrades if automatic security updates fit your release and change-control policy:

sudo apt install unattended-upgrades
sudo dpkg-reconfigure unattended-upgrades

On RHEL-like systems, dnf-automatic can apply or report updates according to its configuration. Automatic patching still needs monitoring, exclusions for special workloads and a recovery plan. Test kernel updates on a disposable or staging host first.

Step 2: Lock down SSH, your primary entry point

SSH is still the service I see probed most often. Changing its port may reduce log noise, but it is not a security control by itself. Authentication, authorization and network restrictions matter more.

Disable direct root login

Create an individual administrator account and give it only the sudo access it needs.

sudo adduser deploy
sudo usermod -aG sudo deploy

On RHEL-based systems, use the wheel group instead:

sudo usermod -aG wheel deploy

Before changing the daemon configuration, validate it with sshd -t. In /etc/ssh/sshd_config or a file under /etc/ssh/sshd_config.d/, use settings similar to:

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitEmptyPasswords no
PubkeyAuthentication yes

Reload the service name used by your distribution. Debian and Ubuntu commonly use ssh; RHEL-family systems commonly use sshd:

sudo sshd -t && sudo systemctl reload ssh

On RHEL-family systems, replace ssh with sshd. Keep an existing session open and test a second key-based session before disabling passwords. I once came close to locking myself out because I changed access rules before testing the new account; the provider console saved me, but it was not a clever recovery plan.

Use key-based authentication, not passwords

Generate a key on your workstation, protect it with a passphrase and keep the private half off the server.

ssh-keygen -t ed25519

Use ssh-copy-id where available, or append the public key to ~/.ssh/authorized_keys. For automation, use a separate restricted key rather than your personal administrator key.

Limit who can log in via SSH

Define the permitted accounts explicitly when that matches your access model:

AllowUsers deploy

For a bastion host, restrict access further with source addresses, a VPN or an upstream firewall. If you use an MFA-capable SSH setup, test the emergency access path before you need it.

Step 3: Enforce strong authentication and authorization

SSH hardening answers who may enter. Sudo policy answers what they can do after entering.

Use sudo with least privilege

Give each administrator a personal account and record privilege changes in version control or configuration management. Use /etc/sudoers.d/ and validate edits with visudo; do not share a root password.

When someone leaves the team, remove the account or disable it, revoke its keys and review any service credentials it owned. Access that exists only in somebody's memory is not an access-control system.

Consider multi-factor authentication for critical access

MFA can protect SSH, VPN and management panels when it is deployed carefully. TOTP through a PAM module is one option; FIDO2 or hardware-backed keys can provide stronger phishing resistance where your SSH clients and policy support them. Keep a documented, tightly controlled break-glass method, and test it without weakening normal access.

Step 4: Put a real firewall in front of every service

A provider firewall and a host firewall solve different problems. I use both when possible, with the host rules kept simple enough to audit.

Allow only what you actually use

List the required ports before writing rules. Usually that means SSH and, for a web server, ports 80 and 443. Database ports should normally be reachable only from the application network or trusted administration addresses.

On Ubuntu, ufw is convenient for a small host:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

On RHEL-based systems with firewalld:

sudo firewall-cmd --permanent --set-default-zone=public
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

For more explicit rule sets, nftables is the current Linux packet-filtering framework behind many distributions' tooling. Whichever layer you choose, confirm the SSH rule and keep a console or active session available before enabling it. The -a flag is not the thing to remember here; the thing to remember is how you will recover from a bad rule.

Step 5: Reduce attack surface: services, ports and software

Every listening daemon expands the amount of code and configuration you must maintain. If a service is unnecessary, remove it or prevent it from starting.

Inventory running services

Start with sockets:

sudo ss -tulpn

Then inspect enabled services:

systemctl list-unit-files --type=service --state=enabled

Check each unfamiliar listener with its package and service documentation. Do not disable something merely because its name is unfamiliar; identify it first.

Disable and remove what you do not need

sudo systemctl disable --now avahi-daemon
sudo systemctl disable --now cups

Those examples are appropriate only when you do not need discovery or printing. Remove unused packages through the distribution package manager and review dependencies afterward. A service that is disabled but never patched can still become a problem if somebody starts it later.

Step 6: Harden the OS and kernel

Kernel settings can reduce exposure, but they are not a substitute for patching, firewall rules or application security. Test them against the way the host actually routes traffic.

Apply secure sysctl settings

For a host that is not a router, a starting point may include:

net.ipv4.ip_forward = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

Save this in a reviewed file such as /etc/sysctl.d/99-custom.conf and apply it with:

sudo sysctl --system

Be careful with reverse-path filtering. Strict rp_filter=1 can break asymmetric routing, policy routing, VPNs or some cloud network designs. Use the distribution default or a tested loose mode where the network requires it; do not paste a large hardening template into production. I have seen a supposedly harmless sysctl change turn a routing problem into a longer investigation because nobody had written down the expected traffic paths.

Use mandatory access control where feasible

SELinux and AppArmor restrict what processes can do even when a process is compromised or running with high privileges. Keep SELinux enforcing or AppArmor enabled where the distribution and application support it. Fix policy denials deliberately; disabling the control to make an application start is usually only hiding the real configuration problem.

For multiple servers, put the chosen MAC policy into images and automation. Verify the policy after application upgrades, since a package update can change required paths or permissions.

Step 7: Protect data with sane permissions and encryption

A network compromise is not the only route to data loss. A stolen credential, an over-permissive service account or an exposed backup can be just as damaging.

Fix ownership and permissions

World-writable files should be unusual and explainable. Check them periodically:

sudo find / -xdev -type d -perm -0002 -print
sudo find / -xdev -type f -perm -0002 -print

Review the results rather than blindly running chmod 777. That changes the symptom, not the ownership or application design that caused it. Protect /etc/shadow, SSH keys, TLS private keys and application secrets with correct owners and the narrowest practical modes.

Encrypt data at rest and in transit

LUKS full-disk encryption can protect data on stolen physical media, but it also changes how a remote server boots and how keys are made available. Plan console access and key handling before deploying it. For databases and application data, field-level or volume encryption may be more appropriate.

For public services, use current TLS configurations supplied by the web server and certificate tooling, and remove obsolete protocols only after checking client compatibility. Encryption does not help if private keys are stored in a world-readable backup.

Step 8: Logging, auditing and knowing what changed

When a server behaves strangely, I want logs before I want a reboot. Restarting first can erase volatile evidence and rarely explains the original fault.

Centralize and retain logs

Forward important journald or syslog records to a separate collector with restricted access and a retention policy. Include authentication events, sudo activity, firewall events, service failures and application logs. A remote copy is useful because an attacker with root can alter local evidence.

Audit critical actions

For high-value systems, use auditd or an equivalent auditing pipeline for changes to identity files, SSH configuration, sudo policy and other sensitive paths. Tune rules to the risks you can investigate; collecting everything without enough storage or alerting creates noise rather than visibility.

Also record configuration changes through Ansible, packages or a Git-backed process. Knowing who changed a file and when is often as useful as knowing that it changed.

Step 9: Backups and disaster recovery that actually work

Security includes recovery. Ransomware, accidental deletion, failed updates and provider incidents all test the same question: can you restore the service and the data?

Follow a practical backup strategy

The 3-2-1 rule remains a useful baseline:

• Keep at least three copies of important data
• Use two different storage systems or media
• Keep at least one copy offsite, and preferably offline or otherwise immutable

Combine filesystem backups with application-consistent database dumps. On cloud servers or virtual datacenter infrastructure, snapshots are useful for short-term recovery but are not a complete backup strategy. Protect backup credentials separately from production credentials.

Test restores, not just backup jobs

A successful backup job proves that a process ran. It does not prove that the files are usable, the database is consistent or the service can start. Restore into an isolated environment, check application behavior and record the recovery time.

I use rsync for straightforward copies and Borg for deduplicated, encrypted repositories, but the tool is less important than the restore test. A backup I have never restored is an assumption. Schedule the test and document where the keys, repositories and recovery instructions live.

Step 10: Continuous monitoring and basic incident response

Hardening is maintenance, not a one-time ceremony. Packages change, users come and go and a previously internal service may become public after one deployment.

Monitor resources, services and security signals

Watch at least:

• CPU, memory, swap and disk trends
• Network traffic and unusual outbound volume
• Service availability and response time
• Authentication failures, sudo use and new accounts
• Certificate expiry and backup freshness

Use alerts that result in an action, not only dashboards. In my homelab, Prometheus and Grafana show trends while Uptime Kuma catches availability failures; I still inspect the underlying logs when an alert fires. A pretty graph is not an incident response plan.

Plan how you respond to incidents

Write down:

• Who is notified and who can make isolation decisions
• How to restrict network access without destroying evidence
• Which logs, process data and disk images should be preserved
• How to rebuild from a known-good image
• How credentials, tokens and keys will be revoked and replaced

Do not reboot automatically just because a process looks suspicious. Capture what you can, isolate the host, then rebuild when you have enough information and a clean recovery path.

Pulling your Linux security plan together

These ten areas will put an internet-facing host in a much better position than an unpatched, over-permissive default installation. They do not replace application security, provider controls or incident response, and they do not guarantee that an attacker will fail.

From SSH and firewalls to backups and monitoring, the hard part is keeping the baseline true after the first deployment. Standardize the controls with images or Ansible once you have repeated the work twice (although I still do not open a YAML file for a one-line emergency fix). Then every new server from providers like VPS.TC VDS can begin from a reviewed baseline instead of a remembered checklist.

If you manage one server, check its exposed ports, privileged accounts and most recent restore test today. If you manage dozens, measure configuration drift and make the repair repeatable. That is usually where security stops being a document and becomes part of operating the system.

When I harden a Linux server, I also check whether memory pressure is being masked by swap; my guide, How to Create Swap Space on a VPS for Linux Memory Management, explains how to configure it safely and investigate the underlying cause.

When I add a database service, I treat it as another security boundary; my guide on How to Install and Secure MySQL on a VPS walks through binding, authentication, TLS, and restore-tested backups.

When I'm hardening a Linux VPS that will host a game, I also apply these practical steps from How to Set Up a Minecraft Server on a VPS, including running the service as a dedicated user and opening only the required ports.

I also recommend reading our practical DDoS protection guide for your VPS, which explains how to identify an attack, collect evidence, and choose the right defense before restarting or resizing your server.

Frequently Asked Questions