Linux Security Hardening for Production Hosts

Most Linux hardening guides are exhaustive lists of settings, each labeled "important" with no prioritization. After running production Linux for years and surviving one audit, this is the working version: the settings that earn their place because of specific threats they prevent, with the production reasons we tightened each.

The mental model#

Most Linux compromises follow a pattern:

1. Initial access (a vulnerable service, leaked credential, or social engineering)
2. Privilege escalation (find a way to get root from a user account)
3. Persistence (install something that survives reboot)
4. Lateral movement (use this host to reach other hosts)

Hardening makes each step harder. We don't pretend to make compromise impossible; we make the bad actor's job materially harder and we make their actions visible to detection.

Authentication: SSH first#

SSH is usually the front door. Hardening it:

# /etc/ssh/sshd_config
PasswordAuthentication no
PermitRootLogin no
PubkeyAuthentication yes
AuthenticationMethods publickey
ChallengeResponseAuthentication no
UsePAM yes
ClientAliveInterval 300
ClientAliveCountMax 0
MaxAuthTries 3
LoginGraceTime 30
AllowUsers admin operations

Specific reasons:

• No passwords: brute-force resistance. Public keys with passphrases are much harder to attack.
• No root login: root attempts hit immediately on most servers; making it impossible removes a class of attempts.
• AllowUsers: explicit allowlist. Users not in the list can't SSH in regardless of what's in /etc/passwd.
• MaxAuthTries 3: limit attempts per connection. Combined with fail2ban (below), brute force becomes impractical.

For our cloud production hosts, we've replaced SSH entirely with AWS SSM Session Manager. No SSH port open, no SSH keys to manage, full audit log of every session. SSH stays available for break-glass via a separate bastion.

fail2ban for what SSH can't prevent#

fail2ban watches logs and blocks IPs that exceed failure thresholds:

# /etc/fail2ban/jail.local
[sshd]
enabled = true
maxretry = 3
findtime = 10m
bantime = 1h

Three failed SSH attempts in 10 minutes → banned for 1 hour. Repeat offenders get longer bans (escalating).

For internet-facing SSH, fail2ban is essential. For SSM-only servers, it's irrelevant (no SSH to attack). Match the tool to the threat.

sudo: limit what users can do as root#

Default sudo: any sudoer can run anything as root. Better:

• Specific commands per user/group, not blanket ALL=(ALL) ALL
• NOPASSWD: only for specific safe commands (like systemctl status)
• Logging of all sudo commands to audit log

# /etc/sudoers.d/operations
%operations ALL=(ALL) /usr/bin/systemctl restart myservice, /usr/bin/journalctl
%operations ALL=(ALL) NOPASSWD: /usr/bin/systemctl status

The operations group can restart services and read journals. They can't cat /etc/shadow. The blast radius of a compromised operations account is bounded.

Filesystem permissions#

Hardening permissions:

• /etc/shadow: 0600, root:root (default; verify)
• /etc/sudoers.d/*: 0440, root:root
• /var/log/: 0750, root:adm
• Application data dirs: 0700 or 0750, owned by service user

Specific things we check:

• No world-writable files outside /tmp and /var/tmp (find offenders with find / -perm -o+w -type f)
• No SUID binaries except known/needed ones (e.g., sudo, passwd)
• Service config files not group-readable if they contain secrets

We have a script that checks these on every host quarterly.

Kernel parameters#

A few sysctl settings worth tightening:

# /etc/sysctl.d/99-security.conf

# Disable IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0

# Don't accept ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

# Log martian packets
net.ipv4.conf.all.log_martians = 1

# Disable IP forwarding (unless needed for routing)
net.ipv4.ip_forward = 0

# TCP SYN cookie protection
net.ipv4.tcp_syncookies = 1

# Disable kernel core dumps (info disclosure)
fs.suid_dumpable = 0

# Restrict ptrace access
kernel.yama.ptrace_scope = 1

# Restrict /proc visibility
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1

Each of these closes a class of attack:

• Source routing / ICMP redirects: defense against routing-based attacks
• ptrace_scope: a compromised process can't attach to another process and steal its memory
• kptr_restrict / dmesg_restrict: kernel pointer info disclosure protection

These don't stop a determined attacker but they raise the bar.

Service exposure: only what's needed#

Audit what's listening:

ss -tlnp

Output lists every TCP listener. Each one is an attack surface. We periodically review:

• Why is this service listening?
• Does it need to be exposed externally, or only locally?
• Can it bind to localhost only?

Common findings:

• Postgres / Redis listening on 0.0.0.0 (should be 127.0.0.1 or VPC-only)
• Default tools (cups, avahi, etc.) running unnecessarily
• Dev/debugging endpoints left enabled in prod

Each exposure is either justified, restricted (firewall rule), or removed.

Firewall: nftables (or ufw)#

Default-deny inbound, allow specific:

# Simplified iptables / nftables; in practice we use AWS Security Groups + host-level
nft add rule inet filter input ct state established,related accept
nft add rule inet filter input ip protocol icmp accept
nft add rule inet filter input tcp dport 22 ip saddr 10.0.0.0/8 accept
nft add rule inet filter input drop

For cloud hosts: AWS Security Groups do the bulk. Host-level firewall is defense-in-depth.

For on-prem / standalone hosts: ufw or nftables is the primary firewall. Configure once via Ansible; review annually.

Auditd for security-relevant events#

Auditd logs syscall-level events:

• Reads of sensitive files (/etc/shadow, AWS credentials)
• Changes to system configuration
• Process executions of suspicious binaries
• Network configuration changes
• User account changes

Sample rules:

-w /etc/shadow -p wa -k shadow_access
-w /etc/sudoers.d/ -p wa -k sudo_changes
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/curl -F auid>=1000 -k user_curl

Logs go to a central audit log collector. Anomalous events alert.

We use auditd selectively (heavy logging hurts performance). For containerized environments, we replaced most of this with eBPF-based tools (Falco, Tetragon) which have similar visibility with less overhead.

SELinux / AppArmor: optional on most hosts#

For server hosts, mandatory access control (SELinux on RHEL family, AppArmor on Debian/Ubuntu) is helpful but real work to maintain.

Our policy:

• Containerized workloads: enforce a default profile via the container runtime; don't customize per-service
• Long-lived hosts (databases, etc.): SELinux in enforcing mode with the default policy
• Non-critical hosts: SELinux in permissive mode (logs violations but doesn't enforce)

Operating SELinux well is a project; many teams opt out. We've found the default policies catch real issues occasionally without much tuning.

Patches: don't fall behind#

Unpatched systems are the most common compromise vector. Discipline:

• Cloud images: rebuild and replace nodes monthly. Karpenter on EKS rotates nodes naturally; AMI versions for non-K8s hosts get bumped quarterly with a forced rotation.
• Kernel patches: live patching where possible (we use kpatch on long-lived database hosts). Otherwise, scheduled reboots.
• Critical CVEs: out-of-cycle patching when something significant lands.

Without this, hosts accumulate unpatched CVEs. We've seen 18-month-old hosts with hundreds of high-severity CVEs in industry incidents. Don't be that team.

Detection: when prevention fails#

Hardening reduces the chance of compromise; it doesn't make compromise impossible. Detection catches the cases that get through:

• Falco / Tetragon: runtime detection (suspicious syscalls, file accesses)
• AIDE / Tripwire: filesystem integrity monitoring (alerts on changes to system files)
• GuardDuty: AWS-specific anomaly detection
• Centralized logs: SSH logins, sudo usage, audit events all pipe to a central log analyzer with alerting on anomalies

A successful hardening + detection setup means: even if an attacker gets in, we know quickly and can respond before damage spreads.

What we don't bother with#

A few things from older hardening guides we don't apply:

Disabling ICMP entirely. Breaks legitimate use cases (path MTU discovery). Limit, don't disable.

Removing all unused user accounts manually. Cloud images don't have many unnecessary accounts. Reviewing once at AMI build is enough.

Custom kernel builds. Mainline kernels with vendor patches are fine. Maintaining a custom kernel is a recipe for missing security updates.

Hand-curated AppArmor profiles per service. The default profiles work; per-service customization is a maintenance burden that doesn't add proportional security.

Disabling TLS protocols beyond TLS 1.2. Already done by default in current OpenSSL versions. Don't pre-emptively block 1.3.

What I'd tell a team starting#

Replace SSH with SSM Session Manager if you're on AWS. Removes the SSH attack surface entirely.

Use cloud Security Groups + host firewall. Both, not either.

Patches are the most important thing. Most compromises target known CVEs. Stay current.

Audit what's listening on every host. Each open port is an attack surface.

Centralize logs and alert on anomalies. Detection is as important as prevention.

Cloud images / golden AMIs. Bake in the hardening; new hosts are hardened from the start.

Don't over-tighten. Restrictive settings can break legitimate workflows. Tighten until something breaks; back off carefully.

Linux hardening is a discipline, not a one-time project. The setup runs in the background; the discipline is in patching, monitoring, and reviewing periodically. Most of the wins come from a small number of high-leverage changes (SSH hardening, patching, default-deny firewalls, audit logging). The rest is incremental and useful but lower-priority. Get the fundamentals right and most threats become much harder.