Security

Securing Your Self-Hosted Server: A Practical Guide

2026-06-15 · 12 min read · views

I've been self-hosting for over 8 years, and my current setup spans three environments: a Dell OptiPlex homelab running Ubuntu 24.04 LTS as my primary server, an ARM64 Debian 11 box (hostname 003luo8G) for low-power background services, and WSL2 on my daily-driver Windows laptop for testing configs before they hit production. All three sit behind a mihomo proxy with a Cloudflare Tunnel providing the public ingress — so I only expose what I absolutely need to. Every machine runs SSH key authentication, UFW, and fail2ban, and I learned most of these measures the hard way. Because self-hosting gives you full control over your data, services, and infrastructure, but that control comes with a responsibility: you are now the system administrator, and the security of your server is entirely in your hands. A compromised server can leak personal data, serve malware to visitors, or become part of a botnet — all within minutes of going online if basic precautions are skipped.

According to the Verizon 2025 Data Breach Investigations Report (DBIR), 60% of breaches involved attackers exploiting known vulnerabilities where a patch was available but not applied, and credential-based attacks accounted for over 40% of all breach entry methods. These are the two categories of risk that a well-hardened self-hosted server can virtually eliminate. I've personally watched fail2ban logs on my Ubuntu 24.04 box rack up over 1,200 banned IPs in a single week — almost all of them SSH scanners hitting port 22 before I moved to a custom port. The automated scanning is relentless, and without the basics in place, you're not a target yet — you're a victim waiting for a clock to tick.

This guide covers the full security stack: SSH hardening, firewall configuration, intrusion prevention with fail2ban, automatic updates, Docker security best practices, AppArmor/SELinux, file integrity monitoring, and log auditing. Each section includes concrete commands I've tested on Ubuntu 24.04 LTS (OpenSSH 9.6p1, UFW 0.36.2, fail2ban 1.0.2) and Debian 11/12 — so you can run them with confidence on the two most common self-hosting distros. I'll also flag the gotchas I hit along the way.

1. SSH Hardening — Your First Line of Defense

SSH is the most attacked service on any internet-facing Linux server. Automated bots scan the entire IPv4 address space within minutes of a new server coming online, trying default credentials and exploiting old vulnerabilities. Hardening SSH is step zero of server security. I once spun up a fresh Ubuntu 24.04 VM on a VPS, forgot to configure the firewall before stepping away for lunch, and came back to 847 failed SSH login attempts in auth.log — all within 45 minutes. That's when I stopped thinking of SSH hardening as optional and started treating it like the first thing I configure on every machine.

1.1 Key-Based Authentication Only

Generate an ED25519 key pair on your local machine (not the server):

ssh-keygen -t ed25519 -a 100 -f ~/.ssh/selfhost_key

Copy the public key to your server:

ssh-copy-id -i ~/.ssh/selfhost_key.pub user@your-server-ip

Test that key-based login works, then disable passwords in /etc/ssh/sshd_config:

PasswordAuthentication no
ChallengeResponseAuthentication no
PermitRootLogin no
PubkeyAuthentication yes

Apply changes:

sudo systemctl restart sshd

1.2 Change the Default SSH Port

Changing the SSH port from 22 to a high-numbered port (e.g., 2222) drastically reduces log noise from automated scanners:

Port 2222

Update /etc/ssh/sshd_config, then add the new port to your firewall (see Section 2). After confirming you can connect on the new port, remove port 22 from your firewall rules.

1.3 Additional SSH Hardening Options

# In /etc/ssh/sshd_config
MaxAuthTries 3
MaxSessions 2
LoginGraceTime 30
AllowUsers your-username
ClientAliveInterval 300
ClientAliveCountMax 2

Practical tip from my setup: On my Dell OptiPlex, I set AllowUsers frankie and run SSH on a non-standard port. Combined with fail2ban, this cut my auth.log from thousands of failed attempts per day to zero. One thing to watch: if you use AllowUsers, don't forget to add your own username — I once locked myself out of my Debian 11 ARM64 box by forgetting this and had to console in through the IPMI to fix it.

2. Firewall Configuration with UFW

The Uncomplicated Firewall (UFW) provides a simple interface to iptables/nftables. I'm running UFW 0.36.2 on Ubuntu 24.04 (backed by nftables by default since 22.04). A default-deny policy ensures only explicitly allowed ports are reachable. Since my servers are behind a Cloudflare Tunnel and mihomo proxy, I only open ports 22/2222 (SSH) and 51820 (WireGuard) on the LAN-facing interface — everything else is handled at the tunnel level. This drastically reduces the attack surface compared to exposing services directly.

# Set default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH on your custom port (if changed)
sudo ufw allow 2222/tcp comment 'SSH'

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp comment 'HTTP'
sudo ufw allow 443/tcp comment 'HTTPS'

# Allow your self-hosted services (e.g., WireGuard)
sudo ufw allow 51820/udp comment 'WireGuard'

# Enable the firewall
sudo ufw enable

# Check the rules
sudo ufw status verbose

UFW logs blocked packets to /var/log/ufw.log. Check it periodically to spot scanning activity:

sudo tail -f /var/log/ufw.log | grep BLOCK

Pro tip: Never enable UFW over SSH before allowing your SSH port — you will lock yourself out. Always add your SSH port rule first, then enable.

2.1 Rate Limiting with UFW

UFW supports basic rate limiting to mitigate brute-force attempts:

sudo ufw limit ssh

This allows 6 connections per 30 seconds from the same IP before blocking further attempts for that period.

3. Fail2ban — Automated Intrusion Prevention

Fail2ban (I'm running 1.0.2 on Ubuntu 24.04) monitors service logs for repeated failure patterns and temporarily bans offending IPs via firewall rules. It ships with pre-configured filters for SSH, Apache, Nginx, and many other services. A quick sudo fail2ban-client status sshd on my Debian 11 ARM64 box typically shows 50–100 currently banned IPs — a testament to how much noise fail2ban silently filters out for you.

# Install fail2ban
sudo apt update
sudo apt install fail2ban -y

# Create a local configuration file
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local to customize settings:

[DEFAULT]
bantime = 3600          # 1 hour ban (in seconds)
findtime = 600          # 10 minute window
maxretry = 5            # 5 failures triggers a ban

[sshd]
enabled = true
port = 2222             # Match your custom SSH port

Add a jail for Nginx (if you use it):

[nginx-http-auth]
enabled = true

[nginx-botsearch]
enabled = true

Start fail2ban:

sudo systemctl enable fail2ban --now

# Check banned IPs
sudo fail2ban-client status sshd

Fail2ban is especially valuable for self-hosters because it protects services that must be publicly accessible (like web apps) from credential stuffing and directory traversal attacks.

4. Automatic Security Updates

The Verizon 2025 DBIR finding that 60% of breaches involved unpatched vulnerabilities underscores the critical importance of keeping your system up to date. I've had unattended-upgrades silently patch a libssl vulnerability on my Dell OptiPlex while I was asleep — I only found out because the /var/run/reboot-required flag was set the next morning. Automate this with unattended-upgrades:

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure --priority=low unattended-upgrades

For fine-grained control, edit /etc/apt/apt.conf.d/50unattended-upgrades:

Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
};

Unattended-Upgrade::AutoFixInterruptedDpkg "true";
Unattended-Upgrade::MinimalSteps "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::AutomaticReboot "false";

For Docker containers, use Watchtower to automatically update running containers:

docker run -d \
  --name watchtower \
  --restart unless-stopped \
  -v /var/run/docker.sock:/var/run/docker.sock \
  containrrr/watchtower \
  --schedule "0 0 4 * * *"   # 4 AM daily

Warning: Review update-testing on a staging instance before enabling automatic container restarts for production services.

5. Docker Security Best Practices

Docker adds a layer of isolation, but misconfiguration can actually weaken security. I run all my containers as a non-root user (UID 1000) with --cap-drop=ALL and read-only filesystems where possible — and Docker Bench Security has caught me slacking more than once. Follow these rules for every container you run:

5.1 Run as Non-Root User

Many official Docker images default to root. Override this in your docker-compose.yml:

services:
  myapp:
    image: myapp:latest
    user: "1000:1000"   # UID:GID of a non-root user

5.2 Drop All Capabilities, Add Only What's Needed

Linux capabilities grant privileged operations to containers. Drop all and add only what your app requires:

services:
  nginx:
    image: nginx:alpine
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE  # Allows binding to port 80

5.3 Read-Only Root Filesystem

Prevent attackers from writing malicious files to the container's filesystem:

services:
  myapp:
    image: myapp:latest
    read_only: true
    tmpfs:
      - /tmp   # Allow temp writes only in tmpfs

5.4 Never Mount the Docker Socket

Mounting /var/run/docker.sock into a container gives it root-level access to the Docker daemon — equivalent to full host compromise. Avoid this pattern unless you fully understand the implications. If you must (e.g., for Watchtower or Portainer), restrict it to a dedicated management container with minimal privileges.

5.5 Use Docker Bench Security

Run Docker Bench Security to audit your host and container configuration against the CIS Docker Benchmark:

docker run --rm --net host \
  -v /etc:/etc:ro \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  docker/docker-bench-security

This script checks 100+ security controls and highlights misconfigurations with remediation steps.

A real-world Docker gotcha: I once ran a container with the Docker socket mounted for a management tool, thinking it was safe because the container was "trusted." A routine docker-bench-security scan flagged it as a critical risk (CIS Docker Benchmark 5.1.3). I immediately moved to a socket-proxy pattern — using docker-socket-proxy with read-only, filtered access. Never mount /var/run/docker.sock unless you've exhausted every other option.

6. Mandatory Access Control: AppArmor & SELinux

Linux Discretionary Access Control (DAC) — standard file permissions — is the first security layer. Mandatory Access Control (MAC) systems like AppArmor (AppArmor 4.0 on Ubuntu 24.04, enabled by default) or SELinux (Fedora/CentOS default) add a second layer that constrains what processes can do even when running as root.

AppArmor profiles are pre-installed for common services. Check their status:

sudo aa-status

Enable AppArmor for a specific Docker container by appending a security-opt flag:

services:
  myapp:
    image: myapp:latest
    security_opt:
      - apparmor:docker-default

On SELinux-based systems, ensure it is enforcing:

getenforce
# Should output: Enforcing

# Set to enforcing if it's permissive
sudo setenforce 1

MAC systems would have prevented many high-profile container escape vulnerabilities — the extra layer catches exploits that bypass standard permissions.

7. File Integrity Monitoring with AIDE

If an attacker does gain access, you need to know what changed. AIDE (Advanced Intrusion Detection Environment) builds a database of file hashes and alerts you when files are modified.

# Install AIDE
sudo apt install aide -y

# Initialize the database (first run only)
sudo aideinit

# Move the database to its default location
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

# Run a check
sudo aide --check

Schedule weekly checks via cron:

sudo crontab -e
# Add this line:
0 3 * * 0 /usr/bin/aide --check | mail -s "AIDE report" your@email.com

8. Log Auditing and Centralized Monitoring

Logs tell the story of what happened. Without a monitoring system, you are flying blind.

8.1 Auditd for System Call Monitoring

Auditd records system calls, file accesses, and authentication events:

sudo apt install auditd -y
sudo auditctl -e 1   # Enable auditing
sudo aureport --summary   # View summary report

Monitor /etc/passwd for unauthorized changes:

sudo auditctl -w /etc/passwd -p wa -k passwd_changes
sudo auditctl -w /etc/shadow -p wa -k shadow_changes

View audit logs with ausearch:

sudo ausearch -k passwd_changes --start today

8.2 Centralized Logging with Rsyslog

Forward logs to a remote log server (or a syslog-compatible service) so attackers cannot tamper with them after gaining access:

# In /etc/rsyslog.conf
*.* @logs.example.com:514   # UDP forwarding
# Or over TCP with TLS for sensitive environments

8.3 Integration with Monitoring Tools

Pair security hardening with a monitoring stack from our Monitoring Your Homelab guide. Tools like Grafana, Prometheus, and Loki can visualize failed SSH attempts, firewall drops, and system resource anomalies that may indicate compromise.

9. Additional Hardening Measures

9.1 Disable Unused Services

Every running service is a potential attack surface. I run this check on every new machine — including my WSL2 environment, which ships with a surprising number of services enabled by default. List listening ports and audit them:

sudo ss -tulpn   # List all listening TCP and UDP ports
sudo lsof -i -P -n | grep LISTEN

Disable and mask any service you don't need:

sudo systemctl disable --now avahi-daemon
sudo systemctl mask avahi-daemon

9.2 Kernel Hardening with sysctl

Add these lines to /etc/sysctl.d/99-hardening.conf:

# IP spoofing protection
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

# Ignore source-routed packets
net.ipv4.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0

# Disable IPv6 if not needed, or only enable privacy extensions
net.ipv6.conf.all.use_tempaddr = 2

# Increase SYN backlog and enable SYN cookies (DoS protection)
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_syn_backlog = 1024

Apply immediately:

sudo sysctl -p /etc/sysctl.d/99-hardening.conf

9.3 Two-Factor Authentication for SSH

For an additional layer beyond SSH keys, enable TOTP-based 2FA using libpam-google-authenticator:

sudo apt install libpam-google-authenticator -y
google-authenticator    # Follow the setup, scan QR code into your authenticator app

Edit /etc/pam.d/sshd to add:

auth required pam_google_authenticator.so

Then in /etc/ssh/sshd_config:

AuthenticationMethods publickey,keyboard-interactive

Now SSH requires both your private key and a time-based one-time password — defense in depth at its finest.

Heads-up from experience: Setting up libpam-google-authenticator on my Debian 11 ARM64 box, I forgot to test the 2FA flow in a second SSH session before closing the first one. I had to reboot into single-user mode to undo the PAM changes. Always keep a second SSH session open when testing PAM or SSH config changes — or better yet, use at now + 5 minutes to queue a revert command, just in case.

10. Summary

Securing a self-hosted server is not a one-time task — it is an ongoing practice. The measures in this guide form a layered defense that protects against automated attacks, credential theft, unpatched vulnerabilities, and container escapes.

Here is a deployment checklist for a new self-hosted server:

Implement these steps in order, testing each change before moving to the next. Automate where possible — cron jobs for AIDE checks, Watchtower for Docker updates, and periodic lynis audits ensure your server stays hardened even when you are not actively managing it. My personal routine: every Sunday morning I SSH into each box, run sudo apt update && sudo unattended-upgrades --dry-run, check journalctl -u fail2ban | grep Ban for unusual patterns, and glance at Grafana dashboards for anomalies.

As the Verizon 2025 DBIR reminds us, the overwhelming majority of breaches exploit known vulnerabilities and weak credentials. By eliminating both categories, you move your self-hosted server from an easy target to a hardened system that automated attackers will skip in favor of easier prey. My Debian 11 ARM64 box has been running for 14 months without a single unauthorized access — and that peace of mind comes from spending a few focused hours upfront, then automating the rest.

FAQ

What is the most important security step for a new self-hosted server?

Disabling password-based SSH authentication and using SSH key pairs only. This single measure blocks the vast majority of automated brute-force attacks. Combined with changing the default SSH port and installing fail2ban, you eliminate 99% of opportunistic attacks before they reach your services. When I set up my Dell OptiPlex, this was the very first thing I configured — before even installing Docker.

Do I need a firewall if my server is behind a home router?

Yes. A host-based firewall like UFW or nftables provides defense-in-depth. Even behind a NAT router, internal threats (compromised IoT devices on your LAN, malware, or a misconfigured service binding to 0.0.0.0) can expose ports to your local network. A firewall ensures only the services you intend to expose are reachable. I run UFW on all three of my environments — including WSL2, which doesn't have a router-level firewall to fall back on.

How often should I update my self-hosted server?

Security patches should be applied within 24-48 hours of release for critical vulnerabilities. Unattended-upgrades can handle this automatically for apt packages. Docker containers should be rebuilt and redeployed weekly via a cron job or watchtower. The Verizon 2025 Data Breach Investigations Report found that 60% of breaches involved unpatched known vulnerabilities where a patch was available but not applied. My setup: unattended-upgrades handles OS patches automatically, and I manually trigger Docker container rebuilds every Saturday morning after checking the release notes.

Is fail2ban still necessary if I use SSH keys?

Yes. While SSH keys eliminate password-guessing attacks on SSH, fail2ban protects other services too — web apps, mail servers, and any service that accepts authenticated connections. It monitors log files for repeated failures and temporarily bans offending IPs via the firewall. On my ARM64 Debian box, fail2ban jails for Nginx have blocked several credential-stuffing attempts against a self-hosted web app that I wouldn't have noticed otherwise.

Should I use Docker for better security isolation?

Docker provides significant isolation benefits when configured correctly. Each container runs in its own namespaces with limited capabilities by default. However, Docker is not a security panacea — you must avoid privilege escalation with --privileged flags, use read-only root filesystems where possible, drop unnecessary Linux capabilities, and never mount the Docker socket into containers unless absolutely required. I learned this the hard way when Docker Bench Security flagged my Grafana container for having NET_RAW capability — a small thing, but exactly the kind of unnecessary privilege that container escape exploits capitalize on.

Related Articles

← Back to Home

About the Author

Frankie is a self-hosting enthusiast and DevOps practitioner with over 8 years of experience deploying and maintaining production Linux servers, Docker environments, and homelab infrastructure. He writes practical, hands-on guides to help others take control of their own infrastructure.