How to Secure SSH on a Linux VPS

The short answer

SSH is how you manage a Linux VPS, and it is the first service attackers look for. Automated bots scan the internet for SSH on TCP port 22 and try common usernames and leaked passwords on anything that answers, all day, every day. No single setting fixes that. What works is a few layers: sign in with keys instead of passwords, stop logging in as root, limit who can reach SSH at all, block repeated failures, keep the server patched, and read the logs.

This guide walks through those layers on a Linux VPS in the order we recommend doing them. The commands are for Ubuntu and Debian, with notes for Rocky Linux, AlmaLinux and other Red Hat family systems where they differ. If you run Windows instead, see our Remote Desktop security guide.

Before you change anything: keep a second way in

SSH changes are the classic way to lock yourself out of your own server. Two habits prevent that:

• Keep your current session open while you make changes, and test every change from a second, new terminal window. If the new login fails, you still have the old session to fix it.
• Know your out of band option. Confirm you can reach the server some other way that does not depend on SSH, such as a console from your hosting provider. If you are not sure what your plan includes, ask us before you start.

Step 1: Switch to SSH keys

A password can be guessed. A modern SSH key cannot be guessed in any practical sense, and it never travels to the server. On your own computer (Windows 10 and 11, macOS and Linux all include an SSH client), create a key:

ssh-keygen -t ed25519 -C "you@yourcompany.com"

Give it a passphrase when asked. The passphrase protects the key file if your laptop is lost or stolen. Then copy the public half to the server:

ssh-copy-id youruser@your-server-address

If ssh-copy-id is not available (it is not built into Windows), paste the contents of your .pub file as a new line in ~/.ssh/authorized_keys on the server. Now open a new terminal and confirm you can sign in with the key before going further.

Step 2: Use a named account and turn off root and password logins

• One person, one account. Give every person who needs access their own Linux user with their own key, and use sudo for admin work. When someone leaves, you remove one account instead of changing a shared password everywhere, and the logs show who did what.
• Turn off direct root login. Attackers know the root username exists on every Linux server, so it is the first one they try.
• Turn off password logins once every user has a working key.

Create a new file for your settings, for example /etc/ssh/sshd_config.d/00-hardening.conf, with these lines:

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes

Why a file in sshd_config.d? On current Ubuntu and Debian releases, the main config file includes that folder near the top, and for most settings SSH uses the first value it reads. Some cloud images ship a file in that folder that turns password logins back on, so a file whose name sorts first wins. On Red Hat family systems the same folder approach works on current releases; if your system has no such folder, put the lines in /etc/ssh/sshd_config itself.

Check the config and see the values SSH will actually use:

sudo sshd -t
sudo sshd -T | grep -Ei '^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication) '

The first command prints nothing when the config is valid. If the second command shows the values you set, restart the service with sudo systemctl restart ssh on Ubuntu and Debian or sudo systemctl restart sshd on Red Hat family systems. Your open session stays connected. Then test a fresh login from a second terminal before you close anything.

To go further, an AllowUsers line lists exactly which accounts may sign in over SSH, and anyone not listed is refused even with a valid key.

Step 3: Limit who can reach SSH

The strongest improvement after keys is making SSH unreachable for everyone except the people who need it.

• Best: put SSH behind a VPN. Your team connects to the VPN first, and SSH only accepts connections from VPN addresses. Nothing on the open internet ever sees the port.
• Good: allow only known IP addresses. If your office has a static IP, allow SSH from that address only.

On Ubuntu with UFW, add the allow rule first and only then turn the firewall on (replace the example address with your own):

sudo ufw allow from 203.0.113.10 to any port 22 proto tcp
sudo ufw enable
sudo ufw status numbered

On Red Hat family systems, firewalld does the same job with a rich rule or a dedicated zone for your trusted addresses. Whichever tool you use, keep your existing session open and test from a second terminal before you log out.

If your team works from home connections whose IP changes, a VPN is a better fit than chasing addresses in firewall rules.

What about changing the port? Moving SSH off port 22 cuts down noise in your logs from the laziest bots, but anyone who scans every port will still find it. Treat a new port as tidying, not as protection.

Step 4: Block repeated failures with Fail2ban

Fail2ban watches your sign in logs and temporarily blocks addresses that keep failing. On Ubuntu and Debian:

sudo apt install fail2ban

Then create /etc/fail2ban/jail.local so your settings survive package updates:

[sshd]
enabled = true

If your system logs to the systemd journal instead of /var/log/auth.log (current Debian releases do this by default), add backend = systemd under [sshd]. Restart Fail2ban and check that the jail is running:

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

On Red Hat family systems, Fail2ban is available from the EPEL repository. Add your own office or VPN address to ignoreip so a few typos never lock you out.

Step 5: Keep the server patched

SSH hardening does not help if the software underneath has a known hole. Turn on automatic security updates:

• Ubuntu and Debian: sudo apt install unattended-upgrades, then sudo dpkg-reconfigure -plow unattended-upgrades and answer yes.
• Rocky, Alma and other Red Hat family systems: install dnf-automatic and enable its timer.

Some updates, such as a new kernel, only take effect after a reboot. Plan a regular maintenance window so reboots happen on your schedule.

Step 6: Read the logs

A few commands tell you who has been knocking and who got in:

sudo journalctl -u ssh --since today
last -a | head
sudo lastb | head

On Red Hat family systems the service is named sshd, so use journalctl -u sshd. Look for successful logins you do not recognize, sign ins at odd hours, or logins from countries where nobody on your team works. A failed login flood on its own is normal background noise once passwords are off; a successful login you cannot explain is what matters.

Optional: a second factor for SSH

Keys with passphrases already give you something you have and something you know. If your policy calls for a separate one time code as well, SSH can require a time based code from an authenticator app through a PAM module. Set it up carefully and keep a working session open while you test, because a mistake here locks out everyone.

Quick checklist

Layer	What to set
Keys	ed25519 key with a passphrase for every person
Accounts	Named users with sudo, no shared logins
Root	PermitRootLogin no
Passwords	PasswordAuthentication no once keys work
Reachability	VPN or firewall allow list for SSH
Failures	Fail2ban sshd jail enabled
Patching	Automatic security updates on, planned reboots
Logs	Review SSH logins regularly

Where GlassHosting fits

These steps apply to any Linux VPS, including GlassHosting KVM and LXC plans. If you would rather not own this checklist yourself, that is exactly what managed hosting is for. Read managed vs unmanaged VPS to see the difference, then tell us what you run or email sales@glasshosting.com, and we will help you lock it down without locking yourself out.