SSH: Secure Remote Access

Why This Matters

Before SSH existed, system administrators used Telnet and rsh to manage remote servers. Every keystroke -- including passwords -- was sent across the network in plain text. Anyone on the same network could read everything with a simple packet capture.

SSH (Secure Shell) changed everything. It encrypts all communication between your machine and the remote server. Today, SSH is so fundamental that it is practically impossible to do Linux administration without it. If you manage even one remote server, you are using SSH. If you manage hundreds, you need to understand SSH deeply -- key-based authentication, tunneling, agent forwarding, and hardening.

This chapter takes you from "I can ssh into a server" to "I understand SSH well enough to build secure, efficient remote access infrastructure."

Try This Right Now

# Check if SSH client is installed
ssh -V

# Check if SSH server is running on your machine
systemctl status sshd

# List your existing SSH keys (if any)
ls -la ~/.ssh/

# Try connecting to localhost (if sshd is running)
ssh localhost

How SSH Works

When you type ssh user@server, a lot happens before you see that command prompt.

The Connection Process

  Client                              Server
    |                                   |
    |  1. TCP connection (port 22)      |
    |---------------------------------->|
    |                                   |
    |  2. Protocol version exchange     |
    |<--------------------------------->|
    |                                   |
    |  3. Key exchange (Diffie-Hellman) |
    |<--------------------------------->|
    |  (Both sides now have a shared    |
    |   session key for encryption)     |
    |                                   |
    |  4. Server authentication         |
    |<----------------------------------|
    |  (Server proves identity with     |
    |   its host key)                   |
    |                                   |
    |  5. User authentication           |
    |---------------------------------->|
    |  (Password, public key, etc.)     |
    |                                   |
    |  6. Encrypted session begins      |
    |<=================================>|
    |                                   |

Key concepts:

Key Exchange: Both sides negotiate a shared secret using Diffie-Hellman. This secret is used to encrypt the session. Even if someone captures all the traffic, they cannot derive the encryption key.
Host Key: The server has a unique key pair. The first time you connect, SSH asks you to verify the fingerprint. This fingerprint is stored in ~/.ssh/known_hosts. If it changes later, SSH warns you -- this could mean a man-in-the-middle attack.
User Authentication: After the encrypted channel is established, you prove your identity (password, key, certificate, etc.).

The "Host Key Changed" Warning

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

This warning means the server's host key is different from what you have stored. This happens when:

The server was reinstalled
The server's IP was reassigned to a different machine
Someone is attempting a man-in-the-middle attack

If you are sure the change is legitimate (e.g., you just reinstalled the server):

# Remove the old key for that host
ssh-keygen -R hostname_or_ip

# Then connect again, and you'll be prompted to accept the new key
ssh user@hostname

SSH Basics

Connecting to a Remote Server

# Basic connection (uses your current username)
ssh 192.168.1.100

# Specify a username
ssh admin@192.168.1.100

# Connect on a non-standard port
ssh -p 2222 admin@192.168.1.100

# Verbose mode (for debugging connection problems)
ssh -v admin@192.168.1.100

# Extra verbose (more detail)
ssh -vv admin@192.168.1.100

Running a Single Command

# Run a command on the remote server without opening a shell
ssh admin@server 'uptime'

# Run multiple commands
ssh admin@server 'hostname && uptime && df -h'

# Run a command that needs a TTY (e.g., sudo, top)
ssh -t admin@server 'sudo systemctl restart nginx'

The -t flag forces a pseudo-terminal allocation, which is needed for interactive commands.

Key-Based Authentication

Password authentication works but has serious drawbacks:

Passwords can be brute-forced
Passwords can be phished or shoulder-surfed
Passwords are annoying to type hundreds of times a day
Passwords cannot be used for automated scripts

Key-based authentication uses public-key cryptography. You generate a key pair:

Private key: Lives on your machine. Never share it with anyone.
Public key: Goes on every server you want to access. Safe to share.

+-------------------+          +-------------------+
|    Your Machine   |          |   Remote Server   |
|                   |          |                   |
|  ~/.ssh/id_ed25519|          | ~/.ssh/            |
|  (PRIVATE KEY)    |  proves  |  authorized_keys  |
|  Keep this secret!|--------->|  (PUBLIC KEY)     |
|                   |          |                   |
+-------------------+          +-------------------+

Generating a Key Pair

# Generate an Ed25519 key (recommended, modern, fast)
ssh-keygen -t ed25519 -C "your_email@example.com"

# You will be prompted:
# Enter file in which to save the key (/home/user/.ssh/id_ed25519):
# Enter passphrase (empty for no passphrase):

You should set a passphrase. It encrypts your private key on disk, so even if someone steals the file, they cannot use it without the passphrase.

If you need RSA compatibility (older systems):

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Copying Your Public Key to a Server

# The easy way (recommended)
ssh-copy-id admin@server

# This copies your public key to the server's ~/.ssh/authorized_keys
# and sets the correct permissions automatically.

If ssh-copy-id is not available:

# Manual method
cat ~/.ssh/id_ed25519.pub | ssh admin@server 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

Verifying It Works

# This should log you in without a password
ssh admin@server

# If you set a passphrase, it will ask for the passphrase (not the server password)

The `authorized_keys` File

On the server, each user's authorized keys live in ~/.ssh/authorized_keys. Each line is one public key:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHx... user@laptop
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJy... user@desktop
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ... admin@workstation

You can add options before each key to restrict it:

# Only allow this key from a specific IP
from="10.0.0.5" ssh-ed25519 AAAAC3...

# Only allow this key to run a specific command
command="/usr/local/bin/backup.sh" ssh-ed25519 AAAAC3...

# Disable forwarding for this key
no-port-forwarding,no-agent-forwarding ssh-ed25519 AAAAC3...

Think About It: Why is it important to set correct file permissions on the .ssh directory and its contents?

SSH is strict about permissions for security. If ~/.ssh is world-readable or authorized_keys is writable by others, SSH will refuse to use them. The correct permissions are:

~/.ssh/ -- 700 (drwx------)
~/.ssh/authorized_keys -- 600 (-rw-------)
~/.ssh/id_ed25519 (private key) -- 600 (-rw-------)
~/.ssh/id_ed25519.pub (public key) -- 644 (-rw-r--r--)

SSH Config File

Typing ssh -p 2222 -i ~/.ssh/special_key admin@long-server-name.example.com every time is tedious. The SSH config file (~/.ssh/config) solves this.

# ~/.ssh/config

# Production web server
Host web-prod
    HostName 203.0.113.50
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_work

# Database server (only reachable through web-prod)
Host db-prod
    HostName 10.0.0.50
    User dba
    ProxyJump web-prod

# Development machines
Host dev-*
    User developer
    Port 22
    IdentityFile ~/.ssh/id_ed25519_dev

Host dev-app
    HostName 192.168.1.10

Host dev-api
    HostName 192.168.1.11

# Default settings for all hosts
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentitiesOnly yes

Now you can simply type:

ssh web-prod     # instead of ssh -p 2222 -i ~/.ssh/id_ed25519_work admin@203.0.113.50
ssh db-prod      # automatically jumps through web-prod
ssh dev-app      # uses developer@192.168.1.10

Important Config Options

Option	Purpose
`HostName`	The actual hostname or IP
`User`	Username to connect as
`Port`	SSH port
`IdentityFile`	Path to private key
`ProxyJump`	Jump through another host (bastion/jump box)
`ServerAliveInterval`	Send keepalive every N seconds
`ServerAliveCountMax`	Disconnect after N missed keepalives
`ForwardAgent`	Forward SSH agent to remote host
`LocalForward`	Set up a local port forward
`IdentitiesOnly`	Only use the specified key, not all in agent

SSH Agent

If you have a passphrase on your key (and you should), typing it every time gets old fast. The SSH agent stores your decrypted private key in memory so you only type the passphrase once per session.

# Start the SSH agent (often already running in desktop environments)
eval "$(ssh-agent -s)"

# Add your key to the agent
ssh-add ~/.ssh/id_ed25519
# Enter passphrase once

# List keys in the agent
ssh-add -l

# Remove all keys from the agent
ssh-add -D

Agent Forwarding

Agent forwarding lets you use your local SSH keys on a remote server without copying the private key to that server.

You (laptop)  --->  Jump box  --->  Internal server
   [key]         [no key needed]    [authenticates with
                 [agent forwarded]    your key via agent]

# Enable agent forwarding for a connection
ssh -A user@jumpbox

# Or in ~/.ssh/config:
Host jumpbox
    ForwardAgent yes

Safety Warning: Only enable agent forwarding to servers you trust. A compromised server with access to your forwarded agent could use your keys to connect to other servers. Use ProxyJump instead of agent forwarding when possible -- it is more secure because the jump host never sees your keys.

Port Forwarding (SSH Tunnels)

SSH tunnels are one of the most useful and underappreciated features of SSH. They let you securely access services that are behind firewalls or only listening on localhost.

Local Port Forwarding

"Make a remote service available on my local machine."

Your machine (localhost:8080) --[SSH tunnel]--> Remote (localhost:5432)

You access localhost:8080, traffic goes through the SSH tunnel
to the remote server, which connects to its own localhost:5432.

# Forward local port 8080 to remote's localhost:5432 (PostgreSQL)
ssh -L 8080:localhost:5432 admin@remote-server

# Now you can connect to PostgreSQL at localhost:8080 from your machine
psql -h localhost -p 8080 -U myuser mydb

More specific syntax:

# Forward local port 3307 to a database at 10.0.0.50:3306 via jump-server
ssh -L 3307:10.0.0.50:3306 admin@jump-server

# The database is not directly reachable from your machine,
# but jump-server can reach it.
# You connect to localhost:3307, which tunnels through.

Remote Port Forwarding

"Make a local service available on the remote machine."

# Make your local web server (port 3000) available on remote port 9000
ssh -R 9000:localhost:3000 admin@remote-server

# Anyone who can reach remote-server:9000 can now access
# your local machine's port 3000 through the tunnel.

This is useful for:

Exposing a local development server to a remote tester
Giving remote access to a service behind NAT

Dynamic Port Forwarding (SOCKS Proxy)

This creates a SOCKS proxy that routes all traffic through the SSH connection.

# Create a SOCKS5 proxy on local port 1080
ssh -D 1080 admin@remote-server

# Configure your browser or application to use SOCKS proxy localhost:1080
# All traffic will be routed through remote-server

This is useful for:

Browsing the web as if you were on the remote network
Accessing internal web applications
Bypassing geographic restrictions

Persistent Tunnels

# Keep the tunnel open in the background
ssh -f -N -L 8080:localhost:5432 admin@remote-server

# -f: Go to background after authentication
# -N: Don't execute a remote command (just the tunnel)

In your SSH config:

Host tunnel-db
    HostName remote-server
    User admin
    LocalForward 8080 localhost:5432
    # Connect with: ssh -f -N tunnel-db

File Transfer over SSH

`scp` -- Secure Copy

# Copy a file to a remote server
scp file.txt admin@server:/home/admin/

# Copy a file from a remote server
scp admin@server:/var/log/syslog ./

# Copy a directory recursively
scp -r ./project/ admin@server:/home/admin/

# Use a specific port
scp -P 2222 file.txt admin@server:/home/admin/

`sftp` -- Secure FTP

# Start an interactive SFTP session
sftp admin@server

# Inside sftp:
sftp> ls
sftp> cd /var/log
sftp> get syslog
sftp> put localfile.txt
sftp> mkdir new-directory
sftp> exit

`rsync` over SSH

rsync is the best tool for transferring files over SSH because it only transfers what has changed.

# Sync a directory to a remote server
rsync -avz ./project/ admin@server:/home/admin/project/

# Sync from remote to local
rsync -avz admin@server:/var/log/ ./logs/

# Use a specific SSH port
rsync -avz -e 'ssh -p 2222' ./data/ admin@server:/backup/

# Dry run (show what would be transferred)
rsync -avzn ./project/ admin@server:/home/admin/project/

# Delete files on destination that don't exist on source
rsync -avz --delete ./project/ admin@server:/home/admin/project/

Flags breakdown:

-a: Archive mode (preserves permissions, timestamps, symlinks, etc.)
-v: Verbose
-z: Compress during transfer
-n: Dry run
--delete: Remove files on destination not in source

Think About It: You need to transfer a 10 GB directory to a remote server. The connection drops halfway through. With scp, what happens? With rsync, what happens?

With scp, you have to start over. The partial transfer is there but scp does not know where it left off. With rsync, you run the same command again and it picks up where it left off, only transferring the remaining files.

Hardening SSH: `sshd_config`

The SSH server configuration lives at /etc/ssh/sshd_config. Here are the essential hardening steps for production servers.

# /etc/ssh/sshd_config
PermitRootLogin no

This forces admins to log in as a regular user and use sudo. It also eliminates the "root" username as a brute-force target.

Key-Only Authentication (Disable Passwords)

# /etc/ssh/sshd_config
PasswordAuthentication no
PubkeyAuthentication yes

Safety Warning: Before disabling password authentication, make ABSOLUTELY sure that key-based authentication works. Test it in a separate terminal first. If you disable passwords and your key does not work, you will be locked out.

Change the Default Port

# /etc/ssh/sshd_config
Port 2222

This is security through obscurity and does not stop determined attackers, but it eliminates 99% of automated brute-force attempts that target port 22.

Limit Users and Groups

# Only allow specific users
AllowUsers admin deployer

# Or allow by group
AllowGroups ssh-users

Other Hardening Options

# Disable empty passwords
PermitEmptyPasswords no

# Set a login grace period (time to authenticate)
LoginGraceTime 30

# Limit authentication attempts per connection
MaxAuthTries 3

# Disable X11 forwarding (if not needed)
X11Forwarding no

# Disable TCP forwarding (if not needed)
AllowTcpForwarding no

# Use only protocol 2 (protocol 1 is insecure and obsolete)
Protocol 2

# Specify allowed key exchange algorithms (modern only)
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org

# Specify allowed ciphers (modern only)
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com

# Specify allowed MACs (modern only)
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

Applying Changes

After editing sshd_config:

# Test the configuration for syntax errors (critical step!)
sudo sshd -t

# If no errors, reload the service
sudo systemctl reload sshd

Safety Warning: Always test the config with sshd -t before reloading. And always keep your current SSH session open while testing. Open a NEW terminal and try to connect. If the new connection works, you are safe. If it does not, you still have your existing session to fix things.

Hands-On: Complete SSH Setup

Let's walk through a complete SSH setup from scratch.

Step 1: Generate a key pair on your local machine

ssh-keygen -t ed25519 -C "admin@company.com"
# Accept the default path, set a strong passphrase

Step 2: Copy the public key to the server

ssh-copy-id admin@192.168.1.100
# Enter the server password one last time

Step 3: Test key-based login

ssh admin@192.168.1.100
# Should ask for your key passphrase, NOT the server password

Step 4: Set up the SSH agent

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
# Enter passphrase once

ssh admin@192.168.1.100
# No passphrase prompt this time

Step 5: Create an SSH config entry

cat >> ~/.ssh/config << 'EOF'
Host myserver
    HostName 192.168.1.100
    User admin
    IdentityFile ~/.ssh/id_ed25519
EOF

chmod 600 ~/.ssh/config

# Now just:
ssh myserver

Step 6: Harden the server (keep your current session open!)

ssh myserver
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
sudo vim /etc/ssh/sshd_config

Add or modify:

PermitRootLogin no
PasswordAuthentication no
MaxAuthTries 3

sudo sshd -t && sudo systemctl reload sshd

Step 7: Test from a new terminal

ssh myserver    # Should work with key
ssh -o PubkeyAuthentication=no myserver   # Should FAIL (password disabled)

Debug This

Scenario: You cannot SSH into a server. You have already verified that the server is reachable (ping works). Running ssh -v admin@server shows:

debug1: Connecting to server [192.168.1.100] port 22.
debug1: Connection established.
debug1: identity file /home/user/.ssh/id_ed25519 type 3
debug1: Authentications that can continue: publickey
debug1: Trying private key: /home/user/.ssh/id_ed25519
debug1: Authentication failed.
Permission denied (publickey).

What is happening and how do you fix it?

Diagnosis: The server only accepts public key authentication (no password). Your key is being offered but rejected. Possible causes:

Your public key is not in the server's ~/.ssh/authorized_keys
The permissions on the server's ~/.ssh directory are wrong
You are using the wrong key or the wrong username
The server's authorized_keys file is owned by root or has wrong permissions

Fix: If you have out-of-band access (console):

# On the server, check the auth log
sudo tail -20 /var/log/auth.log       # Debian/Ubuntu
sudo tail -20 /var/log/secure         # RHEL/CentOS

# Common log messages:
# "Authentication refused: bad ownership or modes for directory /home/admin/.ssh"
# Fix:
chmod 700 /home/admin/.ssh
chmod 600 /home/admin/.ssh/authorized_keys
chown -R admin:admin /home/admin/.ssh

What Just Happened?

+-------------------------------------------------------------------+
|                     Chapter 36 Recap                               |
+-------------------------------------------------------------------+
|                                                                   |
|  * SSH encrypts all communication using key exchange and          |
|    symmetric encryption. It replaced insecure protocols like      |
|    Telnet and rsh.                                                |
|                                                                   |
|  * Key-based auth is more secure than passwords. Use              |
|    ssh-keygen (Ed25519) and ssh-copy-id.                         |
|                                                                   |
|  * The SSH config file (~/.ssh/config) saves you from typing     |
|    long commands and organizes access to many servers.            |
|                                                                   |
|  * SSH agent caches your key passphrase in memory.               |
|    Agent forwarding lets you use your keys on remote servers.    |
|                                                                   |
|  * Port forwarding creates encrypted tunnels:                    |
|    -L (local), -R (remote), -D (dynamic/SOCKS).                 |
|                                                                   |
|  * File transfers: scp (simple), sftp (interactive),             |
|    rsync (incremental, resumable -- best for large transfers).   |
|                                                                   |
|  * Harden sshd_config: disable root login, disable passwords,   |
|    limit users, change port, use modern algorithms.              |
|                                                                   |
|  * Always test config changes from a new terminal while keeping  |
|    your current session open.                                    |
|                                                                   |
+-------------------------------------------------------------------+

Try This

Key rotation: Generate a new Ed25519 key pair. Add it to a server alongside your existing key. Verify both work. Remove the old key. This simulates key rotation.
SSH tunnel: Start a simple web server on a remote machine (python3 -m http.server 8000 on localhost). Use local port forwarding to access it from your laptop at localhost:9000. Verify with curl localhost:9000.
Jump host: If you have three machines (or VMs), configure the middle one as a jump host. Use ProxyJump in your SSH config to transparently reach the third machine through the second.
Security audit: Examine your current sshd_config. How many of the hardening recommendations from this chapter are already in place? Apply the ones that are missing on a test system.
Bonus challenge: Set up a dynamic SOCKS proxy with SSH and configure your web browser to use it. Visit a site like ifconfig.me to verify that your traffic is exiting from the remote server's IP address, not your own.

Linux Book: From First Boot to Production