SSH Tutorial — Keys, Config, and Working Remotely

SSH Tutorial: Keys, Config, and Working Remotely#

By the end of this post you'll have generated an SSH key, set up passwordless login to a remote server, configured ~/.ssh/config so you can type ssh prod instead of remembering hostnames and usernames, and you'll know how to debug the most common SSH errors. About 25 minutes.

You'll need a Linux or macOS terminal and access to a remote server (a VPS, a cloud VM, or a coworker's machine — anything you can SSH to).

What SSH actually does#

SSH (Secure Shell) gives you a remote terminal session over an encrypted connection. The encryption is the easy part — every modern OS handles it. The interesting part is how you authenticate: prove to the remote server that you are who you claim to be.

Two common methods:

• Password — type a password every time. Works, but vulnerable to brute force, phishing, and reuse.
• Public key — you have a key pair (private key on your machine, public key on the server). The server challenges your client; the client signs the challenge with the private key; the server verifies with the public key. No password is ever sent.

Public-key auth is the right default. Faster (no typing), more secure (the private key never leaves your machine), and the foundation for almost all server automation.

Step 1: Generate an SSH key#

Check if you already have one:

ls ~/.ssh/

If you see id_ed25519 and id_ed25519.pub (or id_rsa and id_rsa.pub), you have a key pair already. Skip to Step 2 with that key.

Otherwise, generate one:

ssh-keygen -t ed25519 -C "your-email@example.com"

When it asks where to save, accept the default (~/.ssh/id_ed25519). When it asks for a passphrase, set one — a passphrase encrypts the private key on disk. If your laptop is stolen, the attacker still can't use the key without the passphrase. (We'll set up ssh-agent so you don't type it constantly.)

ed25519 is a modern, fast, secure key type. Use it instead of rsa for new keys.

You should now have two files:

ls -l ~/.ssh/id_ed25519*

-rw-------  1 alice  staff   411 Apr 28 10:00 id_ed25519
-rw-r--r--  1 alice  staff   100 Apr 28 10:00 id_ed25519.pub

The private one is 600 (only you can read it). The public one (.pub) is 644 — fine for it to be readable; that's its job.

Step 2: Copy the public key to the remote server#

The simplest way:

ssh-copy-id user@your-server.example.com

It'll prompt for the password one last time, then append your public key to ~/.ssh/authorized_keys on the remote server. Done.

If ssh-copy-id isn't available (some minimal systems), do it manually:

cat ~/.ssh/id_ed25519.pub | ssh user@your-server.example.com \
  'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

Now test:

ssh user@your-server.example.com

You should get a shell on the remote server with no password prompt (you may be asked for your passphrase once, which is different — that's unlocking the local key, not authenticating remotely).

Step 3: Use ssh-agent so you don't type the passphrase#

ssh-agent is a small program that holds your unlocked private key in memory for the session, so you only type the passphrase once.

On macOS, the agent runs by default. On Linux, you may need to start it:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Type your passphrase when prompted. Now ssh user@server works without a passphrase prompt for the rest of your session.

Persistent setup: most modern shells (zsh, bash with sshd config) auto-start the agent. If yours doesn't, add eval "$(ssh-agent -s)" to ~/.zshrc or ~/.bashrc.

On macOS, you can have the agent remember the passphrase in Keychain:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

Now you don't even type it once per session — Keychain unlocks the key.

Step 4: Configure ~/.ssh/config#

Typing ssh -p 2222 -i ~/.ssh/staging_key alice@staging.example.com every time is painful. The config file lets you alias.

Create or edit ~/.ssh/config:

Host prod
    HostName prod-web-1.example.com
    User alice
    IdentityFile ~/.ssh/id_ed25519

Host staging
    HostName staging.example.com
    User alice
    Port 2222
    IdentityFile ~/.ssh/staging_key

Host bastion
    HostName bastion.example.com
    User alice

Host *.internal
    User alice
    ProxyJump bastion

Set permissions:

chmod 600 ~/.ssh/config

Now:

• ssh prod connects to prod-web-1.example.com as alice
• ssh staging connects on port 2222 with the staging key
• ssh db1.internal automatically jumps through bastion first (the ProxyJump line)

ssh prod is much easier to remember than the full command. Plus you can scp file prod:/tmp/, rsync ... prod:, etc — all the SSH-using tools respect the config.

Step 5: Common operations#

Run a single command, no interactive shell:

ssh prod 'df -h'

Copy files (note: scp is being deprecated in favor of rsync or modern scp):

rsync -av ./localdir/ prod:/tmp/remotedir/
rsync -av prod:/var/log/myapp.log .

Port forwarding — make a remote port available locally:

ssh -L 5432:localhost:5432 prod
# Now your laptop's localhost:5432 forwards to prod's localhost:5432

Useful for tunneling to a remote database for debugging.

Background SSH — run a command and disconnect:

ssh prod 'nohup ./long-running-job.sh &'

Step 6: When something goes wrong#

The -v flag (or -vv, -vvv) shows what SSH is actually trying:

ssh -v prod

You'll see which keys it's offering, which auth methods the server accepts, where authentication fails. Surprisingly readable once you've seen it once.

The most common errors:

"Permission denied (publickey)." Server rejected your key. Either the public key isn't in the server's ~/.ssh/authorized_keys, the server's authorized_keys has wrong permissions (must be 600), or ~/.ssh on the server has wrong permissions (must be 700). Use ssh-copy-id again or fix permissions manually.

"Connection refused." Nothing's listening on the port. Either the server is down, sshd isn't running, or you have the wrong port. Try nc -zv server.example.com 22 to test reachability.

"Connection timed out." Network problem. Firewall blocking port 22, server unreachable, or wrong IP. Verify with ping server.example.com.

"Host key verification failed." The server's host key has changed (someone reinstalled, or you're being attacked). Don't blindly accept — verify with the server admin first. If legitimate, remove the old entry: ssh-keygen -R server.example.com.

Common mistakes#

Setting ~/.ssh or ~/.ssh/id_* to too-loose permissions. SSH refuses to use a private key with permissions other than 600 (or 700 on the directory). If SSH says "permissions are too open," chmod 600 ~/.ssh/id_ed25519.

Sharing a private key. The private key is yours alone. If a teammate needs access, they generate their own key pair and add their public key to the server. Sharing private keys means you can't audit who did what.

Disabling host key checking with -o StrictHostKeyChecking=no. That defeats a real security check. The error usually means the server's identity changed — investigate, don't bypass.

Using rsa for new keys. RSA still works but ed25519 is faster, smaller, and more secure. Use ed25519 for anything new.

What to read next#

You've got the basics. The next levels:

• Linux security hardening — the broader server-hardening checklist (which includes SSH config tightening)
• Linux file permissions — for when SSH errors trace back to permission issues
• Cloud security best practices: securing AWS infrastructure — replacing SSH with SSM Session Manager on AWS, and other modern alternatives

SSH is one of those skills that compounds. The 25 minutes you spent setting up keys and config saves you a few seconds every single SSH command for the rest of your career. Worth it.