How to Set Up SSH Config, Multiple Keys, and Host Aliases for Safer Daily Admin Work

Before you begin

• A Linux, macOS, or Windows terminal with OpenSSH available.
• At least one working SSH connection already set up to a server or Git host.
• Permission to create or use separate SSH keys for different systems.
• Basic familiarity with editing text files and using your home directory.

This guide is intentionally about the client side of SSH, not how to install SSH on a server from scratch. The goal is to make your daily operator workflow cleaner once SSH already exists in your life.

Step 1: Understand what SSH config solves

Without a config file, people often rely on shell history or memory:

ssh -i ~/.ssh/id_ed25519_clientA -p 2222 ubuntu@203.0.113.10

That works until you manage several servers, several Git accounts, or several usernames. Then mistakes become common:

• You connect to the wrong host because two IPs look similar.
• You offer the wrong key and hit confusing authentication failures.
• You copy a command from history and accidentally run production work on staging.

~/.ssh/config lets you define those details once. After that, you connect using stable host aliases that are easier to recognize under pressure.

Step 2: Create or review separate SSH keys

If you already use one key everywhere, this is a good time to improve separation. A practical pattern is one key for personal Git hosting, one for work Git hosting, and separate keys for especially sensitive servers when needed.

Create a new Ed25519 key:

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_prod_admin -C "prod-admin"

You can repeat that pattern with different filenames:

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_staging -C "staging"
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_github_work -C "github-work"

Then set strict permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519_prod_admin ~/.ssh/id_ed25519_staging ~/.ssh/id_ed25519_github_work
chmod 644 ~/.ssh/*.pub

Copy each public key to the correct destination. For servers, that usually means adding the .pub content to the target user’s ~/.ssh/authorized_keys. For Git hosts, add the public key in the account’s SSH keys settings.

Step 3: Build a clean ~/.ssh/config

Create the file if it does not exist:

touch ~/.ssh/config
chmod 600 ~/.ssh/config

Start with a small, readable structure:

Host prod-web
  HostName 203.0.113.10
  User ubuntu
  Port 22
  IdentityFile ~/.ssh/id_ed25519_prod_admin
  IdentitiesOnly yes

Host staging-web
  HostName 203.0.113.20
  User ubuntu
  Port 22
  IdentityFile ~/.ssh/id_ed25519_staging
  IdentitiesOnly yes

Host github-work
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_github_work
  IdentitiesOnly yes

What these lines do:

• Host is your local alias, the short name you will type.
• HostName is the real IP or DNS name.
• User is the remote login name.
• IdentityFile points to the matching private key.
• IdentitiesOnly yes tells SSH not to spray every loaded key at the server.

If you want a small set of safe defaults for all hosts, add a general block at the top:

Host *
  ServerAliveInterval 30
  ServerAliveCountMax 3
  AddKeysToAgent yes
  IdentitiesOnly yes

Be careful with global settings. Anything in Host * applies everywhere unless a later block overrides it. On some platforms, AddKeysToAgent behaves a little differently depending on which SSH agent is running, so treat it as a convenience option rather than a requirement.

For GitHub personal and work accounts on the same machine, aliases are especially useful:

Host github-personal
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_github_personal
  IdentitiesOnly yes

Host github-work
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_github_work
  IdentitiesOnly yes

Then use the alias in the Git remote URL, for example:

git remote set-url origin git@github-work:company/project.git

Step 4: Test and verify that each alias uses the right key

Start with SSH’s built-in config inspection:

ssh -G prod-web | grep -E 'hostname|user|identityfile'

Then make a verbose test connection:

ssh -v prod-web

Look for lines showing the selected identity file and the target host. For Git hosts, test authentication without opening a shell:

ssh -T github-work

If you use ssh-agent, list currently loaded keys:

ssh-add -l

Add a key manually if needed:

ssh-add ~/.ssh/id_ed25519_prod_admin

Expected outcome: Each alias resolves to the correct hostname, correct username, and correct key without extra command flags.

Step 5: Use habits that reduce wrong-host mistakes

Good SSH config helps, but naming and workflow matter too. A few practical habits make a big difference:

• Use explicit aliases like prod-web, staging-db, and home-nas instead of vague names like server1.
• Keep production aliases visually obvious so they stand out in shell history.
• Prefer DNS names when stable, but aliases still help when the real names are long or inconsistent.
• Store notes about what each host is for in comments inside ~/.ssh/config.

You can also use small quality-of-life options carefully. For example:

Host prod-web
  HostName 203.0.113.10
  User ubuntu
  IdentityFile ~/.ssh/id_ed25519_prod_admin
  IdentitiesOnly yes
  LocalForward 127.0.0.1:9000 127.0.0.1:9000

That kind of setup can be helpful, but keep it deliberate. Hidden forwards or unusual defaults can confuse future-you if they are not documented.

Step 6: Confirm the final setup

By the end, you should have:

• A readable ~/.ssh/config file with named host aliases
• Separate keys where separation meaningfully improves safety or clarity
• Verified login behavior for each important host
• A simpler daily workflow for SSH and Git over SSH

This is one of the highest-value small upgrades for anyone who touches multiple systems regularly.

Troubleshooting common SSH config problems

The alias still uses the wrong key.
Check for another matching Host block earlier in the file, and make sure IdentitiesOnly yes is present for the host.

I get “Permission denied (publickey)”.
Confirm the public key was added to the correct server account or Git account, and check that the private key path in IdentityFile is correct.

The config file is ignored.
Verify the file is at ~/.ssh/config and has reasonable permissions like 600.

Git still pushes to the wrong account.
Check the remote URL with git remote -v. If it still points to git@github.com instead of your alias such as git@github-work, SSH cannot apply the alternate host block.

I changed a key and locked myself out of a server.
Use another working session or your provider console, restore the old public key entry, and only remove it after the replacement is verified.

What to do next

Once your SSH client workflow is clean, the next useful skill is securely reaching private tools through the same connection. Continue with How to Use SSH Port Forwarding for Secure Access to Private Web UIs and Databases.