SSH Tips

Client configuration

Files and file modes

  • Per-person directory ~/.ssh
  • Per-person configure file ~/.ssh/config
  • known hosts
  • authorized keys
  • private key
  • File mode (permission):
    • The directory and file mode should be correct, which generally means no one else can either peek or fake your id.
    • The file mode of home directory matters as well (? that seemed true once, but never reproduced).
    • NOTE If using a private key file owned by others (by -o IdentityFile=/path/to/id_rsa), its file mode doesnot matter.
    • An example settings:
      $ ls -ld $HOME $HOME/.ssh $HOME/.ssh/*
      drwx------ 23 lgfang typhoon  4096 Feb 18 11:54 /home/lgfang
      drwx------  2 lgfang typhoon  4096 Feb 10 11:23 /home/lgfang/.ssh
      -rw-r--r--  1 lgfang typhoon   402 Dec 14 09:49 /home/lgfang/.ssh/authorized_keys
      -rw-r--r--  1 lgfang typhoon   531 Feb 10 11:23 /home/lgfang/.ssh/config
      -rw-------  1 lgfang typhoon  1675 Dec  2 15:16 /home/lgfang/.ssh/id_rsa
      -rw-r--r--  1 lgfang typhoon   418 Dec  2 15:16 /home/lgfang/.ssh/
      -rw-r--r--  1 lgfang typhoon 12510 Feb 10 11:00 /home/lgfang/.ssh/known_hosts

An example of ~/.ssh/config

Here is my configuration file. For detailed explanation, read the following sections.

Host mydesktop
     User lungangfang
     # NOTE: do NOT set "ControlMaster Auto" as default for all
     ControlMaster auto

Host vm*
     User root
     IdentityFile ~/.ssh/id_autoit
     StrictHostKeyChecking no
     UserKnownHostsFile /dev/null
     LogLevel ERROR

Host *
     Protocol 2,1
     ServerAliveInterval 180
     ForwardX11 no
     ControlPath /tmp/%r_ssh_%h_%p
     User lgfang
     # Compression yes

It is a multi-match rather than a single-match.

  • Every entry matches takes effect.

    For instance, configurations for "host 135.*" and that for "host 135.252.41.*" both apply to

  • The first matched options take effect.

    If several entries all match and they all contain the option "user", then the value in the first matched entry is taken.

    NOTE: Hence, put general options at the of configure file.

Host alias

Do not like entering long, hard-to-remember server IPs? Or, want to hide the IP from you scripts?

You can set up host alias. Refer to my ~/.ssh/config

Default user

Want to save the key-stikes for user name?

Setup default user. Refer to my ~/.ssh/config


Enter password only once for multiple ssh sessions by "ControlMaster auto".

For example, assume "Host 135.*" has option "ControlMaster auto". The first ssh session to request password entered. After that, so long as the session alive, I can ssh that host without entering password again.


  1. "ControlPath" must set as well.
  2. Git, rysnc etc. would fail with 'ControlMaster auto'. Therefore, don not put "ControlMaster auto" under "Host *"

Keep sessions alive

It turned out that almost all of us (my teammates and I) overlooked this feature while we were suffering from broken sessions :), even though it seems we all knew for sure that it must be certain firewall or alike that closes sessions being idle for an hour.

The solution is actually just at hand: have ssh clients send keep-alive messages regularly, say, every 180 seconds.

  • If you ssh to a server from certain linux box, edit your SSH config at your linux box (the ssh client side) as the following.
    # in either /etc/ssh/ssh_config or ~/.ssh/config
    Host *
         ServerAliveInterval 180

    Of course, specifying the option in command line (as in ssh -o ServerAliveInterval=180 remote_host) also works.

  • If you are a PuTTY user instead, in the "connection" panel, set seconds between keepalives to 180.

NOTE: quote from here

Note that keepalives are not always helpful. They help if you have a firewall which drops your connection after an idle period; but if the network between you and the server suffers from breaks in connectivity then keepalives can actually make things worse. If a session is idle, and connectivity is temporarily lost between the endpoints, but the connectivity is restored before either side tries to send anything, then there will be no problem - neither endpoint will notice that anything was wrong. However, if one side does send something during the break, it will repeatedly try to re-send, and eventually give up and abandon the connection. Then when connectivity is restored, the other side will find that the first side doesn't believe there is an open connection any more. Keepalives can make this sort of problem worse, because they increase the probability that PuTTY will attempt to send data during a break in connectivity. (Other types of periodic network activity can cause this behaviour; in particular, SSH-2 re-keys can have this effect. See section 4.19.2.)

Host key conflict

What is this

The remote host (ssh server) sends a host key (The default is /etc/ssh/ for protocol version 1, and /etc/ssh/ or /etc/ssh/ for protocol version 2.).

The ssh client will compare that with the one stored in ~/.ssh/known_hosts. If they do not match, you'll get an error like the following.

$ ssh
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
Offending key in /home/lungangfang/.ssh/known_hosts:191
RSA host key for has changed and you have requested strict checking.
Host key verification failed.

Solution 1: delete the stored key

Just delete corresponding entry (as shown in this line) in known_hosts. This is actually a one-liner:

sed -i 191d /home/lungangfang/.ssh/known_hosts

Solution 2: ignore the conflict

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null


StrictHostKeyChecking no
tells ssh client to connect regardless of host keys check result.
UserKnownHostsFile /dev/null
makes ssh client store host keys to nowhere (/dev/null).

Since we do not care host keys of labs at all, you may want to put above options into corresponding section of ~/.ssh/config.

Manipulating keys and fingerprints

# Retrieve remote host key of HOST without login it
ssh-keyscan -t rsa,dsa HOST

# List fingerprints of every public key in a file
ssh-keygen -lf ~/.ssh/known_hosts
ssh-keygen -lf /dev/stdin <<< $(ssh-keyscan HOST)
for each in /etc/ssh/ssh_host_*; do ssh-keygen -lf $each; done

# Delete a public key from a file (default ~/.ssh/known_hosts)
ssh-keygen -R HOST

Public-key authentication

Login without entering password.

Set up

  1. Server side should have already enabled PubkeyAuthentication.

    That is usually the case. Otherwise, you'll have to do it youself as "root"

    1. edit /etc/ssh/sshd_conf
      RSAAuthentication yes
      PubkeyAuthentication yes
    2. restart /etc/init.d/sshd restart
  2. Check permissions of configuration directory and files on client side (refer to *Files).
  3. Generate public/private keys

    For putty, refer to ./putty.html

    To set/clear/change passphrase of private keys

    ssh-keygen -p

Choosing type of keys

Usually, the type of keys just does not matter. The default type is RSA. And, some believe it is better than DSA. For more detailed comparison and analysis, please turn to google.

SSH agent

It is kind of a dilemma when using pub-key auth that: if we set passphrase for a private key file we will have to re-enter it every time we use that private key; on the other hand, if we do not set passphrase everyone has access to the private key file, say a nasty administrator, can use it.

Here is when ssh-agent steps in. Note that ssh-agent bring in a security whole as well. For example, when the ssh-agent is running, root may "su" as you and take advantage of that agent.

Hence, based on how paranoid you are, you may choose to:

  1. Do not use ssh-agent at all.
  2. Use ssh-agent occasionally and kill it right away.
  3. Make ssh-agent a daemon (always running)

Here are the commands involved:

# start ssh-keygen, set corresponding env
eval $(ssh-agent -s)            # for csh, eval `ssh-agent -c`

# add private keys, will prompted for the passphrase for each key file

# ssh as usual, but without entering passphrase

# stop ssh-agent
ssh-agent -k

And a light-weight wrapper

# get a command-line ready for being copied/pasted to other terminals to access
# the ssh-agent
function get_ssh_agent {

    if [ -z "$SSH_AGENT_PID" -o -z "$SSH_AUTH_SOCK" ]; then
        # cmd=$(ssh-agent -s)
        # eval $cmd
        # echo $cmd
        echo "No ssh agent in this terminal"
        for each in SSH_AGENT_PID SSH_AUTH_SOCK; do
            eval "echo export $each=\${$each}"

Get public key from prviate key

This is useful to test if a pair of keys matches.

ssh-keygen -y [-f input_keyfile]

Force Password Authentication

Sometimes, for debugging, we do want password authentication for a short period. This can be done without modifying configuration files.

ssh -o PreferredAuthentications=password user@server

For SSH1 connections, try -o RSAAuthentication=no.

SSH scripting

Options for ssh within a script

-o BatchMode=yes
-o ConnectTimeout=2
-o ConnectionAttempts=2
-o LogLevel=error

Redirect IO between local and remote machines

Use case 1

# Package on remote server and save to local host, need no temporary files
ssh lgfang@ "tar cvfz - vlcs" > vlcs.tar.gz

# Upload a number of files in one command, need no temporary files
tar hcvfz -                     \
    .hosts                      \
    .tmux.conf                  \
    .emacs.d/init.el            \
    | ssh lgfang@ "tar xvfz -"

Use case 2

The "/backup" directory locates on blade 0-0-2, which is only accessible from the "pilot" blade. To download the "docs",

ssh root@pilot_pub_ip 'ssh 0-0-2 "tar cvfz - /backup"' \
    | tar xvfz -

Use case 3

"Deploy" new binary to a virutal blade in openstack.

cat a.jar | ssh me@host 'ssh root@pilot "cat - >a.jar && scp ./a.jar 0-0-2:./"'

Force pseudo-tty allocation

  • "-t" Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine. To name a few:
    • Run tmux directly
      ssh -t lungangfang@qdbuild2 tmux attach
    • SSH via a machine in the middle
      ssh -t host_middle ssh dest_host
    • Sudo shutdown
      $ ssh lgfang@ sudo shutdown now
      lgfang@'s password:
      stty: standard input: Inappropriate ioctl for device
      sudo: sorry, you must have a tty to run sudo
      $ ssh -t lgfang@ sudo shutdown now
      lgfang@'s password:
      Broadcast message from ...
      The system is going down for power-off NOW!
      Connection to closed.
  • "-tt" (Multiple -t options) force tty allocation, even if ssh has no local tty. i.e. force tty allocation when calling ssh in a script.

SSH processes eat stdin

That is why SSH in a "while read xxx" loop makes the loop run only once. The SSH drains the stdin!

$ seq 1 3 | while read id;do ssh user@ "echo hi"; done

Note that while block looped only once.

There are a couple of solutions:

  1. Redirect ssh within the loop
    $ seq 1 3 | while read line;do ssh user@ "echo hi" </dev/null; done
  2. Use -n option of ssh
    $ seq 1 3 | while read line;do ssh -n user@ "echo hi"; done
  3. Read from another file descriptor
    $ seq 1 3 > list.txt
    $ while read -u 99 line;do ssh user@ "echo hi"; done 99<list.txt

Run command on a remote server on background

Use -f. Note however, this time you needn't (and should not) append "&" to the end of your command.

ssh -f -ttq $OTHER_SSH_OPTIONS "mycmd"

Set up PS1 "before" login

spawn ssh -q -o StrictHostKeyChecking=no \
    -o UserKnownHostsFile=/dev/null $username@$vm \
    -t "PS1='AUTOIT ' bash -i"

SSH remote commands get fewer environment variables

Please do bear in mind that personal settings ($HOME/.profile etc.) are not executed when you run ssh user@remote cmd (because it is run in a non-interactive, non-login shell).

Some of the consequences are:

  1. Your customization of environment variables does not take effect at all.
  2. Some commands fail only when being evoked as ssh remote commands
  3. Some commands behave "strangely" if evoked as ssh remote commands.

NOTE: Sometimes, it is really hard to link the unexpected output to the missing of environment variables.


  • On server side:
    1. Make PermitUserEnvironment=yes in sshd_config (and restart sshd)
    2. Set up environment in $HOME/.ssh/environment.
  • Or, "manually" set up env in command line:
    ssh -t lungangfang@ 'PATH=/my/path:$PATH; echo $PATH'
  • Or, on server side, set up environment in /etc/profile which always gets executed (NOTE: this is not true for KSH).

X11 Forwarding

It is way more convenient than "export DISPLAY".

  1. Make sure X forwarding allowed by server side ("sshd").
  2. Connect with "ForwardX11" enabled
    • "ssh -X xxx", or
    • In configure file, "ForwardX11 yes"

    If got error like "error in locking authority file", simply delete that file on server (so that it is regenerated). Or, chmod 0600 it.

  3. Run "xeyes" or whatever X application to test.

Seems needn't worry about xhost + and -nolisten tcp etc.


  • Nested ssh connections
    ssh -t ssh
    1. Simple.
    2. ssh config on the intermidiate host takes effect.
  • Tunneling

    The tunnel can be set either locally (for you only) or on the intermidiate server (for everyone)

    # 1. set a local tunnel
    ssh -L 8000:target_host:22 intermidiate_user@intermidiate_host
    # 2. connect to via the tunnel
    ssh -p 8000 target_user@localhost
    scp -P 8000 ./test.file target_user@localhost:./
    1. You have to set up the tunnel beforehand and keep it open.
    2. scp directly from/to the target host works.
  • iptables DNAT

    You need root privilege to setup iptables rules.

Forced command

This can be done by putting commands into authorized_keys (refer to For instance:

command="LANG=en_US.UTF-8 tmux -2 attach || tmux -2 new -s foo" ..key..

Or, configure it in sshd_config as shown in sftp jail.

sftp jail

This procedure tested on CentOS6.5.

  1. Configure and sshd
    • Change subsystem sftp to internal-sftp
    • For users in certain group
      • force command to internal sftp
      • chroot to their home

    Edit /etc/ssh/sshd_config

    Subsystem     sftp     internal-sftp
    # NOTE: must put this section to the end of sshd_config. Refer to
    # description of "Match" in (man "sshd_config") for more.
    Match Group sftponly
    ChrootDirectory %h
    ForceCommand internal-sftp
    AllowTcpForwarding no
    sudo service restart sshd
  2. Create(setup) accounts
    sudo groupadd sftponly
    sudo useradd -g sftponly lgfang
    sudo usermod -s /dev/null lgfang
    # important ! file owner and permision must be correct
    sudo chown root:root /home/lgfang
    sudo chmod 755 /home/lgfang
  3. Done. Reference: here and here.

Limit bandwidth

scp -l limit you@host:/home/you/* .

where limit is in Kb


  • On client side: ssh -vvv ...
  • On server side, if don't know where the log goes, start a not-daemonized debug-mode sshd on an ad-hoc port:
    sshd -dddp 10222

Created: 2015-11-05 Thu 13:23 by Emacs 24.5.1 (Org mode 8.2.10)

comments powered by Disqus