SSH autocompletion, jump host automation and other tips

This post appeared originally in our sysadvent series and has been moved here following the discontinuation of the sysadvent microsite

OpenSSH is a flexible tool for not only logging into other servers, but to also help tunnel other network traffic. The following article is a grab-bag of useful SSH tips.

SSH per-user configuration file

Using the per-user configuration file, ~/.ssh/config you can make your life a bit easier. One common scenario is that a SSH server you commonly use is listening on a non-standard port. To save you from the strain of typing out -p 1234 every time you want to connect to this server you can add a couple of lines in your ~/.ssh/config file:

Host some.somehost.invalid
  Port 1234

You can also configure SSH to e.g. use compression if you have a restricted amount of bandwidth available, set up X forwarding, set longer timeout values and send keep-alive packages in order to hold the TCP connection in this file.

SSH through a jump host

It is convenient to have the SSH port open for connection from the entire internet, but it’s not without risks. There may be vulnerabilities in SSH, and if you allow SSH with password login you’re risking entry by brute force password cracking. The typical ways to reduce the risk includes:

• Limit to login by ssh-key only (strongly recommended)
• Port-knocking. Send some package to port 222, and port 22 magically opens up
• Rate-limiting, through iptables, denyhosts, sshguard, fail2ban or similar.
• Using an obscure port instead of the standard port 22 (this is security-through-obscurity but may reduce the amount of brute-force password guesses)
• Fail2ban and other similar blocking mechanisms after repeated login failures
• Firewalls allowing access only from white-listed IP addresses

The last one works well as long as those who have legitimate reason to log in always sits at the same location with the same IP address - but one would often like to be able to log in to the systems from anywhere. This can be achieved either through VPN or through a “jump host”.

One may argue that forcing logins to go via a “jump host” is “security-through-obscurity”. A jump host will probably not stop determined and skilled crackers targeting your site using some zero-day vulnerability in SSH. It will also not stop someone that has already gained control over the users account on his laptop or desktop from getting to the target servers. It will however give some benefits:

• With only one host exposed, the probability of getting unwanted visitors i.e. through “broad” brute-force login attempts is reduced
• If only the jump host gets compromised, the damage control is easier
• Forcing login traffic through one central place makes monitoring, security hardening and incident response handling easier

It easy to configure SSH to work through a jump host - assuming connections to anyhost.mydoma.in should be routed through jump.mydoma.in:

Host *.mydoma.in !jump.mydoma.in
  ProxyCommand  ssh -T jump.mydoma.in netcat -q 0 %h %p

(“-q 0” may not be supported by all versions of Netcat - skip it if it’s causing you problems)

Now, all SSH connections to anyhost.mydoma.in will be routed through jump.mydoma.in

From a resource utilization point of view, this is pretty bad - we’re doing the SSH encryption and authorization twice, causing extra CPU usage both at the local work station and on the jump host, and going through the jump host may be a detour - however, for most real-world applications that cost is really negligible.

Auto-complete SSH host names

This is an easy way of using tab to auto-complete SSH host names.

Simply add the following to your ~/.bashrc:

complete -W "$(echo `cat ~/.ssh/known_hosts | cut -f 1 -d ' ' | \
sed -e s/,.*//g | uniq | grep -v "\["`;)" ssh

This will auto-complete the host name you ssh to, by pressing tab, based on the current entries in your ~/.ssh/known_hosts file.

Keep in mind that this will only work as intended with clear-text host names in ~/.ssh/known_hosts - which is not the default behavior on all distributions.

The relevant SSH configuration:

Host *
  HashKnownHosts no

SSH hidden shell

Most people probably know about the ~. to terminate a connection but OpenSSH also provides a number of other useful commands like f.ex ~? and ~C

$ ~?
Supported escape sequences:
 ~.   - terminate connection (and any multiplexed sessions)
 ~B   - send a BREAK to the remote system
 ~C   - open a command line
 ~R   - request rekey
 ~V/v - decrease/increase verbosity (LogLevel)
 ~^Z  - suspend SSH
 ~#   - list forwarded connections
 ~&   - background SSH (when waiting for connections to terminate)
 ~?   - this message
 ~~   - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)