This post appeared originally in our sysadvent series and has been moved here following the discontinuation of the sysadvent microsite

The other day, as I just had updated my workstation to Fedora 27, I realized maybe the Include statement in ssh_config(5) had been implemented. And indeed it had.

So it’s time to reorganize my ssh-config-generate script, FoxyProxy browser plugin for tunneling web traffic through SSH, and maybe even setting up systemd so I don’t have to manually initiate the SSH sessions!

Now why would I need this? Consider the following problem: I need to connect to the web interface of a switch at a customers location.

  • The customer has set up firewall, and only allows SSH from one specific host on the Internet - my_jumphost.me.example.com
  • On the other hand, I can only connect to one server in the customers DMZ - ssh.customer-a.example.com
  • Only a server deep in the customers backend network can connect to the web interface on the switch - server_123.secure.customer-a.example.com
  • Finally, the switch has an alternative DNS name in the customer’s internal DNS - switch_123.internal.example.com

In order to solve this, we have to make some assumptions for the above scenario:

  • ssh-server on all machines are set up, and you can manually SSH from one to the other.
  • authentication using SSH keys, SSH agent, sshpass or any other mechanism is just working.

ssh_config(5)

First lets look at the main SSH client configuration file: ~/.ssh/config

Include config.d/*.conf
Include ~/Customers/*/.ssh/config

Host *
  ControlMaster auto
  ControlPath ~/.ssh/session-%r@%h:%p
  ControlPersist 120
  ForwardAgent no
  VisualHostKey no

The first Include includes configuration files from /home/me/.ssh/config.d with file names ending with .conf. The file name ending with .conf is just so that I can rename a file to disable it, which is handy. Also notice the relative path, so SSH defaults to ~/.ssh/.

The second Include reads the ssh_config files from where I keep my customer specific config files. An example customer would have its config file in /home/me/Customers/customer-a/.ssh/config. This allows me to move the complete customers directory to /home/me/Customers/00_inactive to disable the configuration.

Then comes the default host configuration. This sets parameters for all hosts, unless overridden per host. Here i set up ControlMaster. This allows for reuse of connections through the ControlPath while ControlPersist specifies how long the connection will be kept around after going idle.

Next i disallow forwarding of the ssh-agent, as i want my open ssh-keys to stay om my computer. I also disable the VisualHostKey, as I generally don’t want lots of output in my terminal.

Jumphost configuration

Now lets move on to the host configuration for my own jumphost, the server I need to connect through in order to be allowed through the customer firewall.

This is our own server, so I let the configuration live in the file /home/me/.ssh/config.d/my_servers.conf

Host jumpy
  HostName my_jumphost.me.example.com

Put simply, this allows med to type “ssh jumpy”, and SSH will connect using the full host-name.

Customer specific configuration

Now lets focus on customer-a. Here the configuration resides in /home/me/Customers/customer-a/.ssh/config:

Host cust-a-edgy
  HostName ssh.customer-a.example.com
  Port 2222
  User my_user
  ProxyCommand ssh -W %h:%p jumpy
  # ProxyCommand ssh -T jumpy nc -w 420 -q 42 %h %p

Host cust-a-server123
  HostName server_123.secure.customer-a.example.com
  User some_admin_user
  DynamicForward 31001
  LocalForward 31080 1.2.3.4:80
  ProxyJump cust-a-edgy

This defines the two customer servers, and I give them my own names so tab-completion is more logical for me.

As the host cust-a-edgy is listening on a public IP address on the Internet, the customer has changed the port. ProxyCommand defines a command to use instead of the regular SSH, which lets me control the previous server in the jump-chain that should be connected to first. Also included in a comment is the older way of doing the same in “netcat mode”.

The second host definition is the last server in the SSH jump-chain, so the forwarding configuration resides here.

DynamicForward will listen with a SOCKS PROXY on port 31001 on a local interface on my workstation, and all traffic to this port will be forwarded through the SSH jump-chain all the way to server_123.

I also added the line with LocalForward just to show that I also can listen on any given local port, forward every packet through the jump-chain to server_123, and from this host, the packets will be sent to the 1.2.3.4, port 80.

This host configuration block also uses the brand new ProxyJump statement. This statement accepts a comma separated list of jump hosts to go through to get to this host. SWEET!

With these few lines in the correct configuration files I can now type “ssh cust-a-server123” and have a shell on server_123!

Before I leave the SSH configuration, just let me say that all of this is documented in the ssh_config(5) man page. Also if you have not configured ssh-keys, you will be prompted for password for every host.

FoxyProxy

Now that we have a SOCKS PROXY on localhost port 31001, we can tell my web browser to use this for connecting to the desired web interface on switch_123.internal.example.com .I could set this as the default proxy server in my browser’s configuration, but since I am using Firefox, and there is a add-on for managing multiple proxies, FoxyProxy, I would rather use that.

First install the add-on.

Then set up a proxy in FoxyProxy:

  • Click the small Foxy icon somewhere in your browser.
  • Make sure to select “Use Enabled Proxies By Patterns and Priority”
  • Click the same icon again, and select “Options”, and then “Add”
  • Select “SOCKS5” as “Proxy Type”
  • Insert “127.0.0.1” or “localhost” as “IP address, DNS name, server name”
  • Put “31001” as “Port”. This is the DynamicForward from the SSH configuration above.
  • Also make sure the “Send DNS through SOCKS5 proxy?” is toggled to “On”. This will ask for host-names through the SOCKS connection, and resolve the host-name inside the customers internal network.
  • Click “Save & Edit Patterns” to adjust what traffic will use this proxy.
  • Remove the default wildcard for “all URLs” by clicking the garbage can below “Delete”
  • Click “New” to add a new pattern
  • Type some name, and insert “switch_123.internal.example.com” in the pattern field.

You are now ready to point your browser to http://switch_123.internal.example.com !

Wouldn’t it be nice if…

SystemD was configured to listen on the DynamicForward proxy port, and start the SSH session as it would be needed? Or if you could start new SSH sessions through a SOCKS proxy? Oh wait; you can use Corkscrew for that ;)

Bjørn Ove Grødal Wærås

a Systems Consultant at Redpill Linpro

Bjørn Ove has worked with us for ten years now and is a Red Hat Certified Architect (RHCA). He is also certified for holding most of Red Hat's courses. After spending an inordinate amount of time holding courses far and wide, he won the Red Hat Cloud Instructor of the Year award this year

Thoughts on the CrowdStrike Outage

Unless you’ve been living under a rock, you probably know that last Friday a global crash of computer systems caused by ‘CrowdStrike’ led to widespread chaos and mayhem: flights were cancelled, shops closed their doors, even some hospitals and pharmacies were affected. When things like this happen, I first have a smug feeling “this would never happen at our place”, then I start thinking. Could it?

Broken Software Updates

Our department do take responsibility for keeping quite a lot ... [continue reading]

Alarms made right

Published on June 27, 2024

Just-Make-toolbox

Published on March 22, 2024