How to SSH Reverse Tunnel to a Remote DB Server in an AWS Private Subnet using an EC2 Bastion Jump Host

December 7, 2020

Do you have publicly inaccessible database that you need to SSH into? Using SSH Reverse Tunneling, aka SSH Reverse Port Forwarding, you can securely connect to the database without directly opening it up to a vector of attack.

Resources in private subnets are notoriously hard to connect to (by design). With AWS you have VPNs and AWS Direct Connect as options, but the overhead isn't worth it unless you have very specific requirements. That's where SSH port forwarding/tunneling + Bastion Hosts come in.

This post goes hand in hand with another piece I will be uploading soon; the purpose of that post being a beginner's DevOps/Cloud Architect's guide to bringing up the foundational infrastructure featured below. I will post a link here when that goes up.

What is SSH Reverse Tunneling?

With a traditional SSH, your machine can connect to a remote instance.

SSH reverse tunneling on the other hand, sets up an omnidirectional connection between a port on your local machine and a port on the remote instance.

Your local machine initiates a connection by forwarding a port on the remote instance to your local machine. You can then use that established connection to set up a new connection from your local machine back to the remote instance. The end-state of this interaction being that you can connect your local machine to the remote instance, so that you can use the tunnel (in reverse) to connect from the server to your local machine. Fortunately for us, this process is way easier to do than it is to understand.

Our Infrastructure Topology

Zooming out, we can see the individual resources we're dealing with. In your AWS cloud you have a VPC (Virtual Private Cloud) that has one public subnet and one private subnet.

The public subnet has a lightweight EC2 instance (t2.nano in our case) whose only function is to act as a jump server. It doesn't even have PSQL installed. This bastion host's security group allows inbound connections on port 22 (SSH) and already has my public key and user profile on it.

The private subnet has a PostgresSQL database with an attached security group (sg-00d981dc294ce24f0) that allows inbound connections from our bastion host's security group (sg-096afe51bf07b5e3c)

Using SSH Reverse Tunneling

First, we want to establish a reverse tunnel through our bastion host (75.101.188.93) to our PostgresSQL DB (postgrestest.cewfon7xqsuk.us-east-1.rds.amazonaws.com). To do so, we want to open up a terminal window on our local machine and write the following command:

ssh -i [private key] -N -L [local port]:[database host]:[remote port] [remote username]@[remote host]

where

• "ssh -i" is what we use to tell SSH what private key to use
• [private key] is your private ssh key
• -N sets up the tunnel without running any remote commands
• -L tells the tunnel to answer on the local side of the runnel
• [local port] should match the port number of your database. For PSQL this is 5432
• [database host] is your database's endpoint listed in the Connectivity and Security tab of of your database
• [remote port] should match the port number of your database.
• [remote username] is the username
• [remote host] is your ec2 IP address

That's it, our reverse tunnel is now sitting open for as long as we keep this terminal window open. Now all we have to do is to open up a new terminal window on our local machine and connect to our PostgresSQL database using:

psql -U [db username] -p [local port] -h localhost

where

• psql is the command we use to talk to postgres
• -U says what user to use
• [db username] is the existing user on the psql db we want to connect as
• -p says what port we want to use
• [local port] is the local port we used to connect from before
• -h says which host to connect to
• localhost is the same as 127.0.0.1, since were connecting to a locally forwarded port that leads to the tunnel

And we're in. A super complicated concept that only takes two short alphanumerical lines to put into practice.

Conclusion

Aside from the theory behind SSH Reverse Tunneling, the hardest part is setting up the infrastructure to support it. I will be covering just that in my next post.