Native and Distant Port Forwarding

2023-01-12 00:27:55

Do not miss new posts within the collection! Subscribe to the weblog updates and get deep technical write-ups on Cloud Native matters direct into your inbox.

TL;DR SSH Port Forwarding as a printable cheat sheet.

SSH is yet another example of an historic expertise that’s nonetheless in large use at this time. It might very effectively be that studying a few SSH tips is extra worthwhile in the long term than mastering a dozen Cloud Native instruments destined to grow to be deprecated subsequent quarter.

One in all my favourite elements of this expertise is SSH Tunnels. With nothing however commonplace instruments and sometimes utilizing only a single command, you possibly can obtain the next:

Entry inner VPC endpoints via a public-facing EC2 occasion.
Open a port from the localhost of a improvement VM within the host’s browser.
Expose any native server from a house/non-public community to the skin world.

And extra ????

However even though I exploit SSH Tunnels each day, it at all times takes me some time to determine the best command. Ought to or not it’s a Native or a Distant tunnel? What are the flags? Is it a local_port:remote_port or the opposite approach round? So, I made a decision to lastly wrap my head round it, and it resulted in a collection of labs and a visible cheat sheet ????

Stipulations

SSH Tunnels are about connecting hosts over the community, so each lab under expectedly includes a number of “machines”. Nonetheless, I am too lazy to spin up full-blown cases, particularly when containers can be utilized as a substitute. That is why I ended up utilizing simply a single vagrant VM with Docker on it.

In concept, any Linux field with Docker Engine on it ought to do. Nonetheless, working the under examples as-is with Docker Desktop will not be doable as a result of the flexibility to entry the ~~machines~~ containers by their IPs is assumed.

Alternatively, the labs may be performed with Lima (QEMU + nerdctl + containerd + BuildKit), however remember to limactl shell bash first.

Each instance requires a sound passphrase-less key pair on the host that’s then mounted into the containers to simplify entry administration. If you do not have one, producing it is so simple as simply ssh-keygen on the host.

Vital: SSH daemons within the containers listed here are solely for instructional functions – containers on this submit are supposed to symbolize full-blown “machines” with SSH purchasers and servers on them. Beware that it is hardly ever a good suggestion to have SSH stuff in real-world containers!

Native Port Forwarding

Ranging from the one which I exploit essentially the most. Oftentimes, there is perhaps a service listening on localhost or a non-public interface of a machine that I can solely SSH to by way of its public IP. And I desperately must entry this port from the skin. A number of typical examples:

Accessing a database (MySQL, Postgres, Redis, and so forth) utilizing a elaborate UI software out of your laptop computer.
Utilizing your browser to entry an online software uncovered solely to a non-public community.
Accessing a container’s port out of your laptop computer with out publishing it on the server’s public interface.

The entire above use instances may be solved with a single ssh command:

ssh -L [local_addr:]local_port:remote_addr:remote_port [user@]sshd_addr

The -L flag signifies we’re beginning a native port forwarding. What it really means is:

In your machine, the SSH shopper will begin listening on local_port (doubtless, on localhost, however it relies upon – check the GatewayPorts setting).
Any visitors to this port will likely be forwarded to the remote_private_addr:remote_port on the machine you SSH-ed to.

Right here is the way it appears to be like on a diagram:

SSH Tunnels visualized - local port forwarding.

Lab 1: Utilizing SSH Tunnels for Native Port Forwarding ????‍????

The lab reproduces the setup from the diagram above. First, we have to put together the server – a machine with the SSH daemon and a easy internet service listening on 127.0.0.1:80:

$ docker buildx construct -t server:newest -<<'EOD'
# syntax=docker/dockerfile:1
FROM alpine:3

# Set up the dependencies:
RUN apk add --no-cache openssh-server curl python3
RUN mkdir /root/.ssh && chmod 0700 /root/.ssh && ssh-keygen -A

# Put together the entrypoint that begins the daemons:
COPY --chmod=755 <<'EOF' /entrypoint.sh
#!/bin/sh
set -euo pipefail

for file in /tmp/ssh/*.pub; do
  cat ${file} >> /root/.ssh/authorized_keys
performed
chmod 600 /root/.ssh/authorized_keys

# Minimal config for the SSH server:
sed -i '/AllowTcpForwarding/d' /and so forth/ssh/sshd_config
sed -i '/PermitOpen/d' /and so forth/ssh/sshd_config
/usr/sbin/sshd -e -D &

python3 -m http.server --bind 127.0.0.1 ${PORT} &

sleep infinity
EOF

# Run it:
CMD ["/entrypoint.sh"]
EOD

Beginning the server and noting its IP handle:

$ docker run -d --rm 
   -e PORT=80 
   -v $HOME/.ssh:/tmp/ssh 
   --name server 
   server:newest

SERVER_IP=$(
  docker examine 
    -f '{{vary.NetworkSettings.Networks}}{{.IPAddress}}{{finish}}' 
  server
)

Because the internet service listens on the localhost, it will not be accessible from the skin (i.e., from the host system on this specific case):

$ curl ${SERVER_IP}
curl: (7) Failed to connect with 172.17.0.2 port 80: Connection refused

However from the within of the “server”, it really works simply superb:

$ ssh -o StrictHostKeyChecking=no root@${SERVER_IP}
7b3e49181769:$# curl localhost
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
...

And right here is the trick: bind the server’s localhost:80 to the host’s localhost:8080 utilizing native port forwarding:

$ ssh -o StrictHostKeyChecking=no -f -N -L 8080:localhost:80 root@${SERVER_IP}

Now you need to have the ability to entry the net service on a neighborhood port of the host system:

$ curl localhost:8080
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
...

A barely extra verbose (however extra express and versatile) approach to obtain the identical objective – utilizing the local_addr:local_port:remote_addr:remote_port type:

$ ssh -o StrictHostKeyChecking=no -f -N -L 
  localhost:8080:localhost:80 
  root@${SERVER_IP}

Native Port Forwarding with a Bastion Host

It won’t be apparent at first, however the ssh -L command permits forwarding a neighborhood port to a distant port on any machine, not solely on the SSH server itself. Discover how the remote_addr and sshd_addr could or could not have the identical worth:

ssh -L [local_addr:]local_port:remote_addr:remote_port [user@]sshd_addr

Undecided how reliable the usage of the time period bastion host is right here, however that is how I visualize this state of affairs for myself:

SSH Tunnels visualized - local port forwarding with a bastion host.

I usually use the above trick to name endpoints which can be accessible from the bastion host however not from my laptop computer (e.g., utilizing an EC2 occasion with non-public and public interfaces to connect with an OpenSearch cluster deployed totally inside a VPC).

Lab 2: Native Port Forwarding with a Bastion Host ????‍????

Once more, the lab reproduces the setup from the diagram above. First, we have to put together the bastion host – a machine with solely the SSH daemon on it:

$ docker buildx construct -t bastion:newest -<<'EOD'
# syntax=docker/dockerfile:1
FROM alpine:3

# Set up the dependencies:
RUN apk add --no-cache openssh-server
RUN mkdir /root/.ssh && chmod 0700 /root/.ssh && ssh-keygen -A

# Put together the entrypoint that begins the SSH daemon:
COPY --chmod=755 <<'EOF' /entrypoint.sh
#!/bin/sh
set -euo pipefail

for file in /tmp/ssh/*.pub; do
  cat ${file} >> /root/.ssh/authorized_keys
performed
chmod 600 /root/.ssh/authorized_keys

# Minimal config for the SSH server:
sed -i '/AllowTcpForwarding/d' /and so forth/ssh/sshd_config
sed -i '/PermitOpen/d' /and so forth/ssh/sshd_config
/usr/sbin/sshd -e -D &

sleep infinity
EOF

# Run it:
CMD ["/entrypoint.sh"]
EOD

Beginning the bastion host and noting its IP:

$ docker run -d --rm 
    -v $HOME/.ssh:/tmp/ssh 
    --name bastion 
    bastion:newest

BASTION_IP=$(
  docker examine 
    -f '{{vary.NetworkSettings.Networks}}{{.IPAddress}}{{finish}}' 
  bastion
)

Now, beginning the goal internet service on a separate “machine”:

$ docker run -d --rm 
    --name server 
    python:3-alpine 
    python3 -m http.server 80

SERVER_IP=$(
  docker examine 
    -f '{{vary.NetworkSettings.Networks}}{{.IPAddress}}{{finish}}' 
  server
)

Let’s fake that calling curl ${SERVER_IP} straight from the host is just not an choice for some cause (e.g., as if there have been no route from the host to that IP handle). So, we have to begin the port-forwarding:

$ ssh -o StrictHostKeyChecking=no -f -N -L 8080:${SERVER_IP}:80 root@${BASTION_IP}

Discover that the SERVER_IP and the BASTION_IP variables have totally different values within the above command.

Checking that it really works:

$ curl localhost:8080
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
...

Distant Port Forwarding

One other widespread (however somewhat inverse) state of affairs is while you need to momentarily expose a neighborhood service to the skin world. In fact, for that, you may want a public-facing ingress gateway server. However concern not! Any public-facing server with an SSH daemon on it may be used as such a gateway:

ssh -R [remote_addr:]remote_port:local_addr:local_port [user@]gateway_addr

The above command appears to be like no extra difficult than its ssh -L counterpart. However there’s a pitfall…

By default, the above SSH tunnel will enable utilizing solely the gateway’s localhost because the distant handle. In different phrases, your native port will grow to be accessible solely from the within of the gateway server itself, and extremely doubtless it is not one thing you really need. As an illustration, I sometimes need to use the gateway’s public handle because the distant handle to reveal my native providers to the general public Web. For that, the SSH server must be configured with the GatewayPorts yes setting.

Here’s what distant port forwarding can be utilized for:

Exposing a dev service out of your laptop computer to the general public Web for a demo.
Hmm… I can suppose of some esoteric examples, however I doubt it is price sharing them right here. Curious to listen to what different individuals could use distant port forwarding for!

Right here is how the distant port forwarding appears to be like on a diagram:

SSH Tunnels visualized - remote port forwarding.

Lab 3: Utilizing SSH Tunnels for Distant Port Forwarding ????‍????

The lab reproduces the setup from the diagram above. First, we have to put together a “dev machine” – a pc with an SSH shopper and a neighborhood internet server:

$ docker buildx construct -t devel:newest -<<'EOD'
# syntax=docker/dockerfile:1
FROM alpine:3

# Set up dependencies:
RUN apk add --no-cache openssh-client curl python3
RUN mkdir /root/.ssh && chmod 0700 /root/.ssh

# Put together the entrypoint that begins the net service:
COPY --chmod=755 <<'EOF' /entrypoint.sh
#!/bin/sh
set -euo pipefail

cp /tmp/ssh/* /root/.ssh
chmod 600 /root/.ssh/*

python3 -m http.server --bind 127.0.0.1 ${PORT} &

sleep infinity
EOF

# Run it:
CMD ["/entrypoint.sh"]
EOD

Beginning the dev machine:

$ docker run -d --rm 
    -e PORT=80 
    -v $HOME/.ssh:/tmp/ssh 
    --name devel 
    devel:newest

Getting ready an ingress gateway server – a easy SSH server with GatewayPorts set to sure in sshd_config:

$ docker buildx construct -t gateway:newest -<<'EOD'
# syntax=docker/dockerfile:1
FROM alpine:3

# Set up the dependencies:
RUN apk add --no-cache openssh-server
RUN mkdir /root/.ssh && chmod 0700 /root/.ssh && ssh-keygen -A

# Put together the entrypoint that begins the SSH server:
COPY --chmod=755 <<'EOF' /entrypoint.sh
#!/bin/sh
set -euo pipefail

for file in /tmp/ssh/*.pub; do
  cat ${file} >> /root/.ssh/authorized_keys
performed
chmod 600 /root/.ssh/authorized_keys

sed -i '/AllowTcpForwarding/d' /and so forth/ssh/sshd_config
sed -i '/PermitOpen/d' /and so forth/ssh/sshd_config
sed -i '/GatewayPorts/d' /and so forth/ssh/sshd_config
echo 'GatewayPorts sure' >> /and so forth/ssh/sshd_config

/usr/sbin/sshd -e -D &

sleep infinity
EOF

# Run it:
CMD ["/entrypoint.sh"]
EOD

Beginning the gateway server and noting its IP handle:

$ docker run -d --rm 
    -v $HOME/.ssh:/tmp/ssh 
    --name gateway 
    gateway:newest

GATEWAY_IP=$(
  docker examine 
    -f '{{vary.NetworkSettings.Networks}}{{.IPAddress}}{{finish}}' 
  gateway
)

Now from the within of the dev machine, begin the distant port forwarding:

$ docker exec -it -e GATEWAY_IP=${GATEWAY_IP} devel sh
/ $# ssh -o StrictHostKeyChecking=no -f -N -R 0.0.0.0:8080:localhost:80 root@${GATEWAY_IP}
/ $# exit  # or detach with ctrl-p, ctrl-q

And validate that the dev machine’s native port turned uncovered on the gateway’s public interface (from the host system):

$ curl ${GATEWAY_IP}:8080
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
...

Distant Port Forwarding from a Dwelling/Non-public Community

Very similar to native port forwarding, distant port forwarding has its personal bastion host mode. However this time, the machine with the SSH shopper (e.g., your dev laptop computer) performs the position of the bastion. Specifically, it permits exposing ports from a house (or a non-public) community your laptop computer has the entry to to the skin world via the ingress gateway:

ssh -R [remote_addr:]remote_port:local_addr:local_port [user@]gateway_addr

Appears virtually similar to the easy distant SSH tunnel, however the local_addr:local_port pair turns into the handle of a tool within the dwelling community. Right here is how it may be depicted on a diagram:

SSH Tunnels visualized - remote port forwarding to home network.

Since I sometimes use my laptop computer as a skinny shopper and the precise improvement occurs on a house server, I depend on such a distant port forwarding once I want to reveal a dev service from a house server to the general public Web, and the one machine with the gateway entry is my skinny laptop computer.

Lab 4: Distant Port Forwarding from a Dwelling/Non-public Community ????‍????

As at all times, the lab reproduces the setup from the diagram above. First, we have to put together the “skinny dev machine”:

$ docker buildx construct -t devel:newest -<<'EOD'
# syntax=docker/dockerfile:1
FROM alpine:3

# Set up the dependencies:
RUN apk add --no-cache openssh-client
RUN mkdir /root/.ssh && chmod 0700 /root/.ssh

# This time we run nothing (at first):
COPY --chmod=755 <<'EOF' /entrypoint.sh
#!/bin/sh
set -euo pipefail

cp /tmp/ssh/* /root/.ssh
chmod 600 /root/.ssh/*

sleep infinity
EOF

# Run it:
CMD ["/entrypoint.sh"]
EOD

Beginning the “dev machine”:

$ docker run -d --rm 
    -v $HOME/.ssh:/tmp/ssh 
    --name devel 
    devel:newest

Beginning a non-public dev server utilizing a separate “machine” and noting its IP handle:

$ docker run -d --rm 
    --name server 
    python:3-alpine 
    python3 -m http.server 80

SERVER_IP=$(
  docker examine -f '{{vary.NetworkSettings.Networks}}{{.IPAddress}}{{finish}}' 
  server
)

Getting ready the ingress gateway server:

$ docker buildx construct -t gateway:newest -<<'EOD'
# syntax=docker/dockerfile:1
FROM alpine:3

# Set up the dependencies:
RUN apk add --no-cache openssh-server
RUN mkdir /root/.ssh && chmod 0700 /root/.ssh && ssh-keygen -A

# Put together the entrypoint that begins the SSH daemon:
COPY --chmod=755 <<'EOF' /entrypoint.sh
#!/bin/sh
set -euo pipefail

for file in /tmp/ssh/*.pub; do
  cat ${file} >> /root/.ssh/authorized_keys
performed
chmod 600 /root/.ssh/authorized_keys

sed -i '/AllowTcpForwarding/d' /and so forth/ssh/sshd_config
sed -i '/PermitOpen/d' /and so forth/ssh/sshd_config
sed -i '/GatewayPorts/d' /and so forth/ssh/sshd_config
echo 'GatewayPorts sure' >> /and so forth/ssh/sshd_config

/usr/sbin/sshd -e -D &

sleep infinity
EOF

# Run it:
CMD ["/entrypoint.sh"]
EOD

And beginning it:

$ docker run -d --rm 
    -v $HOME/.ssh:/tmp/ssh 
    --name gateway 
    gateway:newest

GATEWAY_IP=$(
  docker examine 
    -f '{{vary.NetworkSettings.Networks}}{{.IPAddress}}{{finish}}' 
  gateway
)

Now, from the within of the “dev machine”, begin the distant port forwarding SERVER-GATEWAY:

$ docker exec -it -e GATEWAY_IP=${GATEWAY_IP} -e SERVER_IP=${SERVER_IP} devel sh
/ $# ssh -o StrictHostKeyChecking=no -f -N -R 0.0.0.0:8080:${SERVER_IP}:80 root@${GATEWAY_IP}
/ $# exit  # or detach with ctrl-p, ctrl-q

Lastly, validate that the dev server turned accessible on the gateway’s public interface (from the host system):

$ curl ${GATEWAY_IP}:8080
<!DOCTYPE HTML>
<html lang="en">
<head>
...

Summarizing

After doing all these labs and drawings, I observed that:

The phrase “native” can imply both the SSH shopper machine or an upstream host accessible from this machine.
The phrase “distant” can imply both the SSH server machine (sshd) of an upstream host accessible from it.
Native port forwarding (ssh -L) implies it is the ssh shopper that begins listening on a brand new port.
Distant port forwarding (ssh -R) implies it is the sshd server that begins listening on an additional port.
The mnemonics are “ssh -L local:distant” and “ssh -R remote:native” and it is at all times the left-hand facet that opens a brand new port.

As an alternative of conclusion

Hope the above supplies helped you a bit with turning into a grasp of SSH Tunnels ???? If you happen to discover the networking matters fascinating however really feel like a lot of the supplies are written for bearded community engineers and virtually indigestible for mere builders, I’ve received a couple of extra articles that you could be discover helpful:

Joyful studying!