Some ssh and screen ninja sauce
local port forwarding
ssh user@remoteserver -L localport:targetserver:targetport
any network traffic hitting
localport on the local machine (the machine you’re ssh’ing from) will be forwarded over the encrypted ssh channel to
remoteserver (the machine you’re ssh’ing to), and then onward to
the traffic coming into the local machine, and going from
targetserver is unencrypted. only the segment between the local machine and
remoteserver is encrypted. usually the tunneled traffic originates from your local machine, and
targetserver is the same as
remoteserver, so in practice the entire communication would be secure.
targetserver is resolved relative to
remoteserver. that means
targetserver could be an internal IP on
remoteserver’s LAN that you could not access directly from the local machine. this is also what lets you specify “
targetserver, since it’s “
localhost” as resolved with respect to
local forwards are useful for the following scenarios:
- access a service on a remote machine that is only available locally (either because only local connections are allowed, or the port is blocked via firewall)
ssh remoteserver -L 4000:localhost:5984access couchdb running on a remote server; we can now access it using the address
- access a firewalled/NAT’ted machine that you cannot access directly
ssh remoteserver -L 8080:192.168.1.15:80
192.168.1.15is only visible on
remoteserver’s LAN. we access its webserver via
localhost:8080; traffic is forwarded through
- encrypting a channel that would otherwise be unencrypted
ssh remoteserver -L 5900:localhost:5900vnc is an insecure protocol; passwords are sent in the clear and desktop contents are visible to anyone snooping the traffic. instead of vnc’ing to
remoteserver, we securely vnc to
localhost(5900 is the standard vnc port).
- masking the origin of traffic
ssh stoogeserver -L 8080:dupedserver:80forwarding a connection in this manner is not always transparent. for example, HTTP includes a
Hostparameter that specifies the hostname the browser wants to connect to. In this case that will be
dupedserverwill see that.
by default, only local traffic can connect to
localport. to enable external hosts to use the port forward through your machine, invoke as
ssh remoteserver -L
localport:targetserver:targetport (note asterisk).
remote port forwarding
ssh user@remoteserver -R remoteport:targetserver:targetport
any network traffic hitting
remoteserver will go to your local machine (via encrypted channel), and then onward to
targetserver from the local machine.
targetserver is now resolved relative to the local machine, but in the remote forwarding scenario
targetserver is nearly always “
- making a public server (visible to the internet at large) that forwards to your local machine
ssh publicserver -R 8053:localhost:8053you’re testing an android app over GPRS against your dev server. the app can only hit public IPs. now it can access your dev server via
there is a catch with remote forwards. for the above scenario (which is really the only useful scenario) to work, external traffic must be allowed to connect to
remoteport. this is only allowed if the
GatewayPorts setting in
yes. this is not the case by default on most installs. therefore, you must have root control on
remoteserver to enable this. the setting is in the config of the ssh server, so no client settings you do can override it.
as for the “
*:” to enable external connections, it usually is implicit for remote forwards. but sometimes not — when in doubt, add it; it can’t hurt anything. it will not override
if you can’t change
sshd_congig, there is a workaround. we can daisy-chain a local forward to the remote forward, so all traffic hitting
remoteport originates from
ssh remoteserver -R dummyport:localhost:targetport
- then, on
ssh localhost -L *:remoteport:localhost:dummyport(note the ‘*’)
needless to say, this is ridiculous.
running a command remotely instead of a shell
ssh user@server command
ssh user@server ls -l ~
ssh escape character
~ typed after a newline is the ssh escape character. it allows you to perform out-of-band actions with the ssh session. some highlights:
~?— print list of available commands
~.— terminate the session (useful if network dropped or session is hung)
~[ctrl-z]— suspend session (instead of suspending the currently running program inside the session)
~C— enter command line to add additional port forwards on-the-fly (type
~~— type a literal
additional ssh sessions can piggyback on an originating session’s connection. no authentication is needed for the piggybacking sessions.
to start the first session (the “master session”):
ssh -M -S /tmp/sshsocket user@server
for piggybacking sessions:
ssh -S /tmp/sshsocket user@server
this is particularly useful when the piggybacking sessions run a command on the remote server instead of a shell.
/tmp/sshsocket can be any file, unique to the master session. anything with access to this file can piggyback on the session. you can also use shorthand like
/tmp/ssh-%r@%h:%p, which ssh will auto-expand to help maintain uniqueness among master sessions.
run GUI programs on a remote server!
ssh -X user@server callofduty
any configuration option available in
ssh_config can be invoked on a per-session basis. this is useful when running ssh from scripts:
ssh -o BatchMode=yes — fail immediately if any interactive prompt is displayed (e.g., password prompt), since these would hang your script forever
ssh -o ExitOnForwardFailure=yes — abort if the desired port forwards could not be set up
ssh -o ServerAliveInterval=60 — ‘ping’ the server every 60 seconds and terminate the session if some consecutive number of pings go unanswered (usually 3)
quickly set up key-based auth
this copies your public keys to the remote server
scp. quick and dirty, but not robust.
sshfs user@server:path localpath
path on the remote server as a filesystem under
localpath. you have to create
localpath yourself, sadly. useful for browsing directory trees. i’m unsure how robust it is against dropped connections.
umount the filesystem with
fusermount -u localpath
sftp often works just as well, with the added benefit that your file manager (e.g., nautilus) may have support built right in.
rsync is great, and can use ssh for its transport:
rsync -ravz -e ssh user@server:path localpath (note that
user@server:path is interpreted according to rsync, not ssh!)
if you want to robustly transfer a 10GB couch database from africa while you sleep:
while [ true ] ; do rsync -ravz --progress --partial -e ssh user@server:path localpath ; sleep 5 ; done
the loop will resume after dropped connections;
--partial allows resuming the transfer;
--progress displays a progress bar;
sleep 5 avoids flooding the server.
make sure you’re set up to log into
user@server using password-less authentication, otherwise resume attempts will hang with a password prompt!
if you want to use a one-off keypair for the transfer, you can specify an arbitrary private key to use (assuming you have placed the corresponding public key on the remote server (see
ssh-copy-id)) by replacing
-e ssh with
-e "ssh -i /path/to/privatekey.key".
there’s a long delay before i get a password prompt or a shell prompt
this is usually either:
- the ssh server is trying to reverse DNS lookup the server IP, and it has to wait for the lookup to time outfix with
- the ssh server is trying to integrate with an authentication library that is not configured properly, and timing outto fix, disable the guilty library in
sshd_config, such as
i’m sure you all know these:
C-a c— new window
C-a n— change windows
C-a k— kill window
C-a d— detach from session
screen -ls— display sessions
screen -r— resume session
how the f#$% do i scroll
enter ‘copy’ mode:
C-a [. in copy mode, you can navigate around the history using arrows and page up/down.
the size of the buffer is pretty limited by default. to make it bigger
screen -h ####(affects all windows in this session)
C-a :scrollback ####(affects current window only)
- set a bigger default in your
in ‘copy’ mode, you can also search with
C-a s (forward) or
copy text using mark and set points. space to start/end at the current char;
y to start/end at the current line.
C-a ]— paste into current window
C-a >— dump copy buffer to file
C-a h— write current window (visible portion only) to file
C-a :hardcopy -h /must/specify/a/file— write the entire scrollback history to
C-a H— start/stop logging of current terminal to file
monitoring a window for activity
C-a M— alert if this window has activity
C-a _— alert if this window has no activity for a while
C-a S— split current region horizontally
C-a |— split current region vertically
C-a— move to next region
C-a X— close this region
C-a Q— close all regions except this one
screen -x [session name] — allows multiple terminals to connect to the same session (must be same user)
you can actually set up multi-user screen sharing, but it’s kind of a bitch, and not that useful, for all of:
- the screen exe must be setuid root, along with other permissions changes
- no one can sudo; they must be running as their logged-in user
- there’s a shitload of commands you have to run
- you can’t really guarantee everyone is seeing the same thing; split-screens, window changes, etc., are all local to each connection
- a user closing a window or the session closes it for everybody
communicate with network sockets using stdin/stdout
nc host port — connect to server
nc -l port — listen for a connection from a client
once the connection is open, commicate with it by typing or piping
Another day, another Zero Day: What all Digital Development organizations should take away from recent IT security news
Even if you don’t work as a software developer, you probably heard about recent, high profile security issues that had IT Admins and developers frantically patching servers over the holidays and again more recently. Dimagi's CTO shares what these recent issues mean for Digital Development organizations.
January 28, 2022
Join the fight to support critical open source infrastructure
Open Source tools are a critical piece of global infrastructure, and need champions for long term investment
March 17, 2020
Two big lessons that Iowa and Geneva can teach us about technology in digital development
Last week brought two high profile technology failures into the global spotlight. Although these two mishaps may seem quite different at first glance, they both highlight challenges that are inherent in providing software in the public sector (regardless of locale) and illustrate cautionary lessons worth discussing for practitioners in Digital Development. The Iowa Caucus Debacle
February 7, 2020