§SSH / Mosh

More SSH awesomeness on https://github.com/moul/awesome-ssh

§SSH or mosh into a host and attach or create a named tmux session

ssh SERVER -t "tmux attach -t mysession || tmux new-session -s mysession || /bin/sh"

and with mosh

mosh SERVER -- /bin/sh -c "tmux attach -t mysession || tmux new-session -s mysession || /bin/sh"

replaces SERVER with your SSH target hostname.

§Unsafe SSH or Mosh connection (temporary servers: CI)

ssh -o ControlMaster=no -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no SERVER

with Mosh:

mosh -ssh="ssh -o ControlMaster=no -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" -- SERVER

§Non-interactive SSH with password

expect -c "set timeout -1; spawn ssh SERVER; match_max 100000; expect *password:*; send -- PASSWORD\r; interact;"

§Networking

§Measure network performances and stability with iperf

Install:

TCP test:

UDP test:

§traceroute & ping with mytraceroute (mtr)

Example:

$ mtr google.com
                                   My traceroute  [v0.86]
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                   Packets               Pings
 Host                                            Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. xxx-xxx-xxx-xxx.rev.poneytelecom.eu           0.0%     5    0.4   0.6   0.4   0.7   0.0
 2. 195.154.2.6                                   0.0%     4    0.9   0.9   0.8   1.0   0.0
 3. 209.85.149.12                                 0.0%     4    1.0   1.0   1.0   1.0   0.0
 4. 108.170.245.1                                 0.0%     4    2.4   2.2   2.0   2.4   0.0
 5. 66.249.95.247                                 0.0%     4    1.1   1.1   1.1   1.1   0.0
 6. par21s12-in-f14.1e100.net                     0.0%     4    1.1   1.1   1.1   1.2   0.0

§Find big files & Cleanup space

§Get aggregated sizes of sub-dirs and files

du -hs .??* *

§Find big files

find / -xdev -type f -size +100M -ls 2>/dev/null

§Cleanup docker

docker system prune

§Git / GitHub

§Git / GitHub essential tools

§Do rebase instead of merge when pulling

git config --global pull.rebase true

Update the following variables when appropriate:

§Close a pull request from command line with “hub”

  1. Install “hub”
  2. Type hub api -X PATCH /repos/$USER/$REPO/pulls/$ID -F state=closed

§List projects in JSON

Or inline

hub api --paginate user/repos | jq '.[] | select(.owner.login == "moul") | with_entries(select(.key|test("_url")|not)) | del(.owner) | del(.permissions) | del(.url) | del(.node_id)'`

§Bonus: find and close a pull-request named “Enabled GuardRails” in the current repository

export prid=$(hub pr list | grep "Enable GuardRails" | cut -d '#' -f 2 | awk '{print $1}')
if [ "$prid" != "" ]; then
    project=$(git remote -v | cut -d: -f2 | sed "s/\\.git.*//" | head -1)
    hub api -X PATCH /repos/$project/pulls/$prid -F state=closed
fi

§Manipulate JSON

§Flatten concatenated JSON files

Sometimes, when you run a command like this: hub api --paginate user/repos, you get multiple JSON outputs in the same file.

jq doesn’t care and you can run any jq command over the aggregated file, however, this .json file won’t be supported by most of the other softwares.

You can “repair” by flattening a concatenated JSON file by doing the following:

cat aggregate.json | sed 's/^]$$/],/;$$s/],/]]/;1s/^/[/' | jq '. | add'

The sed ... command will transfer the standalones arrays into a a valid array of arrays.

The jq . | add will flatten the array of arrays into a big 1 dimension array.

§Apply change on multiple files

§Replace a string by another in a whole Git repository

I’m using my https://github.com/moul/golang-repo-template project as repo template on GitHub.

The first task I need to do immediately after creating the new repo, is to replace every instance of golang-repo-template in the codebase by lorem-ipsum (the project name).

for file in $(git grep golang-repo-template | cut -d: -f1 | sort -u); do sed -i'' s/golang-repo-template/lorem-ipsum/g $file; done

§Terminator on Windows

§Web

§Temporary static web server

python -m SimpleHTTPServer

And then open your browser on http://localhost:8000

§Generate directory listing

tree -H . --charset=utf-8 > index.html

§Proxy to avoid caching

{ nc ${proxy_addr} ${proxy_port} && exec 1>&- ; } < <(
    awk '{ sub(/GET/, "POST"); print; fflush("") }; NR<2{print "Expires: Thu, 01 Dec 2042 16:00:00 GMT"; print "Cache-Control: public"; }
)

§Socat

§Connect to a TTY over TCP

socat stdio,raw,echo=0,escape=0x11 tcp-connect:${HOST}:${PORT}

§TCP to UDP proxy

socat TCP-LISTEN:9887,reuseaddr,fork UDP:127.0.0.1:9887

This TCP proxy can be forwarded over SSH to have access to an UDP port from localhost from a remote host

§Media manipulation

§Loop a video with ffmpeg

  1. calculate the amount of loops needed using https://dan.hersam.com/tools/time-calc.html
  2. for i in {1..383}; do printf "file '%s'\n" "loop1.mp4" >> list.txt; done
  3. ffmpeg -f concat -i list.txt -c copy output.mp4

See this status

§Download YouTube video cover image