Terminal Velocity

I admit it. I was a Microsoft user for a long time. In the DOS days I did my fair share of command line business. I then got suckered into the crutches of window based things. To get really fast and efficient, command line mastery is essential. I would first start with learning about the UNIX philosophy as it totally lays the foundation for further understanding.

Terminal basics - shell

What is a shell? Like most things, the key is in the name. A shell of anything covers the thing inside. A shell then is a program that receives and interprets commands from "outside" and interfaces with the inner workings of that which is "inside" i.e. the inner workings of the operating system. Watch this.

By default your terminal runs a shell called bash. There are flavours of such programs and you can easily change shells. You can see what your shell is by running:

env

The env command lets you look at your environment so it displays a lot of information. For example, I have "oh my zsh" and use iTerm:

...
LANG=en_GB.UTF-8
PWD=/
SHELL=/bin/zsh
TERM_PROGRAM=iTerm.app
...

Help

If you get stuck with a command, help is always at hand. You can type man [command] - to access the manual page. To display all the commands available in the shell:

man source

Use apropos to find what man page is appropriate.

Cutsomisation

You can customise your shell by adding commands, aliases etc into a file, depending on your environment. Sometimes it is a .bashrc file in your home directory. Or, if you use zshell then that is the .zshrc.

Directories

To find out where I am I use pwd which stands for "print working directory". Once you know where you are you can see the contents of the directory using ls and use flags such as -l or -la. cd ../../somewhere changes directory.

Creating a directory:

mkdir test

Creating and moving to that directory:

mkdir test && cd test

Note, the && can be used with any command that follows each other, such as cd and then ls.

Creating a nested folder structure:

mkdir test test/sub1 test/sub1/subsub1

Using the -p flag:

mkdir -p /parent/child/test

A really handy way to create a multiple nested folder structure:

mkdir -p parent/{child1, child2, child3}

This makes documents, with 3 sub folders for years, each having 4 seasons:

mkdir -p /documents/{2011,2012,2013}/{Winter,Spring,Summer,Fall}

"Symlinks" (symbolic or soft links) are so powerful and useful. If you want to group folders in one location, but you don't want to move them from where they already exists, symlinks are for you. You create a symlink by using the -s flag:

ln -s /path/to/existing/file mycurrentfolder

Files

Using touch to create an html file:

touch index.html

Creating multiple files at once

touch index.htm style.css main.js

I've seen TJ Hollowaychuk use something like this (though I can't get it working myself!):

touch !!:2/{file.ext,file.ext}

The echo command prints arguments passed, but we can use those arguments for creating a file and writing to a file:

echo "Your copy" > targetfile.txt

If you want to view the contents of a file use less to paginate through the file or use cat to print out the whole file.

Removing stuff

Remove multiple files having the same name:

rm lib/*/filename.txt

Delete all files and folders under a folder:

rm -rf /var/www/idiots-guide/content/*

Renaming stuff

Mac terminal batch rename file. Just to make sure things don't go tits up, do an echo first:

for filename in *.png; do echo mv "$filename" "${filename//_cyan/}"; done

Then do the real thing if all is ok:

for filename in *.png; do mv "$filename" "${filename//_cyan/}"; done

Mac terminal batch lowercase files:

for i in *; do mv "$i" "$(echo $i|tr A-Z a-z)"; done

Duplicating

The ditto command copies source files and structure into destination:

ditto folder1 folder2

Compress files using Zip

zip -r data.zip /data

Unzip

unzip data.zip -d /data

Using Tar

The tar "tape archive".

tar czf target.tar.gz foldertocompress

and to extract:

tar xzf target.tar.gz -C target

a variation:

tar zxvf target.tar.gz -C /my/folder/

-z runs the file through gzip
-x extract the file
-v verbose output
-f specifies the file name to decompress

File and folder information

To display the sizes of folders in a directory

du -sh */

Find the top ten largest files:

du -hsx * | sort -rh | head -10

Determining how much disk space you are using:

df

or

df -h

The above also shows mounted drives. To see a specific list of mounted drives run:

mount

Open ports

To find out what ports your machine is listening on use:

lsof -i | grep LISTEN

Ownership

Change ownership of files or directories:

chown [flags] owner[:ownergroup] file/folder

Recursively:

sudo chown -Rv username directory

User admin

Creating a user is easy:

useradd -G groupname username

To change your password run passwd. You will be prompted for your current password, and also to enter your new password (if successful) and enter it again to make sure.

To see groups for a user:

groups username

To see who is logged in:

w

or

who

To set user permissions:

visudo

You can also run a one-off command as a user:

su pm2 -c 'pm2 list'

Flags

Flags are parameters that you can pass to the command function you are using.

  • -h = human readable form (so file sizes would be displayed as "1.5G" for example)
  • -s = summary, show total for each
  • -r = reverse
    -10 = 10 = n = number

Running a process in the background

Sometimes we don't want processes taking up terminal windows so a handy command is to run the process in the background by appending an &:

elasticsearch --cluster.name test --node.name local_dev &

This creates a new sub-shell asynchronously, and returns 0 meaning

echo "Hello" &
echo $!

Which returns:

37331
Hello

The 37331 depends on your machine obvs. The $! is a special variable.

/etc/sercurity/limits.conf

UNIX/Linux operating systems have the ability to limit the amount of various system resources available to a user process.

There are 'soft limits' and 'hard limits'. Hard limit is the total max, soft limit lets the user know, and they can 'up' the level if needed. But they cannot use above the hard limit.

See thegeekdiary

SSH tunnel and reverse tunnel

See how does reverse ssh tunneling work.

SSH Keys

Basically, run ssh-keygen, name file (full path) and provide password (optional). See adding public/private key pairs on a Mac

To add to the keychain you also may need to add in .zshrc to add keys into memory.

Others

When using yeoman, grunt, gulp, ionic, brunch, et al, sometimes you get an error saying too many files are being watched. One way around this is to increase the limit of watched files using ulimit

ulimit -n 10000

Use tail to monitor a file (such as a log file):

tail -F nginx.log

To see the current Process list:

ps aux

Or use grep to find things inside files:

ps aux | grep pm2

To count the number of files in a nested directory:

find . -type f | wc -l

To find files from current directory (recursive):

find . -name "*.jpg"

scp

scp [options] username1@source_host:directory1/filename1 username2@destination_host:directory2/filename2

Checkout scp

Crontab

Want to automate certain tasks? Crobtab is for that. You can specify scripts to be run at certain times, certain intervals etc. Here is a cool tutorial on crontab.

List all current tasks:

crontab -l

List tasks for a particular user:

cronatab -l -u bobjonson

To edit tasks:

crontab -e

Edit a specific users' crontab:

crontab -e -u bobjonson

Kill

Sometimes there are runaway processes. Especially in Node/VSCode. In Unix,

lsof -i tcp:3001

netstat -anp | grep 3333 -c
netstat -anp | grep 3001 -c

Fin

Use exit to exit the shell.

Resources: