I admit it. I was a Microsoft user for a long time. In the DOS days I did my fair share of command line business. I then got suckered into the crutches of window based things. To get really fast and efficient, command line mastery is essential. I would first start with learning about the UNIX philosophy as it totally lays the foundation for further understanding.
Terminal basics - shell
What is a shell? Like most things, the key is in the name. A shell of anything covers the thing inside. A shell then is a program that receives and interprets commands from "outside" and interfaces with the inner workings of that which is "inside" i.e. the inner workings of the operating system. Watch this.
By default your terminal runs a shell called bash. There are flavours of such programs and you can easily change shells. You can see what your shell is by running:
env
The env command lets you look at your environment so it displays a lot of information. For example, I have "oh my zsh" and use iTerm:
...
LANG=en_GB.UTF-8
PWD=/
SHELL=/bin/zsh
TERM_PROGRAM=iTerm.app
...
Help
If you get stuck with a command, help is always at hand. You can type man [command] - to access the manual page. To display all the commands available in the shell:
man source
Use apropos to find what man page is appropriate.
Cutsomisation
You can customise your shell by adding commands, aliases etc into a file, depending on your environment. Sometimes it is a .bashrc file in your home directory. Or, if you use zshell then that is the .zshrc.
Directories
To find out where I am I use pwd which stands for "print working directory". Once you know where you are you can see the contents of the directory using ls and use flags such as -l or -la. cd ../../somewhere changes directory.
Creating a directory:
mkdir test
Creating and moving to that directory:
mkdir test && cd test
Note, the && can be used with any command that follows each other, such as cd and then ls.
Creating a nested folder structure:
mkdir test test/sub1 test/sub1/subsub1
Using the -p flag:
mkdir -p /parent/child/test
A really handy way to create a multiple nested folder structure:
mkdir -p parent/{child1, child2, child3}
This makes documents, with 3 sub folders for years, each having 4 seasons:
mkdir -p /documents/{2011,2012,2013}/{Winter,Spring,Summer,Fall}
Symlinks
"Symlinks" (symbolic or soft links) are so powerful and useful. If you want to group folders in one location, but you don't want to move them from where they already exists, symlinks are for you. You create a symlink by using the -s flag:
ln -s /path/to/existing/file mycurrentfolder
Files
Using touch to create an html file:
touch index.html
Creating multiple files at once
touch index.htm style.css main.js
I've seen TJ Hollowaychuk use something like this (though I can't get it working myself!):
touch !!:2/{file.ext,file.ext}
The echo command prints arguments passed, but we can use those arguments for creating a file and writing to a file:
echo "Your copy" > targetfile.txt
If you want to view the contents of a file use less to paginate through the file or use cat to print out the whole file.
Removing stuff
Remove multiple files having the same name:
rm lib/*/filename.txt
Delete all files and folders under a folder:
rm -rf /var/www/idiots-guide/content/*
Renaming stuff
Mac terminal batch rename file. Just to make sure things don't go tits up, do an echo first:
for filename in *.png; do echo mv "$filename" "${filename//_cyan/}"; done
Then do the real thing if all is ok:
for filename in *.png; do mv "$filename" "${filename//_cyan/}"; done
Mac terminal batch lowercase files:
for i in *; do mv "$i" "$(echo $i|tr A-Z a-z)"; done
Duplicating
The ditto command copies source files and structure into destination:
ditto folder1 folder2
Compress files using Zip
zip -r data.zip /data
Unzip
unzip data.zip -d /data
Using Tar
The tar "tape archive".
tar czf target.tar.gz foldertocompress
and to extract:
tar xzf target.tar.gz -C target
a variation:
tar zxvf target.tar.gz -C /my/folder/
-z runs the file through gzip
-x extract the file
-v verbose output
-f specifies the file name to decompress
File and folder information
To display the sizes of folders in a directory
du -sh */
Find the top ten largest files:
du -hsx * | sort -rh | head -10
Determining how much disk space you are using:
df
or
df -h
The above also shows mounted drives. To see a specific list of mounted drives run:
mount
Open ports
To find out what ports your machine is listening on use:
lsof -i | grep LISTEN
Ownership
Change ownership of files or directories:
chown [flags] owner[:ownergroup] file/folder
Recursively:
sudo chown -Rv username directory
User admin
Creating a user is easy:
useradd -G groupname username
To change your password run passwd. You will be prompted for your current password, and also to enter your new password (if successful) and enter it again to make sure.
To see groups for a user:
groups username
To see who is logged in:
w
or
who
To set user permissions:
visudo
You can also run a one-off command as a user:
su pm2 -c 'pm2 list'
Flags
Flags are parameters that you can pass to the command function you are using.
- -h = human readable form (so file sizes would be displayed as "1.5G" for example)
- -s = summary, show total for each
- -r = reverse
-10 = 10 = n = number
Running a process in the background
Sometimes we don't want processes taking up terminal windows so a handy command is to run the process in the background by appending an &:
elasticsearch --cluster.name test --node.name local_dev &
This creates a new sub-shell asynchronously, and returns 0 meaning
echo "Hello" &
echo $!
Which returns:
37331
Hello
The 37331 depends on your machine obvs. The $! is a special variable.
/etc/sercurity/limits.conf
UNIX/Linux operating systems have the ability to limit the amount of various system resources available to a user process.
There are 'soft limits' and 'hard limits'. Hard limit is the total max, soft limit lets the user know, and they can 'up' the level if needed. But they cannot use above the hard limit.
See thegeekdiary
SSH tunnel and reverse tunnel
See how does reverse ssh tunneling work.
SSH Keys
Basically, run ssh-keygen, name file (full path) and provide password (optional). See adding public/private key pairs on a Mac
To add to the keychain you also may need to add in .zshrc to add keys into memory.
Others
When using yeoman, grunt, gulp, ionic, brunch, et al, sometimes you get an error saying too many files are being watched. One way around this is to increase the limit of watched files using ulimit
ulimit -n 10000
Use tail to monitor a file (such as a log file):
tail -F nginx.log
To see the current Process list:
ps aux
Or use grep to find things inside files:
ps aux | grep pm2
To count the number of files in a nested directory:
find . -type f | wc -l
To find files from current directory (recursive):
find . -name "*.jpg"
scp
scp [options] username1@source_host:directory1/filename1 username2@destination_host:directory2/filename2
Checkout scp
Crontab
Want to automate certain tasks? Crobtab is for that. You can specify scripts to be run at certain times, certain intervals etc. Here is a cool tutorial on crontab.
List all current tasks:
crontab -l
List tasks for a particular user:
cronatab -l -u bobjonson
To edit tasks:
crontab -e
Edit a specific users' crontab:
crontab -e -u bobjonson
Kill
Sometimes there are runaway processes. Especially in Node/VSCode. In Unix,
lsof -i tcp:3001
netstat -anp | grep 3333 -c
netstat -anp | grep 3001 -c
Fin
Use exit to exit the shell.