Commands

Updated: 2020-12-29

Not a comprehensive manual, but some notes and examples for quick lookup.

For example

$ ls -l

here ls is the command, -l is the parameter, and the leading $ is a convention:

  • $ <command>: run as a regular user
  • # <command>: run as root. (We do not use it, but use$ sudo <command> instead)

Getting Helps

Most commands have built-in help info, usually with the --help or -h option, e.g. $ ls --help will give you more info about the command ls.

Or open the manual by calling man, e.g. $ man ls will show the manual page of ls.

Notice that at the very beginning of the manual, the command often has a number associated with it, like LS(1), the number is the section of the manual:

  • Section 1: user commands
  • Section 2: system calls
  • Section 3: library functions
  • Section 4: special files
  • Section 5: file formats
  • Section 6: games
  • Section 7: conventions and miscellany
  • Section 8: administration and privileged commands
  • Section L: math library functions
  • Section N: tcl functions

This info can be found in the manual of manual: $ man man.

The same entry may appear in different sections, e.g. to check manual for mount in system calls vs admin command:

$ man 2 mount # the system call
$ man 8 mount # the admin command

Use man -k to search the man pages, e.g. $ man -k mount will not only return mount but also umount, cgroupfs-mount, etc. This is equivalent to apropos mount

Use man -f to list all sections that have the term.

$ man -f mount
mount (8)            - mount a filesystem
mount (2)            - mount filesystem

Current Working Directory

ls lists CWD if given no other parameters.

Check CWD, these 2 are equivalent:

$ pwd # Print Working Directory
$ echo $PWD

Nohup

Run command in background and redirect output to file.

$ nohup command > output &

Quote from Wikipedia: nohup is a POSIX command to ignore the HUP (hangup) signal, enabling the command to keep running after the user who issues the command has logged out. The HUP (hangup) signal is by convention the way a terminal warns depending processes of logout. Then use

$ tail -f output

to print the last few lines of the output.

Jobs / Foreground / Background

List all jobs:

$ jobs

Run something in background

$ <command> &

Foreground to Background:

[Ctrl + z]

$ bg

Background to Foreground:

# something is running in background
$ fg

More or Less

  • more: can only go forward, bash autocompletion uses more
  • less: can go backward, search, and more; almost everything else uses less

dd

The dd Unix utility program reads octet streams from a source to a destination, possibly performing data conversions in the process.

Create File With Zeroes

Creating a 1 MiB file, called foobar, filled with zeroes:

dd if=/dev/zero of=foobar count=1024 bs=1024

Note: The block size value can be given in SI (decimal) values, e.g. in GB, MB, etc. To create a 1 GB file one would simply type:

Test IO Performance

$ dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 3.94954 s, 272 MB/s

Read operations from /dev/zero return as many null characters (0x00) as requested in the read operation.

Unlike /dev/null, /dev/zero may be used as a source, not only as a sink for data.

dd if=/dev/zero of=/dev/<destination partition>

dd if=/dev/zero of=foobar count=1 bs=1GB

diff

diff sdiff vimdiff colordiff

env

Set PATH for sudo in Ubuntu

In Ubuntu, these commands are showing different results:

$ env
$ sudo env

This may cause a problem when you are setting some bin folder in your path while you need root permission to execute, for example, if you install node in your own folder while you need $ sudo node bin/www to serve it, you will get an error saying node cannot be found.

Solution

$ sudo env PATH=$PATH [COMMAND]

this will use your own PATH when executing the COMMAND, e.g.

$ sudo env PATH=$PATH node bin/www

or add this to ~/.bashrc

alias sudo='sudo env PATH=$PATH'

ldd

ldd: print shared library dependencies

$ ldd ./my-program
not a dynamic executable
$ ldd /usr/bin/gzip
linux-vdso.so.1 =>  (0x00007fff39fff000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fb842afa000)
/lib64/ld-linux-x86-64.so.2 (0x00007fb842ea0000)
$ ldd /usr/bin/ssh
linux-vdso.so.1 =>  (0x00007fffd0164000)
libfipscheck.so.1 => /lib64/libfipscheck.so.1 (0x00007fb62de63000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fb62dc43000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007fb62d8a9000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fb62d6a6000)
libz.so.1 => /lib64/libz.so.1 (0x00007fb62d48f000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fb62d276000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fb62d03f000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fb62ce24000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fb62cbe0000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fb62c8fa000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fb62c6cd000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fb62c4c9000)
libnss3.so => /usr/lib64/libnss3.so (0x00007fb62c18d000)
libc.so.6 => /lib64/libc.so.6 (0x00007fb62bdf8000)
libplc4.so => /lib64/libplc4.so (0x00007fb62bbf3000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fb62b9ef000)
/lib64/ld-linux-x86-64.so.2 (0x00007fb62e2da000)
libfreebl3.so => /lib64/libfreebl3.so (0x00007fb62b78c000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fb62b581000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fb62b37e000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb62b160000)
libnssutil3.so => /usr/lib64/libnssutil3.so (0x00007fb62af3a000)
libplds4.so => /lib64/libplds4.so (0x00007fb62ad36000)
libnspr4.so => /lib64/libnspr4.so (0x00007fb62aaf8000)

LD_LIBRARY_PATH to include this path for the application to successfully run.

ls

Set Color

$ ls --color=auto
$ ls --color=tty
$ ls --color=none

seq

Generate Sequence

Generate sequence

$ seq 5856 5859
5856
5857
5858
5859

Set delimiter by -s ('\n' As default)

$ seq -s ',' 5856 5859
5856,5857,5858,5859,

tar

  • x: extract
  • z: compress archive using gzip program
  • c: create archive
  • v: verbose
  • f: archive file name

.tar

Create:

$ tar -cvf archive_name.tar directory_to_compress

Extract the archive:

$ tar -xvf archive_name.tar.gz

Extract the files to a different directory:

$ tar -xvf archive_name.tar -C /tmp/extract_here/

.tar.gz

Create and compress

$ tar -cvzf archive_name.tar.gz directory_to_compress

Decompress and extract:

$ tar -xvzf archive_name.tar.gz

Extract the files to a different directory:

$ tar -zxvf archive_name.tar.gz -C /tmp/extract_here/

.tar.bz2

Create

$ tar -jcvf archive_name.tar.bz2 directory_to_compress

Extract:

$ tar -jxvf archive_name.tar.bz2 -C /tmp/extract_here/

type

Shell Builtin

Linux built in commands:

$ type pwd cd
pwd is a shell builtin
cd is a shell builtin

And type itself is a shell builtin

$ type type
type is a shell builtin

Shell Keyword

Shell keywords are used in shell scripts

$ type if fi
if is a shell keyword
fi is a shell keyword

Alias

If you have alias defined in .bash_profile or .bashrc, for example

alias ls="ls -G"

where -G is set to enable colorized output. Then ls becomes an alias:

$ type ls
ls is aliased to `ls -G'

Executable

If you have docker installed, you will see something like this:

$ type docker
docker is /usr/local/bin/docker

Show all python executables

$ type -a python

xargs

Similar to “map”, apply functions on each item in the list

e.g. kill all ssh connections

$ ps -ax | grep ssh | cut -d ' ' -f 1 | xargs kill -9

zip

Compress

$ zip -r archive_name.zip directory_to_compress

Extract:

$ unzip archive_name.zip

Trouble Shooting

Error

$ unzip *.zip
error: caution: filename not matched:

Solution

$ unzip \*.zip

which vs whereis

which: based on PATH

$ which hadoop
/Users/myname/lib/hadoop

whereis: based on standard binary directories

$ whereis hadoop
hadoop: /usr/bin/hadoop /etc/hadoop /usr/lib/hadoop /usr/share/man/man1/hadoop.1.gz

Check If Command Exists

$ command -v foo >/dev/null 2>&1
$ type foo >/dev/null 2>&1
$ hash foo 2>/dev/null

Example: add HADOOP_CLASSPATH if hadoop exists

command -v hadoop > /dev/null && {
    HADOOP_CLASSPATH=`hadoop classpath`
    CLASSPATH=$HADOOP_CLASSPATH:$CLASSPATH
}

Run at a specific time

$ at 2:00 PM
cowsay 'hello'
(CTRL + D)

the command will run at 2 PM

List all jobs

$ at -l
1	Tue Apr 12 14:00:00 2016

Remove a job

$ at -r 1

Cron

/etc/crontab schedules /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly, /etc/cron.monthly.

Scripts in those folders will be executed.

By default cron jobs logs can be found in /var/log/syslog

$ grep CRON /var/log/syslog

TTY

TTY: teletype, now refers to any device that opens a physical or virtual terminal session.

Serial Port Terminals

Each serial port is considered to be a "device". e.g. /dev/ttys0

Pseudo Terminals

Pairs of devices such as /dev/ptyp3 and /dev/ttyp3; no physical device directly associated with either.

Controlling Terminal

/dev/tty

SSH to a Linux server (Ubuntu)

$ tty
/dev/pts/1

On a Mac

$ tty
/dev/ttys001