Linux / Unix

Updated: 2020-06-29

Unix Philosophy

Do one thing and do it well.

"Everything is a file"

Probably the most important and defining feature of Unix. A text file is of course a file, but a directory is also a file, and a hard drive is a file, a mouse is a file, a socket is a file ...

This is super helpful for learners because everything you encounter would be a file and can be examined by commands like ls.

Flavors

macOS is actually a Unix system, originally branched off BSD.

Linux is another big branch of Unix, some of the popular flavors that you may have heard of:

  • Debian / Ubuntu

    • Ubuntu is derived from Debian.
    • Ubuntu is probably the most popular Desktop Linux.
    • Both use apt as package manager.
  • RedHat / CentOS / Fedora

    • RedHat is for enterprise use; CentOS is the open source version; Fedora is the community version.
    • All use yum as package manager.
  • Gentoo / ChromeOS

    • ChromeOS was originally based on Debian, but later moved to Gentoo. (if you run lsb_release -a in ChromeOS's terminal you will see Debian instead, that is because the terminal and other linux programs are actually running containers inside a VM(on KVM))
  • Android: yes Android is also based on Linux; however Google's new OS, Fuchsia, is not based on Linux, but a microkernel called Zircon.

Commands

You mean find something like this when discussing commands:

$ ls -l

here ls is the command, -l is the parameter, and the leading $ is a convention:

  • $ <command>: run as a regular user
  • # <command>: run as root. (We do not use it, but use$ sudo <command> instead)

Current Working Directory

ls lists CWD if given no other parameters.

Check CWD, these 2 are equivalent:

$ pwd # Print Working Directory
$ echo $PWD

Different From Windows

  • Unix uses / in file path, while Windows uses \
  • there's only one root directory /, so you cannot find C:\ or D:\; however other filesystems can be mounted

    $ mount # lists all mounts
    ...
    C:\ on /mnt/c type drvfs
    D:\ on /mnt/d type drvfs
  • Unix (or more precisely, the file systems like ext4) is case sensitive, a.txt and A.txt are two different files.

Getting Helps

Most commands have built-in help info, usually with the --help or -h option, e.g. $ ls --help will give you more info about the command ls.

Or open the manual by calling man, e.g. $ man ls will show the manual page of ls.

Notice that at the very beginning of the manual, the command often has a number associated with it, like LS(1), the number is the section of the manual:

  • Section 1: user commands
  • Section 2: system calls
  • Section 3: library functions
  • Section 4: special files
  • Section 5: file formats
  • Section 6: games
  • Section 7: conventions and miscellany
  • Section 8: administration and privileged commands
  • Section L: math library functions
  • Section N: tcl functions

This info can be found in the manual of manual: $ man man.

The same entry may appear in different sections, e.g. to check manual for mount in system calls vs admin command:

$ man 2 mount # the system call
$ man 8 mount # the admin command

Use man -k to search the man pages, e.g. $ man -k mount will not only return mount but also umount, cgroupfs-mount, etc. This is equivalent to apropos mount

Use man -f to list all sections that have the term.

$ man -f mount
mount (8)            - mount a filesystem
mount (2)            - mount filesystem

Unix Domain Socket

A.k.a. IPC socket. Similar to an Internet Socket.

Internet Sockets use the IP address and a port number as socket address, e.g. 10.20.30.40:4444; Unix Domain Sockets use the file system as the address space (everything in Unix is a file) e.g. /var/run/docker/containerd/containerd.sock

Standards: SUS vs POSIX vs LSB

These are all standards for operating system interfaces.

  • POSIX: Portable Operating System Interface
  • SUS: Single UNIX Specification
  • LSB: Linux Standard Base

POSIX and SUS converged after 2001.

  • macOS is POSIX-certified.
  • Linux is mostly POSIX-compliant. The LSB is based on the POSIX / SUS, and several other open standards, but extends them in certain areas.

Invoking Bash with the --posix option or stating set -o posix in a script causes Bash to conform very closely to the POSIX

SELinux

SELinux: an implementation of Mandatory Access Control (MAC) as contrasted to the standard Unix model of Discretionary Access Control (DAC)

SELinux comes installed by default on Red Hat distributions

To check your SELinux mode, run sestatus and check the output. For example:

$ sestatus
SELinux status:                 disabled

$ sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing

Mode from config file:          error (Success)
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      28

concepts:

  • Multi-Level Security (MLS)
  • Multi-Category Security (MCS)

Permissive vs Enforcing Mode

  • Permissive Mode: SELinux will log access control infringements but will not enforce them
  • Enforcing Mode: enforce!

PAM

pam - Pluggable Authentication Modules for Linux

TTY

TTY: teletype, now refers to any device that opens a physical or virtual terminal session.

Serial Port Terminals

Each serial port is considered to be a "device". e.g. /dev/ttys0

Pseudo Terminals

Pairs of devices such as /dev/ptyp3 and /dev/ttyp3; no physical device directly associated with either.

Controlling Terminal

/dev/tty

SSH to a Linux server (Ubuntu)

$ tty
/dev/pts/1

On a Mac

$ tty
/dev/ttys001

Ubuntu PPA

PPA: Personal Package Archives

Backward Compatibility

The Linux kernel community promised that no upgrade will ever break anything that is currently working in a previous release

udev

udev (userspace /dev)

/dev is static, udev is for plugable devices running in userspace.

To be able to deal with peripheral devices that are hotplug-capable in a user-friendly way, a part of handling all of these hotplug-capable hardware devices was handed over from the kernel to a daemon running in user-space. Running in user space serves security and stability purposes.

Microkernel

Provides the most basic functionalities: low-level address space management, thread management, IPC, etc.

While other monolithic kernel functionalities are not included and instead run in user space: device drivers, file systems, protocol stacks.

Google's new OS, Fuchsia, is based on a microkernel named Zircon. Fuchsia is NOT Linux.

Other Tools

sed(stream editor) is for programmatically editing files based on lines.

awk is for text processing, especially useful for table-like text files like csv.