Updated: 2019-06-15

What is Container?

Think of "container" as just another packaging format.

Just like .iso files for disk images, .deb/.rpm for linux packages, or .zip/.tgz for binary or arbitrary files.

The ecosystem is more than just a format, it includes:

  • Image(package)
  • Distribute
  • Runtime
  • Orchestration

What is cgroup and namespaces

There are 7 namespaces in Linux:

  • cgroup
  • IPC
  • Network
  • Mount
  • PID
  • User
  • UTS: UNIX Timesharing System, named after the data structure used to store info returned by uname system call. Isolates hostname and NIS domain name.

VM vs (Traditional) Container vs Sandboxed Container

VM: On top of Hypervisor, and each VM has its own guest OS

     VM1        VM2
|   App    |   App    |
|==========|==========| => System Calls
|  Guest   |  Guest   |
|  Kernel  |  Kernel  |
| Virtual  | Virtual  |
| Hardware | Hardware |
|   Hypervisor(VMM)   |
|=====================| => System Calls
|     Host Kernel     |
|    Host Hardware    |

(Traditional) Container: Operating system level virtualization. The kernel imposes limits on resources, implemented through use of cgroups and namespaces

|   App    |   App    |
|   Container Layer   |
|     Host Kernel     |
|    Host Hardware    |

Sandboxed Container(e.g. gVisor): provides a user-space kernel

|   App    |   App    |
|==========|==========| => System Calls
|        gVisor       |
|=====================| => Limited System Calls
|     Host Kernel     |
|    Host Hardware    |

OCI: Open Container Initiative

Defines 2 important specs, so different tools can be used to pack/unpack and run by different runtimes:

Notable Projects

  • Docker: an open source Linux containerization technology. Package, distribute and runtime solution.
  • cgroup: limits and isolates resources(CPU, memory, disk I/O, network, etc)
  • lxc(linuxcontainer)
  • CoreOS
  • containerd: Container Runtime
  • rkt: Container Runtime
  • gVisor: a user-space kernel for containers. It limits the host kernel surface accessible to the application while still giving the application access to all the features it expects. It leverages existing host kernel functionality and runs as a normal user-space process. For running untrusted workloads. Lower memory and startup overhead compared to a full VM.


Docker's default runtime: runC

$ docker run --runtime=runc ...

gVisor can be integrated with Docker by changing runc to runsc("run sandboxed container)

$ docker run --runtime=runsc ...

gVisor runs slower than default docker runtime due to the "sandboxing":


  • Kuberenetes
  • Swarm
  • Mesos
  • Nomad

LXC vs LXD vs cgroups

  • Linux Containers (LXC): on top of cgroups. operating system–level virtualization technology for running multiple isolated Linux systems (containers) on a single control host (CoreOS instance).
  • cgroups: provides namespace isolation and abilities to limit, account and isolate resource usage (CPU, memory, disk I/O, etc.) of process groups
  • LXD: similar to LXC, but a REST API on top of liblxc

Docker vs LXC/LXD

  • Docker: application container; LXC/LXD: system container
  • Docker initially used liblxc but later changed to libcontainer

Who's (Not) Using Containers?

Well it is gaining momentum and popularity. Many companies are adopting it.

Two notable exceptions are: Google and Facebook

Google has its own packaging format: MPM. MPM on Borg is similar to container on Kubernetes, and Kubernetes is the open-source version of Borg.

Facebook use Tupperware. Why not docker or coreos? They didn't exist then.

Tupperware resources:


  • 2 most important APIs: Images and Container APIs



  • Manager quorum: Raft: exchange information with strong consistency
  • Worker: Gossip: share information in bulk, converge fast
  • Between manager and worker: GRPC(on top of HTTP/2, versioned)

Docker for Mac

The Docker for Mac application does not use docker-machine to provision that VM; but rather creates and manages it directly.

Docker Compose, Docker Machine, Docker Engine

Compose is a tool for defining and running multi-container Docker applications.

Docker Machine is a tool for provisioning and managing your Dockerized hosts (hosts with Docker Engine on them). Typically, you install Docker Machine on your local system. Docker Machine has its own command line client docker-machine and the Docker Engine client, docker

Docker Engine, the client-server application made up of the Docker daemon, a REST API that specifies interfaces for interacting with the daemon, and a command line interface (CLI) client that talks to the daemon (through the REST API wrapper). Docker Engine accepts docker commands from the CLI, such as docker run <image>,docker ps to list running containers, docker images to list images, and so on.

Unlike traditional virtualization, containerization takes place at the kernel level. Most modern operating system kernels now support the primitives necessary for containerization, including Linux with openvz, vserver and more recently lxc, Solaris with zones, and FreeBSD with Jails.

Because Docker operates at the OS level, it can still be run inside a VM!


Both CMD and ENTRYPOINT instructions define what command gets executed when running a container. There are few rules that describe their co-operation.

Dockerfile should specify at least one of CMD or ENTRYPOINT commands. ENTRYPOINT should be defined when using the container as an executable. CMD should be used as a way of defining default arguments for an ENTRYPOINT command or for executing an ad-hoc command in a container. CMD will be overridden when running the container with alternative arguments.