Kubernetes - Versus

Last Updated: 2023-08-27

Spec vs Status

• Spec: desired state
• Status: actual state
  • contains any information we want users or other controllers to be able to easily obtain.
  • we never read our own status

Controllers vs Webhook

• Controller: a client of Kubernetes.
• Webhook: Kubernetes is the client and calls out to a remote service. The remote service is called a Webhook Backend.

Controller Manager vs Controllers vs Reconcilers

• a controller manager manages a set of controllers; packaged as a cmd / container image.
• a controller consists a set of reconcilers.
• a reconciler reconciles resources.
  • https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile

Kind vs Resources vs Object

TL;DR: kind vs resource => class vs instances; resource = a collection of objects.

• Kind = the name of an Object Schema: every object returned or accepted by the API must conform to an object schema
• Resources = resource types + resource instances
  • Resource types are organized into API groups, and API groups are versioned. (API Groups e.g. apps/v1)
  • Every resource representation follows a certain schema defined by its kind.
• an Object is a resource instance; the reverse is not true: not every resource represents a Kubernetes Object; Kubernetes Objects are stored in etcd.
• Resources are named in all lower case, plural form, e.g. persistentvolumes; Kinds are named in CamelCase, singular form, e.g. PersistentVolume.
• every API endpoint deals with objects of a certain resource type.

Often, there’s a one-to-one mapping between Kinds and resources. For instance, the pods resource corresponds to the Pod Kind. However, sometimes, the same Kind may be returned by multiple resources. For instance, the Scale Kind is returned by all scale subresources, like deployments/scale or replicasets/scale. With CRDs, however, each Kind will correspond to a single resource. resources are always lowercase, and by convention are the lowercase form of the Kind.

When we refer to a kind in a particular group-version, we’ll call it a GroupVersionKind, or GVK for short. each GVK corresponds to a given root Go type in a package.

Resources vs Objects

Most of the Kubernetes API resources represent Objects.

• Some of the resources only mandate the kind field.
• Objects require kind, apiVersion, metadata (namespace, name, uid).

Kubernetes Objects are persistent entities, which represent the intent (desired state, spec) and the status (actual state, status)

Service Accounts vs User Accounts

Service accounts are managed within k8s (ServiceAccount resource), user accounts are managed outside of k8s.

Level triggered vs edge triggered.

Kubernetes is Level triggered instead of edge triggered. "Edge" means it monitors a change of voltage level, while "Level" means it monitors a consistent voltage level.

Edge triggered: e.g. when the hardware needed attention, it would signal the CPU "interrupt"; interrupts used to be a literal wire from the device to the CPU, the device can change the voltage on the wire, the CPU can detect changes in voltage. Problem: if the CPU doesn't detect the pulse, the interrupt doesn't get serviced.

Level triggered: the device change the voltage and keep it there until the interrupt is serviced. The CPU can't miss edges.

For kubernetes: State is more useful than events; clients can check and re-check state at any time. k8s's controller model

Reconciliation is level-based, meaning action isn't driven off changes in individual Events, but instead is driven by actual cluster state read from the apiserver or a local cache.

All nodes watch the kubernetes API and figure out what they need to do, instead of master sending out instructions

Benefits

• no single point of failure
• simple master components

PV vs PVC

• PersistentVolume (PV): the actual storage.
• PersistentVolumeClaim (PVC): a declaration of need for storage by the user.

Users use PVC to say "I need XXX TB storage" but do not care how that requirement is fulfilled (e.g. which file system or which cloud provider), so the implementation details are abstracted away.

Annotation vs Label

Both are ways to attach metadata to objects in Kubernetes.

• Labels are for Kubernetes, annotations are for humans.
• Labels are used in conjunction with selectors, to group a set of related resources.
• Labels are constrained by RFC 1123 (e.g. a maximum 63 character length).
• Annotations are used for "non-identifying information" i.e., metadata that Kubernetes does not care about. Clients such as tools and libraries can retrieve this metadata.

Schema vs Scheme

• Schema: the go types, e.g. schema.GroupVersionKind

  • k8s.io/apimachinery/pkg/runtime/schema

• Scheme: defines methods for serializing and deserializing API objects.

  • sigs.k8s.io/controller-runtime/pkg/scheme

    // Create a scheme
    scheme *runtime.Scheme = runtime.NewScheme()
    
    // Add to the scheme
    utilruntime.Must(myapigroupv1.AddToScheme(scheme))
    

Note about AddToScheme: Avoid adding to scheme on a running process. This will cause concurrent read and write for the schemes. Consider adding it during registration or in init().

Taints and Tolerations

• Taints: set on a Node; it marks that the node should not accept any pods that do not tolerate the taints.("Node: I'm flawed ...")
• Tolerations: set on a Pod. ("Pod: that's ok, I do not care!")

Group vs Version vs apiVersion

TL;DR: apiVersion = Group + "/" + Version

"k8s.io/apimachinery/pkg/runtime/schema" schema.GroupVersion:

type GroupVersion struct {
  Group string
  Version string
}

k8s.io/apimachinery/pkg/runtime/schema/group_version.go

type GroupVersionKind struct {
  Group string
  Version string
  Kind string
}

e.g.

GroupVersion = schema.GroupVersion {
  Group: "kind.x-k8s.io",
  Version: "v1alpha4",
}

apiVersion = Group + "/" + Version, e.g. kind.x-k8s.io/v1alpha1

Deployment vs Service

• A deployment without a service: the deployment could be scaled up and down and pods could be replicated. Each pod could be accessed individually via direct network requests, rather than abstracting them behind a service.
• A service without a deployment: create each pod individually then the service routes network requests to the pods.

Deployment vs StatefulSet vs DaemonSet

• Deployment for stateless services, e.g. web servers.

  • Pod replicas in deployments share the same persistent volume.

• StatefulSet is for stateful services, e.g. db.

  • Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.

  • If a volume is configured, each pod will be provision its own persistent volume

  • Better use headless services (service without an IP address), clients can connect directly to what lies behind it. Headless services has clusterIP: None.

    kind: Service
    spec:
      clusterIP: None
    

• DaemonSet: one copy per worker node.

Service vs Pod

• Service has a stable IP address that lasts for the life of the service; each pod has its own address but pods may come and go and their IP addresses change.
• Service provides load balancing across pods.

Gateway vs Ingress

The Gateway API project is the successor to the Ingress API.

# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress

# Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway

generic PLEG vs evented PLEG

PLEG: Pod lifecycle event generator. For kubelet to get container status updates.

• generic PLEG: polling based container status updates.
  • incurs non-negligible overhead due to frequent polling of container statuses.
• evented PLEG: since 1.27, event based updates for container status.
  • to reduce unnecessary work during inactivity by replacing periodic polling.