Tech Stacks
    Overview
    System Design Patterns
    CAP Theorem
    C10K and C10M
    Network Programming Models
    Infrastructure as Code
    Examples

Google Cloud Platform

Updated: 2022-09-10

GCP Services

  • Cloud Run: the latest iteration on the "serverless" concept, based on (Docker) containers; under the hood, Cloud Run is actually using Knative; the other serverless products are Cloud Function and App Engine. Cloud Run vs Cloud Functions vs App Engine
  • API Gateway: still in beta but looks like a replacement for Cloud Endpoint. API Gateway vs Cloud Endpoint
  • DataFlow vs Dataproc
  • Cloud Build: uses Docker to execute builds. For each build step, Cloud Build executes a Docker container as an instance of docker run.

GCE

General: n2 > e2. E2 VMs are more tightly packed into physical cores and the idea is to continiously monitor the VM’s usage and “live migrate” it to another hypervisor as and when needed.

Every virtual machine (VM) instance stores its metadata on a metadata server. Your VM automatically has access to the metadata server API without any additional authorization. Metadata is stored as key:value pairs.

Why Googles uses Power?

From this article

Google found that the performance of its web search algorithm, the heart and soul of the company, scaled well with both the number of cores and the number of threads available to it. IBM's POWER9 processor is a many-core, many-thread beast. Variants of the chip range from 12 to 24 cores, with eight threads per core for the 12-core version and four threads per core for the 24-core version. Intel's chips support only two threads per core via hyperthreading.

However

They're not well suited for workloads that don't benefit from more threads, which is why the market-share ceiling for POWER isn't all that high.

Google Cloud Datastore vs Cloud Firestore vs Firebase Realtime Database

  • Firestore is the successor of Datastore; Datastore is deprecated.
  • Firebase Realtime Database was the original database offering for Firebase.
  • Cloud Firestore, as the name suggests, is a "joint-venture" of Google Cloud and Firebase, it is the new flagship database offering of Firebase.
  • Both Firestore and Realtime Database are NoSQL, using JSON as data format.

Container Registry vs Artifact Registry

Artifact Registry:

  • the recommended service for container image storage and management on Google Cloud.
  • extends the capabilities of Container Registry.
  • support for both container images and non-container artifacts.

Supported artifact formats:

  • Container images: Docker, Helm
  • Language packages: Java, Node.js, Python
  • OS packages: Debian, RPM

Anthos

Control plane runs on GCP for upgrades, policy management, telemetry

What is Anthos:

  • Anthos = GKE (Kubernetes) + Service Mesh (Istio) + Cloud Run (Knative) + Management (UI/CLI, policy, configs, etc) + logging / monitoring

Components:

  • Rebranded as GDC Virtual:
    • Anthos on VMware: run GKE in a vSphere environment.
      • OS->vSphere->Kubernetes
      • Anthos manages os and packages.
    • Anthos on Bare Metal: run GKE in on-prem data centers. WITHOUT VMware.
      • OS->(KubeVirt)->Kubernetes
      • Customer manages os and packages.
      • KubeVirt: optional; enabless Kubernetes orchestration of VMs running on KVM. For legacy virtualized workloads.
      • CLI:
        • bmctl: manages admin cluster
        • kubectl and Anthos K8s API: manage user clusters
  • Anthos Service Mesh (ASM): a Google productionized version of OSS Istio.
  • Anthos Config Management (ACM)
  • Anthos Identity Service (AIS): just a proxy, not a full IdP. Implemented as a webhook authentication.
  • Anthos obervability based on GCP: Stackdriver API, Cloud Monitoring API, Cloud Logging API

Anthos bare metal

Managing Kubernetes on customer managed Operating Systems. Uses Cilium, MetalLB, and Ansible for machine management.

ABM Kubevirt support: support for managing VMs on top of ABM (server nodes are often too beefy to be one node for customers) Customers bring and manage their nodes, while Google is responsible for installing / upgrading Anthos components (e.g., GKE, ACM, ASM, Cloud Run).

From the user’s point of view, Anthos Bare Metal has an API in the form of Kubernetes resource model. But under the hood, Anthos Bare Metal utilizes Ansible as the mechanism to manage nodes.

Uhy ansible:

  • To reduce the cost of supporting different operating system distributions, since it provides some level of abstraction and framework.
  • To avoid interfering with the customer’s existing management tools, since it does not require the installation of an agent.

GDC

GDC = Google Distributed Cloud. Extensions of GCP.

  • GDC Virtual: connected, software only, running on customer's infra. (Basically a rebranded Anthos on-prem + Anthos Bare Medel)
    • GDCV customers can bring Google Cloud’s software stack to their own data center servers. Moreover, with the service, they can operate various on-premises applications using the same Google Cloud APIs, control planes, hardware, and tooling as their cloud-hosted apps.
    • uses Google Cloud Console to provision Anthos clusters on vSphere or bare metal:
  • GDC Edge: connected, Google managed hardware + software; for low latency, data residency hybrid workloads.
  • GDC Hosted: air-gapped, pre-configured hardware + software, managed by Google or a trusted partner, for the most sensitive workloads.
    • does not require connectivity to Google Cloud at any time to manage infrastructure, services, APIs, or tooling, and uses a local control plane provided by Anthos for operations.
    • diff from ABM: airgapped from the Internet, i.e. no direct access to public cloud, so need special ops personnel to manage.

marketplace service vs managed service

Marketplace: user gets a container and a guide only, need to manage the whole lifecycle on their own; no SLA; no data backup