logo

Google Cloud Platform

Last Updated: 2023-10-21

GCP Services

  • Cloud Run: the latest iteration on the "serverless" concept, based on (Docker) containers; under the hood, Cloud Run is actually using Knative; the other serverless products are Cloud Function and App Engine. Cloud Run vs Cloud Functions vs App Engine
  • API Gateway: still in beta but looks like a replacement for Cloud Endpoint. API Gateway vs Cloud Endpoint
  • DataFlow vs Dataproc
  • Cloud Build: uses Docker to execute builds. For each build step, Cloud Build executes a Docker container as an instance of docker run.

GCE

General: n2 > e2. E2 VMs are more tightly packed into physical cores and the idea is to continiously monitor the VM’s usage and “live migrate” it to another hypervisor as and when needed.

Every virtual machine (VM) instance stores its metadata on a metadata server. Your VM automatically has access to the metadata server API without any additional authorization. Metadata is stored as key:value pairs.

Why Googles uses Power?

From this article

Google found that the performance of its web search algorithm, the heart and soul of the company, scaled well with both the number of cores and the number of threads available to it. IBM's POWER9 processor is a many-core, many-thread beast. Variants of the chip range from 12 to 24 cores, with eight threads per core for the 12-core version and four threads per core for the 24-core version. Intel's chips support only two threads per core via hyperthreading.

However

They're not well suited for workloads that don't benefit from more threads, which is why the market-share ceiling for POWER isn't all that high.

Google Cloud Datastore vs Cloud Firestore vs Firebase Realtime Database

  • Firestore is the successor of Datastore; Datastore is deprecated.
  • Firebase Realtime Database was the original database offering for Firebase.
  • Cloud Firestore, as the name suggests, is a "joint-venture" of Google Cloud and Firebase, it is the new flagship database offering of Firebase.
  • Both Firestore and Realtime Database are NoSQL, using JSON as data format.

Container Registry vs Artifact Registry

Container Registry:

  • GCR is a private Docker registry backed by Cloud Storage.
  • container images only
  • gcr.io

Artifact Registry:

  • a.k.a. GCR v2
  • the recommended service for container image storage and management on Google Cloud.
  • extends the capabilities of Container Registry.
  • support for both container images and non-container artifacts:
    • Container images: Docker, Helm
    • Language packages: Java, Node.js, Python
    • OS packages: Debian, RPM
  • pkg.dev

GDC

GDC = Google Distributed Cloud. Extensions of GCP.

https://cloud.google.com/distributed-cloud

  • GDC Virtual: connected, software only, running on customer's infra. (Basically a rebranded Anthos on-prem + Anthos Bare Medel)
    • GDCV customers can bring Google Cloud’s software stack to their own data center servers. Moreover, with the service, they can operate various on-premises applications using the same Google Cloud APIs, control planes, hardware, and tooling as their cloud-hosted apps.
    • uses Google Cloud Console to provision Anthos clusters on vSphere or bare metal:
  • GDC Edge: connected, Google managed hardware + software; for low latency, data residency hybrid workloads.
  • GDC Hosted: air-gapped, pre-configured hardware + software, managed by Google or a trusted partner, for the most sensitive workloads.
    • does not require connectivity to Google Cloud at any time to manage infrastructure, services, APIs, or tooling, and uses a local control plane provided by Anthos for operations.
    • diff from ABM: airgapped from the Internet, i.e. no direct access to public cloud, so need special ops personnel to manage.

GDC Virtual / Anthos

Control plane runs on GCP for upgrades, policy management, telemetry.

What is Anthos:

  • Anthos = GKE (Kubernetes) + Service Mesh (Istio) + Cloud Run (Knative) + Management (UI/CLI, policy, configs, etc) + logging / monitoring

Components:

  • Rebranded as GDC Virtual:
    • Anthos on VMware: run GKE in a vSphere environment.
      • OS->vSphere->Kubernetes
      • Anthos manages os and packages.
    • Anthos on Bare Metal: run GKE in on-prem data centers. WITHOUT VMware.
      • Customer manages os and packages.
      • KubeVirt: optional; enables Kubernetes orchestration of VMs running on KVM. For legacy virtualized workloads.
      • CLI:
        • bmctl: manages admin cluster
        • kubectl and Anthos K8s API: manage user clusters
  • Anthos Service Mesh (ASM): a Google productionized version of OSS Istio.
  • Anthos Config Management (ACM)
  • Anthos Identity Service (AIS): just a proxy, not a full IdP. Implemented as a webhook authentication.
  • Anthos obervability based on GCP: Stackdriver API, Cloud Monitoring API, Cloud Logging API

Anthos Bare-metal

Managing Kubernetes on customer managed Operating Systems. Uses Cilium, MetalLB, and Ansible for machine management.

ABM Kubevirt support: support for managing VMs on top of ABM (server nodes are often too beefy to be one node for customers) Customers bring and manage their nodes, while Google is responsible for installing / upgrading Anthos components (e.g., GKE, ACM, ASM, Cloud Run).

From the user’s point of view, Anthos Bare Metal has an API in the form of Kubernetes resource model. But under the hood, Anthos Bare Metal utilizes Ansible as the mechanism to manage nodes.

Why ansible:

  • To reduce the cost of supporting different operating system distributions, since it provides some level of abstraction and framework.
  • To avoid interfering with the customer’s existing management tools, since it does not require the installation of an agent.

bmctl upgrade

# Node is put in maintenance mode before upgrading
$ bmctl upgrade

$ bmctl upgrade cluster --kubeconfig=<admin_kubeconfig> -c cluster.

# upgrade the cluster without creating a temporary bootstrap clusters
$ bmctl upgrade cluster --kubeconfig=<admin_kubeconfig>:

In-place upgrades will leverage kubeadm upgrade to do the heavy-lifting.

The flow is to upgrade the master node pool first followed by worker node pools. Upgrade will be performed on one node at a time to minimize the capacity reduction.

Will bootstrap a temporary cluster to act as temporary parent of the cluster and this leaves the rest of upgrade process similar to upgrade cluster with parents.

GKE vs Kubernetes

  • Kubernetes is a piece of software, for container orchestration. You still need to acquire hardware, config and bootstrap the systems, install add-ons, monitor the systems, upgrade to keep the system up-to-date.
  • GKE is a managed service that takes care of those tasks, and you can start utilizing the container orchestration functionality right away.

GKE

GKE migrated from calico (dataplane v1, based on iptables) to cilium (dataplane v2, based on eBPF, pod: anetds)

GKE Dataplane V2 does not use kube-proxy: GKE Dataplane V2 uses cilium instead of kube-proxy to implement Kubernetes Services.

https://cloud.google.com/kubernetes-engine/docs/concepts/dataplane-v2#kube-proxy

GKE additional features comparing to open source k8s: networking, backup, auth, configmanagement, addon (log, metric), configsync

GKE vs Anthos (GDCV) vs GDCH

  • GKE: all running on GCP.
  • Anthos (GDCV): control plane running on GCP, data plane can be on other clouds (AWS, Azure, etc), VMWare, or Bare Metal. Your cluster needs to be connected to GCP all the time.
  • GDCH: air-gapped, meaning there's no connection to GCP at any time, both control plane and data plane run on customer's hardware (reusing Anthos Bare Metal)

Marketplace services vs Managed services

  • Marketplace services: user gets a container and a guide only, need to manage the whole lifecycle on their own; no SLA; no data backup.
  • Managed services: all taken care of by GCP.

Cloud Identity vs IAM

  • Cloud Identity: to manage users and groups. The identity provider (IdP) for GCP, also the Identity-as-a-Service (IDaaS) solution that powers Google Workspace.
  • IAM: to manage policies and to apply permissions / roles to those users and groups. Can use an external identity provider (IdP).

CLI

gcloud

gcloud auth login (no --update-adc) would populate your user credentials into a sqlitedb under .config/gcloud/

If you use gcloud cli, by default it'll use those creds it finds there.

gsutil

Not the recommended CLI for Cloud Storage. Use gcloud storage commands in the Google Cloud CLI instead.

Storage

  • local SSD: attached to a specific VM, fast, data may be lost.
  • PD Persistent disks: durable network storage devices that your instances can access like physical disks. use case: accessed by a single VM, or content does not change (attach it to a read-only disk to hundreds of VMs) can be HDD (pd-standard) or SSD (pd-balanced, pd-ssd, pd-extreme)
  • managed filestorage: filestore

BeyondCorp

Enables employees to work from untrusted networks without the use of a VPN.

"BeyondCorp Enterprise is a modern zero trust platform which allows your employees and extended workforce to access applications in the cloud or on-premises and work from anywhere without a traditional remote-access VPN."

https://cloud.google.com/beyondcorp

Google Cloud Build

Using Tekton under the hood.

Cloud Storage FUSE

Msount and access Cloud Storage buckets as local file systems.

logo
PRIVACY POLICYABOUT中文