Cloud
"Cloud is the new OS"
Cloud is just a fancy name for systems (in this modern world, they are most likely distributed systems), with the hardware procurement and maintenance and many other things abstracted away from software developers.
Cloud can be public, meaning Amazon or Microsoft or Google is running your infrastructure. Or private, meaning you still fully own your data center but resource request and allocation is done through software UI. Or hybrid, taking the best parts of public and private cloud.
Based on the abstraction level, cloud offerings can be infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS).
Computation evolution: bare metal -> virtualized -> containerized -> cloud -> serverless.
Cloud Infrastructure
3 big players (60%+ combined market share):
- Amazon AWS
- Microsoft Azure
- Google Cloud Platform
Other players (less than 10% of market share each):
- Outside of China:
- IBM / Redhat
- Oracle
- China: (the top 4 has >80% marketshare in China)
- Alibaba Cloud
- Huawei Cloud
- Tencent Cloud
- Baidu AI Cloud
Specialized:
- Okta: identity.
- DigitalOcean: compute.
- Backblaze: storage / backup.
- Snowflake: data warehouse.
- Hashicorp: infrastructure-as-code (IaC), enable IT collaboration and automation.
- CloudFlare: CDN, security.
- ServiceNow: task management.
- PagerDuty: Incident Response.
- Workday
- Salesforce
3 Key Categories
Each public cloud offers dozens of services. To make it easier to understand, think of these 3 key categories:
- compute: VM, containers, serverless functions, etc.
- storage: databases, datawarehouses, object stores, etc.
- networking: DNS, VPC, load balancing, etc.
Virtualization
Virtualization is the key enabler of the Cloud, so that the resources can be split and re-packaged to sell: virtual machine (hypervisor), virtualized (software-defined) storage and network.
Hyper-converged infrastructure (HCI): software-defined infrastructure, virtualizes all of the elements of conventional "hardware-defined" systems. HCI is mostly a marketing term.
Read more about Virtualization.
Data Plane vs Control Plane vs Management Plane
The 3 Planes in distributed systems / clouds:
- Management Plane:
- manage and push configs to the Control Plane and Data Plane.
- directly used by admins, either through code (e.g. check in config files to Git repo to be picked up by CI/CD) or UI (e.g. modify configs in cloud console web UI), or CLI / API.
- slow rate of change.
- Control Plane:
- input: configs from the Management Plane and signals from the Data Plane.
- state transition initiated by events:
- dataplane signals.
- config change triggered by the user (from the Management Plane).
- time-triggered jobs, e.g. backups.
- state transition initiated by events:
- output: control instructions to the Data Plane.
- determine which path the data plane will take.
- ensures the data plane is operating with correct state as provided by the management plane (the configs) or a dynamic control algorithm (like network routing protocol).
- medium rate of change.
- input: configs from the Management Plane and signals from the Data Plane.
- Data Plane:
- the actual services, databases etc.
- controlled by the Control Plane.
- fast rate of change.
Availability Requirements: Data Plane > Control Plane > Management Plane.
How to provision cloud resources
- CLI (e.g.
aws,gcloud). - Web UI (e.g. AWS Management Console, Google Cloud Concole).
- REST API.
- Infrastructure-as-code tools (e.g. Terraform).
Events / Conferences
Follow the latest developments in the Cloud world:
- AWS re:Invent: usuall at the end of the year, in Las Vegas.
- Google Cloud Next: usually at the end of summer; used to be in San Francisco, moved to Las Vegas in 2024.
Comparison Charts
https://cloud.google.com/docs/get-started/aws-azure-gcp-service-comparison