Compute
Trends
Cloud providers are investing in some of the following areas to increase efficiency and profitability:
- NIC Offload
- ARM chips
- Bare Metal Instances
- GPU / APU / DPU for AI / ML / Data processing.
- VMware support
- containerized / serverless
Bare Metal
bare metal = no hypervisor, no vmware; (instead can use KVM or Kubevirt.)
ARM for Servers
- AWS Graviton
- Ampere Altra:
- Customers: Oracle, Microsoft Azure and Google Cloud Platform, HPE
NIC offload
(NIC based encryption for RPC)
Why?
- integrity, privacy
- improve RPC performance (QPS, bandwidth and latency)
- improve disk performance
- reduce CPU and memory usage
- reduce power usage for crypto operations
Servers vs Services
Servers are long-lived pieces of software that provide services. A server is a collection of running services. The most common kinds of services: HTTP and RPC request-handling services.
Server Traffic management
- load balancing
- load shedding (throttling)
- RPC client behaviors
Integration Patterns
- API: contract driven
- Event Driven
- Data Stream Driven
What is Serverless
Serverless or Function as a Service (FaaS) describes the idea that the deployment unit is a single function. A function takes input and returns output.
Serverless Manifesto
http://blog.rowanudell.com/the-serverless-compute-manifesto/
- Function are the unit of deployment and scaling.
- No machines, VMs, or containers visible in the programming model.
- Permanent storage lives elsewhere.
- Scales per request; Users cannot over- or under-provision capacity.
- Never pay for idle (no cold servers/containers or their costs).
- Implicitly fault-tolerant because functions can run anywhere.
- BYOC - Bring Your Own Code.
- Metrics and logging are a universal right.
Why not serverless:
- Cold-start latency: serverless functions must be instantiated on demand. versus traditionally virtual machines or containers are preprovisioned.
- Hard to debug and monitor: cannot hop on the server to debug.
- Cost management: since serverless systems dynamically allocate resources behind the scenes, it is hard to manage cloud resource costs directly.
Examples
- AWS Lambda
- Google Cloud Functions
- Webtask
- Firebase
- Knative: open source, based on Kubernetes