Google Internal vs Open Source
Updated: 2022-06-26
Some of Google's open source projects are born open-source, but others are siblings to the original internal projects, their names, even technical details may be different; also some famous projects are not open sourced by Google, but as an implementation of Google's published papers.
Google Open-sourced
Blaze and Bazel
- Blaze: Google's internal build system.
- Bazel: the open source version.
Borg and Kubernetes
- Borg: Google's internal cluster manager.
- package format: MPM.
- Kubernetes: modeled after Borg. Written in Go.
- package format: containers.
Public info about Borg and MPM:
- Borg: https://ai.google/research/pubs/pub43438
- MPM: https://www.usenix.org/conference/lisa14/conference-program/presentation/mcnutt
Knative vs Cloud Run
- Knative is an open-source serverless platform on top of Kubernetes, now part of CNCF.
- Cloud Run is a managed Knative service.
Flume vs Apache Beam
- Flume: Google's internal data pipelines; originally on top of MapReduce, now moved to a faster underlying engine.
- Apache Beam: the open-source version of Flume.
Google's Flume is totally different from Apache Flume.
Stubby vs gRPC
- Stubby: Google's internal RPC framework.
- gRPC: the next generation of Stubby, and open sourced.
Both depend on protobuf.
Google Influenced
GFS and HDFS
- GFS: Google's (previous generation) distributed file system, and Google published a paper about it.
- HDFS: part of Hadoop, an open source implementation of that paper.