Google Internal vs Open Source

Updated: 2019-01-24

Some of Google's open source projects are born open-source, but others are siblings to the original internal projects, their names, even technical details may be different; also some famous projects are not open sourced by Google, but as an implementation of Google's published papers.

blaze and bazel

blaze is Google's internal build system, bazel is the open source version

GFS and HDFS

Google published a paper about its distributed file system. HDFS, part of Hadoop, is an open source implementation of that paper.

Borg and Kubernetes

Borg is Google's internal cluster manager, Kubernetes is modeled after Borg but works with containers, while Borg's package format is called MPM

Flume vs Apache Beam

Google's Flume is totally different from Apache Flume. It is used for authoring data pipelines; originally on top of MapReduce, now moved to a faster underlying engine. The open source version is called Apache Beam.

Stubby vs gRPC

Stubby is Google's internal RPC framework. gRPC is called the next generation of Stubby, and open sourced. Both depend on protobuf.

https://cloud.google.com/blog/products/gcp/grpc-a-true-internet-scale-rpc-framework-is-now-1-and-ready-for-production-deployments