Distributed Systems - API

Updated: 2018-12-10

APIs are used to glue web client and servers, or mobile app and its backend, or different backend services within the microservice architecture.

Server listens on a port and respond to the requests that are sent to it, e.g. listen on HTTP port for URL requests, or on RPC port for RPC requests, sometimes both.

The 3 most popular flavors: REST, GraphQL, and RPC

REST

  • REST is not a protocol, a file format, or a development framework. It’s a set of design constraints: statelessness, HATEOAS(Hypermedia As The Engine Of Application State), and so on.
  • rely on HTTP verbs and organize things as "resources"

GraphQL

  • solves over-fetching and under-fetching problem in REST
  • easy to change api
  • created by Facebook, originally used by mobile app calling backends. Github started to use GraphQL for its API since v4, replacing v3 REST API.

RPC

Usually used in microservices, among the backend services.

3 pupular choices: ProtoBuf, Thrift, Avro

  • Thrift and ProtoBuf are statically typed, while Avro uses a more dynamic approach
  • Protocol Buffers: came from Google, and used by all Google internal services
  • Thrift: came from Facebook and used by all Facebook internal services
  • Thrift has data serialization and RPC framework in one; ProtoBuf is the data serialization, gRPC(open source) or Stubby(Google's internal) is the RPC framework

About gRPC:

  • high-performance, compact, lightweight
  • utilize protobuf, HTTP/2
  • for microservices
  • used by Docker/Kubernetes, etcd
  • from Google

Protocol Buffers

  • developed by Google. Google internally uses Protocol Buffers as its primary file format.
  • language neutral.
  • Protocol Buffers is similar to Thrift. It provides a compiler and a set of libraries that a developer can use to serialize data. A developer defines the structure or schema of a dataset in a file and compiles it with the Protocol Buffers compiler, which generates the code that can then be used to easily read or write that data.
  • Compared to Thrift, Protocol Buffers support a smaller set of languages. Currently, it supports C++, Java, and Python. In addition, unlike Thrift, which provides tools for both data serialization and building remote services, Protocol Buffers is primarily a data serialization format. It can be used for defining remote services, but it is not tied to any RPC (remote procedure call) protocol.

Thrift

  • language-independent
  • supports a variety of languages, including C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml, Delphi, and other languages.
  • Thrift provides a code-generation tool and a set of libraries for serializing data and transmitting it across a network. It abstracts the mechanism for serializing data and transporting it across a network. Thus, it allows an application developer to focus on core application logic, rather than worry about how to serialize data and transmit it reliably and efficiently across a network.
  • With Thrift, an application developer defines data types and service interface in a language-neutral interface definition file. The services defined in an interface definition file is provided by a server application and used by a client application. The Thrift compiler compiles this file and generates code that a developer can then use to quickly build client and server applications.
  • A Thrift-based server and client can run on the same computer or different computers on a network. Similarly, the server and client application can be developed using the same programming language or different programming languages.

Avro

  • language-independent.
  • supports rich data structures, including nested data.
  • schema(in JSON) is stored along with data. Therefore, an Avro file can be later read by any application. In addition, since schema is stored along with data, each datum is written without per-value overheads, making serialization fast and compact.
  • When data is exchanged over a network using Avro, the sender and receiver exchange schemas during an initial connection handshake.
  • Avro automatically handles field addition and removal, and forward and backward compatibility—all without any awareness by an application.

Readings