PubSub vs Message Queue

Updated: 2021-11-26

TL;DR

| Message Queue | PubSub | | ---------------- | ---------------------------------- | ----------------------------------------------- | | Consumers | one and only one consumer | allows multiple consumers | | Message Deletion | after ONE consumer ack consumption | after ALL consumers ack consumption | | Message Order | may be out of order | messages are received in order by each consumer |

Message Queue

  • Each message for a given topic or channel is delivered to and processed by exactly one consumer. (e.g. every paycheck is issued once and only once)
  • can support high rates of consumption by adding multiple consumers for each topic, but only one consumer will receive each message on the topic
  • the message will be deleted once a consumer has acknowledged consumption of the message
  • in the case of network or consumer failures, the message queue will try resending an affected message at a later time (not necessarily to the same consumer) and as a result, that message may be processed out of order.

PubSub

  • decoupling publishers and subscribers: publishers need not target their messages to specific subscribers nor even know how many subscribers there will be. Publishers and subscribers need not even be running at the same time, and can be implemented in different languages / frameworks. Decoupling often simplifies application design and supports more flexible application scalability.
  • message persistence: messages are kept in permanent storage until they have been acknowledged by all interested subscribers. Persistence allows client applications to be resilient in the face of intermittent application, network, and machine failures.

How to choose

  • Use message queues if a consumer application instance goes down then another consumer will be able to handle the message instead.
  • Use pubsub, if a subscriber is down then once it has recovered the messages it has missed will be available for consumption in its subscribing queue.

Options

Message Queues:

  • ActiveMQ
  • RabbitMQ
  • ZeroMQ

Distributed PubSub:

  • Kafka

Cloud:

  • GCP PubSub
  • Amazon Kinesis