Message Queues

• Receives, hold and deliver messages

  • server cannot handle the message at once
  • such as payments
  • should be durable
  • usually FIFO
  • Pull by the server or push from queue to server
  • queue would continuously sent same data until received acknowledgement from the server
  • so the data are durable

• Work flow

  • An application publishes a job to the queue, then notifies the user of job status
  • A worker picks up the job from the queue, processes it, then signals the job is complete

• Use cases

  • Sending many emails
  • data post processing
  • recommender system

• Redis is useful as a simple message broker but messages can be lost.

• RabbitMQ is popular but requires you to adapt to the 'AMQP' protocol and manage your own nodes.

• Amazon SQS is hosted but can have high latency and has the possibility of messages being delivered twice.

Disadvantage(s): asynchronism

• Use cases such as inexpensive calculations and realtime workflows might be better suited for synchronous operations, as introducing queues can add delays and complexity.