Design a messaging queue
Functional requirements
- Queue creation: The client should be able to create a queue and set some parameters—for example, queue name, queue size, and maximum message size.
- Send message: Producer entities should be able to send messages to a queue that’s intended for them.
- Receive message: Consumer entities should be able to receive messages from their respective queues.
- Delete message: The consumer processes should be able to delete a message from the queue after a successful processing of the message.
- Queue deletion: Clients should be able to delete a specific queue.
Non-functional requirements
- Durability: The data received by the system should be durable and shouldn’t be lost. Producers and consumers can fail independently, and a queue with data durability is critical to make the whole system work, because other entities are relying on the queue.
- Scalability: The system needs to be scalable and capable of handling the increased load, queues, producers, consumers, and the number of messages. Similarly, when the load reduces, the system should be able to shrink the resources accordingly.
- Availability: The system should be highly available for receiving and sending messages. It should continue operating uninterrupted, even after the failure of one or more of its components.
- Performance: The system should provide high throughput and low latency.
Design considerations
- Ordering of the messages
- best effort ordering
- order the message according to the sequence received
- strict ordering
- according to message order when produced
- tactics for ordering
- increasing number
- causality based sorting on server side
- time stamps based on synchronised clocks