Message Queue
A component that stores messages sent between applications, enabling asynchronous communication and decoupling between services.
In-Depth Explanation
A message queue is a form of asynchronous service-to-service communication. Messages are stored in the queue until they are processed and deleted. This decouples senders from receivers, improving reliability and scalability.
Message queue concepts:
- Producer: Sends messages to the queue
- Consumer: Receives and processes messages
- Queue: Stores messages until consumed
- Message: Data payload with metadata
Benefits:
- Decoupling: Services don't need to know about each other
- Reliability: Messages persist if consumer is down
- Scalability: Add consumers to handle more load
- Load levelling: Smooth out traffic spikes
- Asynchronous: Producer doesn't wait for processing
Popular message queues:
- RabbitMQ (open source)
- Amazon SQS (managed)
- Azure Service Bus (managed)
- Apache Kafka (for high throughput)
- Redis Streams
Business Context
Message queues enable building reliable, scalable systems where components can fail independently without losing data or blocking each other.
How Clever Ops Uses This
We use message queues in Australian business AI architectures to handle async processing, batch AI requests, and ensure reliability.
Example Use Case
"Queuing AI processing requests: user submits document, gets immediate confirmation, document is processed asynchronously, user notified when complete."