Message Queues

Message Queues are a important category of indirect communication systems. Groups and publish subscribe provide systems are provide one to many method of communication. Message queues provide point to point service using the concept of message queue as in indirection, so it achieving the desired properties of space and time uncoupling. Message queues are point to point in that sender places message into queue, and its removed by a single process. Message queues also referred to as message oriented middle-ware.

Programming model

Programming model offered by message queues is simple. It offers an approach to communication in distributed systems through queues. Producer process can send message to specific queue and other process can receive that message from that queue. There are three style of receive:

1. Blocking receive: It will block until relevant message is available.
2. Non blocking receive: It will check status of queue and return a message if available, or not available indication otherwise.
3. Notify operation: It will issue event notification when message is available in associated queue.

Many processes can send message to the same queue, and many receivers can remove messages from the queue. Queuing policy is normally first in first out, but more message queue implementations support the concept of priority, with higher priority messages delivered first. Consumer process also can select message from the queue based on properties of the message like destination, metadata, priority of the message, delivery mode and body of the message.

Implementation issues

Implementation issue for message queue system is choice between centralized and distributed implementation of the concepts. Some implementation are centralized with one or more message queues managed by queue manager located at a given node. Advantage of this implementation is simplicity, but this managers become heavy weight components and have potential to become a bottleneck or single point of failure. So more distributed implementation have been proposed to overcome from this issues.

Example with RabitMQ Work Queue using Java

Main idea behind work queue is avoid doing resource intensive task immediately and having to wait for it to complete. Instead schedule task to be done later. A task can be encapsulate as message and send to a queue. A Worker process running in background will pop the tasks and eventually execute the job. When many workers are running the tasks will be shared between them.

Lets see the example.

Create file NewTask.java to allow arbitrary messages to be sent from command line. This program will schedule tasks to work queue.

Create file Worker.java to handle delivered messages and perform the task.

One of the advantage of using Task Queue is the availability to easily parallelized work. If there building up a backlog , can just add more workers and that way, scale easily. RabitMQ supports message acknowledgments. An acknowledgment is sent back by the consumer to tell RabitMQ that a particular message has been received, processed and that RabitMQ is free to delete it.