message queue

Core concept

A **message queue** is a software mechanism that stores messages from sending applications and delivers them to receiving applications asynchronously. Senders place messages onto the queue and continue their work without waiting for an immediate response, while receivers read and process messages from the queue at their own pace.

Message queues are typically part of a broader messaging or integration platform (for example, a message broker, ESB, or iPaaS) and are used to decouple systems so they do not need to be online, available, or operating at the same speed at the same time.

How message queues work

In a typical implementation:

– **Producers (senders)** create and send messages and post them to a named queue or topic.
– **The queue** temporarily stores messages, usually in a reliable, ordered structure.
– **Consumers (receivers)** read messages from the queue and process them, often acknowledging successful processing.
– **Message handling rules** (e.g., retry, dead-letter queues, time-to-live) govern what happens when messages cannot be delivered or processed.

Messages are often small, structured payloads (for example, JSON or XML) containing business data or event notifications.

Use in manufacturing and regulated environments

In industrial and regulated manufacturing contexts, message queues commonly support integration between:

– **MES and ERP**: exchanging production orders, material movements, inventory updates, and quality results.
– **OT and IT systems**: sending events or measurements from control systems or data historians to higher-level applications.
– **Quality and compliance systems**: communicating deviations, electronic signatures, or release decisions between specialized applications.

They are used to:

– Decouple systems with different release cycles or maintenance windows.
– Smooth out load when one system produces data faster than another can consume it.
– Buffer messages during brief network or system outages, then deliver once connectivity is restored.

What a message queue is and is not

**Includes:**

– A logical or physical structure that holds messages until they are consumed.
– Software implementing queuing semantics such as FIFO ordering, persistent storage, acknowledgements, and retries.
– Cloud or on-premises services like RabbitMQ, IBM MQ, ActiveMQ, Azure Service Bus queues, or AWS SQS.

**Excludes:**

– **File drops** or shared folders used for batch file transfers (these may act as buffers but lack standard queuing semantics).
– **Databases** used only for storing transactional records without explicit queuing mechanisms (e.g., no consumer offset, no dequeue/ack model).
– **Email queues** managed by mail servers; these are a different domain even though they technically queue messages.

Common patterns and features

Message queues are used in several integration patterns that are relevant for manufacturing systems:

– **Point-to-point queuing**: one producer sends to one consumer via a queue.
– **Competing consumers**: multiple consumers read from the same queue, each processing different messages for throughput.
– **Publish/subscribe (via topics)**: a producer sends a message once, and multiple subscribers receive their own copies (often implemented using related but distinct constructs like topics or streams rather than a single queue).
– **Guaranteed delivery**: queues often support message persistence and acknowledgement to reduce risk of message loss.
– **Dead-letter queues**: separate queues for messages that cannot be processed successfully after a configured number of attempts.

Common confusion and related terms

– **Message queue vs. message broker**: a *message broker* is the broader messaging server or service that can host many queues, topics, and routing rules; a *message queue* is one specific logical channel within that broker.
– **Message queue vs. event stream**: event streaming platforms (for example, Apache Kafka) manage ordered logs of events and consumer offsets. They can implement queuing-like behavior but are optimized for high-volume streaming and replay, not just simple point-to-point queuing.
– **Message queue vs. ESB/iPaaS**: an ESB or iPaaS may use queues internally but also adds routing, transformation, protocol mediation, and orchestration capabilities.

Connection to MES–ERP integration (site context)

When integrating MES and ERP systems, message queues are often one of several integration methods used alongside file-based transfers, database views, and web services/APIs. In this context, message queues provide an asynchronous channel for:

– Sending production order data from ERP to MES without tightly coupling their availability.
– Returning production confirmations, consumption data, or quality results from MES back to ERP.
– Mediating communication through an ESB or iPaaS to standardize message formats and routing while isolating each system from direct point-to-point connections.