Advanced Message Queueing Protocol (AMQP) 0–9–1

Messaging protocol standard that advises message brokers and event sourcing systems such as RabbitMQ and ApacheKafka

10 min readJan 17, 2020

AMQP 0–9–1 is a messaging protocol that enables conforming client applications to communicate with conforming messaging middleware brokers.

Brokers and their role

Messaging brokers receive messages from publishers(applications that publish messages, also known as producers) and route them to consumers (applications that process messages).

Since it is a network protocol the publisher, consumer and broker can all reside on different machines.

AMQP 0–9–1 Model in Brief

The AMQP 0–9–1 Model has the following view of the world; messages are published to exchanges, which are often compared to post offices or mailboxes. Exchanges then distribute message copies to queues using rules called bindings. Then the broker either deliver messages to consumers subscribed to queues, or consumers fetch/pull messages from queues on demand.

When publishing a message, publishers may specify various message attributes(message meta-data). Some of this meta-data may be used by the broker, however, the rest of it is completely opaque to the broker and is only used by applications that receive the message.

Networks are unreliable and applications may fail to process messages, therefore, the AMQP 0–9–1 model is a notion of message acknowledgements: when a message is delivered to a consumer the consumer notifies the broker, either automatically or as soon as the application developer chooses to do so. When message acknowledgements are in use, a broker will only completely remove a message from a queue when it receives a notification for that message (or group of messages).

In certain situations, for example, when a message cannot be routed, messages may be returned to publishers, dropped, or, if the broker implements an extension, placed into a so-called “dead-letter queue”. Publishers choose how to handle situations like this by publishing messages using certain parameters.

Queues, exchanges, and bindings are collectively referred to as AMQP entities.

AMQP 0–9–1 is a Programmable Protocol

AMQP is a programmable protocol in the sense that AMQP 0–9–1 entities and routing schemes are primarily defined by applications themselves, not broker administrators. Accordingly, provision is made for protocol operations that declare queues and exchanges, define bindings between them, subscribe to queues and so on.

This gives application developers a lot of freedom but also requires them to be aware of potential definition conflicts. In practice, definition conflicts are rare and often indicate a misconfiguration.

Applications declare the AMQP entities that they need, define necessary routing schemes and may choose to delete AMQP entities when they are no longer used.

Exchanges and Exchange Types

Exchanges are AMQP entities where messages are sent. Exchanges take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings.

AMQP brokers provide four exchange types:

Exchange types

Direct exchange — (Empty string) and amq.direct
Fanout exchange — amq.fanout
Topic exchange — amq.topic
Headers exchange — amq.match (amq.header in RabbitMQ)

Besides exchange type, exchanges are declared with a number of attributes, the most important of which are:

Name
Durability (exchanges survive broker restart)
Auto-delete (exchange is deleted when the last queue is unbound from it)
Arguments (Optional, used by plugins and broker-specific features)

Exchanges can be durable or transient. Durable exchanges survive broker restart whereas transient exchanges do not (they have to be redeclared when broker comes back online). Not all scenarios and use cases require exchanges to be durable.

Default Exchange

The default exchange is a direct exchange with no name(empty-string) pre-declared by the broker. It has one special property that makes it very useful for simple applications: every queue that is created is automatically bound to it with a routing key which is the same as the queue name.

Direct exchange

A direct exchange delivers messages to queues based on the routing key. A direct exchange is ideal for the unicast routing of messages. Here is how it works:

A queue binds to the exchange with a routing key K
When a new message with routing key R arrives at the direct exchange, the exchange routes to the queue if K = R.

Direct exchanges are often used to distribute tasks between multiple workers (instances of the same application) in a round robbin manner. When doing so, it is important to understand that, in AMQP 0–9–1, messages are load-balanced between consumers and not between queues.

Fanout Exchanges

A fanout exchange routes messages to all the queues that are bound to it and routing is ignored. If N queues are bound to a fanout exchange when a new message is published to that exchange a copy of the message is delivered to all N queues. Fanout exchanges are ideal for the broadcast routing of messages.

Because a fanout exchange delivers a copy of a message to every queue bound to it, its use cases are quite similar.

Massively multiplayer online (MMO) games can use it for leader board updates or other global events.
Sport news sites can use fanout exchanges for distributing score updates to mobile clients in nearly real-time.
Distributed systems can broadcast various state and configuration updates
Group chats can distribute messages between participants using a fanout exchange (although AMQP does not have a built-in concept of presence, so XMPP may be a better choice)

Topic Exchanges

Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange. The topic exchange type is often used to implement various publish/subscribe pattern variations. Topic exchanges are commonly used for the multicast routing of messages.

Topic exchanges have a very broad set of use cases. Whenever a problem involves consumers/applications that selectively choose which type of messages they want to receive, the use of topic exchanges should be considered.

Header Exchanges

A header exchange is designed for routing on multiple attributes that are more easily expressed as message headers than a routing key. Headers exchanges ignore the routing key attribute. Instead, the attributes used for routing are taken from the headers attribute. A message is considered matching if the value of the header equals the value specified upon binding.

It is possible to bind a queue to a headers exchange using more than one header for matching. In this case, the broker needs one more piece of information from the application developer, namely, should it consider messages with any of the headers matching or all of them? This is what the “x-match” binding argument is for.

When the “x-match” argument is set to “any”, just one matching header value is sufficient. Alternatively, setting “x-match” to “all” mandates that all the values must match.

Queues

Queues in the AMQP model are very similar to queues in other messages- and task-queueing systems: they store messages that are consumed by applications. Queues share some properties with exchanges, but also have some additional properties.

Name
Durable (the queue will survive a broker restart)
Exclusive (used by only one connection and the queue will be deleted when that connection closes)
Auto-delete (Queue that has had at least one consumer is deleted when the last consumer unsubscribes)
Arguments (Optional: used by plugins and broker specific features such as message TTL, queueing length limit ,etc)

Before a queue can be used it has to be declared. Declaring a queue will cause it to be created if it does not already exist. The declaration will have no effect if the queue does already exist and its attributes are the same as those in the declaration. When the existing queue attributes are not the same as those in the declaration a channel-level exception with code 406 (PRECONDITION_FAILED).

Queue Names

Applications may pick queue names or ask the broker to generate a name for them. Queue names may be up to 255 bytes of UTF-8 characters. An AMQP broker can generate a unique name on behalf of an app. To use this feature, pass an empty string as the queue name argument. The generated name will be returned to the client with the queue declaration response.

Queue names generated with “amq.” are reserved for internal use by the broker. Attempts to declare a queue with a name that violates this rule will result in a channel-level exception with reply code 403 (ACCESS_REFUSED).

Queue Durability

Durable queues are persisted to disk and thus survive broker restarts. Queues that are not durable are called transient. Not all scenarios and use cases mandate queues to be durable.

The durability of a queue does not make messages that are routed to that queue durable. If the broker is taken down and then brought back up, the durable queue will be re-declared during broker startup, however, only persistent messages will be recovered.

Bindings

Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q. Q has to be bound to E. Bindings may have an optional routing key attribute used by some exchange types. The purpose of the routing key is to select certain messages published to an exchange to be routed to the bound queue. In other words, the routing key acts like a filter.

Consumers

Storing messages in queues is useless unless applications can consume them. In the AMQP Model, there are two ways for applications to do this;

Have messages delivered to them (“push API”)
Fetch messages as needed (“pull API”)

With the “push API”, applications have to indicate interest in consuming messages from a particular queue. When they do so, we say that they register a consumer or simply put, subscribe to a queue. It is possible to have more than one consumer per queue or to register an exclusive consumer (excludes all other consumers from the queue while it is consuming).

Each consumer (subscription) has an identifier called a consumer tag. It can be used to unsubscribe from messages. Consumer tags are just strings.

Message Acknowledgments

Consumer applications — that is applications that receive and process messages — may occasionally fail to process individual messages or will sometimes just crash.

There is also the possibility of network issues causing problems. This raises a question: when should the broker remove messages from queues? The AMQP specification gives consumers control over this.

There are two acknowledgement modes:

After a broker sends a message to an application (using either basic.deliver or basic.get-ok method)
After the application sends back an acknowledgement (using either the basic.ack method)

If a consumer dies without sending an acknowledgement, the broker will redeliver it to another consumer or, if none are available at the time, the broker will wait until at least one consumer is registered for the same queue before attempting re-delivery.

Rejecting Messages

When a consumer application receives a message, processing of that message may or may not succeed. An application can indicate to the broker that message processing has failed (or cannot be accomplished at the time) by rejecting a message. When rejecting a message, an application can ask the broker to discard or re-queue it.

When there is only one consumer on a queue, make sure you do not create infinite message delivery loops by rejecting and re queueing a message from the same consumer over and over again.

Messages are rejected with basic.reject method.

Prefetching Messages

For cases when multiple consumers share a queue, it is useful to be able to specify how many messages each consumer can be sent at once before sending the next acknowledgement. This can be used as a simple load balancing technique or to improve throughput if messages tend to be published in batches.

Message Attributes and Payload

Messages in the AMQP model have attributes. Some attributes are so common that the AMQP specification defines them and application developers do not have to think about their exact attribute name.

Some examples are:

Content-type
Content encoding
Routing key
Delivery mode (persistent or not)
Message priority
Message publishing timestamp
Expiration period
Publisher application id

Some attributes are used by AMQP brokers, but most are open to interpretation by applications that receive them. Some attributes are optional and known as headers. They are similar to X-Headers in HTTP. Message attributes are set when a message is published.

Messages also have a payload (the data that they carry), which AMQP brokers treat as an opaque byte array. The broker will not inspect or modify the payload.

It is possible for messages to contain only attributes and no payload. It is also common to use serialization formats like JSON, Thrift, Protocol Buffers and MessagePack to serialize structured data in order to publish it as the message payload.

Protocol peers typically use the “content-type” and “content-encoding” fields to communicate this information, but this is by convention only.

Messages may be published as persistent which makes the broker persist them on disk. If the server is restarted, the system ensures that received persistent messages are not lost.

Publishing messages as persistent affects performance.

AMQP Methods

AMQP is structured as a number of methods. Methods are operations (like HTTP methods) and have nothing in common with methods in object-oriented programming languages. Protocol methods in AMQP are grouped into classes. Classes are just logical groupings of AMQP methods.

Let us take a look at the exchange class, a group of methods related to operations on exchanges. It includes the following operations: