Anypoint MQ dead-letter queue configuration in Mule 4

  • November 09, 2020

Anypoint MQ is a multitenant, cloud messaging service that enables customers to perform advanced asynchronous messaging scenarios between their applications.

The dead-letter queue (or undelivered message queue) is the queue where messages are sent if they can’t be routed to their correct destination.

By the end of this article, you’ll have a clear picture of the dead-letter queue (DLQ), how to configure a queue and assign a DLQ to the parent queue and how the message gets routed from the parent queue to the DLQ after failure.

Configuration

Open Anypoint platform and select MQ in that:

Anypoint MQ1

Note: To access Anypoint MQ, you need to buy it separately from MuleSoft.

In the MQ window, we need to first select a region where we want to create our queues. Select a region that contains the data center nearest to our geographical area. Click the plus ( + ) symbol to create the queues. Every region has its own dedicated url, which will be used at the time or configuring the queue in the Mule flow.

Anypoint MQ2
Anypoint MQ3

Here, we’ll create two queues. One will be named demo-DLQ (DLQ), and the other will be the demo-MQ (parent queue) in which the DLQ will be assigned.

DLQ Configuration MQ Configuration

Anypoint MQ4

The demo-DLQ will have a normal configuration.

While creating the demo, we toggle the assign a dead-letter queue to on state. Once the state is on, we’ll get a dropdown option to select the dead-letter queue name where we can find the name of all DLQ present. Here we’ll select our DLQ to be assigned in the parent queue. There’s an option to assign the number of attempts. This option defines how many retries the parent queue will do to process the message if there is any failure while sending the message to the destination.

Once the queues are created with the provided configurations, we can find it in the main window. To use the queues in our Mule flow, we also need to have Client App Id and Client Secret. To achieve this, select the client Apps option at the left corner over the queue windows and then click the plus ( + ) to create credentials.

Anypoint MQ5
Anypoint MQ6

Once the credentials are created, it can be seen in the Client App window.

The configuration and creation of MQ are done. Now, we’ll create flow and use the MQ.

We’ll drop an Anypoint MQ Subscriber as a source in our flow and configure the required details.

Anypoint MQ7

MQ Configuration:

Anypoint MQ8
Anypoint MQ9

The Complete Demo Flow:

Anypoint MQ10

The Request Payload logger will display the payload received from the queue and DateTime logger will display the current execution DateTime.

Request Payload logger DateTime logger

Anypoint MQ11

Http Requester Configuration:

Anypoint MQ12

Note: The requestor path is pointed to a non-existing resource on purpose so that the flow will through error so we can demonstrate the functionality of DLQ.

The Response Payload logger is to display the payload received as a response from Http Requester.

The configuration and creation of the MQ and Flows are done. Now, we'll look at the working.

Working:

Click on the name of the MQ. A new window will open, and in that we’ll add the message we want to process through the Subscriber.

Anypoint MQ13
Anypoint MQ14

The message processed through the flow and failed at the Http Requester as the endpoints are not valid.

Anypoint MQ15

While configuring the demo-MQ (parent queue), we’ve configured the retry attempts at two, so upon failure the MQ will try to process the message twice. The same can be observed in the console.

Anypoint MQ16

Let’s check the message status in the demo-MQ (parent queue) and demo-DLQ (DLQ).

demo-MQ: demo-DLQ:

Anypoint MQ17

The message from the demo-MQ is processed, but since the flow has an error, the message is routed to DLQ.

Anypoint MQ18

The message is saved in the DLQ. Hence, the message wasn’t lost.

Whenever we create any flow, we also create error handling strategies. So let’s check how the DLQ will work if the error handling strategies are configured.

With error handling:

In the existing flow, we’ll add on Error Propagate and a Publisher queue config to save the message in an error-demo-MQ (error queue), which is a normal queue and not a DLQ.

Anypoint MQ19

Error Logger: Display the Request Payload

Anypoint MQ20

error-demo-MQ:

Anypoint MQ21

MQ Window:

Anypoint MQ22
Anypoint MQ23

Currently, the error queue is empty.

Now will again send a message in the demo-MQ (parent queue) and execute the flow.

Anypoint MQ24

Again, since the retry attempt for the demo-MQ is two, the message will be processed twice if an error is encountered. In this case, the message will be processed two times.

Anypoint MQ25

The error-demo-MQ will have two messages:

Anypoint MQ26
Anypoint MQ27

The DLQ has one message only:

Anypoint MQ28
Anypoint MQ29

The reason demo-DLQ has one message and error-demo-MQ has two messages is:

The DLQ is a part of the parent queue, so it inherits the policies and properties of the parent queue. The error queue is a separate normal queue, so the message is processed two times. The error handler captures the error two times, so the message is saved two times in the error queue.

— By Abhishek Bathwal