Export MuleSoft application logs to AWS CloudWatch

  • July 01, 2022

This tech article describes the steps needed to configure the Anypoint Platform to post the logs on AWS CloudWatch.

In this information era, logging plays a crucial role in any software development project. Application logging of data provides insights on the current state of the API, depending on the level of logging configured. A good logging strategy is important because it contains information about application events, messages, errors and warnings, along with a few other informational events.

What is AWS CloudWatch?

AWS CloudWatch is used to store, access and get meaningful insights from log files. It enables you to centralize the logs from all your systems, applications and AWS services in a single highly scalable service. You can then easily view, search for specific error codes or patterns, filter based on specific fields or archive them securely for future analysis.

Why CloudWatch?

With CloudWatch, you can collect and access all your performance and operational data in the form of logs and metrics from a single platform. It has the following features:

  • Query your log data
  • Monitor logs from Amazon EC2 instances
  • Monitor AWS CloudTrail logged events
  • Log retention
  • Archive log data

Section 1: AWS CloudWatch stream creation 

1. CloudWatch configuration 

  1. In CloudWatch, create a Log Group where the Log Stream resides 
  2. Navigate to Console and search for CloudWatch
  3. Select Log Group 
  4. Click Create Log Group 
  5. Select Log Group “AnypointLogs”
  6. Click Create Log Stream

2. IAM configuration — you need the access key, secret key and region where the log steam is created to access the stream from Anypoint; you can get it on IAM by following these steps: 

  1. Navigate to Console and search for IAM
  2. Select Manage Access Key 
  3. Create New Access Key and write down the Secret Key (generated once while creating the access key)

Section 2: Anypoint Configuration 

1. Add the following two dependencies in the POM file of the project:

XML 

1 

<dependencies> 

2 

<dependency> 

<groupId>com.kdgregory.logging</groupId> 

<artifactId>log4j2-aws-appenders</artifactId> 

<version>2.4.1</version> 

</dependency> 

<dependency> 

<groupId>com.amazonaws</groupId> 

<artifactId>aws-java-sdk-logs</artifactId> 

<version>1.11.908</version> 

</dependency> 

12 

</dependencies>

2.Update the Log4j2.xml file of the project:

a. Add the kdgregory configuration package:

<Configuration packages=”com.mulesoft.ch.logging.appender,com.kdgregory.log4 j2.aws” status=”debug”> 

b. Add CloudWatch Appender under :

XML 

1 

<CloudWatchAppender name=”CLOUDWATCH”> 

2 

<logGroup>${sys:log.group.name}</logGroup> 

3 

<logStream>${sys:log.stream.name}</logStream> 

4 

<dedicatedWriter>true</dedicatedWriter> 

5

<PatternLayout pattern=”%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %p – %c – %m” /> 6 

</CloudWatchAppender>

c. Reference it in :

XML 

1 

<AsyncRoot level=”INFO”> 

2 

<AppenderRef ref=”CLOUDWATCH” /> 

3 

</AsyncRoot>  

Section 3: Deployment

Logs can be deployed on all Mule application deployments where custom logging can be configured. We discuss the deployment steps for Local and CloudHub below.

1. Deployment on Anypoint Studio (Local) — following configuration, you must pass the access key, secret access key, region, log group name and log stream name as a runtime argument on Anypoint Studio

Arguments 

1 

-Daws.accessKeyId= ********* 

2 

-Daws.secretKey=******** 

3 

-Daws.region=******** [example: “us-east-1” ] 

4 

-Dlog.group.name=your-Log-Group-Name [example: “AnypointLogs”] 

5 

-Dlog.stream.name=your-Log-Steam-Name [example: “cloudhubtest”]

2.Deployment on Anypoint Platform (CloudHub):

a. In Runtime Manager Settings, disable CloudHub Logs

b. During the deployment, pass the access key, secret access key and region as runtime properties

That’s it, all done!

Now, all your MuleSoft application logs deployed to CloudHub or on-premises environments will be sent to Amazon CloudWatch.

NOTES:

  1. If you don’t find the Disable CloudHub logs checkbox on Runtime Manager, send a ticket to MuleSoft.
  2. IAM role (access key and secret access key) should have the necessary permissions to push logs to CloudWatch. For example (logs:PutLogEvents, logs:CreateLogStream,logs:CreateLogGroup, logs:Describe, logs:DescribeLogSteams).

— By Santosh Hazari