How to create Bitbucket custom build and deploy pipelines for MuleSoft applications
- July 04, 2023
What is Bitbucket?
Bitbucket is a source code repository platform that enables us to manage our repositories and empowers organizations to build, test and deploy applications using Bitbucket Pipelines.
We can create either automated pipelines or custom pipelines using Bitbucket Pipelines.
Automated pipelines: They automatically trigger when a commit is made on a branch for which the pipeline is set up or enabled.
Custom pipelines: They allow us to customize CI/CD pipelines, enabling us to manually run them as per our requirements with custom inputs.
Let’s begin with the Bitbucket Pipeline implementation for Mule applications.
Bitbucket code
- Create repository variables that will be used in the Bitbucket Pipeline.
From the repository, navigate to the Repository settings → Repository variables.Repository variable section snapshot - Follow the steps below to create a two-step custom pipeline (build & deploy) in Bitbucket.
- Build Pipeline
- Create bitbucket-pipelines.yml file.
- In the bitbucket-pipelines.yml, create a custom build pipeline named ‘MSFT-build-pipeline’ using a custom keyword.
- In the build pipeline, we have the following three steps:
- Step 1 (Invalidate Cache): invalidate the cache if there are any changes in the pom.xml file. To invalidate the existing cache, we are using the built-in bitbucket pipe “atlassian/bitbucket-clear-cache”.
- Step 2 (Test): Run MUnits for the application and cache the Maven dependencies.
- Step 3 (Package): We’re packaging the application and uploading the generated build artifact to the Bitbucket Downloads artifactory using the Bitbucket built-in pipe ‘atlassian/bitbucket-upload-file’.
- Build Pipeline
Build pipeline code snippet
image: maven:3.6.3-jdk-8
definitions:
steps:
- step: &invalidate-cache
name: Invalidate Cache
script:
- echo "Delete cache if changes in the pom.xml"
- pipe: atlassian/bitbucket-clear-cache:3.1.1
variables:
BITBUCKET_USERNAME: $BITBUCKET_USERNAME
BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
CACHES: ["maven"]
condition:
changesets:
includePaths:
- bitbucket-pipeline-poc/pom.xml
pipelines:
custom: # Pipelines that are triggered manually
MLSFT-build-pipeline:
- step: *invalidate-cache
- step:
name: Test
#trigger: 'manual'
caches:
- maven
script:- echo "Runnning MUnits for $application_name on branch $BITBUCKET_BRANCH"
- mvn -f $application_name/pom.xml test -s settings.xml #-DskipTests
- step:
name: Package
script:
- echo "Build number is $BITBUCKET_BUILD_NUMBER"
- mvn -f $application_name/pom.xml clean package -DskipTests -s settings.xml -Djenkins.version=$BITBUCKET_BUILD_NUMBER
- export artifactName=$(ls $application_name/target/*.jar | head -1)
- echo "Artifact $artifactName created"
- echo "Uploading artifact to Bitbucket Downloads"
- pipe: atlassian/bitbucket-upload-file:0.1.2
variables:
BITBUCKET_USERNAME: $BITBUCKET_USERNAME
BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
FILENAME: $artifactName
- echo "Artifact $artifactName is succeesfully uploaded to Bitbucket Downloads"Deploy Pipeline
- In the bitbucket-pipelines.yml, create a custom deploy pipeline named ‘MSFT-deploy-pipeline’ using a custom keyword.
- The deploy pipeline takes the CloudHub environment name to which the artifact needs to be deployed/redeployed, and the artifact/jar build number to download it from Bitbucket Downloads.
- In the deploy pipeline, we have the following two steps:
- Step 1 (Get Artifact): Retrieve the Bitbucket Downloads artifact name using the build number by making a Bitbucket REST API call. Once we have the artifact filename, download the artifact from Bitbucket Downloads using the REST API, and share this artifact and artifact name with the next step, ‘Deploy Application,’ by using the Bitbucket artifacts keyword.
- Step 2 (Deploy Application): We set up some environment variables based on the CloudHub environment selected by executing the external shell script. Once the environment variables are set, we deploy the artifact downloaded in the previous step to CloudHub using the Mule Maven command. In the after-script section, we send success/error notifications via Email & Slack channels by leveraging the built-in pipe provided by Bitbucket.
Deploy pipeline code snippet
MLSFT-deploy-pipeline:
- variables:
- name: cloudhub_env_c
default: Dev
allowed-values:
- "Dev"
- "Test"
- "Uat"
- "Prod"
- name: build_number_c
- step:
name: Get Artifact
script: - echo "Build number is $build_number_c"
- |
export filename=$(curl -v --request GET "https://$BITBUCKET_USERNAME:$BITBUCKET_APP_PASSWORD@api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG/downloads" | grep -o "[^\"/]*-B$build_number_c-[^\"/]*.jar" | head -1)
- echo $filename
- mkdir -p target1/
- |
curl --location -v --request GET "https://$BITBUCKET_USERNAME:$BITBUCKET_APP_PASSWORD@api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG/downloads/$filename" -o target1/$filename
- ls target1/*.jar | head -1 > ./file.txt
artifacts:
- target1/*.jar
- file.txt
- step:
name: Deploy Application
trigger : 'manual'
script:
- echo "Deploy Application"
- . set-environment-deployment-variable.sh # to execute external bash script
- echo "config file is $config_file and application name is $application_name_c"
- export file=$(cat file.txt)
- mvn -f $application_name/pom.xml mule:deploy -s settings.xml -DskipTests -Dmule.artifact=$file -Dconnectedapp.clientid=$connected_app_client_id -Dconnectedapp.clientsecret=$connected_app_client_secret -Danypoint.platform.environment=$cloudhub_env_c -Dmule.env=$config_file -Dapplication.muleVersion=$mule_version -Dapplication.name=$application_name_c -
-Dapplication.region=$region -Dvault.key=$key -Dapplication.workers=$workers -Dapplication.workerType=$worker_type -Danypoint.platform.client.id=$anypoint_platform_client_id -Danypoint.platform.client.secret=$anypoint_platform_client_secret
after-script:
- ALERT_TYPE="SUCCESS"
- export file=$(cat file.txt | cut -d / -f 2)
- app_env_suffix=$( echo "$cloudhub_env_c" | tr -s '[:upper:]' '[:lower:]' )
- content="$(echo \"$(cat bitbucket-email-template.html)\")"
- eval echo "$content" > email-notification.html
- if ! [[ "$emailEnabledEnvironment" == *"$cloudhub_env_c"* && $email == "true" && $BITBUCKET_EXIT_CODE == "0" ]]; then
ALERT_TYPE="ERROR" ;fi
- pipe: atlassian/email-notify:0.7.0
variables:
USERNAME: 'replaceme@gmail.com'
PASSWORD: $GPASSWORD
FROM: 'replaceme@gmail.com'
TO: 'receipient1@gmail.com,mulesoft-poc-receipient@***.slack.com'
HOST: 'smtp.gmail.com'
DEBUG: 'true'
SUBJECT: '${ALERT_TYPE}:${application_name} Bitbucket Pipeline Notification for branch ${BITBUCKET_BRANCH}'
BODY_HTML: 'email-notification.html'
Shell script code snippet
#!/bin/bashif [ $cloudhub_env_c == "Dev" ]; then
config_file="dev"
key=$vault_key_nonprod
application_name_c=${application_name}-dev
anypoint_platform_client_id=$anypoint_platform_client_id_dev
anypoint_platform_client_secret=$anypoint_platform_client_secret_dev;
elif [ $cloudhub_env_c == "Test" ]; then
config_file="test"
key=$vault_key_nonprod
application_name_c=${application_name}-test
anypoint_platform_client_id=$anypoint_platform_client_id_test
anypoint_platform_client_secret=$anypoint_platform_client_secret_test;
elif [ $cloudhub_env_c == "Uat" ]; then
config_file="uat"
key=$vault_key_nonprod
application_name_c=${application_name}-uat
anypoint_platform_client_id=$anypoint_platform_client_id_uat
anypoint_platform_client_secret=$anypoint_platform_client_secret_uat;
elif [ $cloudhub_env_c == "Production" ]; then
config_file="prod"
key=$vault_key
application_name_c=${application_name}
anypoint_platform_client_id=$anypoint_platform_client_id_prod
anypoint_platform_client_secret=$anypoint_platform_client_secret_prod;
else
echo "error"
exit 0;
fi HTML email notification code snippet
<html>
<head>
<style>
table {
font-family: arial, sans-serif;
border-collapse: collapse;
width: 100%;
}
tr,
td {
border: 1px solid #000000;
text-align: left;
padding: 8px;
}
</style>
</head>
<body>
<h3>Dear Team,<br><br>Build number ${build_number_c} has been successfully deployed to ${cloudhub_env_c} environment</h3>
<h3>
Please find below additional details<br></h3>
<table>
<tr>
<td>CloudHub Application Name</td> <td>${application_name}-${app_env_suffix}</td>
</tr>
<tr>
<td>Bitbucket Repository Name</td>
<td>${BITBUCKET_REPO_SLUG}</td>
</tr>
<tr>
<td>Branch Name</td>
<td>${BITBUCKET_BRANCH}</td>
</tr>
<tr>
<td>CloudHub Environment</td>
<td>${cloudhub_env_c}</td>
</tr>
<tr>
<td>Bitbucket Artifact File Name</td>
<td>${file}</td>
</tr>
<tr>
<td>Build Pipeline Build Number</td>
<td>${build_number_c}</td>
</tr>
<tr>
<td>Deploy Pipeline Build Number</td>
<td>${BITBUCKET_BUILD_NUMBER}</td>
</tr>
</table>
</body>
</html>MuleSoft code
- Create a simple Mule Application.
- Add CloudHubDeployment configuration in Mule Maven Plugin.
Mule Maven Plugin Snippet
- Add the settings.xml file to the repository’s root directory.
settings.xml file snapshot
Test the bitbucket pipeline:
- Go to the Pipelines section, and click on ‘Run pipeline.’ A pop-up window will appear, allowing you to select the branch on which you want to run the pipeline and the pipeline (build/deploy) to run. First, we are going to run a build pipeline on the feature branch ‘feature/feature-314’ to create the artifact and upload it to the Bitbucket Downloads.
Build pipeline UI snapshot
- The build pipeline has been successfully completed, and the artifact has been uploaded to the Bitbucket Downloads. The build pipeline build number is 131.
Build Pipeline run status snapshotBitbucket Downloads snapshot
- Run the deploy pipeline to deploy the generated artifact/jar to the CloudHub target environment. To run the build pipeline, we need to provide the CloudHub environment name on which we want to deploy our application and the build pipeline build number, so that we can download that artifact from Downloads.
Build pipeline UI snapshot
- The deploy pipeline has been successfully completed, and the application is deployed to the CloudHub Dev environment. Additionally, notifications are triggered to the email and Slack channel.
Deploy pipeline run status snapshotEmail notification snapshotSlack notification snapshot
Custom pipeline advantages
- It saves pipeline build minutes in Bitbucket, as we won’t be running the pipeline on every commit.
- You can run a custom pipeline on any branch (e.g., working, hotfix, feature.).
- It allows us to build the application on any branch and deploy that build artifact to multiple target environments.
Custom pipeline disadvantages
- Manual intervention is required to run the pipeline.
Current pipeline limitations
To obtain the artifact filename using the build number in the deploy pipeline step, we’re using the REST API. However, this API supports pagination, and currently, I haven’t implemented pagination logic. As a result, if our build number isn’t present on the first page, the pipeline will fail. To address this issue, we have two solutions: either implement pagination logic or, instead of providing the build number in the deploy pipeline, provide the complete artifact/jar name and remove the logic to get the artifact filename based on the build number.
References:
https://support.atlassian.com/bitbucket-cloud/docs/get-started-with-bitbucket-pipelines/
— By Saddam Shaikh