Shave Days off Azure Marketplace Publishing with Automated Testing

The Microsoft Azure Marketplace is a catalog with thousands of solutions that work on or integrate with the Azure cloud.  To ensure users have a quality experience with products in its Marketplace, Microsoft works with software vendors to certify each product before it is offered publicly. This process ensures that any product offered through the Azure Marketplace is compatible with the Azure cloud. However, ensuring quality can be a lengthy process. Wanting to streamline its publishing time, our client reached out for help to build a better approach. 

Our client develops security software for enterprises and releases images via the Azure Marketplace for its customers to use in their environment. The development team had two goals. First, they wanted to improve the publishing process with automation so that it was straight-forward and streamlined. Second, they wanted to make specific images (vs. one image with all code) for each platform where they published — e.g., Google image for Google cloud, Azure image for Azure cloud, and an AWS image for the AWS cloud.  

Publishing to the Azure Marketplace 

To publish its Azure image to the Azure Marketplace, the customer must follow Microsoft’s process to ensure the compatibility of its image. Once an image is submitted to Microsoft, the developer waits several days as Microsoft runs a series of tests, in what it calls Step One. The developer is notified of the results within a handful of days. If the image passes, it moves from this first gate to Phase Two, and so on through each of more than five steps. Publishing to the Marketplace is not a single process for developers as Microsoft has built in these multiple guardrails to ensure only quality products publish in its catalog.  

If an image fails any of the tests, the developer is alerted, and the solution is sent back for correction, only to begin the process over again. Unfortunately, Microsoft only shares a high-level report with the developer making it difficult to determine the exact cause of the issue. This results in the developer spending extra cycles to determine what went wrong and why. And, only then, can they begin addressing the issue that caused the failed test.  

Given these parameters, a few weeks is the best–case scenario for an image to publish to the Marketplace; a few weeks in today’s world can make or break a product. After all, does a developer really want their end users to wait several weeks after development is complete for the solution to be made available? 

Automated Testing Shaves Days Off the Process 

While our client wholeheartedly supports the goal of quality offerings, it sought to significantly impact the response times built into the publishing process. To do so, we helped them build a high-level testing regime using Microsoft’s API; the API allows us to run all the tests that the Microsoft team runs during its validation and certification steps. Applying automation to the process, we can run the image through each of the Microsoft tests in just five minutes. 

With this approach, the customer gets a full report which enables us to proactively identify when something fails, why it fails, and its expected behavior. With this information in hand, we can remediate the issue before solution is submitted to Microsoft, ensuring that the image flows through the publication process as quickly as possible.  

Developer Benefits 

The automated testing process removes downtime in the hand-offs between corrections and re-tests, thus giving the client added confidence that its image will pass through Microsoft test gates. In addition, the new approach streamlines the steps, removing days from the publishing process. Indeed, as our automated tests take only five minutes, it shaves 3-4 days from the process per test hurdle.

In addition, the approach decreases response times because when an issue is found, the test pinpoints exactly where it occurred as well as the expected behavior. This saves the customer needless hours finding the issue, allowing them to focus one hundred percent of their efforts on remediation – all of which means they can get an image that is sure to successfully satisfy Microsoft’s quality controls. Most importantly, the new process helps the client get their image into the Azure Marketplace – and in front of their users – as quickly as possible. 

Could automation help speed your processes and grow quality outcomes?  Reach out to our team today to learn more. Or stay tuned for our next blog on how to publish managed images to the Microsoft Azure Marketplace.