Digital Financial Services Provider Grows Robustness, Security, and Optimizes AWS Costs

Profile

A digital financial services provider, this corporation supplies its financial services customers with a data aggregation and data analytics platform. Thousands of companies, including many of the largest U.S. banks, subscribe to this company’s platform to power personalized financial apps and services.

Business Needs

• Assure new product is highly available
• Ensure security and compliance
• Effectively optimize costs

Challenge

This company has started a new product that offers a large revenue opportunity. With demand growing quickly and the service evolving rapidly, this company needed to make sure that the new product was highly available while ensuring data security and optimizing for cost.

Solution

Already working in AWS, this firm called in AWS Consulting Partner, Flux7, to help expand upon its infrastructure to address its product challenges. Specifically, the product is being offered in the form of panels to this firm’s customers, with a small number of standard panels that its customers can choose from. Panels are delivered either as files in S3 buckets or as a RedShift cluster account for them to be able to query the data using Tableau on their desktop.

Grow Robustness:

With growing demand, it was important that the system supporting the panels had high availability and a failover plan in order to consistently deliver panels promised on-demand. To answer these concerns, Flux7 recommended a series of changes to how the company operated. First, it created a disaster recovery plan with the use of multiple AWS Availability Zones (AZ) to ensure that the service could still be accessible and available should a downtime event occur. With the vast majority of its servers located in a single AZ, this was a critical first step.

Following that, the team introduced Infrastructure as Code to the environment with automation driven by a trio of technologies: CloudFormation, the AWS CLI, and Jenkins. CloudFormation enables the team to use automation to easily create infrastructure. It can create backups with the AWS CLI. And, Jenkins enables scripting to automate continuous integration. As Flux7 helped this organization automate its processes, it also grew availability by removing risk in the process as automation removes the potential for fat finger and similar errors. And, this level of automation allows the software development team to much more easily add new features and functionality for greater customer satisfaction and future product robustness.

Taking automation to the next level, Flux7 helped set up ‘easy buttons’ for starting and stopping environments, giving the team greater control and agility. And, it established AWS Lambda for S3 Polling, allowing batch and real-time enrichment to behave similarly for the financial services organization. Last, working together, the two teams implemented autoscaling lifecycle events for autoscaling groups, which allowed them to create hooks for additional robustness in the system and create a foundation that helped tee the ball for cost optimization.

Optimize Costs:

Once the teams set up AWS autoscaling, it was much easier to reach the firm’s two cost-related goals: to have no idle servers; and to make the most effective possible use of AWS pricing models, like reserved and spot instances. To ensure this financial industry company was maximizing the use of its AWS resources, Flux7 closely monitored the firm’s existing servers and clusters for use and demand. In doing so, it became apparent that the company had a large number of servers with utilization below 50 percent.

After watching for daily activity, Flux7 made several recommendations to optimize this firm’s servers, again ensuring that it was not paying for resources it didn’t need and wasn’t using. It conducted the same exercise with the firm’s clusters, designing autoscaling such that the firm saved costs by only paying for resources it needed when they were needed. Last, Flux7 recommended that this organization implement Elasticsearch with high replication as it lends itself to the use of spot instances. In a hybrid cluster where a fraction of them can use Spot instances, it is possible to see a savings of 20 to 30 percent.

Secure:

This organization already had a diligent security team that was creating AWS policies to ensure systems met regulatory standards and the strict security policies of its customers. To increase security further, Flux7 instituted several changes that immediately tightened security controls at this firm:

• Strengthened this firm’s passwords, instituted multifactor authentication, and restricted access by IP.
• Built a secure VPC-based environment as code, with servers accessed via a Bastion.
• Separated production to a new AWS account, reducing accidental exposure to production data — and the potential to inadvertently delete it.
• Reduced IAM permissions.

These changes helped ensure that this financial services company could meet its security goals of having no personally identifiable information visible to any of its customers, that no customer could infer information about other customers, and that the company’s data was safe from modification or being stolen.

Benefits

In the end, this group was able to achieve its goals by implementing infrastructure as code, tightening several security standards, and implementing a DR plan. The company’s new product is now highly available, scalable, and developers are able to easily add new features and functionality, growing the product to meet customer demand. Simultaneously, the security team is confident that the company’s IP is safe and the CFO is greatly satisfied at the cost optimization achieved.