The Challenge

Traiana enables global market participants to automate cross-asset risk management and pre-trade/post-trade processing, for both listed and over the counter transactions. They solve complex problems facing both the markets and individual organizations so that firms can implement the most cost-effective solution. Traiana worked with Spot to reduce their cloud computing operational costs, which in turn benefit its customers.

The company uses Amazon Web Services (AWS) EC2 cloud infrastructure with Terraform and backed by auto-scaling groups (ASG). Their Terraform deployments consist of Apache Cassandra and Apache Kafka clusters that are built automatically. Cassandra and Kafka clusters can become very large over time and utilize larger instance types optimized for memory and processing power.

Why Spot

Traiana was looking for a solution that would allow them to use spot instances while retaining the advantages of their autoscaling groups, integrate with how they deploy machines on AWS with Terraform, and improve their CI/CD workflow.  

Traiana was interested in Spot because with a simple import from the Spot Console, they were able to import their existing Auto Scaling Group on AWS and have their workloads run on spot instances and start reducing costs.  For organizations that run an application or service behind AWS Elastic Load Balancer (ELB), Elastigroup by Spot provides the freedom to run blended clusters from multiple instance types and with balanced usage of spot instances, reserved capacity and on-demand instances when needed.  

Using mixed instance types is essential to maximize savings and most importantly, availability and continuity when using spot instances. When spot instances are terminated, Elastigroup will choose the most cost-effective instances available at the time. After using Elastigroup in production for a month, Traiana ended up saving over 70% on their cloud computing costs.

Traiana also worked with the Spot team for three months to convert their current code in Terraform to use the Elastigroup plugin in their development environments to deploy Cassandra clusters. With the Terraform Elastigroup plugin, Traiana were able to deploy their workloads onto spot instances and reduce costs without developers changing their workflows. Besides deploying clusters, Traiana was able also able to improve their CI/CD workflow.

For CI/CD, Traiana uses Jenkins to build, test, and ship their code on EC2 on-demand instances. In their journey to reduce costs, they took a look at the Spot Jenkins plugin which automatically scales up and down instances based on the number of jobs in its queue. The nodes that are being provisioned run a startup script and connect as agent nodes to the main node and immediately start to run jobs. With the Jenkins Spot Plugin, Traiana was able to simplify the management of running a Jenkins cluster and reduce costs by using spot instances.

Here is how Elastigroup helped Traiana reduce costs and operate more efficiently:

  • Seamless Integration – Elastigroup integrates with dozens of applications and services including Kubernetes, Amazon EKS, Elastic Load Balancer, Application Load Balancer, Autoscaling Groups, Amazon ECS, Terraform, AWS CloudFormation, Beanstalk, OpsWorks, EMR, CodeDeploy, Jenkins, and more.
  • Intelligent Auto-Scaling – Elastigroup provides tailor-made auto-scaling for your application. Metric-based or Event-driven, Elastigroup will always make sure that you have the optimal mix of Right-sized VMs running to meet your workloads needs.
  • Spot Instance Market – Elastigroup reliably leverages the spot instance market to optimize your underlying infrastructure for cost without compromising availability.
  • Always Up – Elastigroup uses predictive algorithms to identify and drain VMs that are about to be preempted. Prior to VM termination, Elastigroup seamlessly launches a new VM and replace it seamlessly. Elastigroup will also make sure to distribute your VMs across different sizes and types to optimize cost reduction and VM longevity.
  • Jenkins Plugin – Elastigroup will automatically scale up and down agent instances on spot instances to reduce costs.
  • Terraform Plugin – Have Elastigroup provision new instances on AWS EC2 using spot instances to reduce costs versus using on-demand.

The Results

In the end, Traiana was able to easily import their existing AWS Auto Scaling Groups into Elastigroup to take advantage of spot instances to reduce their operating costs by over 70%. Using the Elastigroup Terraform plugin, their developers did not need to make any changes to their workflow to start reducing costs deploying Cassandra clusters on spot instances. They are also currently working on their Terraform scripts to support deploying Kafka clusters on spot instances.  With the Spot Jenkins Plugin, they were able to improve their CI/CD workflow by having Elastigroup automatically spin up Jenkins agent instances on spot instances and scale the idle ones down as needed.

A CME Group company, Traiana operates leading market infrastructure for cross-asset pre- and post-trade processing, risk management, and regulatory compliance across the world.

https://www.traiana.com/