CI/CD on Spot Instances
What is CI/CD?
Continuous Integration) and CD (
Continuous Deployment/Delivery) are related concepts designed to improve efficiency in developer teams, featuring both a cultural and technical element. CI/CD aims to improve efficiency by ensuring that code across an entire development team is always integrated with the central line of code.
CD differs from CI through deploying all code to a testing environment to make sure that changes are fully tested and ready for release. There is also a difference between
Continuous Deployment and
Continuous Delivery, even though both are sometimes referred to as CD. In
Continuous Deployment, the moving of code changes from staging to production environments happens automatically, whereas for
Continuous Delivery this process must be manually approved.
Cultural and Technical CI/CD
The cultural element of
CI/CD is a focus on frequent code integrations across a developer team, often multiple times per day. This is designed to make sure that any bugs or flaws in the code can be found much more quickly. If a developer’s code is frequently integrated then there is a lot less possibly “bad” code to check for issues. Put simply, if you can check every 6 hours of work for errors in code that would make it un-integratable, there’s a lot less code to try and fix if anything goes wrong compared to if you check every 6 days.
The technical element of CI/CD is software which can automatically verify the health of developer code as it is regularly merged into the main line of code. Without an automated element, CI/CD would not be possible as the constant manual health-checking would slow down the developing process to a near stand-still.
Possible Challenges with the CI/CD Pipeline
One issue with the CI/CD for many companies is that, whilst it undoubtedly improve the lives of the developers, it can lead to headaches for company budgets as growth leads to increased expenditure in the test/staging environments.
Increasing number of developers (always a good thing) leads to increasing amount of tests, therefore increasing amount of servers required to run tests and increasing costs of doing so (always a bad thing). This is where companies often face a toss-up between staying Cloud-based to allow ease of scaling or else move to On-Prem environments to allow reduced cost. You can read a detailed case study on this process written by Capriza.
Cost-effectiveness of CI/CD processes
Using Cloud providers spare capacity instances (i.e. Spot Instances on AWS) is a great way to radically increase the cost efficiency of Cloud environments. CI/CD processes are some of the best use-cases for Spot Instances, as testing environments need be intensely scalable whilst being as cost-efficient as possible.
As CI/CD processes are not directly customer-facing, the damage caused by any interruptions to Spot Instances is also much smaller than production or mission-critical environments.
IDC Study Says..
With costs so much lower on
Spot Instances, developer teams don’t have to be restrained as much by budget when it comes to their testing, meaning that using Spot Instances can actually speed up development. A 2017 study by IDC found that using Spot Instances could speed up application development by
13% along with increasing developer productivity by
Using Spot Instances for CI/CD does have its risks, however. For one, connectivity to Spot Instances cannot be guaranteed as AWS provide no SLA for Spot Instances. This means that instances may go offline suddenly, resulting in a slow-down of the CI/CD process. Some integrations which can greatly improve the efficiency of the CI/CD process (Chef, Jenkins, CodeDeploy, etc.) can run into serious issues when a Spot Instance is terminated during the CI/CD process. These issues can lead to running CI/CD on Spot becoming increasingly time-consuming for developer teams, greatly reducing both cost and time efficiency.
This is where Elastigroup can be of help to reduce costs for the
CI/CD pipeline automation whilst simultaneously allowing a company to expand their testing by utilizing Elastigroup’s ML algorithm to predict instance terminations and prevent 99% of Spot interruptions from actually happening.
With Spotinst providing Spot Instances the same SLA as On-Demand Instances for a greatly reduced price, companies can enjoy all the benefits of running a
CI/CD pipeline on
Spot Instances without having to worry about the issues that Spot can raise.
Checkout Elastigroup for Jenkins.