fbpx

Run Stateful applications on Spot Instances in 6 Steps

Introduction

 

In this tutorial, you will learn how to run stateful workloads such as NoSQL databases, Elasticsearch clusters, and dev environments on Amazon EC2 Spot instances with Spotinst Elastigroup.
Spotinst Elastigroup strips away the risk and complexity of running compute workloads on cloud providers’ excess capacity (e.g. Amazon EC2 Spot instances), letting customers pay up to 80% less on their compute, without compromising on availability.

Spot instances are offered at up to 90% off EC2 On-Demand prices, providing you with a great avenue to drastically reduce compute costs. Of course, Spot instances can be taken away by AWS at any time with just a short notice, narrowing the candidate pool to workloads which are fault tolerant and/or ephemeral by nature.

Stateful Elastigroup lets you persist the instance’s volumes and IP, making Spot instance replacements seem like a simple system reboot. Stateful Elastigroup changes the paradigm and widens the workload candidate pool, letting you run any workload without a single point of failure on Spot Instances. To read more about Stateful Elastigroup, visit our stateful documentation.

You are now minutes away from running your stateful workload on Spot Instances. Let’s get started!

Prerequisites:

  1. A verified Spotinst Account.

 

Step 1: Open The Elastigroup Creation Wizard

 

Login to the Elastigroup Console (console.spotinst.com) and navigate to the Creation Wizard by clicking the Create button in the Elastigroups tab.

 

 

In the Creation Wizard select Stateful:

 

Next, you can choose to create an Elastigroup from scratch, and manually enter configurations such as the AMI, VPC, subnets, and key pair, or alternatively, select an existing instance to import the configuration from.

In this tutorial, we will save some time by basing our new group on the configuration of an existing instance.

 

 

 

Step 2: Set the Elastigroup orientation to Availability

(best practice)

 

The Elastigroup cluster orientation influences the Spotinst algorithms that work behind the scenes to keep your group highly available (i.e. minimize instance replacements), while also maximizing cost savings.

By default, the cluster orientation will be set to ‘balanced’, indexing equally on availability and cost savings. For stateful applications, given the nature of data-dependent applications normally that run on stateful Elastigroups, it is recommended to change the cluster orientation of your group to ‘availability’. This will influence Spotinst’s internal logic to index higher on minimizing the number of instance replacements experienced by your group.

To Change the cluster orientation, scroll down to the ‘advanced’ section, click on the ‘cluster orientation’ drop-down, and select ‘Availability’.

 

Once you’ve selected an ASG to import configuration from, click on ‘Create

 

NOTE: Steps 3-5 below each describe how to configure a specific portion of Elastigroup’s stateful feature set. Once one (or more) of the steps below is configured, your group is considered to be 'Stateful'.You can choose to configure the stateful features that are right for your workload.

 

Step 3: Persist the instance’s root volume (optional)

 

With stateful Elastigroup, you can configure each instance in the group to persist its root volume, meaning that upon instance replacement the OS, and any installed software packages and libraries are seamlessly migrated to the new instance.

To configure root volume persistence for your Elastigroup, simply navigate to the compute tab, scroll down to the ‘Stateful’ section, and check ‘Persist Root Volume’.

 

 

Step 4: Persist the instance’s data volume (optional)

 

In this step, you can configure the data attached to your instances to be carried over to newly launched instances when instance replacements occur. This can be done in one of two ways:

  1. Reattach volumes – Instance-attached volumes will be detached from the interrupted instance and reattached to the new instance upon launch.
  2. Snapshot backups – EBS volumes attached to the running instance will be continuously snapshotted, with the latest being used to create new volumes upon instance replacement. 

To configure data volume persistence for your Elastigroup, simply check the ‘Persist Data Volumes’ checkbox.

NOTE: the ‘Reattach volumes’ option will be selected by default. If you prefer Elastigroup to snapshot the existing volumes and create new ones upon instance replacement, select ‘Snapshot Backups’.

 

Step 5: Persist the instance’s Private IP (optional)

 

In this step, you may configure Elastigroup to persist your private IP (using the same ENI) upon instance replacement.

To persist each instance private IP, check the ‘Maintain Private IP’ box.

NOTE: If required, you can specify a pool of private IPs to be used by the instances within your Stateful Elastigroup. To do so, check the ‘Assign Specific Private IPs’ box, and enter as many IP addresses as needed (by clicking on the button)

Step 6: Review and Create your group

 

In the review step, once you have finished reviewing your group’s Json config file, simply click ‘create’.

 

Congratulations!

 

You have now learned how to run a stateful application on Spot instances with Spotinst, letting you:

  • Persist the root volume to carry over any added software or altered configuration
  • Persist the data volumes to keep all of the instance data across instance replacements.
  • Persist the instance’s private IP so its network configuration remains the same.

If you have followed all the steps above, any instance replacement within your Elastigroup should seem like a mere system reboot, persisting your system definitions and software, data, and network information.

 

Next Steps

Now that you have learned how to a stateful application on Spot Instances with Spotinst, try Spotinst for other workloads.