Coal, Hyperscale Computing and Cloud Workload Automation

Around 250 years ago, the Industrial Revolution ushered in an unprecedented era of productivity and growth. Society transitioned from painstaking, manual, hand-made craftsmanship of all goods, to automated, machine-driven, mass manufacturing of everything.

This incredible shift was fueled by the invention and enhancements of coal-powered steam engines. 

Today, we are living in a technological revolution that is driven by hyperscale cloud computing and while the changes I personally have seen in the last 30+ years are amazing, I believe this is just the beginning. 

In 1865, English economist William Stanley Jevons was researching the future supply of coal and noted, “Whatever, therefore, conduces to increase the efficiency of coal, and to diminish the cost of its use, directly tends to augment the value of the steam-engine, and to enlarge the field of its operations.” 

Simply put, the more efficient people become in consuming a resource, more of that resource is consumed. 

Coming back to 2019, today’s coal is cloud computing. The more cost-effective it is AND the easier it is to use, the more it will be used. 

But just how easy is cloud computing? Let’s take a look at some areas in the AWS cloud that can be challenging.

The Finest Blend of EC2 On-Demand, Reserved and Spot Instances…..in One Cluster

Many companies are seeking a platform that automatically places workloads on the most suitable EC2 instances, both in terms of pricing models AND best matching CPU / Memory for the workload.  

On the cost side of things, if you purchased Reserved Instances, you are locked-in for either 1 or 3 year. In the event that projects end sooner than projected, you will end up with fully-paid for (or contractually committed to), completely unutilized resources. 

If you dream of using Spot instances that can slash EC2 pricing by 90%, ensuring availability is a real challenge (AKA a cluster@#$%^&*!) as AWS can pull the plug at any time with just a 2 minute warning. 

So by and large, you end up paying On-Demand rates and crying at the end of each month when you get the bill.

Keep reading to find out how you can utilize all your RIs and run mission-critical workloads on Spot Instances. 

Real-Time Requirements Should Determine Infra Deployment and Workload Placement

Wouldn’t it be great if infra deployment and workload placement was driven by actual application or system requirements? Unfortunately this is not the case. 

Take for example the golden standard in software development –  containers and container orchestration platforms. 

Amazon’s Elastic Container Service (ECS), Elastic Kubernetes Service (EKS) and other orchestration tools are great, but their autoscalers are driven purely by node metrics which can be completely misleading and unrelated to the reality of the actual resource consumption of tasks or pods 

In Kubernetes, the master node is responsible for maintaining the desired state for the cluster. Even if the cluster has a surplus of resources, once a single pod requires more CPU or RAM than a single node has available, the node won’t start and you’ll receive an error message such as:

Similarly in ECS, imagine your Auto Scaling Group has a policy to scale up anytime that the Task’s CPU goes over 70%.

If the Task CPU is at 65% avg, there is no scale-up, even if the ECS scheduler has a new Task that requires, say 40% CPU. This scenario will result in the Task not being scheduled.

Clearly, Tasks or Pods not running due to this auto-scaling gap, is a major issue that must be handled to ensure your workloads run when they should. 

Surprise Ending – There is a Solution That Can Handle it All

Of course, I wasn’t going to leave you hanging with all these problems. In these final paragraphs, I’ll briefly outline how Spotinst allows you to easily launch clusters with Spot Instances, Reservations and if desired, On-Demand. I’ll also share how Spotinst can abstract away all the infra challenges that are inherent in all container orchestration platforms.

Launch Spot, On-Demand and Reserved Instances in a Single Cluster

While AWS Spot Fleet is a great option for a nicely blended cluster (it allows you to define the percentage of On-Demand vs. Spot Instances in your cluster), if any Spot Instances are interrupted, Spot Fleet will not replace them with On-Demand (to ensure high availability). 

Additionally, even if you have available Reserved Instances, Spot Fleet will not actively seek to launch instances of the same type, thereby missing out on additional savings and cost efficiency.

This is where Spotinst’s flagship product, Elastigroup, takes your cluster to the next level. 

For example, with Spotinst Elastigroup you can:

  • Launch your cluster with 100% Spot Instances and if there is any interruption, Elastigroup will automatically launch other Spot types or On-Demand Instances to ensure availability.
  • Fully utilize any pre-purchased Reserved Instances as Elastigroup knows to automatically launch matching instances (and will do so before spinning up Spot Instances). 
  • Increase cluster availability (beyond what Spot Fleet can do) by fulfilling Spot requests across all subnets in selected Availability Zone(s). 

NOTE: While Spot Fleet and Spotinst both provide percentage distribution of Spot vs. On-Demand, it’s NOT at the level of the Task/Pod but only at the infrastructure level of the cluster. Once AWS adds lifecycle to service definition, this capability will be feasible.

Say Goodbye to “No Nodes are available” with Tetris Scaling

As we mentioned above, the available container orchestration platforms don’t properly handle scaling based on Pod or Task requests which often results in workloads not running as scheduled. 

Here too Spotinst provides a comprehensive scaling solution, AKA Tetris scaling as a part of  Spotinst Ocean, a “serveless container” deployment option for Kubernetes and ECS.  

Spotinst Ocean automatically detects when a Pod or Task has specific CPU or RAM requirements and will spin up the appropriate instance size or type.  

Additionally, Spotinst Ocean offers Right-sizing (Vertical Pod Autoscaling or VPA) which essentially looks at the actual Pod or Task resource utilization and based on that decides if it needs to increase or decrease available CPU and RAM as relevant.

These two features complement each other. As the Pods and Tasks themselves becomes more optimized, the Pod/Task-driven scaling can act more precisely for even greater cost savings and efficient placement of workloads in your cluster.

Let’s Hyperscale Your Cloud 

Circling back to the beginning of this blog post, Spotinst is providing cloud consumers an easier and more affordable way to automate workloads with any of the popular cloud providers (AWS, GCE, Azure, etc.) which in turn is driving our new era of technological innovation and growth. 

In closing, here is Jevons’ famous economic principle, slightly modified for our purposes: “Whatever, therefore, conduces to increase the efficiency of cloud-computing, and to diminish the cost of its use, directly tends to augment the value of the cloud, and to enlarge the field of its operations.”

Explore Spotinst today!