Optimizing Costs for Container-as-a-Service Architectures Using Zero Scaling

By Gcore

April 16, 2024

6 min read

Optimizing Costs for Container-as-a-Service Architectures Using Zero Scaling

In container-as-a-service (CaaS) architectures, balancing product performance and cost optimization hinges on eliminating unnecessary cloud expenditure. One method to reduce such spending is to deactivate unused containers and reactivate them as needed, a process called zero scaling. This article examines scaling to zero within the CaaS framework, explains how it contrasts with autoscaling, and uncovers how it can be used to optimize your CaaS costs.

What Is Zero Scaling?

Zero scaling, or scaling to zero, involves reducing resources to zero when idle and scaling to the required number of replicas as needed. This approach allows for the optimization of cloud resources and their associated costs, ensuring that containers only run when necessary.

Why Is Zero Scaling Essential for Businesses Using CaaS?

Since CaaS providers typically operate on a pay-per-use model, zero scaling of unused containers is prudent, particularly as data and traffic demands fluctuate over time. Without scaling to zero, idle containers with no demand continue to use computing resources, leading to unnecessary costs for unused capabilities.

Here’s an analogy to illustrate the value of scaling to zero: Imagine an examination body that conducts biannual tests. Its website is active only during the weeks surrounding each test period. If this website is left running throughout the year, or even if it is scaled down to a single replica, the idle container(s) for the registration service will still accrue costs, eating into the enterprise’s cloud budget. Zero scaling solves this problem by ensuring the enterprise only operates and incurs costs for the containers twice a year, during the active test periods.

Autoscaling vs. Zero Scaling

Although scaling to zero and autoscaling to one replica both aim for cost optimization, they differ in their approach, efficacy, and implementation. While autoscaling does generate some cost savings, zero scaling does so more effectively.

The table below offers a bird’s eye view of the similarities, differences, upsides, and downsides of each approach.

Parameters	Autoscaling	Zero Scaling
Description	Allows you to scale down/up or in/out automatically based on preconfigured schedules (date/time), user request metrics, or artificial intelligence predictions.	Lets you scale in (to zero) and out by tracking and predicting inbound requests, and increasing or removing the number of container instances accordingly.
Differences	Keeps at least one container invocation running during idle periods. Supports both vertical and horizontal scaling.	Deletes all containers when idle. Supports horizontal scaling only.
Use case	Applications with regular periods of use, but with highly varied peak and non-peak usage.	Applications with regular periods of disuse.
Benefits	No latency or performance glitches once inbound requests start to arrive.	Lower baseline utilization and cost; you do not pay for idle resources at all. You fully maximize resource utilization and save on cost.
Restrictions	Comparatively higher cost; since at least one container invocation is left running you continue to pay for compute resources that are not being used.	Some lag time during which new pods are created and booted up, usually 1–2 seconds.

The differences between zero scaling and autoscaling

How Does Zero Scaling Work in a CaaS Architecture?

Zero scaling for CaaS first requires a decision about whether to use the manual or automatic method. A four-step process is then required to set up zero scaling.

Manual vs Automatic

Zero scaling can be implemented either manually or automatically. The manual scaling method involves toggling the ON/OFF icon for each container manually, based on inbound requests. This approach may be impractical due to the significant effort it demands.

In the automatic approach, the architecture is configured to detect new requests automatically and activate the necessary number of containers. Four key steps are involved in zero scaling: configuration, tracking, triggering, and scaling. Although most CaaS providers offer autoscaling by default, you can configure scaling to zero yourself.

Four-Step Process

Having determined whether to use manual or automatic implementation, zero scaling can be set up using a four-step process.

Step 1: Configuration

In this step, you define your scaling limits, rules, and behavior. Zero scaling requires that you set minimum and maximum target replica counts. Typically, there is a default number, which varies across different CaaS providers. For example, in the following table, the default is autoscaling to 1 replica, while the newly defined minimum value is scaling to zero.

Scale Property	Description	Default value	Minimum value	Maximum value
minReplicas	Minimum replica count	1	0	30
maxReplicas	Maximum replica count	10	1	30

When setting the scale limit to zero, keep in mind that there’s usually a default cooldown period ranging from 30 to 300 seconds. During this time, it is not possible to launch or terminate any replicas. This cooldown period prevents autoscaling groups (ASGs)—a key component of zero scaling in CaaS—from adding or removing replicas before completing the previously scheduled activities on the replicas.

You can set scale rules using one of these three methods:

HTTP requests: When defining scale rules based on HTTP requests, you configure your CaaS provider to collect HTTP metrics every 15 seconds and launch a new replica when a specific number of HTTP requests is reached. For example, if you set the maximum value of HTTP requests per replica at 100 and the minimum to 0, a new replica will be launched once HTTP requests hit 101. If no requests are incoming and the cooldown period has passed, all existing replicas will be terminated.
TCP connections: This method is similar to HTTP request-based scaling, but it relies on the number of concurrent TCP connections.
Event-driven or custom metrics: Custom or event-driven metrics are application-specific metrics collected via an API. Integrating a third-party autoscaler, such as Knative or KEDA, makes this method more complex. Not all custom metrics are suitable for every application type.

Step 2: Monitoring

In zero scaling, software acts as an intermediary between containers and inbound requests, monitoring traffic flow and collecting metrics based on the scale rules you set in step 1.

Step 3: Triggering

Also known as scale behavior, triggering uses an algorithm to compare the collected metrics with your preset scale limits. It thereby determines the appropriate number of replicas to operate at any time. The algorithm must also detect new requests in zero-scaled containers, initiating a new replica to avoid request timeouts or HTTP 500 errors.

Step 4: Scaling

This step adjusts the number of required replicas, scaling in or out to achieve the precise quantity needed.

Zero Scaling Pros and Cons

Let’s look at the pros and cons of zero scaling.

Advantages of Zero Scaling

Beyond the clear cost-saving benefits, zero scaling offers several additional advantages:

Waste elimination: Cloud resources are billed by the hour or second, regardless of actual use. An idle instance costing $0.0105 per second could lead to $163,296 in wasted expenses over six months. Zero scaling ensures you pay only for the computing load you use, with minimal maintenance fees possibly incurred.
Budgeting accuracy and better investments: Zero scaling eliminates unnecessary expenses by terminating idle instances, thereby reducing bills. Staying within budget allows for better resource allocation, enhanced return on investment (ROI), and the opportunity to invest in innovation with the saved funds.
Improved security: Idle containers can become backdoors for cybersecurity attacks. With zero scaling, all containers, including potentially vulnerable ones, are deleted, thus shrinking the attack surface. Also, launching a new replica typically involves using a freshly pulled image, unless configured otherwise, which helps to eliminate existing vulnerabilities.

Disadvantages of Zero Scaling

Zero scaling has one significant drawback: the cold start time, which is the time it takes for a new or previously terminated app or service to boot up from scratch. In well-optimized CaaS and Kubernetes environments, this cold start period lasts 1–2 seconds because new containers do not receive requests until they are marked “Ready”. This can be a problem because web requests normally need to be processed in less than 100 milliseconds.

To mitigate the cold start-time issue, consider the following strategies:

Prefetch and cache images locally or at the edge, rather than pulling them from a remote registry for each replica launch. This can significantly reduce start times.
Use zero scaling wisely, understanding when and how it’s best applied. For example, with a situation like the exam registration portal mentioned earlier, preemptively sending a few requests to the container just before peak times can help avoid cold start delays for users.
In development/testing environments, implement scheduled zero scaling by turning off idle containers during non-working hours, such as nights, weekends, or holidays, and reactivating them when the team returns.
Where zero scaling isn’t feasible, employ a keep-warm strategy. This involves maintaining one running instance to handle immediate requests, with the capacity to scale up or out in response to increased traffic, similar to autoscaling.

Gcore for CaaS Zero Scaling

Gcore CaaS offers an intuitive scale-to-zero feature alongside the default autoscaling option, providing you with the option to decide which of your containers should be scaled to zero and which must be scaled to one.

Gcore Customer Portal UI for setting scale rules

Gcore CaaS delivers high availability and 24/7 technical support to ensure a seamless zero-scaling process.

Conclusion

Zero scaling is based on the fact that not all applications or services receive constant traffic. Allowing container resources to remain active without traffic leads to unnecessary expenditure, impacting an organization’s revenue and diminishing ROI. For applications that don’t have constant activity, zero scaling can result in substantial cost savings within the CaaS framework, where customers pay for resource consumption regardless of actual use. However, the cold start time is a notable compromise, effectively managed by selectively applying zero scaling to appropriate applications while scaling others to a minimum of one.

Gcore CaaS enables easy configuration of both zero scaling and autoscaling with just a few clicks. Get started for free to explore how it works.

Explore Gcore CaaS

Cloud computing: types, deployment models, benefits, and how it works

Cloud computing is a model for enabling on-demand network access to a shared pool of configurable computing resources, such as networks, servers, storage, applications, and services that can be rapidly provisioned and released with minimal management effort or service provider interaction. According to research by Gartner (2024), the global cloud computing market size is projected to reach $1.25 trillion by 2025, reflecting the rapid growth and widespread adoption of these services.The National Institute of Standards and Technology (NIST) defines five core characteristics that distinguish cloud computing from traditional IT infrastructure. These include on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service.Each characteristic addresses specific business needs while enabling organizations to access computing resources without maintaining physical hardware on-premises.Cloud computing services are organized into three main categories that serve different business requirements and technical needs. Infrastructure as a Service (IaaS) provides basic computing resources, Platform as a Service (PaaS) offers development environments and tools, and Software as a Service (SaaS) delivers complete applications over the internet. Major cloud providers typically guarantee 99.9% or higher uptime in service level agreements to ensure reliable access to these services.Organizations can choose from four primary use models based on their security, compliance, and operational requirements. Public cloud services are offered over the internet to anyone, private clouds are proprietary networks serving limited users, hybrid clouds combine public and private cloud features, and community clouds serve specific groups with shared concerns. Each model provides different levels of control, security, and cost structures.Over 90% of enterprises use some form of cloud services as of 2024, according to Forrester Research (2024), making cloud computing knowledge important for modern business operations. This widespread adoption reflects how cloud computing has become a cornerstone of digital change and competitive advantage across industries.What is cloud computing?Cloud computing is a model that delivers computing resources like servers, storage, databases, and software over the internet on demand, allowing users to access and use these resources without owning or managing the physical infrastructure. Instead of buying and maintaining your own servers, you can rent computing power from cloud providers and scale resources up or down based on your needs.Over 90% of enterprises now use some form of cloud services, with providers typically guaranteeing 99.9% or higher uptime in their service agreements.The three main service models offer different levels of control and management. Infrastructure as a Service (IaaS) provides basic computing resources like virtual machines and storage. Platform as a Service (PaaS) adds development tools and runtime environments, and Software as a Service (SaaS) delivers complete applications that are ready to use. Each model handles different aspects of the technology stack, so you only manage what you need while the provider handles the rest.Cloud use models vary by ownership and access control. Public clouds serve multiple customers over the internet, private clouds operate exclusively for one organization, and hybrid clouds combine both approaches for flexibility. This variety lets organizations choose the right balance of cost, control, and security for their specific needs while maintaining the core benefits of cloud computing's flexible, elastic infrastructure.What are the main types of cloud computing services?The main types of cloud computing services refer to the different service models that provide computing resources over the internet with varying levels of management and control. The main types of cloud computing services are listed below.Infrastructure as a service (IaaS): This model provides basic computing infrastructure, including virtual machines, storage, and networking resources over the internet. Users can install and manage their own operating systems, applications, and development frameworks while the provider handles the physical hardware.Platform as a service (PaaS): This service offers a complete development and use environment in the cloud, including operating systems, programming languages, databases, and web servers. Developers can build, test, and use applications without managing the underlying infrastructure complexity.Software as a service (SaaS): This model delivers fully functional software applications over the internet through a web browser or mobile app. Users access the software on a subscription basis without needing to install, maintain, or update the applications locally.Function as a service (FaaS): Also known as serverless computing, this model allows developers to run individual functions or pieces of code in response to events. The cloud provider automatically manages server provisioning, scaling, and maintenance while charging only for actual compute time used.Database as a service (DBaaS): This service provides managed database solutions in the cloud, handling database administration tasks like backups, updates, and scaling. Organizations can access database functionality without maintaining physical database servers or hiring specialized database administrators.Storage as a service (STaaS): This model offers flexible cloud storage solutions for data backup, archiving, and file sharing needs. Users can store and retrieve data from anywhere with internet access while paying only for the storage space they actually use.What are the different cloud deployment models?Cloud use models refer to the different ways organizations can access and manage cloud computing resources based on ownership, location, and access control. The cloud use models are listed below.Public cloud: Services are delivered over the internet and shared among multiple organizations by third-party providers. Anyone can purchase and use these services on a pay-as-you-go basis, making them cost-effective for businesses without large upfront investments.Private cloud: Computing resources are dedicated to a single organization and can be hosted on-premises or by a third party. This model offers greater control, security, and customization options but requires higher costs and more management overhead.Hybrid cloud: Organizations combine public and private cloud environments, allowing data and applications to move between them as needed. This approach provides flexibility to keep sensitive data in private clouds while using public clouds for less critical workloads.Community cloud: Multiple organizations with similar requirements share cloud infrastructure and costs. Government agencies, healthcare organizations, or financial institutions often use this model to meet specific compliance and security standards.Multi-cloud: Organizations use services from multiple cloud providers to avoid vendor lock-in and improve redundancy. This plan allows businesses to choose the best services from different providers while reducing dependency on any single vendor.How does cloud computing work?Cloud computing works by delivering computing resources like servers, storage, databases, and software over the internet on an on-demand basis. Instead of owning physical hardware, users access these resources through web browsers or applications, while cloud providers manage the underlying infrastructure in data centers worldwide.The system operates through a front-end and back-end architecture. The front end includes your device, web browser, and network connection that you use to access cloud services.The back end consists of servers, storage systems, databases, and applications housed in the provider's data centers. When you request a service, the cloud infrastructure automatically allocates the necessary resources from its shared pool.The technology achieves its flexibility through virtualization, which creates multiple virtual instances from single physical servers. Resource pooling allows providers to serve multiple customers from the same infrastructure, while rapid elasticity automatically scales resources up or down based on demand.This elastic scaling can reduce resource costs by up to 30% compared to fixed infrastructure, according to McKinsey (2024), making cloud computing both flexible and cost-effective for businesses of all sizes.What are the key benefits of cloud computing?The key benefits of cloud computing refer to the advantages organizations and individuals gain from using internet-based computing services instead of traditional on-premises infrastructure. The key benefits of cloud computing are listed below.Cost reduction: Organizations eliminate upfront hardware investments and reduce ongoing maintenance expenses by paying only for resources they actually use. Cloud providers handle infrastructure management, reducing IT staffing costs and operational overhead.Flexibility and elasticity: Computing resources can expand or contract automatically based on demand, ensuring best performance during traffic spikes. This flexibility prevents over-provisioning during quiet periods and under-provisioning during peak usage.Improved accessibility: Users can access applications and data from any device with an internet connection, enabling remote work and global collaboration. This mobility supports modern work patterns and increases productivity across distributed teams.Enhanced reliability: Cloud providers maintain multiple data centers with redundant systems and backup infrastructure to ensure continuous service availability.Automatic updates and maintenance: Software updates, security patches, and system maintenance happen automatically without user intervention. This automation reduces downtime and ensures systems stay current with the latest features and security protections.Disaster recovery: Cloud services include built-in backup and recovery capabilities that protect against data loss from hardware failures or natural disasters. Recovery times are typically faster than traditional backup methods since data exists across multiple locations.Environmental effectiveness: Shared cloud infrastructure uses resources more effectively than individual company data centers, reducing overall energy consumption. Large cloud providers can achieve better energy effectiveness through economies of scale and advanced cooling technologies.What are the drawbacks and challenges of cloud computing?The drawbacks and challenges of cloud computing refer to the potential problems and limitations organizations may face when adopting cloud-based services. They are listed below.Security concerns: Organizations lose direct control over their data when it's stored on third-party servers. Data breaches, unauthorized access, and compliance issues become shared responsibilities between the provider and customer. Sensitive information may be vulnerable to cyber attacks targeting cloud infrastructure.Internet dependency: Cloud services require stable internet connections to function properly. Poor connectivity or outages can completely disrupt business operations and prevent access to critical applications. Remote locations with limited bandwidth face particular challenges accessing cloud resources.Vendor lock-in: Switching between cloud providers can be difficult and expensive due to proprietary technologies and data formats. Organizations may become dependent on specific platforms, limiting their flexibility to negotiate pricing or change services. Migration costs and technical complexity often discourage switching providers.Limited customization: Cloud services offer standardized solutions that may not meet specific business requirements. Organizations can't modify underlying infrastructure or install custom software configurations. This restriction can force businesses to adapt their processes to fit the cloud platform's limitations.Ongoing costs: Monthly subscription fees can accumulate to exceed traditional on-premise infrastructure costs over time. Unexpected usage spikes or data transfer charges can lead to budget overruns. Organizations lose the asset value that comes with owning physical hardware.Performance variability: Shared cloud resources can experience slower performance during peak usage periods. Network latency affects applications requiring real-time processing or frequent data transfers. Organizations can't guarantee consistent performance levels for mission-critical applications.Compliance complexity: Meeting regulatory requirements becomes more challenging when data is stored across multiple locations. Organizations must verify that cloud providers meet industry-specific compliance standards. Audit trails and data governance become shared responsibilities that require careful coordination.Gcore Edge CloudWhen building AI applications that require serious computational power, the infrastructure you choose can make or break your project's success. Whether you're training large language models, running complex inference workloads, or tackling high-performance computing challenges, having access to the latest GPU technology without performance bottlenecks becomes critical.Gcore's AI GPU Cloud Infrastructure addresses these demanding requirements with bare metal NVIDIA H200. H100. A100. L40S, and GB200 GPUs, delivering zero virtualization overhead for maximum performance. The platform's ultra-fast InfiniBand networking and multi-GPU cluster support make it particularly well-suited for distributed training and large-scale AI workloads, starting from just €1.25/hour. Multi-instance GPU (MIG) support also allows you to improve resource allocation and costs for smaller inference tasks.Discover how Gcore's bare metal GPU performance can accelerate your AI training and inference workloads at https://gcore.com/gpu-cloud.Frequently asked questionsPeople often have questions about cloud computing basics, costs, and how it fits their specific needs. These answers cover the key service models, use options, and practical considerations that help clarify what cloud computing can do for your organization.What's the difference between cloud computing and traditional hosting?Cloud computing delivers resources over the internet on demand, while traditional hosting provides fixed server resources at dedicated locations. Cloud offers elastic growth and pay-as-you-go pricing, whereas traditional hosting requires upfront capacity planning and fixed costs regardless of actual usage.What is cloud computing security?Cloud computing security protects data, applications, and infrastructure in cloud environments through shared responsibility models between providers and users. Cloud providers secure the underlying infrastructure while users protect their data, applications, and access controls.What is virtualization in cloud computing?Virtualization in cloud computing creates multiple virtual machines (VMs) on a single physical server using hypervisor software that separates computing resources. This technology allows cloud providers to increase hardware effectiveness and offer flexible, isolated environments to multiple users simultaneously.Is cloud computing secure for business data?Yes, cloud computing is secure for business data when proper security measures are in place, with major providers offering encryption, access controls, and compliance certifications that often exceed what most businesses can achieve on-premises. Cloud service providers typically guarantee 99.9% or higher uptime in service level agreements while maintaining enterprise-grade security standards.How much does cloud computing cost compared to on-premises infrastructure?Cloud computing typically costs 20-40% less than on-premises infrastructure due to shared resources, reduced hardware purchases, and lower maintenance expenses, according to IDC (2024). However, costs vary primarily based on usage patterns, with predictable workloads sometimes being cheaper on-premises while variable workloads benefit more from cloud's pay-as-you-go model.How do I choose between IaaS, PaaS, and SaaS?Choose based on your control needs. IaaS gives you full infrastructure control, PaaS handles infrastructure so you focus on development, and SaaS provides ready-to-use applications with no technical management required.