AWS Announces 3 Serverless Innovations to Help Customers Analyze and Manage Data at Any Scale

LAS VEGAS, Nov. 28, 2023 — At AWS re:Invent, Amazon Web Services, Inc. (AWS) announced three new serverless innovations across its database and analytics portfolio that make it faster and easier for customers to scale their data infrastructure to support their most demanding use cases.

Credit: Michael-Vi/Shutterstock

Today’s announcement introduces Amazon Aurora Limitless Database, a new capability that automatically scales beyond the write limits of a single Amazon Aurora database, making it easy for developers to scale their applications and saving them months compared to building custom solutions. Additionally, Amazon ElastiCache Serverless helps customers create highly available caches in under a minute and instantly scales vertically and horizontally to support customers’ most demanding applications, without needing to manage the infrastructure.

AWS is also releasing a new Amazon Redshift Serverless capability that uses artificial intelligence (AI) to predict workloads and automatically scale and optimize resources to help customers meet their price-performance targets. These announcements build on AWS’s pioneering work with serverless technologies to help customers manage data at any scale and dramatically simplify their operations, so they can focus on innovating for their end users—without spending time and effort provisioning, managing, and scaling their data infrastructure. To learn more about unlocking the value of data using AWS, visit aws.amazon.com/data.

“Since its earliest days, AWS has focused on removing undifferentiated heavy lifting for customers, and we have continued to build on that legacy through serverless offerings that dramatically simplify what it takes to build, run, and manage applications at scale,” said Dr. Swami Sivasubramanian, vice president of Data and Artificial Intelligence at AWS. “Data is the cornerstone of every organization’s digital transformation, and harnessing data to its full potential requires an end-to-end strategy that can scale with a customer’s needs while accommodating all types of use cases. The dynamic nature of data makes it perfectly suited to serverless technologies, which is why AWS offers a broad range of serverless database and analytics offerings that help support our customers’ most demanding workloads. The new serverless innovations announced today build on this foundation to make it easier for customers to scale to millions of transactions per second, quickly add capacity at a moment’s notice, and dynamically adapt workload patterns to optimize for performance and cost.”

Organizations create and store petabytes of data from a growing number of sources. To get the most value out of this data, these companies need an end-to-end strategy that can help them analyze and manage the data at any scale. Many AWS customers are already using a wide variety of purpose-built data services to support their most critical applications and make data-driven decisions, including Amazon Aurora for relational databases, Amazon ElastiCache for running in-memory caches, and Amazon Redshift for data warehousing.

These services remove much of the heavy lifting that customers have to go through if they run their own database and analytics solutions, allowing them to focus on creating differentiated experiences for their end users. AWS continues to simplify operations for customers by releasing serverless technologies across its service portfolio, from some of AWS’s earliest offerings like Amazon Simple Storage Service (Amazon S3) to pioneering serverless, event-driven computing with AWS Lambda.

Today, AWS offers the broadest set of serverless data analytics offerings in the cloud, making it easy for customers to take advantage of benefits like automatic provisioning, on-demand scaling, and pay-for-use pricing while using the right tool for the job. The new innovations announced today further AWS’s commitment to reimagining its database and analytics portfolio through serverless technologies, by making it even easier for customers to optimize costs and maximize their data’s value.

Amazon Aurora Limitless Database powers petabyte-scale applications with millions of writes per second

Today, hundreds of thousands of customers use Amazon Aurora, a fully managed MySQL- and PostgreSQL-compatible relational database that provides the performance and availability of commercial databases at up to one-tenth the cost. These organizations rely on Amazon Aurora Serverless v2 to power their applications because it is capable of scaling to support hundreds of thousands of transactions in a fraction of a second. As it scales, it adjusts capacity up and down in fine-grained increments to provide the right amount of database resources for the application.

However, there are some use cases, such as online gaming and financial transaction processing, with workloads that need to process and manage hundreds of millions of global users, handle millions of transactions, and store petabytes of data. Today, these organizations must scale horizontally by splitting data into smaller subsets and distributing them across multiple distinct database instances in a process known as “sharding,” which requires months—or even years—of upfront developer effort to build custom software that routes requests to the correct instance or makes changes across multiple instances.

Organizations also need to continuously monitor database activity and adjust capacity, which can be time-consuming and impact availability. The ongoing maintenance effort for these workloads is high, as organizations need to coordinate routine maintenance operations—such as adding a column to a table, taking consistent backups across all compute instances, or applying upgrades and patches—and constantly tune and balance the load across multiple instances. As a result, organizations need ways to automatically scale their applications beyond the limits of a single database without spending time building their scaling solutions.

Amazon Aurora Limitless Database scales to millions of write transactions per second and manages petabytes of data while maintaining the simplicity of operating inside a single database. Amazon Aurora Limitless Database automatically distributes data and queries across multiple Amazon Aurora Serverless instances based on a customer’s data model, eliminating the need to build custom software to route requests across instances.

As compute or storage requirements increase, Amazon Aurora Limitless Database automatically scales resources vertically within serverless instances and horizontally across instances to meet workload demand, providing customers with consistently high performance while saving them months or years of effort in building custom software to scale their databases. Maintenance operations and changes can be made in a single database and automatically applied across instances, eliminating the need for managing routine tasks across dozens, or even hundreds, of database instances manually.

Amazon ElastiCache Serverless makes it faster and easier to create a cache and instantly scale to meet application demand—without needing to provision, plan for, or manage capacity

Organizations building applications store frequently accessed data in caches to improve application response times and reduce database costs. These customers use open source, in-memory data stores like Redis and Memcached for caching because of their high performance and scalability. To simplify the process of building and running a cache, AWS offers Amazon ElastiCache, a fully managed Redis- and Memcached-compatible service that is used by hundreds of thousands of customers today for real-time, cost-optimized performance.

Today, Amazon ElastiCache scales to hundreds of terabytes of data and hundreds of millions of operations per second with microsecond response times, and organizations use it to deploy highly available, mission-critical applications across multiple Availability Zones. While many organizations appreciate the fine-grained configuration options Amazon ElastiCache offers, some companies building a new application or migrating existing workloads want to get started quickly without designing and provisioning cache infrastructure, a process that requires specialized expertise and deep familiarity with application traffic patterns.

Organizations also need to constantly monitor and scale their capacity to maintain consistently high performance, or overprovision for peak capacity, which results in excess costs. As a result, they need a solution that can help them manage the underlying infrastructure, making it faster and easier to create and operate a cache.

With Amazon ElastiCache Serverless, customers can now create a highly available cache in under a minute without infrastructure provisioning or configuration. Amazon ElastiCache Serverless eliminates the complex, time-consuming process of capacity planning by continuously monitoring a cache’s compute, memory, and network utilization and instantly scaling vertically and horizontally to meet demand without downtime or performance degradation.

With Amazon ElastiCache Serverless, customers no longer need to rightsize or fine-tune their caches. Amazon ElastiCache Serverless automatically replicates data across multiple Availability Zones and provides customers with 99.99% availability for all workloads. Customers only pay for the data they store and the compute their application uses. Amazon ElastiCache Serverless is generally available today for both Redis- and Memcached-compatible deployment options. To get started, visit aws.amazon.com/elasticache/features/#Serverless.

Next-generation, AI-driven scaling and optimizations in Amazon Redshift Serverless deliver better price-performance for variable workloads

Tens of thousands of customers collectively process exabytes of data with Amazon Redshift every day. Many of these customers rely on Amazon Redshift Serverless, which automatically provisions and scales data warehouse capacity to meet demand based on the number of concurrent queries. While customers enjoy the ease of running analytics workloads of all sizes on Amazon Redshift Serverless without needing to manage data warehouse infrastructure, they would benefit further from the ability to easily adapt to changes in their workloads along additional dimensions, such as the amount of data or query complexity, to achieve consistently high performance while optimizing cost.

With the new AI-driven scaling and optimizations, Amazon Redshift Serverless automatically scales resources up and down across multiple workload dimensions and performs optimizations to meet price-performance targets. Amazon Redshift Serverless uses AI to learn customer workload patterns along dimensions such as query complexity, data size, and frequency and continuously adjusts capacity based on those dynamic patterns to meet customer-specified, price-performance targets. Amazon Redshift Serverless now also proactively adjusts resources based on those customer workload patterns. Customers can set their own price-performance targets in the AWS Console, choosing to optimize between cost and performance. Amazon Redshift Serverless with AI-driven scaling and optimizations is available in preview. To learn more, visit aws.amazon.com/redshift/redshift-serverless.

About Amazon Web Services

Since 2006, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud. AWS has been continually expanding its services to support virtually any workload, and it now has more than 240 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, virtual and augmented reality (VR and AR), media, and application development, deployment, and management from 102 Availability Zones within 32 geographic regions, with announced plans for 15 more Availability Zones and five more AWS Regions in Canada, Germany, Malaysia, New Zealand, and Thailand. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—trust AWS to power their infrastructure, become more agile, and lower costs.

Source: AWS