How Aurora Serverless v1 works - Amazon Aurora

How Aurora Serverless v1 works

Amazon Aurora offers two different DB engine modes aimed at two broadly different usage models.

The provisioned DB engine mode is designed for predictable workloads. When you work with Aurora provisioned DB clusters, you choose your DB instance class size and several other configuration options. For example, you can create one or more Aurora Replicas to increase read throughput. If your workload changes, you can modify the DB instance class size and change the number of Aurora Replicas. The provisioned model works well when you can adjust capacity in advance of expected consumption patterns.

The serverless DB engine mode is designed for a different usage pattern entirely. For example, your database usage might be heavy for a short period of time, followed by long periods of light activity or no activity at all. Some examples are retail websites with intermittent sales events, databases that produce reports when needed, development and testing environments, and new applications with uncertain requirements. For cases such as these and many others, configuring capacity correctly in advance isn't always possible with the provisioned model. It can also result in higher costs if you overprovision and have capacity that you don't use.

By using Aurora Serverless v1, you can create a database endpoint without specifying the DB instance class size. You specify only the minimum and maximum range for the Aurora Serverless v1 DB cluster's capacity. The Aurora Serverless v1 database endpoint makes up a router fleet that supports continuous connections and distributes the workload among resources. Aurora Serverless v1 scales the resources automatically based on your minimum and maximum capacity specifications.

You don't need to change your database client application code to use the router fleet. Aurora Serverless v1 manages the connections automatically. Scaling is fast thanks to a "warm" resources pool that's always ready to service requests. Storage and processing are separate, so your Aurora Serverless v1 DB cluster can scale down to zero when it's finished processing workloads. When your Aurora Serverless v1 DB cluster scales to zero, you're charged only for storage.

Aurora Serverless v1 architecture

The following image shows an overview the Aurora Serverless v1 architecture.


                  Aurora Serverless v1 Architecture

Instead of provisioning and managing database servers, you specify Aurora capacity units (ACUs). Each ACU is a combination of approximately 2 gigabytes (GB) of memory, corresponding CPU, and networking. Database storage automatically scales from 10 gibibytes (GiB) to 128 tebibytes (TiB), the same as storage in a standard Aurora DB cluster.

You can specify the minimum and maximum ACU. The minimum Aurora capacity unit is the lowest ACU to which the DB cluster can scale down. The maximum Aurora capacity unit is the highest ACU to which the DB cluster can scale up. Based on your settings, Aurora Serverless v1 automatically creates scaling rules for thresholds for CPU utilization, connections, and available memory.

Aurora Serverless v1 manages the warm pool of resources in an AWS Region to minimize scaling time. When Aurora Serverless v1 adds new resources to the Aurora DB cluster, it uses the router fleet to switch active client connections to the new resources. At any specific time, you are only charged for the ACUs that are being actively used in your Aurora DB cluster.

Autoscaling for Aurora Serverless v1

The capacity allocated to your Aurora Serverless v1 DB cluster seamlessly scales up and down based on the load generated by your client application. Here, load is CPU utilization and the number of connections. When capacity is constrained by either of these, Aurora Serverless v1 scales up. Aurora Serverless also scales up when it detects performance issues that can be resolved by doing so.

You can view scaling events for your Aurora Serverless cluster in the AWS Management Console. During autoscaling, Aurora Serverless v1 resets the EngineUptime metric. The value of the reset metric value doesn't mean that seamless scaling had problems, nor does it mean Aurora Serverless dropped connections. It's simply the starting point for uptime at the new capacity. To learn more about metrics, see Monitoring Amazon Aurora metrics with Amazon CloudWatch.

When your Aurora Serverless v1 DB cluster has no active connections, it can scale down to zero capacity (0 ACUs). To learn more, see Pause and resume for Aurora Serverless v1.

When it does need to perform a scaling operation, Aurora Serverless v1 first tries to identify a scaling point, a moment when no queries are being processed. Aurora Serverless might not be able to find a scaling point for the following reasons:

  • Long-running queries

  • In-progress transactions

  • Temporary tables or table locks

To increase your Aurora Serverless DB cluster's success rate when finding a scaling point, we recommend that you avoid long-running queries and long-running transactions. To learn more about scale-blocking operations and how to avoid them, see Best practices for working with Amazon Aurora Serverless.

Currently, Aurora Serverless v1 tries to find a scaling point for 5 minutes (300 seconds). If Aurora Serverless can't find a scaling point within 5 minutes, the autoscaling operation times out.

By default, if autoscaling doesn't find a scaling point before timing out, Aurora Serverless v1 keeps the cluster at the current capacity. This default behavior can be changed when you create or modify your Aurora Serverless DB cluster by selecting the "Force capacity change..." option. For more information, see Timeout action for capacity changes.

Timeout action for capacity changes

If autoscaling times out with finding a scaling point, by default Aurora keeps the current capacity. You can choose to have Aurora force the change by enabling the Force the capacity change option. This option is available in the Capacity settings section of the Create database page, when you create the cluster.

  • [ ] Force the capacity change – By default, this option is deselected. Leave this option unchecked to have your Aurora Serverless DB cluster's capacity to remain unchanged if the scaling operation times out without finding a scaling point.

  • [X] Force the capacity change – Choosing this option causes your Aurora Serverless DB cluster to enforce the capacity change, even without a scaling point. Before enabling this option, be aware of the consequences of this choice.

    • Any in-process transactions are interrupted, and the following error message appears.

      Aurora MySQL 5.6ERROR 1105 (HY000): The last transaction was aborted due to an unknown error. Please retry.

      Aurora MySQL 5.7ERROR 1105 (HY000): The last transaction was aborted due to Seamless Scaling. Please retry.

      You can resubmit the transactions as soon as your Aurora Serverless v1 DB cluster is available.

    • Connections to temporary tables and locks are dropped.

Note

We recommend that you choose the "force" option only if your application can recover from dropped connections or incomplete transactions.

The choices you make in the AWS Management Console when you create an Aurora Serverless DB cluster are stored in the ScalingConfigurationInfo object, in the TimeoutAction property. The value of the TimeoutAction property is set to one of the following values when you create your cluster:

  • RollbackCapacityChange – This is the default behavior. It's set by leaving the "Force" option deselected.

  • ForceApplyCapacityChange – This value is set when you choose the "Force" option.

You can get the value of this property on an existing Aurora Serverless DB cluster by using the describe-db-clusters AWS CLI command, as shown following.

For Linux, macOS, or Unix:

aws rds describe-db-clusters --region region \ --db-cluster-identifier your-cluster-name \ --query '*[].{ScalingConfigurationInfo:ScalingConfigurationInfo}'

For Windows:

aws rds describe-db-clusters --region region ^ --db-cluster-identifier your-cluster-name ^ --query "*[].{ScalingConfigurationInfo:ScalingConfigurationInfo}"

As an example, the following shows the query and response for an Aurora Serverless v1 DB cluster named "west-coast-sles" in the US West (N. California) Region.

$ aws rds describe-db-clusters --region us-west-1 --db-cluster-identifier west-coast-sles --query '*[].{ScalingConfigurationInfo:ScalingConfigurationInfo}' [ { "ScalingConfigurationInfo": { "MinCapacity": 1, "MaxCapacity": 64, "AutoPause": false, "SecondsUntilAutoPause": 300, "TimeoutAction": "RollbackCapacityChange" } } ]

As the response shows, this Aurora Serverless v1 DB cluster uses the default setting.

For more information, see Creating an Aurora Serverless v1 DB cluster. After creating your Aurora Serverless v1, you can modify the timeout action and other capacity settings at any time. To learn how, see Modifying an Aurora Serverless v1 DB cluster.

Pause and resume for Aurora Serverless v1

You can choose to pause your Aurora Serverless v1 DB cluster after a given amount of time with no activity. You specify the amount of time with no activity before the DB cluster is paused. When you select this option, the default inactivity time is five minutes, but you can change this value. This is an optional setting.

When the DB cluster is paused, no compute or memory activity occurs, and you are charged only for storage. If database connections are requested when an Aurora Serverless DB cluster is paused, the DB cluster automatically resumes and services the connection requests.

When the DB cluster resumes activity, it has the same capacity as it had when Aurora paused the cluster. The number of ACUs depends on how much Aurora scaled the cluster up or down before pausing it.

Note

If a DB cluster is paused for more than seven days, the DB cluster might be backed up with a snapshot. In this case, Aurora restores the DB cluster from the snapshot when there is a request to connect to it.

Parameter groups and Aurora Serverless v1

When you create your Aurora Serverless v1 DB cluster, you choose a specific Aurora DB engine and an associated DB cluster parameter group. Unlike provisioned Aurora DB clusters, an Aurora Serverless DB cluster has a single read/write DB instance that's configured with a DB cluster parameter group only—it doesn't have a separate DB parameter group. During autoscaling, Aurora Serverless needs to be able to change parameters for the cluster to work best for the increased or decreased capacity. Thus, with an Aurora Serverless DB cluster, some of the changes you might make to parameters for a particular DB engine type might not apply.

For example, an Aurora PostgreSQL–based Aurora Serverless DB cluster can't use apg_plan_mgmt.capture_plan_baselines and other parameters that might be used on provisioned Aurora PostgreSQL DB clusters for query plan management.

You can get a list of default values for the default parameter groups for the various Aurora DB engines by using the describe-engine-default-cluster-parameters CLI command and querying the AWS Region. The following are values you can use for the --db-parameter-group-family option.

Aurora MySQL 5.6

aurora5.6

Aurora MySQL 5.7

aurora-mysql5.7

Aurora PostgreSQL 10.12 (and later)

aurora-postgresql10

We recommend that you configure your AWS CLI with your AWS Access Key ID and AWS Secret Access Key, and that you set your AWS Region before using AWS CLI commands. Providing the Region to your CLI configuration saves you from entering the --region parameter when running commands. To learn more about configuring AWS CLI, see Configuration basics in the AWS Command Line Interface User Guide.

The following example gets a list of parameters from the default DB cluster group for Aurora MySQL 5.6.

For Linux, macOS, or Unix:

aws rds describe-engine-default-cluster-parameters \ --db-parameter-group-family aurora5.6 --query \ 'EngineDefaults.Parameters[*].{ParameterName:ParameterName,SupportedEngineModes:SupportedEngineModes} | [?contains(SupportedEngineModes, `serverless`) == `true`] | [*].{param:ParameterName}' \ --output text

For Windows:

aws rds describe-engine-default-cluster-parameters ^ --db-parameter-group-family aurora5.6 --query ^ "EngineDefaults.Parameters[*].{ParameterName:ParameterName,SupportedEngineModes:SupportedEngineModes} | [?contains(SupportedEngineModes, 'serverless') == `true`] | [*].{param:ParameterName}" ^ --output text

Modifying parameter values for Aurora Serverless v1

As explained in Working with DB parameter groups and DB cluster parameter groups, you can't directly change values in a default parameter group, regardless of its type (DB cluster parameter group, DB parameter group). Instead, you create a custom parameter group based on the default DB cluster parameter group for your Aurora DB engine and change settings as needed on that parameter group. For example, you might want to change some of the settings for your Aurora Serverless DB cluster to log queries or to upload DB engine specific logs to Amazon CloudWatch.

To create a custom DB cluster parameter group

  1. Sign in to the AWS Management Console and then open the Amazon RDS console at https://console.aws.amazon.com/rds/.

  2. Choose Parameter groups.

  3. Choose Create parameter group to open the Parameter group details pane.

  4. Choose the appropriate default DB cluster group for the DB engine you want to use for your Aurora Serverless v1 DB cluster. Be sure you choose the following options:

    1. For Parameter group family, choose the appropriate family for your chosen DB engine. Be sure your selection has the prefix aurora- in its name.

    2. For Type, choose DB Cluster Parameter Group.

    3. For Group name and Description, enter meaningful names for you or others who might need to work with your Aurora Serverless v1 DB cluster and its parameters.

    4. Choose Create.

Your custom DB cluster parameter group is added to the list of parameter groups available in your AWS Region. You can use your custom DB cluster parameter group when you create new Aurora Serverless DB clusters, and you can modify an existing Aurora Serverless DB cluster to use your custom DB cluster parameter group. Once your Aurora Serverless DB cluster starts using your custom DB cluster parameter group, you can change values for dynamic parameters using either the AWS Management Console or the AWS CLI. You can also use the Console to view a side-by-side comparison of the values in your custom DB cluster parameter group compared to the default DB cluster parameter group, as shown in the following screenshot.


          Logs published to CloudWatch Logs for Aurora MySQL and Aurora PostgreSQL Aurora Serverless v1 DB clusters

When you change parameter values on an active DB cluster, Aurora Serverless starts a seamless scale in order to apply the parameter changes. If your Aurora Serverless DB cluster is in a "paused" state, it resumes and starts scaling so that it can make the change. The scaling operation for a parameter group change always forces the capacity change, so be aware that modifying parameters might result in dropped connections if a scaling point can't be found during the scaling period.

Logging for Aurora Serverless v1

By default, error logs for Aurora Serverless are enabled and automatically uploaded to Amazon CloudWatch. You can also have your Aurora Serverless DB cluster upload Aurora database-engine specific logs to CloudWatch by enabling configuration parameters in your custom DB cluster parameter group. Your Aurora Serverless DB cluster then uploads all available logs to Amazon CloudWatch, and you can use CloudWatch to analyze log data, create alarms, and view metrics.

For Aurora MySQL, you can enable the following logs have them automatically uploaded from your Aurora Serverless DB cluster to Amazon CloudWatch.

Aurora MySQL Description

general_log

Creates the general log. Set to 1 to turn on. Default is off (0).

log_queries_not_using_indexes

Logs any queries to the slow query log that don't use an index. Default is off (0). Set to 1 to turn on this log.

long_query_time

Prevents fast-running queries from being logged in the slow query log. Can be set to a float between 0 and 31536000. Default is 0 (not active).

server_audit_events

The list of events to capture in the logs. Supported values are CONNECT, QUERY, QUERY_DCL, QUERY_DDL, QUERY_DML, and TABLE.

server_audit_logging

Set to 1 to turn on server audit logging. If you turn this on, you can specify the audit events to send to CloudWatch by listing them in the server_audit_events parameter.

slow_query_log

Creates a slow query log. Set to 1 to turn on the slow query log. Default is off (0).

For more information, see Using advanced auditing with an Amazon Aurora MySQL DB cluster.

For Aurora PostgreSQL, you can enable the following logs on your Aurora Serverless DB cluster and have them automatically uploaded to Amazon CloudWatch along with the regular error logs.

Aurora PostgreSQL Description

log_connections

Enabled by default, and can't be changed. It logs details for all new client connections.

log_disconnections

Enabled by default, and can't be changed. Logs all client disconnections.

log_lock_waits

Default is 0 (off). Set to 1 to log lock waits.

log_min_duration_statement

The minimum duration (in milliseconds) for a statement to run before it's logged.

log_min_messages

Sets the message levels that are logged. Supported values are debug5, debug4, debug3, debug2, debug1, info, notice, warning, error, log, fatal, panic. To log performance data to the postgres log, set the value to debug1.

log_temp_files

Logs the use of temporary files that are above the specified kilobytes (kB).

log_statement

Controls the specific SQL statements that get logged. Supported values are none, ddl, mod, and all. Default is none.

After you enable logs for Aurora MySQL 5.6, Aurora MySQL 5.7, or Aurora PostgreSQL for your Aurora Serverless DB cluster, you can view the logs in CloudWatch.

Viewing Aurora Serverless v1 logs with Amazon CloudWatch

Aurora Serverless v1 automatically uploads ("publishes") to Amazon CloudWatch all logs that are enabled in your custom DB cluster parameter group. You don't need to choose or specify the log types. Uploading logs starts as soon as you enable the log configuration parameter. If you later disable the log parameter, further uploads stop. However, all the logs that have already been published to CloudWatch remain until you delete them.

For more information on using CloudWatch with Aurora MySQL logs, see Monitoring log events in Amazon CloudWatch.

For more information about CloudWatch and Aurora PostgreSQL, see Publishing Aurora PostgreSQL logs to Amazon CloudWatch Logs.

To view logs for your Aurora Serverless DB cluster

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose your AWS Region.

  3. Choose Log groups.

  4. Choose your Aurora Serverless DB cluster log from the list. For error logs, the naming pattern is as follows.

    /aws/rds/cluster/cluster-name/error

For example, in the following screenshot you can find listings for logs published for an Aurora PostgreSQL Aurora Serverless DB cluster named "western-sles." You can also find several listings for Aurora MySQL Aurora Serverless DB cluster, "west-coast-sles." Choose the log of interest to start exploring its content.


                    Logs published to CloudWatch Logs for Aurora MySQL and Aurora PostgreSQL Aurora Serverless v1 DB clusters

Aurora Serverless v1 and maintenance

Maintenance for Aurora Serverless v1 DB cluster, such as applying the latest features, fixes, and security updates, is performed automatically for you. Unlike provisioned Aurora DB clusters, Aurora Serverless doesn't have user-settable maintenance windows. However, it does have a maintenance window that you can view in the AWS Management Console in Maintenance & backups for your Aurora Serverless DB cluster. You can find the date and time that maintenance might be performed and if any maintenance is pending for your Aurora Serverless DB cluster, as shown following.


              Maintenance window for an example Aurora Serverless DB cluster, not user settable, no pending maintenance

Whenever possible, Aurora Serverless performs maintenance in a non-disruptive manner. When maintenance is required, your Aurora Serverless DB cluster scales its capacity to handle the necessary operations. Before scaling, Aurora Serverless looks for a scaling point and it does so for up to seven days if necessary.

At the end of each day that Aurora Serverless can't find a scaling point, it creates a cluster event. This event notifies you of the pending maintenance and the need to scale to perform maintenance. The notification includes the date when the Aurora Serverless can force the DB cluster to scale.

Until that time, your Aurora Serverless DB cluster continues looking for a scaling point and behaves according to its TimeoutAction setting. That is, if it can't find a scaling point before timing out, it abandons the capacity change if it's configured to RollbackCapacityChange. Or it forces the change if it's set to ForceApplyCapacityChange. As with any change that's forced without an appropriate scaling point, this might interrupt your workload.

For more information, see Timeout action for capacity changes.

Aurora Serverless v1 and failover

If the DB instance for an Aurora Serverless v1 DB cluster becomes unavailable or the Availability Zone (AZ) it is in fails, Aurora recreates the DB instance in a different AZ. We refer to this capability as automatic multi-AZ failover.

This failover mechanism takes longer than for an Aurora provisioned cluster. The Aurora Serverless v1 failover time is currently undefined because it depends on demand and capacity availability in other AZs within the given AWS Region.

Because Aurora separates computation capacity and storage, the storage volume for the cluster is spread across multiple AZs. Your data remains available even if outages affect the DB instance or the associated AZ.

Aurora Serverless v1 and snapshots

The cluster volume for an Aurora Serverless v1 cluster is always encrypted. You can choose the encryption key, but you can't disable encryption. To copy or share a snapshot of an Aurora Serverless v1 cluster, you encrypt the snapshot using your own AWS Key Management Service customer master key (CMK). For more information, see Copying a DB cluster snapshot . To learn more about encryption and Amazon Aurora, see Encrypting Amazon Aurora resources.