Amazon Neptune Engine Version 1.1.1.0 (2022-04-19) - Amazon Neptune

Amazon Neptune Engine Version 1.1.1.0 (2022-04-19)

As of 2022-04-19, engine version 1.1.1.0 is being generally deployed. Please note that it takes several days for a new release to become available in every region.

Important

Upgrading to this engine release from a version earlier than 1.1.0.0 also triggers an operating-system upgrade on all the instances in your DB cluster. Because active write requests that occur during the operating-system upgrade will not be processed, you must pause all write workloads to the cluster being upgraded, including bulk data loads, before starting the upgrade.

In order to complete the upgrade successfully, each subnet in every availability zone (AZ) must have at least one IP address available per Neptune instance. For example, if there is one writer instance and two reader instances in subnet 1, and two reader instances in subnet 2, subnet 1 must have at least 3 IP addresses free and subnet 2 must have at least 2 IP addresses free before starting the upgrade.

At the start of the upgrade, Neptune generates a snapshot with a name composed of preupgrade followed by an autogenerated identifier based on your DB cluster information. You will not be charged for this snapshot, and you can use it to restore your DB cluster if anything goes wrong during the upgrade process.

When the engine upgrade itself has completed, the new engine version will be available briefly on the old operating system, but in less than 5 minutes all the the instances in your cluster will simultaneously begin an operating-system upgrade. Your DB cluster will be unavailable at this point for a number of minutes. You may resume write workloads after the upgrade completes.

This process generates the following events:

  • Per-cluster event messages:

    • Upgrade in progress: Creating pre-upgrade snapshot [preupgrade-(autogenerated snapshot ID)]

    • Database cluster major version has been upgraded

  • Per-instance event messages:

    • Applying off-line patches to DB instance

    • DB instance shutdown

    • Finished applying off-line patches to DB instance

    • DB instance restarted

Subsequent Patch Releases for This Release

New Features in This Engine Release

  • The openCypher query language is now generally available for production use.

    Warning

    There is a breaking change in this release for code that uses openCypher with IAM authentication. In the Neptune preview for openCypher, the host string in the IAM signature included the protocol, such as bolt://, like this:

    "Host":"bolt://(host URL):(port)"

    Starting with this engine release, the protocol must be omitted:

    "Host":"(host URL):(port)"

    See Using the Bolt protocol for examples.

  • Added support for TinkerPop 3.5.2. Among the changes in this release are support for remote transactions and bytecode support for sessions (using g.tx), and the addition of the datetime() function to the Gremlin language.

    Warning

    There are several breaking changes introduced in TinkerPop 3.5.0, 3.5.1, and 3.5.2 which may affect your Gremlin code. For example, using traversals spawned by a GraphTraversalSource as children like this will no longer work: g.V().union(identity(), g.V()).

    Now instead, use an anonymous traversal like this: g.V().union(identity(), __.V()).

  • Added support for AWS global condition keys that you can use in IAM data-access policies that control access to data stored in Neptune a Neptune DB cluster.

  • The Neptune DFE query engine is now generally available for production use with the openCypher query language, but not yet for Gremlin and SPARQL queries. You now enable it using its own neptune_dfe_query_engine instance parameter rather than the lab-mode parameter.

Improvements in This Engine Release

  • Added new features to openCypher such as parameterized query support, abstract syntax tree (AST) caching for parameterized queries, variable length path (VLP) improvements, and new operators and clauses. See openCypher specification compliance in Amazon Neptune for the current level of language support.

  • Made significant performance improvements to openCypher for simple read and write workloads, resulting in higher throughput when compared to Release 1.1.0.0.

  • Removed openCypher bi-directional and depth limitations handling variable-length paths.

  • Completed support in the DFE engine for Gremlin within and without predicates, including cases where they are combined with other predicate operators. For example:

    g.V().has('age', within(12, 15, 18).or(gt(30)))
  • Extended support in the DFE engine for the Gremlin order step when the scope is global (that is, not order(local)), and when by() modulators are not used. For example, this query would now have DFE support:

    g.V().values("age").order()
  • Added an isLastOp field to the Neptune streams change-log response format, to indicate that a record is the last operation in its transaction.

  • Significantly improved the performance of audit logging and reduced latency when audit logging is enabled.

  • Converted Gremlin WebSocket bytecode and HTTP queries into a user-readable format in audit logs. Queries can now be directly copied from the audit logs to be executed in Neptune notebooks and elsewhere. Note that this change to the current audit log format constitutes a breaking change.

Defects Fixed in This Engine Release

  • Fixed a rare Gremlin bug where no results were returned when using nested filter() and count() steps in combination, such as in the following query:

    g.V("1").filter(out("knows") .filter(in("knows") .hasId("notExists")) .count())
  • Fixed a Gremlin bug where an error was returned when using a vertex stored by an aggregate step in either to() or from() traversals in conjunction with an addE step. An example of such a query is:

    g.V("id").aggregate("v").out().addE().to(select("v").unfold()))
  • Fixed a Gremlin bug where the not step was failing in edge cases when using the DFE engine. For example:

    g.V().not(V())
  • Fixed a Gremlin bug where sideEffect values were not available within to() and from() traversals.

  • Fixed a bug that occasionally caused a fast reset to trigger an instance failover.

  • Fixed a bulk loader bug where a failed transaction would not be closed before beginning the next load job.

  • Fixed a bulk loader bug where a low memory condition could cause a crash of the system.

  • Added a retry to fix a bulk loader bug where the loader did not wait long enough for IAM credentials to become availble after a failover.

  • Fixed a bug where the internal credential cache was not being cleared properly for non-query endpoints such as the status endpoint.

  • Fixed a streams bug to ensure that stream commit sequence numbers are properly ordered.

  • Fixed a bug where long-running connections were terminated sooner than ten days on IAM-enabled clusters.

Query-Language Versions Supported in This Release

Before upgrading a DB cluster to version 1.1.1.0, make sure that your project is compatible with these query-language versions:

  • Gremlin earliest version supported: 3.5.2

  • Gremlin latest version supported: 3.5.4

  • openCypher version: Neptune-9.0.20190305-1.0

  • SPARQL version: 1.1

Upgrade Paths to Engine Release 1.1.1.0

You can manually upgrade any previous Neptune engine release to this release. Note that versions prior to the major version engine (1.1.0.0) will take longer to upgrade to this release.

You will not be automatically upgraded to this release.

Upgrading to This Release

Important

Upgrading to this engine release from any version prior to 1.1.0.0 also triggers an operating-system upgrade on all the instances in your DB cluster. Because active write requests that occur during the operating-system upgrade will not be processed, you must pause all write workloads to the cluster being upgraded, including bulk data loads, before starting the upgrade.

At the start of the upgrade, Neptune generates a snapshot with a name composed of preupgrade followed by an autogenerated identifier based on your DB cluster information. You will not be charged for this snapshot, and you can use it to restore your DB cluster if anything goes wrong during the upgrade process.

When the engine upgrade itself has completed, the new engine version will be available briefly on the old operating system, but in less than 5 minutes all the the instances in your cluster will simultaneously begin an operating-system upgrade. Your DB cluster will be unavailable at this point for around 6 minutes. You may resume write workloads after the upgrade completes.

This process generates the following events:

  • Per-cluster event messages:

    • Upgrade in progress: Creating pre-upgrade snapshot [preupgrade-(autogenerated snapshot ID)]

    • Database cluster major version has been upgraded

  • Per-instance event messages:

    • Applying off-line patches to DB instance

    • DB instance shutdown

    • Finished applying off-line patches to DB instance

    • DB instance restarted

If a DB cluster is running an engine version from which there is an upgrade path to this release, it is eligible to be upgraded now. You can upgrade any eligible cluster using the DB cluster operations on the console or by using the SDK. The following CLI command will upgrade an eligible cluster immediately:

For Linux, OS X, or Unix:

aws neptune modify-db-cluster \ --db-cluster-identifier (your-neptune-cluster) \ --engine neptune \ --engine-version 1.1.1.0 \ --allow-major-version-upgrade \ --apply-immediately

For Windows:

aws neptune modify-db-cluster ^ --db-cluster-identifier (your-neptune-cluster) ^ --engine neptune ^ --engine-version 1.1.1.0 ^ --allow-major-version-upgrade ^ --apply-immediately

Instead of --apply-immediately, you can specify --no-apply-immediately. To perform a major version upgrade, the allow-major-version-upgrade parameter is required. Also, be sure to include the engine version or your engine may be upgraded to a different version.

If your cluster uses a custom cluster parameter group, be sure to include this paramater to specify it:

--db-cluster-parameter-group-name (name of the custom DB cluster parameter group)

Similarly, if any instances in the cluster use a custom DB parameter group, be sure to include this parameter to specify it:

--db-instance-parameter-group-name (name of the custom instance parameter group)

Always test before you upgrade

When a new major or minor Neptune engine version is released, always test your Neptune applications on it first before upgrading to it. Even a minor upgrade could introduce new features or behavior that would affect your code.

Start by comparing the release notes pages from your current version to those of the targeted version to see if there will be changes in query language versions or other breaking changes.

The best way to test a new version before upgrading your production DB cluster is to clone your production cluster so that the clone is running the new engine version. You can then run queries on the clone without affecting the production DB cluster.

Always create a manual snapshot before you upgrade

Before performing an upgrade, we strongly recommend that you always create a manual snapshot of your DB cluster. Having an automatic snapshot only offers short-term protection, whereas a manual snapshot remains available until you explicitly delete it.

In certain cases Neptune creates a manual snapshot for you as a part of the upgrade process, but you should not rely on this, and should create your own manual snapshot in any case.

When you are certain that you won't need to revert your DB cluster to its pre-upgrade state, you can explicitly delete the manual snapshot that you created yourself, as well as the manual snapshot that Neptune might have created. If Neptune creates a manual snapshot, it will have a name that begins with preupgrade, followed by the name of your DB cluster, the source engine version, the target engine version, and the date.

Note

If you are trying to upgrade while a pending action is in process, you may encounter an error such as the following:

We're sorry, your request to modify DB cluster (cluster identifier) has failed. Cannot modify engine version because instance (instance identifier) is running on an old configuration. Apply any pending maintenance actions on the instance before proceeding with the upgrade.

If you encounter this error, wait for the pending action to finish, or trigger a maintenance window immediately to let a previous upgrade complete.

For more information about upgrading your engine version, see Maintaining your Amazon Neptune DB Cluster. If you have any questions or concerns, the AWS Support team is available on the community forums and through AWS Premium Support.