Parameter groups for DB clusters in Timestream for InfluxDB 3

Database parameters specify how the database is configured. You manage your database configuration by associating your DB instances with parameter groups.

Amazon Timestream for InfluxDB defines parameter groups with default settings. You can also define your own parameter groups with customized settings. Parameter groups for Core and Enterprise editions while similar are not identical or interchangeable.

For InfluxDB 3, cluster configurations are managed through parameter groups. These parameter groups contain engine configuration values that determine how your InfluxDB 3 cluster operates.

Parameter group characteristics

• Immutability: Parameter groups are immutable once created. You cannot modify the parameters after creation.

• Cloning: You can create a new parameter group by cloning an existing one and modifying the parameters during the cloning process. This is a AWS Management Console only feature.

• Deletion: Parameter groups cannot be deleted.

• Default Groups: Timestream for InfluxDB provides default parameter groups with pre-configured settings optimized for common use cases.

Key parameters for cluster configuration

Parameter groups for InfluxDB 3 allow you to configure various aspects of your cluster, including:

• Engine Types (InfluxDBV2, InfluxDBV3Core, InfluxDBV3Enterprise)

• InfluxDB 3 Enterprise Parameters

• Service-Defined Parameter Groups

• Query concurrency limits

• Memory allocation for queries

• Storage cache settings

• HTTP timeout configurations

• Log levels and verbosity

• Compaction settings (Enterprise version)

When creating a new cluster, you must associate it with a parameter group. This association determines the operational characteristics of your cluster.

Common Parameters (Available in both Core and Enterprise)

Parameter	Description	Range/Format
Query Configuration
queryFileLimit	Maximum number of query files	0-1024
queryLogSize	Size of query log	1-10000
logFilter	Log filtering configuration	Max 1024 characters
logFormat	Format for logs	"full"
DataFusion Engine
dataFusionNumThreads	Number of DataFusion threads	1-2048
dataFusionRuntimeType	Runtime execution model	current-thread, multi-thread, multi-thread-alt
dataFusionMaxParquetFanout	Maximum Parquet file fanout	1-1000000
dataFusionUseCachedParquetLoader	Enable Parquet loader caching	Boolean
Memory Management
bufferMemLimitMb	Buffer memory limit in MB	1024-16777216000
execMemPoolBytes	Execution memory pool size	Percentage or absolute value
parquetMemCacheSize	Parquet memory cache size	Percentage or absolute value
forceSnapshotMemThreshold	Memory threshold for forced snapshots	Percentage or absolute value
WAL Configuration
walSnapshotSize	WAL snapshot size	1-10000
walMaxWriteBufferSize	Maximum WAL write buffer size	1-1000000
snapshottedWalFilesToKeep	Number of snapshotted WAL files to retain	0-10000
walReplayConcurrencyLimit	WAL replay concurrency limit	1-100
Cache Settings
lastCacheEvictionInterval	Last value cache eviction interval	Duration
distinctCacheEvictionInterval	Distinct value cache eviction interval	Duration
tableIndexCacheMaxEntries	Maximum table index cache entries	1-1000
parquetMemCachePrunePercentage	Parquet cache prune percentage	0-1
HTTP Configuration
maxHttpRequestSize	Maximum HTTP request size	1024-16777216
Retention & Deletion
retentionCheckInterval	Retention policy check interval	Duration
deleteGracePeriod	Grace period for deletions	Duration
hardDeleteDefaultDuration	Default hard delete duration	Duration

Enterprise-only parameters

Parameter	Description	Range/Format
Node Configuration
ingestQueryInstances	Number of writer/reader nodes	1-2 (required)
queryOnlyInstances	Number of dedicated reader nodes	0-1 (required)
dedicatedCompactor	Enable dedicated compactor node	Boolean (required)
Compaction Settings
compactionRowLimit	Row limit for compaction	1-100000000
compactionMaxNumFilesPerPlan	Maximum files per compaction plan	1-10000
compactionGen2Duration	Generation 2 compaction duration	Duration
compactionMultipliers	Compaction level multipliers	Format: "n,n,n,n"
compactionCleanupWait	Wait time before cleanup	Duration
compactionCheckInterval	Compaction check interval	Duration
Cache Configuration
lastValueCacheDisableFromHistory	Disable LVC from historical data	Boolean
distinctValueCacheDisableFromHistory	Disable DVC from historical data	Boolean
Replication
replicationInterval	Data replication interval	Duration
catalogSyncInterval	Catalog synchronization interval	Duration

Managing parameter groups

To manage parameter groups:

1. Create: Create a new parameter group with custom settings.

2. Associate: Associate a parameter group with a new or existing cluster.

3. Delete or Modify: Parameter groups cannot be modified or deleted.

Since parameter groups are immutable, plan your configuration carefully before creating a cluster. If you need to change parameters later, you'll need to create a new parameter group and associate it with your cluster, which can require a cluster restart.

Creating parameter groups

Create an InfluxDB 3 Core parameter group:

aws timestream-influxdb create-db-parameter-group \
  --name "my-influxdb3-core-pg" \
  --description "Custom parameter group for InfluxDB 3 Core" \
  --parameters '{
    "InfluxDBv3Core": {
      "queryFileLimit": 512,
      "dataFusionNumThreads": 16,
      "bufferMemLimitMb": 8192,
      "maxHttpRequestSize": 8388608
    }
  }'

Create an InfluxDB 3 Enterprise parameter group (3-node configuration):

aws timestream-influxdb create-db-parameter-group \
  --name "my-influxdb3-enterprise-pg" \
  --description "Custom parameter group for 3-node InfluxDB 3 Enterprise cluster" \
  --parameters '{
    "InfluxDBv3Enterprise": {
      "ingestQueryInstances": 2,
      "queryOnlyInstances": 0,
      "dedicatedCompactor": true,
      "compactionRowLimit": 1000000,
      "compactionCheckInterval": "30m",
      "dataFusionNumThreads": 32,
      "bufferMemLimitMb": 16384
    }
  }'

Create an InfluxDB 3 Enterprise parameter group (single-node configuration):

aws timestream-influxdb create-db-parameter-group \
  --name "my-influxdb3-enterprise-single-pg" \
  --description "Custom parameter group for single-node InfluxDB 3 Enterprise" \
  --parameters '{
    "InfluxDBv3Enterprise": {
      "ingestQueryInstances": 1,
      "queryOnlyInstances": 0,
      "dedicatedCompactor": false,
      "dataFusionNumThreads": 16
    }
  }'

Using service-defined parameter groups

Create cluster with default core parameter group:

aws timestream-influxdb create-db-cluster \
  --name "my-core-cluster" \
  --db-parameter-group-identifier "InfluxDBv3Core" \
  --vpc-subnet-ids subnet-12345 subnet-67890 \
  --vpc-security-group-ids sg-abcdef \
  --db-instance-type db.influxIOIncluded.large \

Create cluster with default enterprise parameter group (3-node):

aws timestream-influxdb create-db-cluster \
  --name "my-enterprise-cluster" \
  --db-parameter-group-identifier "InfluxDBv3Enterprise" \
  --vpc-subnet-ids subnet-12345 subnet-67890 \
  --vpc-security-group-ids sg-abcdef \
  --db-instance-type db.influxIOIncluded.xlarge \

Create cluster with single-node enterprise parameter group:

aws timestream-influxdb create-db-cluster \
  --name "my-enterprise-single" \
  --db-parameter-group-identifier "InfluxDBv3Enterprise1Node" \
  --vpc-subnet-ids subnet-12345 subnet-67890 \
  --vpc-security-group-ids sg-abcdef \
  --db-instance-type db.influxIOIncluded.large \

Listing and describing parameter Groups

List all parameter groups:

aws timestream-influxdb list-db-parameter-groups

Describe a specific parameter group:

aws timestream-influxdb get-db-parameter-group \
  --identifier "my-influxdb3-enterprise-pg"

Service-defined parameter groups Reference

Amazon Timestream for InfluxDB provides three service-defined parameter groups:

1. InfluxDBv3Core: Default configuration for single-node InfluxDB 3 Core clusters.

2. InfluxDBv3Enterprise: Default configuration for 3-node InfluxDB 3 Enterprise clusters (2 writer/reader nodes + 1 dedicated compactor).

3. InfluxDBv3Enterprise1Node: Default configuration for single-node InfluxDB 3 Enterprise clusters (no dedicated compactor).

These service-defined parameter groups are immutable and can be used directly or as templates for creating custom parameter groups using the --source-db-parameter-group-id option.