Logging and monitoring

Amazon FinSpace provides a variety of Amazon CloudWatch metrics that you can monitor to determine the health and performance of your FinSpace resources and instances. You can view these metrics using various tools, including the FinSpace console, the AWS CLI, the Amazon CloudWatch console.

Metrics and dimensions

The AWS/FinSpace namespace includes the following metrics.

Metric	Description
CPUUtilization	The average CPU utilization across all the nodes in a cluster.
MemoryUtilization	The average memory utilization across all the nodes in a cluster.
DatabaseCacheDataReadBytes	The operations for reading database cache.
DatabaseCacheDataWriteBytes	The operations for writing to database cache.
DatabaseCacheDataReadOperations	The operations for reading database cache.
DatabaseCacheDataWriteOperations	The operations for writing to database cache.
DatabaseCacheFreeDataStorageCapacity	Database cache free storage.
LocalStorageVolumeReadBytes	The volume for read operation in local storage.
LocalStorageVolumeWriteBytes	The volume for read operation in local storage.
LocalStorageVolumeReadOps	The operations for reading local storage volume.
LocalStorageVolumeWriteOps	The operations for writing to local storage volume.
MemoryHeapBytes	The memory available in the heap in bytes for the component container.
MemoryHeapPeakBytes	The maximum heap size so far in bytes for the component container.
MemoryHeapLimitBytes	The memory limit heap in bytes for the component container as set by -w.
MemoryMappedBytes	The mapped memory in bytes for the component container.
MemoryPhysicalBytes	The physical memory available in bytes for the component container.
KdbSymsTotal	The number of symbols for the component container.
KdbSymsMemoryBytes	The memory use of symbols in bytes for the component container.
KdbHandlesTotal	The number of active connection handles.
KdbIpcOpenedTotal	The total number of IPC sockets that have been opened to the component container.
KdbIpcClosedTotal	The total number of open handles (IPC and WebSocket) for the component container.
KdbWsOpenedTotal	The total number of WebSocket connections opened to the component container.
KdbWsClosedTotal	The total number of WebSocket connections closed to the component container.
KdbSyncTotal	The sync requests made to the component container.
KdbHttpGetTotal	The HTTP GET requests made to the component container.
KdbHttpPostTotal	The HTTP POST requests made to the component container.
KdbWsTotal	The WebSocket messages received by the component container.
KdbTsTotal	The timer calls made within the component container.
KdbSyncErrTotal	The number of errors returned in sync requests.
KdbAsyncErrTotal	The number of errors returned in async requests.
KdbHttpGetErrTotal	The number of errors returned in http GET requests.
KdbHttpPostErrTotal	The number of errors returned in http POST requests.
KdbWsErrTotal	The number of errors returned in WebSocket calls.
KdbTsErrTotal	The number of errors returned in timer calls.
KdbSyncSeconds	The count and time taken by the sync requests.
KdbAsyncSeconds	The count and time taken by the async requests.

The AWS/Usage namespace includes the following metrics.

Metric	Description
KxNodeSglargePerEnvironment	The maximum number of kx.sg.large Managed kdb scaling group nodes per environment.
KxNodeSgxlargePerEnvironment	The maximum number of kx.sg.xlarge Managed kdb scaling group nodes per environment.
KxNodeSg2xlargePerEnvironment	The maximum number of kx.sg.2xlarge Managed kdb scaling group nodes per environment.
KxNodeSg4xlargePerEnvironment	The maximum number of kx.sg.4xlarge Managed kdb scaling group nodes per environment.
KxNodeSg8xlargePerEnvironment	The maximum number of kx.sg.8xlarge Managed kdb scaling group nodes per environment.
KxNodeSg16xlargePerEnvironment	The maximum number of kx.sg.16xlarge Managed kdb scaling group nodes per environment.
KxNodeSg32xlargePerEnvironment	The maximum number of kx.sg.32xlarge Managed kdb scaling group nodes per environment.
KxNodeSgOne16xlargePerEnvironment	The maximum number of kx.sg1.16xlarge Managed kdb scaling group nodes per environment.
KxNodeSgOne24xlargePerEnvironment	The maximum number of kx.sg1.24xlarge Managed kdb scaling group nodes per environment.
KdbFileStorageVolumesPerEnvironment	The maximum number of Managed kdb volumes per environment.
ReadMountsPerKdbVolume	The maximum number of read mounts per Managed kdb volume per environment.
WriteMountsPerKdbVolume	The maximum number of write mounts per Managed kdb volume per environment.
ManagedKdbVolumeStorage	The maximum amount of storage for Managed kdb volumes per environment.
KdbScalingGroupsPerEnvironment	The maximum number of Managed kdb scaling groups per environment.
KdbDataviewsPerEnvironment	The maximum number of Managed kdb dataviews per environment.
ConcurrentKdbDataviewsVersionsPerEnvironment	The maximum number of concurrent Managed kdb dataview version processing.
TotalKdbEnvironments	The maximum number of environments per AWS account.
ManagedKdbClusterUsers	The maximum number of cluster users per environment.
ManagedKdbClusters	The maximum number of clusters allowed per environment.
ManagedKdbDatabases	The maximum number of databases allowed per environment.
ManagedKdbConcurrentChangesetIngestions	The maximum number of concurrent changeset ingestions allowed per environment.
ManagedKdbDatabaseClusterCacheSize	The maximum amount of database cluster cache allowed per environment.
ManagedKdbSavedownStorage	The maximum amount of savedown storage allowed per environment.
KxSlargeNodes	The maximum number of kx.s.large nodes allowed per environment.
KxSxlargeNodes	The maximum number of kx.s.xlarge nodes allowed per environment.
KxS2xlargeNodes	The maximum number of kx.s.2xlarge nodes allowed per environment.
KxS4xlargeNodes	The maximum number of kx.s.4xlarge nodes allowed per environment.
KxS8xlargeNodes	The maximum number of kx.s.8xlarge nodes allowed per environment.
KxS16xlargeNodes	The maximum number of kx.s.16xlarge nodes allowed per environment.
KxS32xlargeNodes	The maximum number of kx.s.32xlarge nodes allowed per environment.
ManagedKdbSingleAzClusters	The maximum number of Single-AZ clusters per environment.
ManagedKdbMultiAzClusters	The maximum number of Multi-AZ clusters per environment.

Note

Amazon CloudWatch aggregates these metrics at one-minute intervals.

Monitoring Managed kdb cluster metrics

You can monitor the health and activity of Managed kdb Insights clusters by using CloudWatch. When you interact with Managed kdb clusters, the following metrics are sent to CloudWatch in the AWS account that you used to create the Managed kdb environment. You can use the following procedures to view the metrics for Managed kdb clusters.

To view metrics using the CloudWatch console

Metrics are grouped first by the service namespace, and then by the various dimension combinations within each namespace.

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. In the navigation pane, choose Metrics.

3. Under the All metrics tab, choose FinSpace and then choose the KxClusterId.

Monitoring Managed kdb scaling groups metrics

You can monitor the health and activity of Managed kdb scaling groups by using CloudWatch. When you interact with Managed kdb scaling groups, the following metrics are sent to CloudWatch in the AWS account that you used to create the Managed kdb environment. You can use the following procedures to view the metrics for Managed kdb scaling groups.

To view metrics using the CloudWatch console

Metrics are grouped first by the service namespace, and then by the various dimension combinations within each namespace.

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. In the navigation pane, choose Metrics.

3. Under the All metrics tab, choose FinSpace and then choose the KxClusterId.

FinSpace supports the following metrics for Managed Kdb scaling groups.

Metric	Description
ScalingGroupCpuUtilization	The total CPU utilization of the Managed kdb scaling group hosts across all clusters.
ScalingGroupMemoryUtilization	The total memory utilization of the Managed kdb scaling group hosts across all clusters.

Monitoring Managed kdb volume metrics

You can monitor the health and activity of Managed kdb volumes by using CloudWatch. When you interact with Managed kdb volumes, the following metrics are sent to CloudWatch in the AWS account that you used to create the Managed kdb environment. You can use the following procedures to view the metrics for Managed kdb volumes.

To view metrics using the CloudWatch console

Metrics are grouped first by the service namespace, and then by the various dimension combinations within each namespace.

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. In the navigation pane, choose Metrics.

3. Under the All metrics tab, choose FinSpace and then choose the KxVolumeName.

FinSpace supports the following metrics for Managed Kdb volumes.

Metric	Description
KdbVolumeDataReadBytes	The data in bytes read from managed kdb volumes.
KdbVolumeDataWriteBytes	The data in bytes written from managed kdb volumes.
KdbVolumeDataReadOperations	The operations for reading Managed kdb volumes.
KdbVolumeDataWriteOperations	The operations for writing to Managed kdb volumes.
KdbVolumeFreeDataStorageCapacity	The Managed kdb volumes free storage.

Logging

The KDB application logs from Managed kdb clusters are captured in Amazon CloudWatch Logs. There is a separate CloudWatch Log Group created for each cluster. You can access cluster logs for FinSpace in the following CloudWatch log group.

/aws/vendedlogs/finspace/<Managed kdb Environment ID>/<Managed kdb Cluster Name>

You can view data directly in CloudWatch using CloudWatch reporting or CloudWatch Insights. You can also extract the data from CloudWatch into other logging tools like Splunk or Datadog. Using the Logs, you can view stdout and stderr logs the for KDB processes. From this tab, you can navigate to CloudWatch Logs Insights to run custom queries on the logs.

To view and query logs for Managed kdb clusters

1. Sign in to the AWS Management Console and open the Amazon FinSpace console at https://console.aws.amazon.com/finspace.

2. In the left pane, under Managed kdb Insights, choose Kdb environments.

3. From the kdb environments table, choose the name of the environment.

4. On the environment details page, choose the Clusters tab. The table under this tab displays a list of clusters.

5. Choose a cluster name to view its logs. The cluster details page opens.

6. Choose the Logs tab. The logs are displayed in a list.

7. Choose the @logstream links to open the CloudWatch Logs Insights where you can run custom queries on the logs.