IoT SiteWise publisher - AWS IoT Greengrass

IoT SiteWise publisher

The IoT SiteWise publisher component (aws.iot.SiteWiseEdgePublisher) enables AWS IoT SiteWise gateways to export data from the edge to the AWS Cloud.

For more information about AWS IoT SiteWise gateways, see Using AWS IoT SiteWise at the edge in the AWS IoT SiteWise User Guide.

Versions

This component has the following versions:

  • 3.1.x

  • 3.0.x

  • 2.4.x

  • 2.3.x

  • 2.2.x

  • 2.1.x

  • 2.0.x

Type

This component is a generic component (aws.greengrass.generic). The Greengrass nucleus runs the component's lifecycle scripts.

For more information, see Component types.

Operating system

This component can be installed on core devices that run the following operating systems:

  • Linux

  • Windows

Requirements

This component has the following requirements:

  • The Greengrass core device must run on one of the following platforms:

    • OS: Ubuntu 18.04 or later

      Architecture: x86_64 (AMD64) or ARMv8 (Aarch64)

    • OS: Red Hat Enterprise Linux (RHEL) 8

      Architecture: x86_64 (AMD64) or ARMv8 (Aarch64)

    • OS: Amazon Linux 2

      Architecture: x86_64 (AMD64) or ARMv8 (Aarch64)

    • OS: Debian 11

      Architecture: x86_64 (AMD64) or ARMv8 (Aarch64)

    • OS: Windows Server 2019 or later

      Architecture: x86_64 (AMD64)

  • The Greengrass core device must connect to the Internet.

  • The Greengrass core device must be authorized to perform the iotsitewise:BatchPutAssetPropertyValue action. For more information, see Authorize core devices to interact with AWS services.

    Example permissions policy
    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "iotsitewise:BatchPutAssetPropertyValue", "Resource": "*" } ] }

Endpoints and ports

This component must be able to perform outbound requests to the following endpoints and ports, in addition to endpoints and ports required for basic operation. For more information, see Allow device traffic through a proxy or firewall.

Endpoint Port Required Description

data.iotsitewise.region.amazonaws.com

443 Yes

Publish data to AWS IoT SiteWise.

Dependencies

When you deploy a component, AWS IoT Greengrass also deploys compatible versions of its dependencies. This means that you must meet the requirements for the component and all of its dependencies to successfully deploy the component. This section lists the dependencies for the released versions of this component and the semantic version constraints that define the component versions for each dependency. You can also view the dependencies for each version of the component in the AWS IoT Greengrass console. On the component details page, look for the Dependencies list.

The following table lists the dependencies for versions 2.0.x to 2.2.x of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.3.0<3.0.0 Hard
Stream manager >=2.0.10<3.0.0 Hard

For more information about component dependencies, see the component recipe reference.

Configuration

This component doesn't have any configuration parameters.

You can use the AWS IoT SiteWise console or API to configure the IoT SiteWise publisher component. For more information, see Step 3: Configure publisher - optional in the AWS IoT SiteWise User Guide.

Input data

This component reads PutAssetPropertyValueEntry messages from AWS IoT Greengrass stream manager. For more information, see PutAssetPropertyValueEntry in the AWS IoT SiteWise API Reference.

Local log file

This component uses the following log file.

Linux
/greengrass/v2/logs/aws.iot.SiteWiseEdgePublisher.log
Windows
C:\greengrass\v2\logs\aws.iot.SiteWiseEdgePublisher.log
To view this component's logs
  • Run the following command on the core device to view this component's log file in real time. Replace /greengrass/v2 or C:\greengrass\v2 with the path to the AWS IoT Greengrass root folder.

    Linux
    sudo tail -f /greengrass/v2/logs/aws.iot.SiteWiseEdgePublisher.log
    Windows (PowerShell)
    Get-Content C:\greengrass\v2\logs\aws.iot.SiteWiseEdgePublisher.log -Tail 10 -Wait

Troubleshooting and debugging

This component includes a new events log to help customers identify and fix problems. The log file is separate from the local log file, and is found in the following location. Replace /greengrass/v2 or C:\greengrass\v2 with the path to the AWS IoT Greengrass root folder.

Linux
/greengrass/v2/work/aws.iot.SiteWiseEdgePublisher/logs/IotSiteWisePublisherEvents.log
Windows
C:\greengrass\v2\work\aws.iot.SiteWiseEdgePublisher\logs\IotSiteWisePublisherEvents.log

This log includes detailed information and troubleshooting instructions. Troubleshooting information is provided alongside the diagnostics, with a description of how to remedy the issue, and sometimes with links to further information. Diagnostic information includes the following:

  • Severity level

  • Timestamp

  • Additional event-specific information

Example log
accountBeingThrottled: Summary: Data upload speed slowed due to quota limits Level: WARN Timestamp: '2023-06-09T21:30:24.654Z' Description: The IoT SiteWise Publisher is limited to the "Rate of data points ingested" quota for a customers account. See the associated documentation and associated metric for the number of requests that were limited for more information. Note that this may be temporary and not require any change, although if the issue continues you may need to request an increase for the mentioned quota. FurtherInformation: - https://docs.aws.amazon.com/iot-sitewise/latest/userguide/quotas.html - https://docs.aws.amazon.com/iot-sitewise/latest/userguide/troubleshooting-gateway.html#gateway-issue-data-streams AssociatedMetrics: - Name: TotalErrorCount Description: The total number of errors of this type that occurred. Value: 327724.0 AssociatedData: - Name: AggregatePropertyAliases Description: The aggregated property aliases of the throttled data. FileLocation: /greengrass/v2/work/aws.iot.SiteWiseEdgePublisher/./logs/data/AggregatePropertyAliases_1686346224654.log

Licenses

This component is released under the Greengrass Core Software License Agreement.

Changelog

The following table describes the changes in each version of the component.

Version

Changes

3.1.4

Bug fixes and improvements
  • Fixes issues that could cause longer-than-expected startup times after being offline.

3.1.3

Bug fixes and improvements
  • Fixes an issue where the events log file located at /greengrass/v2/work/aws.iot.SiteWiseEdgePublisher/logs/IoTSiteWisePublisherEvents.log was created but no events were being logged.

  • Adds the following CloudWatch metrics for monitoring the connection with the MQTT broker:

    • IoTSiteWisePublisher.IsConnectedToMqttBroker

    • IoTSiteWisePublisher.NumberOfSubscriptionsToMqttBroker

    • IoTSiteWisePublisher.NumberOfUniqueMqttTopicsReceived

    • IoTSiteWisePublisher.MqttMessageReceivedSuccessCount

    • IoTSiteWisePublisher.MqttReceivedSuccessBytes

    For more information about these metrics, see AWS IoT Greengrass Version 2 gateway metrics.

  • Fixes an issue where the BatchCreateJob API is still called even if uploading a parquet file to S3 fails.

3.1.2

Bug fixes and improvements
  • Fixes the issue of high CPU usage introduced in version 3.1.1.

3.1.1

Warning

Version 3.1.1 was discontinued on March 12, 2024. The improvements in this version are available in later versions of this component.

Bug fixes and improvements
  • Adds additional logging that identifies the affected data aliases when an error occurs.

  • Adds local enforcement of AWS IoT SiteWise API limits on the age of the data ingested.

  • Fixes the issue where Publisher mixes up the checkpoints of the StreamManager streams when there are multiple Amazon S3 destinations.

  • Fixes performance bottleneck with how the Publisher reads from the StreamManager streams.

3.1.0

New features
  • Adds support to publish data as parquet files to Amazon S3.

  • Adds support for AWS IoT SiteWise buffered ingestion.

3.0.0

Bug fixes and improvements
  • Fixes issues related to proxy support.

New features
  • Enables support of data ingestion from an MQTT broker.

2.4.1

Bug fixes and improvements
  • Enable component to work with Java Corretto 11 versions 11.0.20.8.1 and later. Component versions 2.4.0 and 2.3.3 show the "Could not find or load main class" error message when used with Java Corretto version 11.0.20.8.1.

2.4.0

New features
  • Adds a new events log to make it easier to identify and remediate problems.

Bug fixes and improvements
  • Improves Publisher checkpoint recovery.

2.3.3

Bug fixes and improvements
  • Improves ability to support high throughput.

2.3.2

Bug fixes and improvements
  • Fixes HTTP proxy support when downloading Publisher configuration.

2.3.1

New features
  • Adds support for installing Data Collection Pack on Linux ARMv8 architecture.

  • Minimum requirements for Linux ARMv8:

    • Memory: 4 GB

    • CPU: ARM Cortex-A72 or equivalent specification

2.2.3

Bug fixes and improvements
  • Removes retry for generic exception which was not in the retriable exception list.

2.2.2

Bug fixes and improvements
  • Reintroduces data upload support to AWS IoT SiteWise through an HTTP proxy server.

2.2.1

Note

This version doesn't support HTTP proxy configuration. Version 2.2.2 and higher reintroduces support for this feature.

New features
  • Adds support to this component to toggle compression when uploading data to AWS IoT SiteWise.

2.2.0

Note

This version doesn't support HTTP proxy configuration. Version 2.2.2 and higher reintroduces support for this feature.

New features
  • Updates this component to compress data before sending it to the AWS IoT SiteWise service.

    • In most cases, this change reduces bandwidth usage by 75 percent compared to previous versions of this component.

    • In most cases, this change increases CPU usage by up to 5 percent. On gateways that process large amounts of data, this change can increase CPU usage by up to 15 percent.

    • This change doesn't affect AWS IoT SiteWise service charges or service quota usage.

    • Adds support for Windows Server 2019 or higher.

Bug fixes and improvements
  • Fixes an issue that prevents this component from starting when the checkpoint file is corrupted.

2.1.4

Bug fixes and improvements
  • Fixes compatibility with Java version 8.

2.1.3

Warning

This version is no longer available, except in the US East (Ohio), Canada (Central), and AWS GovCloud (US-East) Regions. This component version requires Java version 11 or greater to run. The improvements in this version are available in later versions of this component.

Bug fixes and improvements
  • Improves error messages when you deploy this component on unsupported devices.

  • Updates to log errors when data uploads fail.

2.1.2

Bug fixes and improvements
  • Updates to invoke the expired data export feature as soon as data expires.

2.1.1

Bug fixes and improvements.

2.1.0

New features
  • Adds support for publishing the newest data to the cloud first.

  • Adds support for not publishing expired data to the cloud.

  • Adds support for storing expired data locally.

Bug fixes and improvements
  • Reduces disk I/O and corresponding latency.

2.0.2

Bug fixes and improvements.

2.0.1

Initial version.

See also