Monitor AWS IoT Greengrass logs
AWS IoT Greengrass consists of the cloud service and the AWS IoT Greengrass Core software. The AWS IoT Greengrass Core software can write logs to Amazon CloudWatch Logs and to the core device's local file system. Greengrass components that run on the core device can also write logs to CloudWatch Logs and the local file system. You can use logs to monitor events and troubleshoot issues. All AWS IoT Greengrass log entries include a timestamp, log level, and information about the event.
By default, the AWS IoT Greengrass Core software writes logs to only the local file system. You can view file system logs in real time, so you can debug Greengrass components that you develop and deploy. You can also configure a core device to write logs to CloudWatch Logs, so you can troubleshoot the core device without access to the local file system. For more information, see Enable logging to CloudWatch Logs.
Topics
Access file system logs
The AWS IoT Greengrass Core software stores logs in the
folder on a
core device, where
/logs/greengrass/v2
is the path to the AWS IoT Greengrass root folder. The logs folder has the
following structure./greengrass/v2
└── logs ├── greengrass.log ├── greengrass_
/greengrass/v2
2021_09_14_15_0
.log ├──ComponentName
.log ├──ComponentName
_2021_09_14_15_0
.log └── main.log
-
greengrass.log
– The AWS IoT Greengrass Core software log file. Use this log file to view real-time information about components and deployments. This log file includes logs for the Greengrass nucleus, which is the core of the AWS IoT Greengrass Core software, and plugin components, such as log manager and secret manager. -
– Greengrass component log files. Use component log files to view real-time information about a Greengrass component that runs on the core device. Generic components and Lambda components write standard output (stdout) and standard error (stderr) to these log files.ComponentName
.log -
main.log
– The log file for themain
service that handles component lifecycles. This log file will always be empty.
For more information about the differences between plugin, generic, and Lambda components, see Component types.
The following considerations apply when you use file system logs:
-
Root user permissions
You must have root permissions to read AWS IoT Greengrass logs on the file system.
-
Log file rotation
The AWS IoT Greengrass Core software rotates log files every hour or when they exceed a file size limit. Rotated log files contain a timestamp in their file name. For example, a rotated AWS IoT Greengrass Core software log file might be named
greengrass_2021_09_14_15_0.log
. The default file size limit is 1,024 KB (1 MB). You can configure the file size limit on the Greengrass nucleus component. -
Log file deletion
The AWS IoT Greengrass Core software cleans up earlier log files when the size of AWS IoT Greengrass Core software log files or Greengrass component log files, including rotated log files, exceeds a disk space limit. The default disk space limit for the AWS IoT Greengrass Core software log and each component log is 10,240 KB (10 MB). You can configure the AWS IoT Greengrass Core software log disk space limit on the Greengrass nucleus component or the log manager component. You can configure each component's log disk space limit on the log manager component.
To view the AWS IoT Greengrass Core software log file
-
Run the following command to view the log file in real time. Replace
with the path to the AWS IoT Greengrass root folder./greengrass/v2
To view the log file for a component
-
Run the following command to view the log file in real time. Replace
or/greengrass/v2
C:\greengrass\v2
with the path to the AWS IoT Greengrass root folder, and replacecom.example.HelloWorld
with the name of the component.
You can also use the logs
command of the Greengrass CLI to analyze Greengrass logs on a core device. To
use the logs
command, you must configure the Greengrass nucleus to output JSON format log files.
For more information, see Greengrass Command Line Interface and logs.
Access CloudWatch Logs
You can deploy the log manager component to configure the core device to write to CloudWatch Logs. For more information, see Enable logging to CloudWatch Logs. Then, you can view logs on the Logs page of the Amazon CloudWatch console or using the CloudWatch Logs API.
- Log group name
-
/aws/greengrass/
componentType
/region
/componentName
The log group name uses the following variables:
-
componentType
– The type of the component, which can be one of the following:-
GreengrassSystemComponent
– This log group includes logs for the nucleus and plugin components, which run in the same JVM as the Greengrass nucleus. The component is part of the Greengrass nucleus. -
UserComponent
– This log group includes logs for generic components, Lambda components, and other applications on the device. The component isn't part of the Greengrass nucleus.
For more information, see Component types.
-
-
region
– The AWS Region that the core device uses. -
componentName
– The name of the component. For system logs, this value isSystem
.
-
- Log stream name
-
/
date
/thing/thingName
The log stream name uses the following variables:
-
date
– The date of the log, such as2020/12/15
. The log manager component uses theyyyy/MM/dd
format. -
thingName
– The name of the core device.
Note
If a thing name contains a colon (
:
), the log manager replaces the colon with a plus (+
). -
The following considerations apply when you use the log manager component to write to CloudWatch Logs:
-
Log delays
Note
We recommend that you upgrade to log manager version 2.3.0 which reduces log delays for rotated and active log files. When you upgrade to log manager 2.3.0, we recommend you also upgrade to Greengrass nucleus 2.9.1.
The log manager component version 2.2.8 (and earlier) processes and uploads logs from only rotated log files. By default, the AWS IoT Greengrass Core software rotates log files every hour or after they are 1,024 KB. As a result, the log manager component uploads logs only after the AWS IoT Greengrass Core software or a Greengrass component writes over 1,024 KB worth of logs. You can configure a lower log file size limit to cause log files to rotate more often. This causes the log manager component to upload logs to CloudWatch Logs more frequently.
The log manager component version 2.3.0 (and later) processes and uploads all logs. When you write a new log, log manager version 2.3.0 (and later) processes and directly uploads that active log file instead of waiting for it to be rotated. This means that you can view the new log in 5 minutes or less.
The log manager component uploads new logs periodically. By default, the log manager component uploads new logs every 5 minutes. You can configure a lower upload interval, so the log manager component uploads logs to CloudWatch Logs more frequently by configuring the
periodicUploadIntervalSec
. For more information about how to configure this periodic interval, see Configuration.Logs can be uploaded in near real-time from the same Greengrass file system. If you need to observe logs in real time, consider using file system logs.
Note
If you're using different file systems to write logs to, log manager reverts back to the behavior in log manager component versions 2.2.8 and earlier. For information about accessing file system logs, see Access file system logs.
-
Clock skew
The log manager component uses the standard Signature Version 4 signing process to create API requests to CloudWatch Logs. If the system time on a core device is out of sync by more than 15 minutes, then CloudWatch Logs rejects the requests. For more information, see Signature Version 4 signing process in the AWS General Reference.
Access system service logs
If you configure the AWS IoT Greengrass Core software as a system service, you can view system service logs to troubleshoot issues, such as the software failing to start.
To view system service logs (CLI)
-
Run the following command to view AWS IoT Greengrass Core software system service logs.
-
On Windows devices, the AWS IoT Greengrass Core software creates a separate log file for system service errors. Run the following command to view the system service error logs.
On Windows devices, you can also use the Event Viewer application to view system service logs.
To view Windows service logs (Event Viewer)
-
Open the Event Viewer application.
-
Select Windows Logs to expand it.
-
Choose Application to view application service logs.
-
Find and open event logs whose Source is greengrass.
Enable logging to CloudWatch Logs
You can deploy the log manager component to configure a core device to write logs to CloudWatch Logs. You can enable CloudWatch Logs for AWS IoT Greengrass Core software logs, and you can enable CloudWatch Logs for specific Greengrass components.
Note
The Greengrass core device's token exchange role must allow the core device to write to CloudWatch Logs, as shown in the following example IAM policy. If you installed the AWS IoT Greengrass Core software with automatic resource provisioning, your core device has these permissions.
{ "Version": "2012-10-17", "Statement": [ { "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Effect": "Allow", "Resource": "arn:aws:logs:*:*:*" } ] }
To configure a core device to write AWS IoT Greengrass Core software logs to CloudWatch Logs, create a deployment that specifies a configuration
update that sets uploadToCloudWatch
to true
for the
aws.greengrass.LogManager
component. AWS IoT Greengrass Core software logs include logs for the
Greengrass nucleus and plugin components.
{ "logsUploaderConfiguration": { "systemLogsConfiguration": { "uploadToCloudWatch": "true" } } }
To configure a core device to write a Greengrass component's logs to CloudWatch Logs, create a deployment that specifies a configuration update that adds the component to the list of component logging configurations. When you add a component to this list, the log manager component writes its logs to CloudWatch Logs. Component logs include logs for generic components and Lambda components.
{ "logsUploaderConfiguration": { "componentLogsConfigurationMap": { "
com.example.HelloWorld
": { } } } }
When you deploy the log manager component, you can also configure disk space limits and whether the core device deletes log files after writing them to CloudWatch Logs. For more information, see Configure logging for AWS IoT Greengrass.
Configure logging for AWS IoT Greengrass
You can configure the following options to customize logging for Greengrass core devices. To configure these options, create a deployment that specifies a configuration update to the Greengrass nucleus or log manager components.
-
Writing logs to CloudWatch Logs
To remotely troubleshoot core devices, you can configure core devices to write AWS IoT Greengrass Core software and component logs to CloudWatch Logs. To do so, deploy and configure the log manager component. For more information, see Enable logging to CloudWatch Logs.
-
Deleting uploaded log files
To reduce disk space usage, you can configure core devices to delete log files after writing the log files to CloudWatch Logs. For more information, see the log manager component's
deleteLogFileAfterCloudUpload
parameter, which you can specify for AWS IoT Greengrass Core software logs and component logs. -
Log disk space limits
To limit disk space usage, you can configure the maximum disk space for each log, including its rotated log files, on a core device. For example, you can configure the maximum combined disk space for
greengrass.log
and rotatedgreengrass.log
files. For more information, see the Greengrass nucleus component'slogging.totalLogsSizeKB
parameter and the log manager component'sdiskSpaceLimit
parameter, which you can specify for AWS IoT Greengrass Core software logs and component logs. -
Log file size limit
You can configure the maximum file size for each log file. After a log file exceeds this file size limit, the AWS IoT Greengrass Core software creates a new log file. The log manager component version 2.28 (and earlier) writes only rotated log files to CloudWatch Logs, so you can specify a lower file size limit to write logs to CloudWatch Logs more frequently. The log manager component version 2.3.0 (and later) processes and uploads all logs instead of waiting for them to be rotated. For more information, see the Greengrass nucleus component's log file size limit parameter (
logging.fileSizeKB
). -
Minimum log levels
You can configure the minimum log level that the Greengrass nucleus component writes to file system logs. For example, you might specify
DEBUG
level logs to help with troubleshooting, or you might specifyERROR
level logs to reduce the amount of logs that a core device creates. For more information, see the Greengrass nucleus component's log level parameter (logging.level
).You can also configure the minimum log level that the log manager component writes to CloudWatch Logs. For example, you might specify a higher log level to reduce logging costs
. For more information, see the log manager component's minimumLogLevel
parameter, which you can specify for AWS IoT Greengrass Core software logs and component logs. -
Interval to check for logs to write to CloudWatch Logs
To increase or decrease how frequently the log manager component writes logs to CloudWatch Logs, you can configure the interval where it checks for new log files to write. For example, you might specify a lower interval to view logs in CloudWatch Logs sooner than you would with the default 5-minute interval. You might specify a higher interval to reduce costs, because the log manager component batches log files into fewer requests. For more information, see the log manager component's upload interval parameter (
periodicUploadIntervalSec
). -
Log format
You can choose whether the AWS IoT Greengrass Core software writes logs in text or JSON format. Choose text format if you read logs, or choose JSON format if you use an application to read or parse logs. For more information, see the Greengrass nucleus component's log format parameter (
logging.format
). -
Local file system logs folder
You can change the logs folder from
to another folder on the core device. For more information, see the Greengrass nucleus component's output directory parameter (
/logs/greengrass/v2
logging.outputDirectory
).
AWS CloudTrail logs
AWS IoT Greengrass integrates with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or AWS service in AWS IoT Greengrass. For more information, see Log AWS IoT Greengrass V2 API calls with AWS CloudTrail.