AWS IoT Greengrass
Developer Guide

IDT for AWS IoT Greengrass Troubleshooting

IDT for AWS IoT Greengrass writes these errors to various locations based on the type of errors. Errors are written to the console, log files, and test reports.

Error Codes

The following table lists the error codes generated by IDT for AWS IoT Greengrass.

Error Code Error Code Name Possible Root Cause Troubleshooting

101

InternalError

An internal error occurred.

Contact AWS Developer Support.

102

TimeoutError

The test cannot be completed in a limited time range. This can happen if:

  • There is a slow network connection between the test machine and device (for example, if you are using a VPN network).

  • A slow network delays the communication between the device and cloud.

  • The timeout field in test configuration files (test.json) has been mistakenly modified.

  • Check the network connection and speed.

  • Make sure that you did not modify any file under the /test directory.

  • Try running the failed test group with "--group-id" flag manually.

  • Try running the test suite by increasing the test timeouts. For more information, see Timeout Errors.

103

PlatformNotSupportError

Incorrect OS/architecture combination specified in device.json.

Change your configuration to one of the supported combinations:

  • Linux, x86_64

  • Linux, ARMv7l

  • Linux, AArch64

  • Ubuntu, x86_64

  • OpenWRT, ARMv7l

  • OpenWRT, AArch64

For more information, see Device Configuration.

104

VersionNotSupportError

The AWS IoT Greengrass Core software version is not supported by the version of IDT you are using.

Use the device_tester_bin version command to find the supported version of the AWS IoT Greengrass Core software. For example, if you are using macOS, use: ./devicetester_mac_x86_64 version.

To find the version of AWS IoT Greengrass Core software you are using:

  • If you are running tests with pre-installed AWS IoT Greengrass Core software, use SSH to connect to your AWS IoT Greengrass core device and run: <path-to-preinstalled-greengrass-location>;/greengrass/ggc/core/greengrassd --version

  • If you are running tests with a different version of the AWS IoT Greengrass Core software, go to the devicetester_greengrass_ <os>products/greengrass/gcc directory. The AWS IoT Greengrass Core software version is part of the .zip file name.

You can test a different version of the AWS IoT Greengrass Core software. For more information, see Getting Started with AWS IoT Greengrass.

105

LanguageNotSupportError

IDT only supports Python for AWS IoT Greengrass libraries and SDKs.

Make sure:

  • The SDK package under devicetester_greengrass_<os>/products/greengrass/ggsdk is the Python SDK.

  • The contents of the bin folder under devicetester_greengrass_<os> /tests/GGQ_1/suite/resources/run.runtimefarm/bin have not been changed.

106

ValidationError

Some fields in device.json or config.json are invalid.

Check the error message on the right side of the error code in the report.

  • Invalid private key path: Specify the correct path to your private key. For more information, see Device Configuration.

  • Invalid AWS Region: Specify a valid AWS Region in your config.json file. For more information about AWS Regions, see Regions and Endpoints.

  • AWS credentials: Set valid AWS credentials on your test machine (through environment variables or the credentials file). Verify that the auth field is configured correctly. For more information, see Create and Configure an AWS Account

107

SSHConnectionFailed

The test machine cannot connect to the configured device.

Verify the following fields in your device.json file are correct:

  • ip

  • user

  • privKeyPath

For more information, see Device Configuration.

108

RunCommandError

A test failed to execute a command on the device under test.

Verify that root access is allowed for the configured user in device.json.

A password is required by some devices when executing commands with root access. Make sure root access is allowed without a password. For more information, consult the documentation for your device.

Try running the failing command manually on your device to see if an error occurs.

109

PermissionDeniedError

No root access.

Set root access for the configured user on your device.

110

CreateFileError

Unable to create a file.

Check your device's disk space and directory permissions.

111

CreateDirError

Unable to create a directory.

Check your device's disk space and directory permissions.

112

InvalidPathError

The path to the AWS IoT Greengrass Core software is incorrect.

Verify that the path in the error message is valid. Do not to edit any files under the devicetester_greengrass_<os> directory.

113

InvalidFileError

A file is invalid.

Verify that the file in the error message is valid.

114

ReadFileError

The specified file cannot be read.

Verify the following:

  • File permissions are correct.

  • limits.config allows enough files to be open.

  • The file specified in the error message exists and is valid.

If you are testing on macOS, increase the open files limit. The default limit is 256, which is enough for testing.

115

FileNotFoundError

A required file was not found.

Verify the following:

  • The SDK package exists under devicetester_greengrass_<os>/products/greengrass/ggsdk.

  • The files under devicetester_greengrass_<os>/tests have not been modified.

116

OpenFileFailed

Unable to open the specified file.

Verify the following:

  • The file specified in the error message exists and is valid.

  • limits.config allows enough files to be open.

If you are testing on macOS, increase the open files limit. The default limit is 256, which is enough for testing.

117

WriteFileFailed

Failed to write file (can be the DUT or test machine).

Try to manually create a file in the directory specified in the error message.

118

FileCleanUpError

A test failed to remove the specified file or directory or to umount the specified file on the remote device.

If the binary file is still running, the file might be locked. End the process and delete the specified file.

119

InvalidInputError

Invalid configuration.

Verify your suite.json file is valid.

120

InvalidCredentialError

Invalid AWS credentials.

Verify your AWS credentials. Because this error can also be caused by network problems, check your network connection and rerun the test group.

121

AWSSessionError

Failed to create an AWS session.

This error can occur if AWS credentials are invalid or the internet connection is unstable. Try using the AWS CLI to call an AWS API operation.

122

AWSApiCallError

An AWS API error occurred.

This error might be due to a network issue. Check your network before retrying the test group.

123

IpNotExistError

IP address is not included in connectivity information.

Check your internet connection. You can use the AWS IoT Greengrass console to check the connectivity information for the AWS IoT Greengrass core thing that is being used by the test. If there are 10 endpoints included in the connectivity information, you can remove some or all of them and rerun the test. For more information, see Connectivity Information.

124

OTAJobNotCompleteError

An OTA job did not complete.

Check your internet connection and retry the OTA test group.

125

CreateGreengrassServiceRoleError

One of the following occurred:

  • An error occurred while creating a role.

  • An error occurred while attaching a policy to the AWS IoT Greengrass service role.

  • The policy associated with the service role is invalid.

  • An error occurred when associating a role with an AWS account.

Configure the AWS IoT Greengrass service role. For more information, see Configure the AWS IoT Greengrass Service Role.

126

DependenciesNotPresentError

One or more dependencies required for the specific test are not present on the device.

Check the test log (<device-tester-extract-location>/results/ <execution-id>>/logs/<test-case-name>.log>) to see which dependencies are missing on your device.

127

InvalidHSMConfiguration

The provided HSM/PKCS configuration is incorrect.

In your device.json file, provide the configuration required to interact with the HSM using PKCS#11.

Resolving IDT for AWS IoT Greengrass Errors

When you use IDT, you must get the correct configuration files in place before you run IDT for AWS IoT Greengrass. If you are getting parsing and configuration errors, your first step is to locate and use a configuration template appropriate for your environment.

If you are still having issues, see the following debugging process.

Where Do I Look?

High level errors are displayed on the console during execution, and a summary of the failed tests with the error is displayed when all of the tests complete. awsiotdevicetester_report.xml contains a summary of all the errors that caused a test to fail. The log files for each test run is stored in a directory named with a UUID for the test execution that was displayed on the console during the test run.

The test logs directory is located in <device-tester-extract-location>/results /<uuidexecution-id>/logs/. This directory contains the following files which are useful for debugging.

File Description
test_manager.log

Contains all of the logs that were written to the console during the test execution. A summary of the results is located at the end of this file which includes a list of what tests failed.

The warning and error logs in this file can give you some information about the failure(s). If you need additional details see the more specific logs below.

<test-name>.log Contains detailed logs on what happened for the specific test.
<test-name>_ggc_logs.tar.gz A compressed collection of all the logs the AWS IoT Greengrass core daemon generated during the test. For more information, see Troubleshooting AWS IoT Greengrass.
<test-name>_ota_logs.tar.gz This file is only for OTA tests. It is a compressed collection of logs generated by the AWS IoT Greengrass OTA agent during the test.

Parsing Errors

Occasionally, a typo in a JSON configuration can lead to parsing errors. Most of the time, the issue is a result of omitting a bracket, comma, or quote from your JSON file. IDT performs JSON validation and prints debugging information. It prints the line where the error occurred, the line number, and the column number of the syntax error. This information should be enough to help you fix the error, but if you still cannot locate the error, you can perform validation manually in your IDE, a text editor such as Atom or Sublime, or through an online tool like JSONLint.

Required Parameter Missing Error

Because new features are being added to IDT, changes to the configuration files might be introduced. Using an old configuration file might break your configuration. If this happens, the <test_case_id>.log file under /results/<uuid>/logs explicitly lists all missing parameters. IDT also validates your JSON configuration file schemas to ensure that the latest supported version has been used.

Could Not Start Test Error

You might encounter errors that point to failures during test start. There are several possible causes, so do the following:

  • Make sure that the pool name you included in your execution command actually exists. The pool name is referenced directly from your device.json file.

  • Make sure that the devices in your pool have correct configuration parameters.

Permission Denied Errors

IDT performs operations on various directories and files in a device under test. Some of these operations require root access. To automate these operations, IDT must be able to run commands with sudo without typing a password.

Follow these steps to allow sudo access without typing a password.

Note

user and username refer to the SSH user used by IDT to access the device under test.

  1. Use sudo usermod -aG sudo <ssh-username> to add your SSH user to the sudo group

  2. Sign out and then sign in for changes to take effect.

  3. Open /etc/sudoers file and then add the following line to the end of the file: <ssh-username> ALL=(ALL) NOPASSWD: ALL

    Note

    As a best practice, we recommend that you use sudo visudo when you edit /etc/sudoers.

SSH Connection Errors

When IDT cannot connect to a device under test, connection failures are logged in /results/<uuid>/logs/<test-case-id>.log. SSH failure messages appear at the top of this log file because connecting to a device under test is one of the first operations that IDT performs.

Most Windows setups use the PuTTy terminal application to connect to Linux hosts. This application requires that standard PEM private key files are converted into a proprietary Windows format called PPK. When IDT is configured in your device.json file, use PEM files only. If you use a PPK file, IDT cannot create an SSH connection with the AWS IoT Greengrass device and cannot run tests.

Timeout Errors

You can increase the timeout for each test by specifying a timeout multiplier, which will be applied to the default value of each test's timeout. Any value configured for this flag must be greater than or equal to 1.0.

To use the timeout multiplier use the flag --timeout-multiplier when running the tests. For example:

./devicetester_linux run-suite --suite-id GGQ_1 --pool-id DevicePool1 --timeout-multiplier 2.5

For more information, run run-suite --help.

Command Not Found Errors While Testing OTA

You need an older version of the OpenSSL library (libssl1.0.0) to run OTA tests on AWS IoT Greengrass devices. Most current Linux distributions use libssl version 1.0.2 or later (v1.1.0). These versions are not compatible with OTA.

For example, on a Raspberry Pi, run the following commands to install the required version of libssl:

  1. wget http://ftp.us.debian.org/debian/pool/main/o/openssl/libssl1.0.0_1.0.2l-1~bpo8+1_armhf.deb

  2. sudo dpkg -i libssl1.0.0_1.0.2l-1~bpo8+1_armhf.deb