Device Advisor detailed console workflow - AWS IoT Core

Device Advisor detailed console workflow

In this tutorial, you'll create a custom test suite and run tests against the device you want to test in the console. After the tests are complete, you can view the test results and detailed logs.

Prerequisites

To complete this tutorial, you need to complete the steps outlined in Setting up.

Create a test suite definition

  1. In the AWS IoT console, in the navigation pane, expand Test, Device Advisor and then choose Test suites.

  2. Select Create Test Suite. Choose between Use the AWS Qualification test suite and Create a new test suite. Select the device role that was created as part of Setting up.

    Choose Use the AWS Qualification test suite if you'd like to qualify and list your device to the AWS Partner Device Catalog. By choosing this option, test cases required for qualification of your device to the AWS IoT Core qualification program are pre-selected. Test groups and test cases can't be added or removed. However, you'll still need to configure the test suite properties.

    Choose Create a new test suite to create and configure a custom test suite. We recommend starting with this option for initial testing and troubleshooting. A custom test suite must have at least one test group, and each test group must have at least one test case. For the purpose of this tutorial, we'll select this option and choose Next.

  3. Choose Test suite properties. You must add the test suite properties when you create your test suite.

    Under Test suite properties, fill out the following.

    • Test suite name: You can create the suite with a custom name.

    • Timeout (optional): The timeout in seconds for each test case in the current test suite. If you don't specify a timeout value, the default value is used.

    • Tags (optional): Add tags to the test suite that you're going to create.

    When you've finished, choose Update properties.

  4. To modify the group level configuration, under Test group 1, choose Edit. Then, enter a Name to give the group a custom name. Optionally, you can also enter a Timeout value in seconds under the selected test group. If you don't specify a timeout value, the default value is used.

    Choose Done.

  5. Drag one of the available test cases from Test cases into the test group.

  6. To modify the test case level configuration on the test case under your test group, choose Edit. Then, enter a Name to give the group a custom name. Optionally, you can also enter a Timeout value in seconds under the selected test group. If you don't specify a timeout value, the default value is used.

    Choose Done.

    Note

    To add more test groups to the test suite, choose Add test group. Follow the preceding steps to create and configure more test groups or to add more test cases to one or more test groups. Test groups and test cases can be reordered by dragging. Device Advisor runs tests in the order in which you define the test groups and test cases.

  7. Choose Create test suite.

    The test suite should be created successfully and you'll be redirected to the Test suites page where you can view all the test suite that have been created.

    If the test suite creation failed, make sure the test suite, test groups, and test cases have been configured according to the instructions.

Start a test suite run

  1. In the AWS IoT console, in the navigation pane, expand Test, Device Advisor, and then choose Test suites.

  2. Choose the test suite for which you'd like to view the test suite details.

    The test suite detail page displays all the information related to the test suite. The Device Advisor Endpoint displayed on this page can be used to configure the firmware/software on the device that you'll use for testing to connect to the Device Advisor test endpoint for your account.

  3. Choose Actions, then Run test suite.

  4. Under Run configuration, you'll need to select an AWS IoT thing or certificate to test using Device Advisor. If you don't have any existing things or certificates, first create AWS IoT Core resources. After you select a thing or certificate, choose Run test.

  5. Choose Go to results on the top banner for viewing the test run details.

Stop a test suite run (optional)

  1. In the AWS IoT console, in the navigation pane, expand Test, Device Advisor, and then choose Test runs and results.

  2. Choose the test suite in progress that you want to stop.

  3. Choose Actions, then Stop test suite.

  4. The cleanup process takes several minutes to complete. While the cleanup process is happening, the test run status will be STOPPING. Wait for the cleanup process to complete and for the test suite status to change to STOPPED status before starting a new suite run.

View test suite run details and logs

  1. In the AWS IoT console, in the navigation pane, expand Test, Device Advisor and then choose Test runs and results.

    This page displays:

    • Number of IoT things

    • Number of IoT certificates

    • Number of test suites currently running

    • All the test suite runs that have been created

  2. Choose the test suite for which you'd like to view the run details and logs.

    The run summary page displays the status of the current test suite run. This page auto refreshes every 10 seconds. We recommend that you have a mechanism built for your device to try connecting to our test endpoint every five seconds for one to two minutes. Then you can run multiple test cases in sequence in an automated manner.

  3. To access the CloudWatch logs for the test suite run, choose Test suite log.

    To access CloudWatch logs for any test case, choose Test case log.

  4. Based on your test results, troubleshoot your device until all tests pass.

Download an AWS IoT qualification report

If you chose the Use the AWS IoT Qualification test suite option while creating a test suite and were able to run a qualification test suite, you can download a qualification report by choosing Download qualification report in the test run summary page.