Menu
AWS Device Farm
Developer Guide (API Version 2015-06-23)

Create a Test Run in AWS Device Farm

You can use the Device Farm console, AWS CLI, or Device Farm API to create a test run. You can also use a supported plugin, such as the Jenkins or Gradle plugins for Device Farm. For more information about plugins, see Tools and Plugins.

For information about runs, see Runs.

Prerequisites

You must have a project in Device Farm. Follow the instructions in Create a Project, and then return to this page.

Create a Test Run (Console)

  1. Sign in to the Device Farm console at https://console.aws.amazon.com/devicefarm.

  2. If you see the AWS Device Farm console home page, enter a name for your project, and then choose Create project. Refresh the page to see your new project.

  3. If you already have a project, you can upload your tests to it.

    Otherwise, choose Create a new project and enter a name.

  4. Open your project, and then choose Create a new run.

  5. On the Choose your application page, choose Native application (the Android and Apple button) or Web application (the HTML5 button).

  6. Upload your application file. You can also drag and drop your file or choose a recent upload.

    If you are uploading an iOS app, be sure to choose iOS device, as opposed to a simulator.

  7. (Optional) In Run name, enter a name.

    If you don't specify a run name, Device Farm uses the app file name.

  8. Choose Next step.

  9. On the Configure page, choose one of the available test suites.

    Note

    If you do not have any tests available, choose Built-in: Fuzz to run a standard, built-in test suite. If you choose Built-in: Fuzz, and the Event count, Event throttle, and Randomizer seed boxes appear, you can change or keep the values.

    For information about the available test suites, see Working with Test Types in AWS Device Farm.

  10. If you did not choose Built-in: Fuzz, choose Upload, and then browse to and choose the file that contains your tests.

  11. For your execution environment, choose Run test in our standard environment or Customize your test environment. For more information see Test Environments.

  12. If you are using the standard test environment, skip to the step 13. If you are using a custom test environment with the default test spec YAML file, skip to the step 13.

    1. If you want to edit the default test spec in a custom test environment, choose Edit to update the default YAML specification.

    2. If you made changes to the test spec, choose Save as New to update the test spec.

  13. Choose Advanced Configuration if you want to configure the video recording or performance data capture options.

    1. Select Enable Video Recording to enable video recording during test execution.

    2. Select Enable App Performance Data Capture to enable the capture of performance data from the device.

    Note

    If you have private devices, Configuration for Private Devices is also displayed.

  14. Choose Next step.

  15. On the Select devices page, do one of the following:

    • To choose a built-in device pool to run the tests against, for Device pool, choose Top Devices.

    • To create your own device pool to run the tests against, follow the instructions in Create a Device Pool, and then return to this page.

    • If you created your own device pool earlier, for Device pool, choose your device pool.

    For more information, see Devices.

  16. Choose Next step.

  17. On the Specify device state page:

    • To provide other data for Device Farm to use during the run, next to Add extra data, choose Upload, and then browse to and choose the .zip file that contains the data.

    • To install an additional app for Device Farm to use during the run, next to Install other apps, choose Upload, and then browse to and choose the .apk, or .ipa, file that contains the app. Repeat this for other apps you want to install. You can change the installation order by dragging and dropping the apps after you upload them.

    • To specify whether Wi-Fi, Bluetooth, GPS, or NFC is enabled during the run, next to Set radio states, select the appropriate boxes.

    • To preset the device latitude and longitude for the run, next to Device location, type the coordinates.

    • To preset the device locale for the run, in Device Locale, choose the locale.

  18. Choose Review and start run.

  19. On this page, you can specify the execution timeout for your test run.

    Note

    If you are using unlimited testing slots, confirm that Run on unmetered slots is selected.

  20. Enter a value or use the slider bar to change the execution timeout. For more information, see Set the Execution Timeout for Test Runs in AWS Device Farm.

  21. Choose Confirm and start run.

Device Farm starts the run as soon as devices are available, typically within a few minutes. Until the run starts, Device Farm displays a calendar icon. After the run starts, results appear as tests are completed. During this time, Device Farm displays a progress icon.

If you need to stop the test run, see Stop a Run in AWS Device Farm.

Create a Test Run (AWS CLI)

For a tutorial about how to use the AWS CLI to create a test run, see this AWS Mobile blog post.

  1. Make sure that you have created a project. For more information, see Create a Project with the AWS CLI.

  2. Run the create-upload command to upload your application file.

  3. Run the create-upload command to upload your tests.

  4. If you want to run in custom test environment, run the update-upload command to upload a YAML test spec file.

  5. Make sure that you have created a device pool. For more information, see Create a Device Pool with the AWS CLI.

  6. Run the schedule-run command to schedule a test run.

For information about using Device Farm with the AWS CLI, see AWS CLI Reference.

Create a Test Run (API)

  1. Make sure that you have created a project. For more information, see Create a Project with the Device Farm API.

  2. Call the ScheduleRun API.

For information about using the Device Farm API, see API Reference.

Next Steps

The run is complete when the progress icon changes to a result icon. A report for the run appears as soon as tests are complete. For more information, see Reports.

To use the report, follow the instructions in Analyze a Report.