Extending Custom Test Environments in Device Farm - AWS Device Farm

Extending Custom Test Environments in Device Farm

The Device Farm Custom Mode enables you to run more than just your test suite. In this section, you learn how to extend your test suite and optimize your tests.

Setting a PIN

Some applications require that you set a PIN on the device. Device Farm does not support setting a PIN on devices natively. However, this is possible with the following caveats:

  • The device must be running Android 8 or above.

  • The PIN must be removed after the test is complete.

To set the PIN in your tests, use the pre_test and post_test phases to set and remove the PIN, as shown following:

phases: pre_test: - # ... among your pre_test commands - DEVICE_PIN_CODE="1234" - adb shell locksettings set-pin "$DEVICE_PIN_CODE" post_test: - # ... Among your post_test commands - adb shell locksettings clear --old "$DEVICE_PIN_CODE"

When your test suite begins, the PIN 1234 is set. After your test suite exits, the PIN is removed.


If you don't remove the PIN from the device after the test is complete, the device and your account will be quarantined.

Speeding Up Appium-based Tests through Server Capabilities

When using Appium, you might find that the standard mode test suite is very slow. This is because Device Farm applies the default settings and doesn't make any assumptions about how you want to use the Appium environment. While these defaults are built around industry best practices, they might not apply to your situation. To fine-tune the parameters of the Appium server, you can adjust the default Appium Driver capabilities in your test spec. For example, the following sets the usePrebuildWDA capability to true in an iOS test suite to speed up initial start time:

phases: pre_test: - # ... Start up Appium - >- appium --log-timestamp --default-capabilities "{\"usePrebuiltWDA\": true, \"derivedDataPath\":\"$DEVICEFARM_WDA_DERIVED_DATA_PATH\", \"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\", \"app\":\"$DEVICEFARM_APP_PATH\", \"automationName\":\"XCUITest\", \"udid\":\"$DEVICEFARM_DEVICE_UDID_FOR_APPIUM\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\"}" >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 &

Appium capabilities must be a shell-escaped, quoted JSON structure.

The following Appium capabilities are common sources of performance improvements:

noReset and fullReset

These two capabilities, which are mutually exclusive, describe the behavior of Appium after each session is complete. When noReset is set to true, the Appium server doesn't remove data from your application when an Appium session ends, effectively doing no cleanup whatsoever. fullReset uninstalls and clears all application data from the device after the session has closed. For more information, see Reset Strategies in the Appium documentation.

ignoreUnimportantViews (Android only)

Instructs Appium to compress the Android UI hierarchy only to relevant views for the test, speeding up certain element lookups. However, this can break some XPath-based test suites because the hierarchy of the UI layout has been changed.

skipUnlock (Android only)

Informs Appium that there is no PIN code currently set, which speeds up tests after a screen off event or other lock event.

For more information on the capabilities that Appium supports, see Appium Desired Capabilities in the Appium documentation.

Using Webhooks and other APIs after Your Tests Run

You can have Device Farm call a webhook after every test suite finishes using curl. The process to do this varies with the destination and formatting. For your specific webhook, see the documentation for that webhook. The following example posts a message each time a test suite has finished to a Slack webhook:

phases: post_test: - curl -X POST -H 'Content-type: application/json' --data '{"text":"Tests on '$DEVICEFARM_DEVICE_NAME' have finished!"}' https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

For more information on using webhooks with Slack, see Sending your first Slack message using Webhook in the Slack API reference.

You are not limited to using curl to call webhooks. Test packages can include extra scripts and tools, as long as they are compatible with the Device Farm execution environment. For example, your test package may include auxiliary scripts that make requests to other APIs. Make sure that any required packages are installed alongside your test suite's requirements. To add a script that runs after your test suite is complete, include the script in your test package and add the following to your test spec:

phases: post_test: - python post_test.py

Maintaining any API keys or other authentication tokens used in your test package is your responsibility. We recommend that you keep any form of security credential out of source control, use credentials with the fewest possible privileges, and use revokable, short-lived tokens whenever possible. To verify security requirements, see the documentation for the third-party APIs that you use.

If you plan on using AWS services as a part of your test execution suite, you should use IAM temporary credentials, generated outside of your test suite and included in your test package. These credentials should have the fewest granted permissions and shortest lifespan possible. For more information on creating temporary credentials, see Requesting temporary security credentials in the IAM User Guide.