Working with instrumentation for Android and AWS Device Farm

Device Farm provides support for Instrumentation (JUnit, Espresso, Robotium, or any Instrumentation-based tests) for Android.

Device Farm also provides a sample Android application and links to working tests in three Android automation frameworks, including Instrumentation (Espresso). The Device Farm sample app for Android is available for download on GitHub.

What is instrumentation?

Android instrumentation makes it possible for you to invoke callback methods in your test code so you can run through the lifecycle of a component step by step, as if you were debugging the component. For more information, see Instrumented tests in the Test types and locations section of the Android Developer Tools documentation.

Upload your Android instrumentation tests

Use the Device Farm console to upload your tests.

1. Sign in to the Device Farm console at https://console.aws.amazon.com/devicefarm.

2. On the Device Farm navigation panel, choose Mobile Device Testing, then choose Projects.

3. In the list of projects, choose the project that you want to upload your tests to.

   Tip

   You can use the search bar to filter the project list by name.

   To create a project, follow the instructions in Create a project in AWS Device Farm.

4. If the Create a new run button is displayed, choose it.

5. On the Choose application page, select Choose File.

6. Browse to and choose your Android app file. The file must be an .apk file.

7. Choose Next.

8. On the Configure page, in the Setup test framework section, choose Instrumentation, and then select Choose File.

9. Browse to and choose the .apk file that contains your tests.

10. Choose Next, and then complete the remaining instructions to select devices and start the run.

Taking screenshots in Android instrumentation tests

You can take screenshots as part of your Android Instrumentation tests.

To take screenshots, call one of the following methods:

• For Robotium, call the takeScreenShot method (for example, solo.takeScreenShot();).

• For Spoon, call the screenshot method, for example:

  Spoon.screenshot(activity, "initial_state");
  /* Normal test code... */
  Spoon.screenshot(activity, "after_login");

During a test run, Device Farm gets screenshots from the following locations on the devices, if they exist, and then adds them to the test reports:

• /sdcard/robotium-screenshots

• /sdcard/test-screenshots

• /sdcard/Download/spoon-screenshots/test-class-name/test-method-name

• /data/data/application-package-name/app_spoon-screenshots/test-class-name/test-method-name

Additional considerations for Android instrumentation tests

System Animations

Per the Android documentation for Espresso testing, it is recommended that system animations are turned off when testing on real devices. Device Farm automatically disables Window Animation Scale, Transition Animation Scale, and Animator Duration Scale settings when it executes with the android.support.test.runner.AndroidJUnitRunner instrumentation test runner.

Test Recorders

Device Farm supports frameworks, such as Robotium, that have record-and-playback scripting tools.

Standard mode test parsing

In the standard mode of a run, Device Farm parses your test suite and identifies the unique test classes and methods that it will run. This is done through a tool called Dex Test Parser.

When given an Android instrumentation .apk file as input, the parser returns the fully qualified method names of the tests that match JUnit 3 and JUnit 4 conventions.

To test this in a local environment:

1. Download the dex-test-parser binary.

2. Run the following command to get the list of test methods that will run on Device Farm:

   java -jar parser.jar path/to/apk path/for/output