Working with Calabash tests on AWS Device Farm

Device Farm supports Calabash for iOS and Android tests. For a sample test suite using Calabash to test an application using Calabash, see the Device Farm sample app.

What is Calabash?

Calabash is a mobile testing framework you can use to run automated interface acceptance tests. Tests are written in the Cucumber description language and Ruby.

Device farm supports Calabash version 0.20.5.

Preparing your Calabash tests

To bring your Calabash tests to Device Farm, they must be zipped with the following structure:

• A directory called features, containing

  • Feature definitions (my-feature.feature)

  • A directory called step_definitions containing ruby step definitions

  • A directory called support containing ruby support files

  • Any other supporting files

Upload your Calabash tests to AWS Device Farm

Use the Device Farm console to upload your tests.

1. Sign in to the Device Farm console at https://console.aws.amazon.com/devicefarm.

2. Choose Device Testing Projects

3. Choose the project for the test run to be associated with

   Tip

   To create a project, follow the instructions in Create a Project in AWS Device Farm.

4. Choose Create a new run

5. On the Choose your application page, choose Upload.

6. Browse to and choose your app file. The file must be an .ipa file for iOS applications or an .apk file for Android applications.

   Note

   Make sure that your .ipa file is built for an iOS device and not for a simulator.

7. Choose Next step.

8. On the Configure a test page, choose Calabash, and then choose Upload.

9. Browse to and choose the .zip file that contains your tests.

10. Choose Next step, and then complete the remaining instructions to select the devices to run your tests on and start the run.

Taking Screenshots in Android Calabash Tests

You can take screenshots as part of your Android Calabash tests.

Calabash provides a set of predefined steps for taking screenshots. For details, see the "Screenshots" section of the iOS Predefined Steps and Android Predefined Steps page in the Calabash repositories on GitHub.

Alternatively, you can define a custom step inside of a Ruby (.rb) file to call the screenshot_embed function, which creates a screenshot and saves it to a directory you define. For example, the following code example creates a screenshot and saves it to the /my/custom/path directory with a file name of screenshot_seconds-since-Epoch:

screenshot_embed(:prefix => "/my/custom/path", :name => "screenshot_#{Time.now.to_i}")

Additional Considerations for Calabash Tests

Device Farm replaces some Calabash hooks so that Calabash tests can run on devices in Device Farm, but does not modify tests.