AWS Device Farm
Developer Guide (API Version 2015-06-23)

Working with Calabash for iOS and AWS Device Farm

Device Farm provides support for Calabash for iOS. The following information describes how to use this test framework with Device Farm test types.

What is Calabash?

Calabash is a mobile testing framework that enables automated user interface acceptance tests that are written in Cucumber to be run on iOS apps. For more information, see the Welcome to Calabash iOS repository on GitHub.

Version Information

Currently Device Farm supports Calabash version 0.20.5.

Prepare Your iOS Calabash Tests

Before you upload your iOS Calabash tests to Device Farm for testing, make sure that your iOS Calabash tests are contained within a .zip file. This .zip file must contain the following structure: `-- features (directory) |-- my-feature-1-file-name.feature |-- my-feature-2-file-name.feature |-- my-feature-N-file-name.feature |-- step_definitions (directory) | `-- (.rb files) |-- support (directory) | `-- (.rb files) `-- (any other supporting files)

Upload Your iOS Calabash Tests

To run your iOS Calabash tests on a set of iOS devices in Device Farm, you upload your tests with the Device Farm console as follows:

  1. Sign in to the Device Farm console at

  2. In the list of projects, choose the option next to the project that you want to upload your tests to.


    If the list of projects is not displayed, then on the secondary navigation bar, for Projects, choose the name of the project that you want to upload your tests to.

    To create a new project, follow the instructions in Create a Project.

  3. If the Create a new run button is displayed, then choose it.

  4. On the Choose your application page, choose Upload.

  5. Browse to and choose your iOS app file. The file must be an .ipa file.


    Make sure that your .ipa file is built for an iOS device and not for a simulator.

  6. Choose Next step.

  7. On the Configure a test page, choose Calabash, and then choose Upload.

  8. Browse to and choose the .zip file that contains your tests. The .zip file must follow the format as described in Prepare Your iOS Calabash Tests.

  9. Choose Next step, and then complete the remaining on-screen instructions to select the devices to run your tests on and to then start the run.

Taking Screenshots in iOS Calabash Tests

You can take screenshots as part of your iOS Calabash tests.

iOS Calabash provides a predefined step for taking screenshots. For details, see the "Screenshots" section of the Predefined steps page in the Calabash iOS repository on GitHub.

Alternatively, you can define a custom step inside of a Ruby (.rb) file to call the screenshot_embed function, which creates a screenshot and saves it to a directory that you define. For example, the following code example creates a screenshot in PNG format and saves it to the /my/custom/path directory with a file name of screenshot_seconds-since-Epoch:

screenshot_embed(:prefix => "/my/custom/path", :name => "screenshot_#{}")

Additional Considerations for iOS Calabash Tests

Device Farm replaces certain Calabash hooks so that iOS Calabash tests will run on devices in Device Farm, but Device Farm does not modify iOS Calabash tests themselves.