AWS Device Farm
Developer Guide (API Version 2015-06-23)

Working with Calabash for iOS and AWS Device Farm

Device Farm provides support for Calabash for iOS. The following information describes how to use this test framework with Device Farm test types.

What is Calabash?

Calabash is a mobile testing framework that you can use to run automated user interface acceptance tests that are written in Cucumber on iOS apps. For more information, see the Welcome to Calabash iOS repository on GitHub.

Version Information

Currently, Device Farm supports Calabash version 0.20.5.

Prepare Your iOS Calabash Tests

Your iOS Calabash tests must be contained in a .zip file with the following structure:

my-zip-file-name.zip `-- features (directory) |-- my-feature-1-file-name.feature |-- my-feature-2-file-name.feature |-- my-feature-N-file-name.feature |-- step_definitions (directory) | `-- (.rb files) |-- support (directory) | `-- (.rb files) `-- (any other supporting files)

Upload Your iOS Calabash Tests

Use the Device Farm console to upload your tests.

  1. Sign in to the Device Farm console at https://console.aws.amazon.com/devicefarm.

  2. In the list of projects, choose the option next to the project where you want to upload your tests.

    Tip

    If the list of projects is not displayed, then on the secondary navigation bar, for Projects, choose the name of the project that you want to upload your tests to.

    To create a project, follow the instructions in Create a Project.

  3. If the Create a new run button is displayed, choose it.

  4. On the Choose your application page, choose Upload.

  5. Browse to and choose your iOS app file. The file must be an .ipa file.

    Note

    Make sure that your .ipa file is built for an iOS device and not for a simulator.

  6. Choose Next step.

  7. On the Configure a test page, choose Calabash, and then choose Upload.

  8. Browse to and choose the .zip file that contains your tests.

  9. Choose Next step, and then complete the remaining instructions to select the devices to run your tests on and start the run.

Taking Screenshots in iOS Calabash Tests

You can take screenshots as part of your iOS Calabash tests.

iOS Calabash provides a predefined step for taking screenshots. For information, see the "Screenshots" section of the Predefined steps page in the Calabash iOS repository on GitHub.

Alternatively, you can define a custom step inside of a Ruby (.rb) file to call the screenshot_embed function, which creates a screenshot and saves it to a directory that you define. For example, the following code example creates a screenshot in PNG format and saves it to the /my/custom/path directory with a file name of screenshot_seconds-since-Epoch:

screenshot_embed(:prefix => "/my/custom/path", :name => "screenshot_#{Time.now.to_i}")

Additional Considerations for iOS Calabash Tests

Device Farm replaces certain Calabash hooks so that iOS Calabash tests can run on devices in Device Farm, but Device Farm does not modify iOS Calabash tests themselves.