Version support Configure your Appium test package Create a zipped package file Upload your test package to Device Farm Take screenshots of your tests (Optional)

Working with Appium and AWS Device Farm

This section describes how to configure, package, and upload your Appium tests to Device Farm. Appium is an open source tool for automating native and mobile web applications. For more information, see Introduction to Appium on the Appium website.

For a sample app and links to working tests, see Device Farm Sample App for Android and Device Farm Sample App for iOS on GitHub.

Version support

Support for various frameworks and programming languages depends on the language used.

Device Farm supports all Appium server versions 1.6.5 and above. You can choose any Appium version by using the avm command. For example, to use Appium server version 1.9.1, add these commands to your test spec YAML file:


phases:
  install:
    commands:
      # To install a newer version of Appium such as version 1.9.1:
      - export APPIUM_VERSION=1.9.1
      - avm $APPIUM_VERSION
      - ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium  /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js

To use all the features of the framework, like annotations, choose a custom test environment, and use the AWS CLI or the Device Farm console to upload a custom test spec.

Topics

Configure your Appium test package
Create a zipped test package file
Upload your test package to Device Farm
Take screenshots of your tests (Optional)

Configure your Appium test package

Use the following instructions to configure your test package.

Java (JUnit)

Modify pom.xml to set packaging to a JAR file:


<groupId>com.acme</groupId>
<artifactId>acme-myApp-appium</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

Modify pom.xml to use maven-jar-plugin to build your tests into a JAR file.

The following plugin builds your test source code (anything in the src/test directory) into a JAR file:


<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-jar-plugin</artifactId>
  <version>2.6</version>
  <executions>
    <execution>
      <goals>
        <goal>test-jar</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Modify pom.xml to use maven-dependency-plugin to build dependencies as JAR files.

The following plugin copies your dependencies into the dependency-jars directory:


<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <version>2.10</version>
  <executions>
    <execution>
      <id>copy-dependencies</id>
      <phase>package</phase>
      <goals>
        <goal>copy-dependencies</goal>
      </goals>
      <configuration>
        <outputDirectory>${project.build.directory}/dependency-jars/</outputDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

Save the following XML assembly to src/main/assembly/zip.xml.

The following XML is an assembly definition that, when configured, instructs Maven to build a .zip file that contains everything in the root of your build output directory and the dependency-jars directory:


<assembly
    xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
  <id>zip</id>
  <formats>
    <format>zip</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <fileSets>
    <fileSet>
      <directory>${project.build.directory}</directory>
      <outputDirectory>./</outputDirectory>
      <includes>
        <include>*.jar</include>
      </includes>
    </fileSet>
    <fileSet>
      <directory>${project.build.directory}</directory>
      <outputDirectory>./</outputDirectory>
      <includes>
        <include>/dependency-jars/</include>
      </includes>
    </fileSet>
  </fileSets>
</assembly>

Modify pom.xml to use maven-assembly-plugin to package tests and all dependencies into a single .zip file.

The following plugin uses the preceding assembly to create a .zip file named zip-with-dependencies in the build output directory every time mvn package is run:


<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <version>2.5.4</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>single</goal>
      </goals>
      <configuration>
        <finalName>zip-with-dependencies</finalName>
        <appendAssemblyId>false</appendAssemblyId>
        <descriptors>
          <descriptor>src/main/assembly/zip.xml</descriptor>
        </descriptors>
      </configuration>
    </execution>
  </executions>
</plugin>

Note

If you receive an error that says annotation is not supported in 1.3, add the following to pom.xml:


<plugin>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.7</source>
    <target>1.7</target>
  </configuration>
</plugin>

Java (TestNG)

Modify pom.xml to set packaging to a JAR file:


<groupId>com.acme</groupId>
<artifactId>acme-myApp-appium</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

Modify pom.xml to use maven-jar-plugin to build your tests into a JAR file.

The following plugin builds your test source code (anything in the src/test directory) into a JAR file:


<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-jar-plugin</artifactId>
  <version>2.6</version>
  <executions>
    <execution>
      <goals>
        <goal>test-jar</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Modify pom.xml to use maven-dependency-plugin to build dependencies as JAR files.

The following plugin copies your dependencies into the dependency-jars directory:


<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <version>2.10</version>
  <executions>
    <execution>
      <id>copy-dependencies</id>
      <phase>package</phase>
      <goals>
        <goal>copy-dependencies</goal>
      </goals>
      <configuration>
        <outputDirectory>${project.build.directory}/dependency-jars/</outputDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

Save the following XML assembly to src/main/assembly/zip.xml.


<assembly
    xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
  <id>zip</id>
  <formats>
    <format>zip</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <fileSets>
    <fileSet>
      <directory>${project.build.directory}</directory>
      <outputDirectory>./</outputDirectory>
      <includes>
        <include>*.jar</include>
      </includes>
    </fileSet>
    <fileSet>
      <directory>${project.build.directory}</directory>
      <outputDirectory>./</outputDirectory>
      <includes>
        <include>/dependency-jars/</include>
      </includes>
    </fileSet>
  </fileSets>
</assembly>

Modify pom.xml to use maven-assembly-plugin to package tests and all dependencies into a single .zip file.

The following plugin uses the preceding assembly to create a .zip file named zip-with-dependencies in the build output directory every time mvn package is run:


<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <version>2.5.4</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>single</goal>
      </goals>
      <configuration>
        <finalName>zip-with-dependencies</finalName>
        <appendAssemblyId>false</appendAssemblyId>
        <descriptors>
          <descriptor>src/main/assembly/zip.xml</descriptor>
        </descriptors>
      </configuration>
    </execution>
  </executions>
</plugin>

Note

If you receive an error that says annotation is not supported in 1.3, add the following to pom.xml:


<plugin>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.7</source>
    <target>1.7</target>
  </configuration>
</plugin>

Node.JS

To package your Appium Node.js tests and upload them to Device Farm, you must install the following on your local machine:

Node Version Manager (nvm)

Use this tool when you develop and package your tests so that unnecessary dependencies are not included in your test package.
Node.js
npm-bundle (installed globally)

Verify that nvm is present
```
command -v nvm
```
You should see nvm as output.

For more information, see nvm on GitHub.
Run this command to install Node.js:
```
nvm install node
```
You can specify a particular version of Node.js:
```
nvm install 11.4.0
```
Verify that the correct version of Node is in use:
```
node -v
```
Install npm-bundle globally:
```
npm install -g npm-bundle
```

Python

We strongly recommend that you set up Python virtualenv for developing and packaging tests so that unnecessary dependencies are not included in your app package.
```
$ virtualenv workspace
$ cd workspace
$ source bin/activate
```
Tip
- Do not create a Python virtualenv with the --system-site-packages option, because it inherits packages from your global site-packages directory. This can result in including dependencies in your virtual environment that are not required by your tests.
- You should also verify that your tests do not use dependencies that are dependent on native libraries, because these native libraries might not be present on the instance where these tests run.
Install py.test in your virtual environment.
```
$ pip install pytest
```
Install the Appium Python client in your virtual environment.
```
$ pip install Appium-Python-Client
```
Unless you specify a different path in custom mode, Device Farm expects your tests to be stored in tests/. You can use find to show all files inside a folder:
```
$ find tests/
```
Confirm that these files contain test suites you wand to run on Device Farm
```
tests/
tests/my-first-tests.py
tests/my-second-tests/py
```
Run this command from your virtual environment workspace folder to show a list of your tests without running them.
```
$ py.test --collect-only tests/
```
Confirm the output shows the tests that you want to run on Device Farm.

Clean all cached files under your tests/ folder:


$ find . -name '__pycache__' -type d -exec rm -r {} +
$ find . -name '*.pyc' -exec rm -f {} +
$ find . -name '*.pyo' -exec rm -f {} +
$ find . -name '*~' -exec rm -f {} +

Run the following command in your workspace to generate the requirements.txt file:
```
$ pip freeze > requirements.txt
```

Ruby

To package your Appium Ruby tests and upload them to Device Farm, you must install the following on your local machine:

Ruby Version Manager (RVM)

Use this command-line tool when you develop and package your tests so that unnecessary dependencies are not included in your test package.
Ruby
Bundler (This gem is typically installed with Ruby.)

Install the required keys, RVM, and Ruby. For instructions, see Installing RVM on the RVM website.

After the installation is complete, reload your terminal by signing out and then signing in again.

Note
RVM is loaded as a function for the bash shell only.
Verify that rvm is installed correctly
```
command -v rvm
```
You should see rvm as output.
If you want to install a specific version of Ruby (for example, 2.5.1) run the following command:
```
rvm install ruby 2.5.1 --autolibs=0
```
Verify that you are on the requested version of Ruby:
```
ruby -v
```
The bundler gem is usually installed by default. If it is not, install it:
```
gem install bundler
```

Create a zipped test package file

Warning

In Device Farm, the folder structure of files in your zipped test package matters, and some archival tools will change the structure of your ZIP file implicitly. We recommend that you follow the specified command-line utilities below rather than use the archival utilities built into the file manager of your local desktop (such as Finder or Windows Explorer).

Now, bundle your tests for Device Farm.

Java (JUnit)

Build and package your tests:


$ mvn clean package -DskipTests=true

The file zip-with-dependencies.zip will be created as a result. This is your test package.

Java (TestNG)

Build and package your tests:


$ mvn clean package -DskipTests=true

The file zip-with-dependencies.zip will be created as a result. This is your test package.

Node.JS

Check out your project.

Make sure you are at the root directory of your project. You can see package.json at the root directory.
Run this command to install your local dependencies.
```
npm install
```
This command also creates a node_modules folder inside your current directory.

Note
At this point, you should be able to run your tests locally.
Run this command to package the files in your current folder into a *.tgz file. The file is named using the name property in your package.json file.
```
npm-bundle
```
This tarball (.tgz) file contains all your code and dependencies.
Run this command to bundle the tarball (*.tgz file) generated in the previous step into a single zipped archive:
```
zip -r MyTests.zip *.tgz
```
This is the MyTests.zip file that you upload to Device Farm in the following procedure.

Python

Python 2

Generate an archive of the required Python packages (called a "wheelhouse") using pip:


$ pip wheel --wheel-dir wheelhouse -r requirements.txt

Package your wheelhouse, tests, and pip requirements into a zip archive for Device Farm:


$ zip -r test_bundle.zip tests/ wheelhouse/ requirements.txt

Python 3

Package your tests and pip requirements into a zip file:


$ zip -r test_bundle.zip tests/ requirements.txt

Ruby

Run this command to create a virtual Ruby environment:


# myGemset is the name of your virtual Ruby environment
rvm gemset create myGemset

Run this command to use the environment you just created:
```
rvm gemset use myGemset
```
Check out your source code.

Make sure you are at the root directory of your project. You can see Gemfile at the root directory.
Run this command to install your local dependencies and all gems from the Gemfile:
```
bundle install
```
Note
At this point, you should be able to run your tests locally. Use this command to run a test locally:
```
bundle exec $test_command
```

Package your gems in the vendor/cache folder.


# This will copy all the .gem files needed to run your tests into the vendor/cache directory
bundle package

Run the following command to bundle your source code, along with all your dependencies, into a single zipped archive:
```
zip -r MyTests.zip Gemfile vendor/ $(any other source code directory files)
```
This is the MyTests.zip file that you upload to Device Farm in the following procedure.

Upload your test package to Device Farm

You can use the Device Farm console to upload your tests.

Sign in to the Device Farm console at https://console.aws.amazon.com/devicefarm.
On the Device Farm navigation panel, choose Mobile Device Testing, then choose Projects.
If you are a new user, choose New project, enter a name for the project, then choose Submit.

If you already have a project, you can choose it to upload your tests to it.
Open your project, and then choose Create a new run.
For native Android and iOS tests

On the Choose application page, choose Mobile App, then select Choose File to upload your application's distributable package.

Note
The file must be either an Android .apk or an iOS .ipa. iOS Applications must be built for real devices, not the Simulator.

For Mobile Web application tests

On the Choose application page, choose Web App.
Give your test an appropriate name. This may contain any combination of spaces or punctuation.
Choose Next.
On the Configure page, in the Setup test framework section, choose Appium language, then Choose File.
Browse to and choose the .zip file that contains your tests. The .zip file must follow the format described in Configure your Appium test package.
Choose to either Run your test in our standard environment or Run your test in a custom environment. The standard environment has granular, per-test reporting, while the custom environment is capable of running custom test harnesses built on top of any framework.

Using the custom environment allows for full control over test setup, teardown, and invocation, as well as choosing specific versions of runtimes and the Appium server. Some language features (such as Python 3 or custom TestNG execution) are only available through the custom environment:
Java (JUnit)

JUnit runs in both the standard and custom environments without issue

Java (TestNG)

Invoking TestNG using a testng.xml configuration is only available in Custom Mode.

Node.JS

The Custom Mode allows you to specify additional parameters or use nonstandard invocations of your test framework.

Python
Python 2.7 is supported in both the standard environment and using Custom Mode. It is the default in both when specifying Python.

Python 3 is only supported in Custom Mode. To choose Python 3 as your python version, change the test spec to set the PYTHON_VERSION to 3, as shown here:
```
phases:
  install:
    commands:
      # ...
      - export PYTHON_VERSION=3
      - export APPIUM_VERSION=1.14.2
      # Activate the Virtual Environment that Device Farm sets up for Python 3, then use Pip to install required packages.
      - cd $DEVICEFARM_TEST_PACKAGE_PATH
      - . bin/activate
      - pip install -r requirements.txt
      # ...
       
```
Ruby
In Custom Mode, you may specify a specific version of Ruby with the rvm command:
```
phases:
  install:
    commands:
       - rvm install version  --autolibs=0
       - rvm use version
```
Choose Next, and then follow the instructions to select devices and start the run. For more information, see Create a test run in Device Farm.

Note

Device Farm does not modify Appium tests.

Take screenshots of your tests (Optional)

You can take screenshots as part of your tests.

Device Farm sets the DEVICEFARM_SCREENSHOT_PATH property to a fully qualified path on the local file system where Device Farm expects Appium screenshots to be saved. The test-specific directory where the screenshots are stored is defined at runtime. The screenshots are pulled into your Device Farm reports automatically. To view the screenshots, in the Device Farm console, choose the Screenshots section.

For more information on taking screenshots in Appium tests, see Take Screenshot in the Appium API documentation.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Test types and frameworks

Calabash