Troubleshooting Device Farm desktop browser testing

In this section, you'll find some common problems you can run into and how to address them.

Can't connect to a RemoteWebDriver

You might see this error when you have an expired URL from CreateTestGridUrl. The URL is valid only for a short time, as specified in the API call. Even if a session is created before the expiry, it cannot be accessed after the expiration of the RemoteWebDriver URL.

Timeouts when setting up RemoteWebDriver

If your testing framework enforces strict timeouts for test completion, you might see tests time out when setting up a new session. When you start a session, there is a 60 to 120 delay between the request to create the session and its availability for use. In situations where you cannot increase timeouts, we recommend that you request a session and open a page before your test framework sets up your tests, and then re-use that session for your test suite. Make sure to close your session when your tests are complete.

Rate limit errors during session creation

You have too many open sessions, are opening too many sessions per second, or are making too many WebDriver requests per second. This is common in large, highly parallel test suites where many browsers are being controlled at once.

For more information, see Quotas in Device Farm desktop browser testing.

Session creation rate management

If you are opening more than 5 sessions at a time, we recommend adding a random delay before your session creation. A uniformly random delay of 0.5 to 5 seconds will stagger your session creation and avoid rate limiting during this time.

Actions per second rate management

To mitigate the rate limit of 150 actions per second across WebDriver sessions, you can use a custom CommandExecuter that automatically limits the rate of outgoing requests. The following example demonstrates how to accomplish this:

Java

import org.openqa.selenium.remote.HttpCommandExecutor;
import org.openqa.selenium.remote.Response;
import org.openqa.selenium.remote.Command;
import java.io.IOException;
import java.time.Instant;
import java.util.concurrent.TimeUnit;

class RateLimitExecutor extends HttpCommandExecutor {

    public static final int CONCURRENT_SESSIONS = 100;
    public static final int ACTIONS_RATE_LIMIT_PER_SECOND = 150;

    public static final double SECONDS_PER_ACTION = ((double) CONCURRENT_SESSIONS)
            / ((double) ACTIONS_RATE_LIMIT_PER_SECOND);
    private long lastExecutionTime;

    public RateLimitExecutor(URL addressOfRemoteServer) {
        super(addressOfRemoteServer);
        lastExecutionTime = 0;
    }

    public Response execute(Command command) throws IOException {
        long currentTime = Instant.now().toEpochMilli();
        double elapsedTime = TimeUnit.MILLISECONDS.toSeconds(currentTime - lastExecutionTime);
        if (elapsedTime < SECONDS_PER_ACTION) {
            try {
                Thread.sleep(TimeUnit.SECONDS.toMillis((long)(SECONDS_PER_ACTION - elapsedTime)));
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        lastExecutionTime = Instant.now().toEpochMilli();
        return super.execute(command);
    }
}
     

When calling the constructor for the remote WebDriver, use an instance of this new class as the command_executor parameter:

RemoteWebDriver driver = new RemoteWebDriver(new RateLimitExecutor(testGridUrl), new FirefoxOptions());

Python

CONCURRENT_SESSIONS = 100
ACTIONS_RATE_LIMIT_PER_SECOND = 150

class RateLimitExecutor(RemoteConnection):

    def __init__(self, *args):
        super().__init__(*args)
        self.last_execution_time = None
        self.seconds_per_action = CONCURRENT_SESSIONS / ACTIONS_RATE_LIMIT_PER_SECOND

    def execute(self, *args):
        if self.last_execution_time:
            current_time = time.time()
            elapsed_time = current_time - self.last_execution_time
            if elapsed_time < self.seconds_per_action:
                time.sleep(self.seconds_per_action - elapsed_time)
        self.last_execution_time = time.time()
        return super().execute(*args)

When calling the constructor for the remote WebDriver, use an instance of this new class as the command_executor parameter:

driver = webdriver.Remote(command_executor=CustomCommandExecutor(test_grid_url), desired_capabilities=DesiredCapabilities.CHROME)

I get errors during session creation

If you get errors during session creation or are getting a browser that you don't expect, check that your requested capabilities are valid. Make sure:

• the browserName capability contains a supported browser name

• the browserName capability value is lower case

• the browserVersion capability is set to latest, latest-1, or latest-2

• the platform capability is set to a supported platform

• the URL of the WebDriver hub has not expired

For more information, see

• Supported browsers

• Supported platforms

Browser can't connect to my app

You might see this in the following cases:

• Verify the DNS hostname that is being used to connect to your application. If you are using Amazon VPC, verify that private DNS is enabled and configured properly.

• If you are using a Amazon VPC configuration, verify that the correct subnets and security groups have been applied and that the specified VPC is in the us-west-2 region. Additionally, if your application requires access to the internet, for example to access a CDN, those resources must be accessible from within the VPC or through a NAT gateway. For more information on NAT gateways, see NAT gateways in the Amazon VPC user guide.

• You must not block traffic from Amazon EC2 or other AWS services.

Selenium 3 WebDriver testing framework is not working properly with Microsoft Edge (Chromium)

Selenium 3 requires the Selenium Tools for Microsoft Edge package to be able to support the Edge browser. For more information, see Using Selenium 3.

Note that Selenium 4 contains built-in support for Edge.

My tests using XPath are slow

Due to the design of Selenium and the desktop browser testing feature, tests using XPath are slow if they are not optimized properly.

If you use find_elements_by_xpath, consider the following optimizations:

• Use an ID selector instead of XPath.

• In Python, avoid looping over the results multiple times or casting the results to a list, using map, or other late-binding iterators without caching the results. Use str(your-element.getAttribute(…)) when requesting strings.

• If you are using the list multiple times, cache the results if possible.

Selenium makes a full round-trip web request for each action you take on the returned elements on an XPath query. For example, if you request every link in a page (//a) and want to validate that every link points to a real resource, each call to getAttribute from your test results in a full round-trip request to the desktop browser testing feature running your test and the WebDriver automating your browser, even if the content has not changed.