Comprehensive Guide to Cross Browser Testing with Cypress

With Cypress, you can run tests across multiple browsers. Cypress currently has support for Chrome-family browsers (including Electron) and the beta support for Firefox browsers.

Aside Electron, any browser that you want to run Cypress tests in has to be installed on your local machine or CI environment. A full list of the detected browsers in your system will be displayed within the browser selection menu of the Test Runner.

You can specify the desired browser using the –browser flag when using the run command to launch Cypress. For instance, if you want to run Cypress tests in Firefox, you will need to run

Cypress run --browser firefox

If you want to make the launching of Cypress with a specific browser more convenient, you can use npm scripts to create a shortcut:

"scripts": {
  "cy:run:chrome": "cypress run --browser chrome",
  "cy:run:firefox": "cypress run --browser firefox"
}

Continuous Integration Strategies

Whenever you are incorporating testing of multiple browsers within your QA process, you will have to implement a CI strategy that provides an optimal level of confidence while taking into consideration the test duration and the infrastructure costs. This optimal strategy varies by the type and the needs of a particular project. This tutorial will present you with several strategies that you can consider when crafting the strategy for your project.

The CI strategies that will be demonstrated in this tutorial will use the Circle CI Cypress Orb for its concise and readable configuration, however, these same concepts apply for most CI providers.

The CI configuration examples in this tutorial, will use the Cypress Docker images to provision testing environments with desired versions of Node, Chrome and Firefox.

A. Periodic Basics

It is generally designed that you run tests with each pushed commit, however, you may not be necessary to do so for all browsers. For instance, you can choose to run tests within Chrome for each commit, but decide to only run Firefox on a periodic basis (i.e nightly). The periodic frequency depends on the scheduling of your releases, hence, you should consider a test run frequency that is appropriate for the release schedule of your project.

The example below will demonstrate a nightly CI schedule against production (master branch (for Firefox:

'''version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  nightly:
    triggers:
      - schedule:
          cron: "0 0 * * *"
          filters:
            branches:
              only:
                - master
    jobs:
      - cypress/run:
          executor: cypress/browsers-chrome73-ff68
          browser: firefox
          start: npm start
          wait-on: http://localhost:3000
'''

B. Production Deployment

In cases where you have projects that exhibit a consistently stable before across browsers, it may be better for you to run tests against additional browsers only before merging changes in the production deployment branch.

The example below demonstrates how to only run Firefox tests when commits are merged into a specific branch (develop branch in this case) so that any potential Firefox issue can be caught before a production release:

'''version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  test_develop:
    jobs:
      - filters:
        branches:
          only:
            - develop
      - cypress/run:
          executor: cypress/browsers-chrome73-ff68
          browser: firefox
          start: npm start
          wait-on: http://localhost:3000
'''

C. Subset of Tests

You can decide to only run a subset of tests against a given browser. For instance, you can decide to only execute the happy or critical paths related to test files, or a directory of specific “smoke” test files. It is not always necessary for you to have both browsers always running all tests.

In the following example, the Chrome cypress/run job will run all the tests against Chrome and then report the results to the Cypress Dashboard using a (group) that is named chrome.

The Firefox cypress/run job will run a subset of tests that are defined in the spec parameter, against the Firefox browser, then, it will report the results to the Cypress Dashboard under rhe group firefox-critical-path.

'''version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/install
      - cypress/run:
          name: Chrome
          requires:
            - cypress/install
          executor: cypress/browsers-chrome73-ff68
          start: npm start
          wait-on: http://localhost:3000
          record: true
          group: chrome
          browser: chrome
      - cypress/run:
          name: Firefox
          requires:
            - cypress/install
          executor: cypress/browsers-chrome73-ff68
          start: npm start
          wait-on: http://localhost:3000
          record: true
          group: firefox-critical-path
          browser: firefox
          spec: "cypress/integration/signup.spec.js,cypress/integration/login.spec.js"
'''

D. Parallelize per browser

You can parallelize the execution of test files on a per group basis, where the files can be grouped by the browser that is under test.  This versatility gives you the ability to allocate the desired amount of CI resources towards a browser to either improve the test duration or to minimize the CI costs.

You don’t need to run all the browsers at the same parallelization level. In the example shown below, the Chrome dedicated cypress/run job will run all the tests in parallel, across 4 machines, against Chrome and then it will report the results to the Cypress Dashboard under the group name chromium. The Firefox dedicated cypress/run job will run a subset of tests in parallel, across 2 machines, defined by the spec parameter, against the Firefox browser and then it will report the result to the Cypress Dashboard under the group named firefox.

'''version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/install
      - cypress/run:
          name: Chromium
          requires:
            - cypress/install
          executor: cypress/browsers-chromium73-ff68
          record: true
          start: npm start
          wait-on: http://localhost:3000
          parallel: true
          parallelism: 4
          group: chromium
          browser: chromium
      - cypress/run:
          name: Firefox
          requires:
            - cypress/install
          executor: cypress/browsers-chrome73-ff68
          record: true
          start: npm start
          wait-on: http://localhost:3000
          parallel: true
          parallelism: 2
          group: firefox
          browser: firefox
          spec: "cypress/integration/app.spec.js,cypress/integration/login.spec.js,cypress/integration/about.spec.js"
'''

Running Specific Tests by Browser

Sometimes, there may be instances where it is useful to run or ignore one or more tests when in specific browsers. For instance, you can reduce test run duration by only running smoke-tests against Chrome and not Firefox. This type of granular selection of test execution will depend on the type of tests and the level of confidence those specific tests provide to the overall project.

It is possible to specify a browser to run or exclude by passing a matcher to the suite or test within the test configuration. The browser option will accept the same arguments as Cypress.isBrowser().

'''// Runs the test if Cypress is run via Firefox
it('Download an extension in Firefox', { browser: 'firefox' }, () => {
  cy.get('#dl-extension')
    .should('contain', 'Download Firefox Extension')
})

// Runs the happy path tests if Cypress is run via Firefox
describe('the happy path suite', { browser: 'firefox' }, () => {
  it('...')
  it('...')
  it('...')
})

// Ignores test if Cypress is running via Chrome
// This test will not be recorded to the Cypress Dashboard
it('Will show warning outside Chrome', {  browser: '!chrome' }, () => {
  cy.get('.browser-warning')
    .should('contain', 'For optimal viewing, use Chrome browser')
})'''