cypress parallel without dashboard

Introduction

In this article, you’ll learn a method for efficient testing with Cypress:

Parallel tests without Cypress Cloud (Dashboard) using cypress-split.
Consolidated report for failed Cypress tests with screenshots & videos.
Slack notifications for test runs.

Note: We’ve utilized the handy set of working Cypress code snippets from our previous hashnode blog.

Ready to see how we can accomplish all of these in GitHub Actions? Let’s go!

Configuring GitHub Actions:

If you’re not familiar with GitHub Actions, take a look at this reference.

Cypress Split Overview

The Cypress Split plugin, maintained by Gleb Bahmutov, divides Cypress test suites into smaller chunks to run tests in parallel and boost machine processing power.

It comes with features such as test retrying and results merging, simplifying issue detection and resolution.

Note: The cypress-split plugin is also compatible with other CI providers (Github Actions, Gitlab CI, Circle CI)

This plugin is a valuable addition to large-scale Cypress test suites and enhances testing efficiency.

Main Workflow

This is a main YAML file defining a GitHub Actions workflow named "cypress-split-slack" that is triggered on every push event to the repository.

The workflow contains one job named "split" that utilizes a reusable workflow defined in a separate file named reusable.yml located in the .github/workflows directory of the repository (will elaborate in the upcoming points).

# .github/workflows/parallel.yml
name: cypress-split-slack
on: [push]
jobs:
  split:
    uses: ./.github/workflows/reusable.yml
    with:
      n: 6
      browser: chrome
      marge: true
      slack: true
    secrets:
      SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
      SLACK_TOKEN: ${{ secrets.SLACK_TOKEN }} 
      SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
      SLACK_UID: ${{ secrets.SLACK_UID }}

The "split" job has the following inputs:

"n": an integer that specifies the number of parts to split the test suite into.
"browser": a string that specifies the browser to use for running the tests. In this case, it is set to "chrome".
"marge": a boolean value that indicates whether or not to merge the test results.

Note: To attach screenshots & videos to the mochawesome report on failed cases, take a look at this blog.
"slack": a boolean value that indicates whether or not to send test results to a Slack channel.

In addition, the workflow uses four secrets that are defined in the repository's settings and injected into the workflow environment:

"SLACK_WEBHOOK_URL": a URL that specifies the Slack webhook to use for sending notifications.
"SLACK_TOKEN": a token that is used to authenticate with the Slack API.
"SLACK_CHANNEL_ID": an identifier for the Slack channel to send notifications to.
"SLACK_UID": the slack user/member ID.

Please take a look at this page to learn more about Slack API.

This workflow is likely intended for running Cypress tests in parallel for free using a test-splitting tool. The results are also reported to a Slack channel if the "slack" input is set to "true."

Reusable Workflow

We leveraged the 'bahmutov/cypress-workflows' repository in our project, which provided us with pre-built reusable Cypress workflows to enhance our testing process.

Please visit this page to learn about reusable workflows in GitHub Actions.

Note: The article repository is at the end of this blog, where you can find the .github/workflows/reusable.yml file.

A reusable GitHub Actions workflow named "split" installs NPM dependencies and runs Cypress tests across multiple machines using the cypress-split.
The workflow is triggered by workflow_call, and accepts inputs for parallel containers, config values, browser, and whether to run tests.
Workflow has three jobs:
1. "prepare" creates a matrix of container configs
2. "tests" run tests in parallel on multiple containers using a matrix
3. "report" supports features like debug inputs, artifact storage, Mochawesome merging, and Slack notifications (additional one)

Let us walk you through the changes we made to the reusable workflow that notifies Slack for each test run. These updates provide us with information about the status of test runs and even post screenshots of failed test cases.

Initially, we update a JSON file known as report.json using the environment variables. Subsequently, we utilize this updated JSON to publish it to a reporting service named test-results-reporter in the next step.

Note: This step ensures that private data is stored safely and also simplifies codebase maintenance tasks.

For more information on GitHub secrets, visit this page.

 env:
   # Points to the current workflow run's page
   RUN_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
   SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
   SLACK_UID: ${{ secrets.SLACK_UID }}
   SLACK_TOKEN: ${{ secrets.SLACK_TOKEN }}
   SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}

It uses the jq command-line tool to update specific fields in the JSON based on the values of the environment variables. The resulting JSON is saved to a temporary file called report-temp.json.

 - name: Update json file with environment variables
   run: |
      jq --arg run_url "$RUN_URL" \
         --arg slack_webhook_url "$SLACK_WEBHOOK_URL" \
         --arg slack_uid "$SLACK_UID" \            '.reports[].targets[].extensions[]?.inputs?.links[]? |= if .url == "$RUN_URL" then .url = $run_url else . end | 
         .reports[].targets[].inputs? |= if .url == "$SLACK_WEBHOOK_URL" then .url = $slack_webhook_url else . end |                    .reports[].targets[].extensions[]?.inputs?.users[]? |= if .slack_uid == "$SLACK_UID" then .slack_uid = $slack_uid else . end' report.json > report-temp.json

We use test-results-reporter to generate a report of the test results and publish it to slack, including @mentions, a dynamic build log link, and browser details.

These configurations are specified in report.json in the blog's repository.
```
 - name: Slack notification
   run: |
     npx test-results-reporter publish -c report-temp.json
```
In this step, we employ the GitHub Action called file-existence-action to examine whether any failed test screenshots are present in the directory mochawesome/screenshots. If true, the step proceeds; otherwise, it skips.
```
 - name: Check failed tests screenshots exists
   id: failed_screenshots
   uses: andstor/file-existence-action@v2
   with:
     files: "mochawesome/screenshots/**/*.png"
```
Uploads the failed test screenshots to a Slack channel if any are found. This step uses a webhook URL to post a message in the channel with a notification about the failed test screenshots.

It then creates a local directory called failed_screenshots and copies the screenshots to that directory. Finally, it runs a custom Node.js script called slack.js (will see in the following points) to upload the screenshots to the Slack channel.
```
 - name: Upload failed tests screenshots to slack channel
   if: steps.failed_screenshots.outputs.files_exists == 'true'
   run: |
     curl -X POST -H 'Content-type: application/json' --data '{"text":"*_See the failed test screenshots ❌_*"}' $SLACK_WEBHOOK_URL
     mkdir failed_screenshots
     cp -r mochawesome/screenshots/**/*.png failed_screenshots
     npx node slack.js
```

Preserve the merged-mochawesome-report as an artifact so that, if necessary, the team can access it.

 - uses: actions/upload-artifact@v3
   with:
     name: merged-mochawesome-report
     path: mochawesome

Post failed test screenshots to Slack

Upload failed test screenshots to Slack channel using Node.js and @slack/web-api

// slack.js
require('dotenv').config();
const { createReadStream } = require('fs');
const { WebClient } = require('@slack/web-api');
const token = process.env.SLACK_TOKEN;
const channelId = process.env.SLACK_CHANNEL_ID;
const web = new WebClient(token);
var fs = require('fs');
var path = require('path');
var dirPath = path.resolve('failed_screenshots');
var filesList;
fs.readdir(dirPath, function (err, files) {
  filesList = files.filter(function (e) {
    return path.extname(e).toLowerCase() === '.png';
  });
  for (const type of filesList) {
    console.log(`${type}`);
    const uploadFileToSlack = async () => {
      await web.files.upload({
        filename: 'Failed Tests',
        file: createReadStream(`failed_screenshots/${type}`),
        channels: channelId,
      });
    };
    uploadFileToSlack();
  }
});

This code uploads all the PNG files in a directory called failed_screenshots to a Slack channel.
It uses the dotenv library to load environment variables from a .env file, including a Slack API token and a Slack channel ID.
It reads the contents of the failed_screenshots directory using the fs module filters the list to only include PNG files and then uploads each file to the specified Slack channel using the @slack/web-api module.

End Results

This is how our Slack notification appears.

Generated GitHub Actions summary at run-time:

HTML report with attached failed case screenshots and videos:

Key Takeaways

Cypress Split speeds up test runs by distributing them across multiple machines.
Slack notifications keep the team informed of changes and issues in real-time.
Using these tools can help you catch bugs and issues earlier in the development process, leading to better-quality software and a smoother release process.

Overall, incorporating these into your CI/CD pipelines can help you streamline your testing process and improve software quality, increasing customer satisfaction and confidence.

Time to wrap up!

We learned how to efficiently execute a painless Cypress test execution in Github actions with parallel execution and attractive reporting. This makes it easier for Project Managers and non-technical people to understand the results.

Article source code: https://github.com/axelerant/cypress-parallel-slack-report

We hope you found this blog helpful!

References:

Unlocking Free Cypress Parallelization And Automated Slack Reporting with Github Actions

Table of contents