How to create a workflow to execute Selenium tests

In modern software development, automated testing is crucial for ensuring the quality and stability of applications. Integrating Selenium, TestNG, and Allure for reporting, within the GitHub Actions CI/CD pipeline provides a robust and efficient way to automate testing and generate insightful reports.

Selenium and TestNG for automation

Selenium WebDriver is a widely used open-source tool for automating web browser interactions. It allows you to simulate real-user actions and test web applications across different browsers and platforms. TestNG complements Selenium by managing the execution of test cases, supporting annotations, dependencies, parallel execution, and more. This combination makes writing scalable and maintainable test suites more effective.

GitHub Actions: A CI/CD solution

GitHub Actions is a continuous integration and deployment (CI/CD) platform, allowing developers to automate tasks like testing, building, and deploying applications directly from their GitHub repositories. By leveraging GitHub Actions, your Selenium and TestNG tests can be executed automatically after each code change, ensuring that no new issues are introduced into the codebase.

How GitHub Actions and runners work

GitHub Actions operates by running workflows, defined in YAML files, that automate tasks like testing, building, and deploying code. These workflows are triggered by events such as code pushes, pull requests, or manual invocations (e.g., workflow_dispatch). Each workflow consists of jobs and steps. The jobs are executed on runners, which are virtual machines (VMs) that run on GitHub's infrastructure or self-hosted environments.

Runners are the backbone of GitHub Actions, providing the execution environment for your jobs. By default, GitHub provides hosted runners with pre-configured environments, such as ubuntu-latest, windows-latest, and macos-latest. These runners come equipped with commonly used tools (e.g., Java, Node.js, Docker, etc.), which speeds up the setup of workflows. For greater flexibility, you can configure self-hosted runners, allowing you to run jobs on custom machines, offering more control over the environment and resource allocation.

Predefined actions and custom actions

GitHub Actions has a large marketplace of predefined actions, such as actions/checkout (to clone your repository), actions/setup-java (to install Java), and many more for various utilities. These actions are reusable and maintained by the GitHub community or third-party providers. They simplify workflow creation by abstracting common tasks.

If no existing action meets your needs, you can create custom actions. Custom actions can be written in JavaScript or as Docker containers. By defining the behavior and inputs in an action.yml file, you can share them across multiple repositories or the community through the GitHub Marketplace.

Infrastructure and scalability

GitHub's infrastructure is cloud-based, providing automatic scalability for workflows. Runners spin up dynamically for each job, ensuring efficient resource utilization. GitHub manages security updates and infrastructure, so developers can focus on writing code and tests without worrying about the underlying infrastructure. Self-hosted runners offer scalability and flexibility, letting organizations fine-tune their environments to specific needs, such as network configurations or resource-intensive tasks.

Why Use GitHub Actions for Testing?

• Automation: Once set up, tests can be triggered automatically on every pull request, push, or manually through a workflow dispatch event. This ensures early detection of issues.
• Parallelism and Speed: GitHub Actions can run jobs in parallel, reducing the time taken for your tests to complete. With TestNG’s built-in support for parallel execution, this further speeds up the process.
• Cross-Platform: GitHub Actions supports running tests on multiple operating systems, including Linux, Windows, and macOS. This ensures your application works across different environments.
• Customizable Workflows: You can create custom workflows for your specific use case. For instance, in the case of Selenium, you can configure the pipeline to run tests in headless mode or use specific browser versions.

Integrating Selenium, TestNG, and Allure with GitHub Actions

To integrate Selenium-based testing with GitHub Actions:

• Define Your Test Suite: In your Maven or Gradle project, define your TestNG test suite and set up your Selenium WebDriver. You can use tools like WebDriverManager to handle WebDriver binaries.
• Configure Allure Reporting: Ensure Allure is configured in your test suite by adding the necessary dependencies in your pom.xml (for Maven) or build.gradle (for Gradle). Make sure to output Allure results in a specified folder, e.g., target/allure-results.
• Create GitHub Actions workflow: Create a .github/workflows/selenium-tests.yml file in your repository.
• Use the latest Ubuntu LTS image for running tests and configure steps for setting up Java, Maven/Gradle, and browsers. Tag latest will provide this image.
• Execute the tests using mvn test or gradle test.
• After test execution, generate and upload the Allure report, which can be viewed as an artifact or published via GitHub Pages or another hosting platform, maybe on Netlify.

Allure: The ultimate test reporting framework

Allure is a widely recognized test reporting framework that provides comprehensive, visually appealing, and detailed reports for automated test results. It captures essential data like step details, screenshots, logs, and even video recordings (for failed test cases), offering detailed insights into test execution. Whether you’re using Selenium, TestNG, JUnit, or another testing framework, Allure can integrate seamlessly into your testing suite, transforming raw test data into insights.

Why Allure is considered one of the best reporting frameworks

• User-Friendly and Visual Reports: Allure's standout feature is its visually rich and intuitive reports. It generates clean, interactive HTML reports that allow developers, testers, and stakeholders to navigate through test results easily. The user-friendly interface includes sections for: Test status (passed, failed, skipped); Step-by-step execution traces; Screenshots, logs, and video recordings (for failed tests); Graphical overviews of test execution
• Comprehensive Test Information: Allure captures detailed information about test execution, making debugging easier. It logs each step of the test, including inputs, outputs, and any assertions made. This level of detail can save significant time when investigating failures or analyzing test trends.
• Customization and Extensibility: Allure provides great flexibility. You can customize your reports by adding custom labels, parameters, and categories. With support for tagging, you can easily classify tests, track different environments, or isolate tests based on features or teams.
• Integration with Multiple Testing Frameworks: Allure integrates with a wide range of testing frameworks like TestNG, JUnit, Cucumber, Pytest, and more. This makes it an ideal choice for teams using different tech stacks but looking for consistent reporting across projects.
• Historical Tracking: Allure allows you to maintain historical data in your reports. You can view trends over time, track improvements, and compare test results from previous runs, which is invaluable for regression testing and long-term project analysis.

Conclusion

Integrating Selenium, TestNG, and Allure with GitHub Actions automates your testing process and ensures that your code remains stable with every change. The power of GitHub Actions lies in its seamless integration with your repository, the flexibility to customize workflows, and the ability to run tests across different environments. Combined with Allure’s insightful reports, this setup provides an efficient and effective solution for continuous testing.

Demo

Check out a working demo here, what you can also execute! Just click on Actions tab and select workflow!
SUT: A publicly available demo site - https://www.saucedemo.com

thisisnotka / selenium-allure-demo-project

This repository contains a simple demo project to showcase Selenium WebDriver with the Page Object Model (POM) and Allure reporting for visualization of test results.

Selenium TestNG Project with Allure Reporting

This repository contains a demo project showcasing Selenium WebDriver with the Page Object Model (POM) and Allure reporting for visualization of test results.

Prerequisites

Before you start, make sure you have the following installed:

• Java Development Kit (JDK): Download and install JDK
• Maven: Download and install Maven
• Allure: Install Allure
• IntelliJ IDEA: Download and install IntelliJ IDEA

Setup and Execution

Follow these steps to set up and run the project:

1. Clone the Repository

2. Open the Project

   Launch IntelliJ IDEA. Open the project by selecting the cloned repository directory.

3. Run Tests

   Type command to execute testSuite.xml

mvn clean install

4. Open Allure Report on localhost

   Open the terminal in IntelliJ IDEA. Navigate to the project directory if not already there. Execute the following command to serve the Allure report:

allure serve allure-results

This will start a local server on a random port…

View on GitHub

Report is deployed here after successful test execution: https://generated-allure-reports.netlify.app/