With multiple testing frameworks available, choosing which is best for your product becomes more challenging. While various testing tools are available in the market, Selenium has been a well-established player for quite some time. However, newer automation testing frameworks such as Cypress and Playwright have recently provided Selenium with serious competition. I have used Selenium and Cypress, and recently started using Playwright.
Every tool/framework has its pros and cons. When I switched from Selenium to Cypress, I found that Cypress was speedy and less flaky compared to Selenium. I saw test cases that had previously taken a long time and resulted in a failure, now executed well in the local environment and the CI/CD pipeline.
Playwright, comparatively a newer entrant in the market, is an open-source framework primarily used for end to end testing. Built by Microsoft, it supports prominent languages like Python, Java, JavaScript, C#, and more. It comprises features and capabilities more aligned with the modern web.
When a new user plans to develop a framework from scratch, choosing the right tool or framework can sometimes be tricky as they are trying to figure out which framework will work best for their project. As I started my career in automation, I used Java, which allowed me to leverage the benefits of the Selenium framework to automate web application testing.
Nevertheless, I am also facing many challenges with my current project at the moment. To solve these difficulties, I am forced to look into other tools to assist me.
Given that there are several considerations to take into account, including in-house technical expertise, ease of use, and CI/CD integration, it is important to determine which framework or tool would be the most suitable option based on these factors and others that are important in selecting the optimal solution. Needless to mention that the framework must let you run tests in a CI/CD pipeline.
The Cypress framework has been my go-to tool for the last three or four years. It offers many features (Automatic Waiting, Flake Detection, Network Traffic Control, Screenshots and Videos, Debuggability, etc.).
However, several alternatives to Selenium and Cypress are currently challenging them (one of which is Playwright, which offers a variety of functionalities).
In this Playwright vs Selenium vs Cypress blog, you will learn more about each tool/framework, and I’ll also compare Playwright, Selenium, and Cypress. The learnings of this Playwright vs Selenium vs Cypress blog will help you choose the ideal tool for your project.
Below are the npm trends of Playwright vs Selenium vs Cypress.
**Note: **I have used tools and frameworks interchangeably throughout this blog on Playwright vs Selenium vs Cypress.
What is Playwright?
Playwright is an open-source automation framework for end-to-end testing. It supports most modern rendering browsers, including Firefox, Chromium, and Webkit. Playwright is created and maintained by Microsoft and has steadily been gaining in popularity.
It is a Node library that automates the Chromium, WebKit, and Firefox browsers using a single API. It enables quick, efficient, competent, and reliable cross-browser c.
Playwright test framework supports Jest, Mocha, Jasmine, and other prominent CI servers using a single API. Playwright offers cross-language support, which includes TypeScript, JavaScript, Python, .NET, and Java.
Here are some of the prominent features of Playwright automation framework:
One API for any browser/platform: Playwright supports all modern rendering engines, including Chromium, Firefox, and WebKit. It supports Windows, Linux, and macOS and additionally supports native mobile emulation of Google Chrome for Android and Mobile Safari.
Reduced flakiness: Playwright waits for actionable elements before performing actions. To reduce flakiness, features like test retries, execution trace capture, videos, and screenshots are also available.
Test Case Execution in Isolation: Every test that Playwright runs has a unique browser context. The same thing as a new browser profile is browser context. With no additional overhead, this gives complete test isolation.
Powerful Tooling: Playwright comes bundled with tools like Codegen, Playwright inspector, and Trace Viewer:
Codegen: Using Codegen, you can record the test cases and save them in any language (JavaScript, Python, .NET, Java, TypeScript). Recording the test cases is helpful for the beginner from a learning perspective. But if you want to use the recorded script in a live project, it’s not a good approach from a maintainability perspective.
Playwright inspector: Using Playwright inspector, you can inspect the page, see click points, and explore execution logs.
Trace Viewer: Capture all the information that helps investigate the test failure.
As per the Playwright GitHub repository, Playwright continues to attain popularity with:
Stars: 46.4k
Forks: 2.3k
Used By: 20.5k
Releases: 91
Contributors: 345
Playwright Architecture
Playwright works directly with the WebSocket, which means once you trigger the test, the code will be converted into JSON format and will be sent to the server using Web socket protocol. Once the connection or handshake connection is established, commands are sent between your test and the Playwright server.
Client and server connections will remain active until one or both parties discontinue them. After closing the connection, the connection is terminated from both ends. One of the reasons that Playwright is fast is because the connection stays active as long as neither party terminates it.
To learn more about Playwright automation testing, you can refer to this blog on the Playwright framework.
In the next section of this blog on Playwright vs Selenium vs Cypress, we will learn about the Selenium test automation framework.
What is Selenium?
Selenium is an open-source framework for automated testing that is used to verify web applications across many platforms and browsers. Using Selenium, you can automate your application with major browsers in different programming languages such as Java, JavaScript, C#, Ruby, PHP, and Python.
Using the Selenium framework, testers can use a grid of browser instances to run test cases in parallel, which can help to speed up the testing process. By running test cases in parallel, you can complete the testing phase of your CI/CD pipeline more quickly, which can help to accelerate the overall development process.
We’re not going to do a long explanation of Selenium’s tool suite here. Instead, I will give you a brief overview of the Selenium suite:
Selenium WebDriver
Selenium WebDriver is a library whose APIs are called from the code, post which the commands are run on the browser of your choice. Using Selenium locators, elements can be identified and located, which is used to generate test cases. Then, using the WebDriver APIs, you may take action on those items and interact with them.
The script you create interacts directly with the browser, which is one of the reasons it runs significantly faster than Selenium RC, which has been deprecated. There are different drivers for different browsers, which ‘interpret’ the script you write for it.
Selenium IDE
Selenium IDE allows the user to record and edit the test steps and then play those steps. IDE (Integrated Development Environment) is a Firefox, Chrome, and Edge add-on used to build and run test cases. It is a simple tool that allows you to quickly create and execute automated tests without the need to write any code.
With Selenium IDE, you can record your actions in the browser, such as clicking on elements and filling out forms and then play them back to test the functionality of your web application. Selenium IDE also allows you to edit your scripts, add assertions to verify the behavior of your application, and export your scripts to other formats, such as HTML, Java, Python, and C#.
Selenium Grid
Selenium Grid is a tool that allows you to run your tests on different machines against different browsers in parallel. It allows you to run your tests on different machines, operating systems, and browsers. This can be particularly useful if you have a large test suite and want to run your tests as quickly as possible. It’s an environment where we have one ‘master’ system (the hub) which controls the child systems (the nodes).
With Selenium Grid, you can set up a hub, which is a central point that manages the test execution on the connected nodes. You can then connect any number of nodes to the hub, and these nodes can be used to execute tests in parallel. Each node represents a combination of operating systems, browsers, and versions, and you can specify which tests should be run on which nodes.
For example, you might set up a hub with three nodes: one running Chrome on Windows, one running Firefox on macOS, and one running Safari on iOS. You can then use Selenium Grid to run your tests concurrently on all three nodes, allowing you to test your application on all three platforms and browsers simultaneously.
How Selenium Grid Works:
You set up a hub, a central point of communication for the nodes that will execute your tests.
You set up one or more nodes, the machines that will run the tests. Each node can run tests on a different browser or operating system.
You configure the nodes to connect to the hub.
You write tests and use a client library, such as the Selenium WebDriver, to send commands to the hub. The hub will forward the commands to the appropriate node, executing the tests and sending the results back to the hub.
Using Selenium Grid can significantly speed up your test execution time, allowing you to run tests in parallel on multiple machines and browsers.
Selenium Trends on GitHub:
As per the Selenium GitHub repository, Selenium persists in acquiring popularity with:
Stars: 25.5k stars
Forks: 7.3k forks
Watching: 1.3k watching
Used By:175k
Contributors: 673
Selenium Architecture
Selenium is an open-source testing framework used for web automation testing. The JSON Wire Protocol was removed in Selenium 4 in favor of the W3C WebDriver Protocol, which is now the official standard for controlling web browsers. Nevertheless, the Selenium team continues to support the outdated protocol, which may be done via Selenium-supported language Bindings and Selenium Server.
The new protocol is called “WebDriver W3C”. It has acquired the acceptance of the “World Wide Web Consortium” (W3C). In Selenium 4 architecture, we can see direct communication between the client and server without needing the JSON Wire Protocol. The same protocol is used by Selenium WebDriver and web browsers, making test case execution quick and flakiness extremely low.
There are several benefits of using WebDriver W3C Protocol. WebDriver W3C Protocol is richer in comparison to the one JSON Wire Protocol. The Actions API is revamped to work with the WebDriver Spec. Action APIs would now let you perform multi-touch actions, zoom-in, zoom-out, pressing two keys simultaneously, and more.
For example, the Pinch-zoom sequence in W3C Protocol is represented by an action sequence consisting of three ticks and two-pointer devices of type-touch.
In the next section of this blog on Playwright vs Selenium vs Cypress, we will learn about the Cypress test automation framework.
What is Cypress?
Cypress is a front-end testing tool built for the modern web. It supports JavaScript and is used for end-to-end testing with the Mocha test framework. Mocha is a feature-rich JavaScript-based testing framework that simplifies asynchronous testing. Cypress is a powerful automation tool gaining popularity and becoming a viable alternative to Selenium.
In Cypress, you don’t need to install multiple dependencies; most required things for writing the test cases come as a bundle in Cypress. Cypress runs a test suite inside the browser, making it very fast compared to frameworks like Selenium. In the Cypress bundle, we have an in-built Chai assertion library that supports the Mocha framework, which provides the syntax for implementing test cases in Behavior-Driven Development (BDD) style.
Cypress automation tool addresses the following key pain points developers and QA engineers face when testing modern applications.
Easy installation: You can install Cypress with npm or their desktop application. Moreover, you do not need to install additional libraries, dependencies, drivers, nor need to install additional libraries, dependencies drivers.
Real-time reloads: In Cypress, you do not need to reload the test case. When you make changes to your test case, it is smart enough to trigger and run the test automatically.
Write Test case faster: Cypress is faster; within minutes, you can start writing the test case. Cypress runs tests within the browser that give us results almost identical to what users will experience.
Automatic waiting: In Cypress UI automation, you don’t need to add wait times or sleep to your tests. Cypress automatically waits for commands and assertions before moving on to the next step.
Screenshots and videos: Video of the entire suite is automatically captured. You can also capture Screenshots automatically of failure when running headlessly. This is extremely useful when tests fail.
Instructive Dashboard: Using Cypress cloud, we can run your test cases in CI/CD providers and record test results.
Cypress Trends on GitHub:
As per the Cypress GitHub repository, Cypress persists in acquiring popularity with:
Stars: 42.3k
Forks: 2.7k
Used By: 660k
Releases: 293
Contributors: 413
Check this out: Base64 Decode — Base64 decoding is a way of translating text form into binary data so that it can be transmitted more easily via e-mail and HTML form data.
Cypress Architecture
Cypress is a testing framework that is used to automate web applications. It has a unique architecture designed to make testing faster and more reliable. In Cypress UI testing, all commands are executed inside the browser without any driver binaries dependency. In the below screenshot, you can see tests are run inside the browser, so test execution is faster without any network lags.
There is a Node server behind Cypress, and the Node process constantly communicates, synchronizes, and performs jobs on behalf of each other. The Node server and browser communicate through WebSocket, which starts once the proxy is created.
Since tests are running inside the browser, it’s very easy to work directly on:
DOM
Local storage
Network layer
Window object
As the things mentioned above are very simple to access in Cypress, this is a significant advantage when running tests on Cypress in comparison to other test automation frameworks.
To learn more about Cypress testing, you can refer to this blog on Cypress test automation.
Check this out: This Prettify JSON online tool is completely free to use.
Comparing Playwright vs Selenium vs Cypress Features
Below are a few points for comparing Playwright vs Selenium vs Cypress.
Installation & Configuration
In this section, we will take a look at how to install and configure each of these tools, and what benefits they offer in terms of installation and setup.
Playwright:
When comparing Playwright vs Selenium vs Cypress installation and configuration, Playwright is easy to install and set up. There are different ways to install Playwright, some of which are explained below:
Here are some basic prerequisites for Playwright:
Ensure you have NodeJS installed.
Install a code editor. I will be using Visual Studio Code.
There are two ways of installing the Playwright:
Using the VS Code extension.
Using the init command.
Using the VS Code extension
Create a folder, e.g., Install_playwright.
Open the folder in VS Code.
Search Playwright extension in VS Code and install it.
- Now Press command+shift+P (in Mac) and finally click on the OK button.
- As we click on the OK button, the Playwright installation will start.
Using the init command
Create a folder, e.g., playwright_new_script.
Open the folder in VS Code.
Run from the project’s root directory npm init playwright@latest.
As we run the above commands, Playwright starts installing. (At the time of writing this blog, the latest version of Playwright is 1.29 ).
In the below screenshot, you can see we have two options. You can select either TypeScript or JavaScript.
We can see the default .spec file after successful installation.
To know more about how to set up and run the test cases in Playwright, you can follow this blog on end to end testing with Playwright.
Selenium:
To set up Selenium, we usually need to follow the steps below. This applies to the local Selenium Grid. The below steps are also applicable to Python and other Selenium-supported languages. Installation of Selenium Grid and Server is not applicable for cloud Selenium Grids like LambdaTest.
- Install Java. Here’s the link to download and install JDK.
- Install IDE. You can learn more about it through this blog on configuring Eclipse and IntelliJ.
Download Selenium WebDriver and Java Client Driver to be set up in Eclipse.
Create a TestNG project and run the Selenium test script. You can learn more about it through this blog on creating TestNG project in Eclipse.
Cypress:
In Cypress, installation is super easy. The good thing about Cypress is it has a folder structure. Just follow the folder structure and start developing the test cases. Using a predefined folder structure, you can develop the framework for your project.
To set up Cypress, follow the steps below:
Pre-requisites: Node should already be installed.
Generate package.json using the command npm init.
**npm install cypress –save-dev **command installs the latest version of Cypress. At the time of writing this blog, the latest version of Cypress is 12.2.0.
- Once the setup is done, open the Cypress, which will open with all default .spec files.
To perform end-to-end testing using Cypress, follow the blog on Cypress end to end Testing to know in detail.
In the next section of this blog on Playwright vs Selenium vs Cypress, we will learn about different runners available for Playwright, Selenium, and Cypress.
Check this out: Minify JSON helps to minify, compress and reformat JSON data. Just load your JSON and it will automatically get minified.
Runner
Runner is a tool you can use to run/execute test cases. After executing the test cases, you can view and export the test execution results.
Playwright:
Playwright has its runner and has the adaptability to operate with third-party runners such as Playwright Test Jest / Jasmine, AVA, Mocha Vitest, and multiple browsers. You can connect Playwright to your current JavaScript test runner with a few lines of code.
Playwright Test: First-party suggested test runner for Playwright is called Playwright Test.
Jest/Jasmine: You can use jest-playwright for Jest. However, requiring the Playwright directly works well as a simple approach. This also applies to Jasmine, which has a shared syntax with Jest.
**AVA: **A single page variable cannot be shared between tests in AVA since tests run concurrently. Use a macro function to create new pages instead.
**Mocha: **Mocha functions similarly to the Jest/Jasmine configuration and has a similar appearance.
Vitest: Vitest looks similar to the Jest/Jasmine setup and functions similarly.
Multiple Browsers: We can execute the test cases using an environment variable in multiple browsers.
In Playwright, we can run a single test case/set of test cases or all test cases. By default, test cases are run in headless mode. Test cases are run in the terminal, and results can only be seen in the terminal. Below are the commands that we can use to run the test cases.
- Performing Playwright browser testing in headless mode:
npx playwright test
- Running a single Playwright test case:
npx playwright test fileName.spec.js
- Run a set of Playwright test cases:
npx playwright test tests/page/home-page/
- Running Playwright tests in headed mode:
npx playwright test home-page.spec.js --headed
In Playwright, you can run the test case in debug mode. Playwright has a built-in tool, i.e., Playwright Inspector, which permits you to step via Playwright API calls, see their debug logs, and explore selectors. The Playwright Inspector also helps in finding the locators. We can find the locator by hovering over the element during the inspection.
In debug mode, Playwright opens the “Playwright Inspector,” where we can go line-wise and easily check the reason for the failure of the test case in the Inspector itself.
Selenium:
Selenium Runner is a command-line tool that allows you to run Selenium scripts. There are several runners available for Selenium, each with its own set of features and capabilities. Below is a list of a few runners normally used to run test cases in Selenium.
Selenium WebDriver: This is the most widely used runner for Selenium. It allows you to automate web browser actions, such as clicking on elements, filling out forms, and navigating between pages.
Selenium Grid: This runner allows you to run your Selenium scripts on a remote machine or in the cloud. It is useful for running tests in parallel on multiple machines and different browsers and operating system configurations.
**TestNG: **This is a popular testing framework often used with Selenium. It allows you to create test cases, define test suites, and support running tests in parallel.
**JUnit:** This is another popular testing framework often used with Selenium. It allows you to create test cases, define test suites, and support running tests in parallel.
**Jenkins: **This is a continuous integration tool that can run your Selenium tests. It automates the build, test, and deployment process and supports running tests in parallel.
**Maven: **This is a build automation tool that can be used to manage your Selenium tests. It allows you to define dependencies, build and test your code, and manage the deployment process.
Cloud testing tools like LambaTest can be used as a test case runner to execute automated tests on a variety of different operating systems, browsers, and devices. With LambdaTest, you can run end-to-end automated tests using Selenium on a safe, dependable, and scalable Selenium infrastructure. This can help users to ensure that their websites and applications are compatible with a wide range of different platforms and devices and to identify and fix any issues that may arise.
Using LambdaTest, you can automate cross browser testing on more than 3000 distinct operating systems and browsers with Selenium scripts, which leads to greater test coverage and significantly quicker build times.
Cypress:
Cypress has a test runner where we can see the executing commands. Apart from that, in Cypress, you can see the application under test with real-time data. Using Cypress runner, you can see the change in UI after hovering over the log in the command log.
Run your Cypress automation scripts across 3000+ environments. Try LambdaTest Now!
Check this out: JSON Escape removes offending characters that can cause parsing errors. Special characters such as quotes, newlines and backslashes will be replaced with their escaped counterparts.
Record & Playback
When comparing Playwright vs Selenium vs Cypress, most tools typically have a record and playback feature, which is quite beneficial for first-time users. You can change an existing test, record new tests, and run the recorded steps using the Record & Playback tool.
All three tools, i.e., Playwright vs Selenium vs Cypress, include Record & Playback, which are described in more depth below.
Playwright:
In Playwright, you will generate the code for the user interactions. You can use CodeGen to generate the actions we perform in the browser. Playwright will generate the code for the user action performed. You can use the below commands to open the Playwright Inspector.
In the below screenshot, you can see code is generated when you click on the Home, Blog, and Mega Menu link.
npx playwright codegen https://ecommerce-playground.lambdatest.io.
Selenium:
We have Selenium IDE to record and playback the code in Selenium. Selenium IDE doesn’t need any additional setup before usage other than installing the plugin in your browser.
It does not require any programming logic to develop its test scripts, unlike Selenium WebDriver and RC. Instead, you may only record your browser interactions to create test cases. The playback option can then be used to rerun the test scenarios.
Selenium IDE records multiple locators for every element it interacts with. It tries to attempt the locator until one is successful if one fails during playback. You can reuse existing test cases inside of another by using the run command (e.g., allowing you to reuse your login logic in multiple places throughout a suite). In the below screenshot, you can see we have reused VerifyHomePage test steps in another test case.
The Selenium IDE comes preloaded with a robust control flow structure that includes available commands like if, while, and times.
Cypress:
We can use Cypress Studio to capture the test case in Cypress. By capturing interactions with the application being tested, Cypress Studio offers a visual approach to developing tests within Cypress.
Cypress Studio provides a visual approach to create tests within Cypress by recording interactions against the application under test. When interacting with the DOM inside of Cypress Studio, the .type().click(),.check(),.uncheck(), and .select() Cypress commands are available and will produce test code.
To enable the Cypress Studio, we have to do the below setting (experimentalStudio: true) in cypress.config.js.
const { defineConfig } = require("cypress");
module.exports = defineConfig({
e2e: {
experimentalStudio: true,
},
});
After doing the above setting when we run the test cases. We can see two options to record the test cases:
Add New Tests.
Add commands to Test.
Add New Tests
Using Add New Test, we can add a new Test, save it and re-run it. In the below screenshot, we can see steps are recorded by Cypress Studio. Just save the commands, and a new script is created.
Add commands to Test
Using Add commands to Test, we can add more steps in existing test cases.
In the screenshot below, you can see whatever steps added in the first test case are recorded on an existing script. This way, we can update our existing script with new additional steps.
/// <reference types="cypress" />
describe("Cypress Studio -- >", { testIsolation: false }, () => {
it("WHEN User Open the Url", () => {
cy.visit(
"https://ecommerce-playground.lambdatest.io/index.php?route=account/login"
);
/* ==== Generated with Cypress Studio ==== */
cy.get('#widget-navbar-217834 > .navbar-nav > :nth-child(1) > .icon-left > .info > .title').click();
cy.get('#widget-navbar-217834 > .navbar-nav > :nth-child(3) > .icon-left > .info > .title').click();
/* ==== End Cypress Studio ==== */
});
it("AND Login into the application", () => {
cy.get('[id="input-email"]').type("lambdatest@yopmail.com");
cy.get('[id="input-password"]').type("lambdatest");
cy.get('[type="submit"]').eq(0).click();
});
it("AND After login Search the Product", () => {
cy.get('[name="search"]')
.eq(0)
.type("Sony VAIO")
.should("have.value", "Sony VAIO");
cy.get('[type="submit"]').eq(0).click();
});
it("THEN Verify Correct Product should display after search ", () => {
cy.contains("Sony VAIO");
});
});
In the next section of this blog on Playwright vs Selenium vs Cypress, we will learn about different ways to select elements in Playwright, Selenium, and Cypress.
Check this out: Unescape JSON removes traces of offending characters that could prevent parsing. To unescape JSON strings, several libraries and tools are available in different programming languages.
Element Selection
Element selection is a way where we filter out all elements with a given tag in documents and make sure the element you are making assertions is detached from the DOM. We have different types of tags (e.g., Id, CSS, Class, Attribute, Tag Name, Link Text, Name Attribute)
Playwright:
In Playwright, we have an auto-wait concept that makes the command fast and the test case execution fast. Playwright auto-waits for all the relevant checks to pass. Once these checks pass, then only perform the requested action.
Here is the list of actionability checks performed for each action on the elements.
| ACTION | ATTACHED | VISIBLE | STABLE | RECEIVES EVENTS | ENABLED | EDITABLE |
| check | Yes | Yes | Yes | Yes | Yes | – |
| press | Yes | – | – | – | – | – |
| setInputFiles | Yes | – | – | – | – | – |
| selectOption | Yes | Yes | – | – | Yes | – |
| textContent | Yes | – | – | – | – | – |
| type | Yes | – | – | – | – | – |
| click | Yes | Yes | Yes | Yes | Yes | – |
| dblclick | Yes | Yes | Yes | Yes | Yes | – |
| setChecked | Yes | Yes | Yes | Yes | Yes | – |
| tap | Yes | Yes | Yes | Yes | Yes | – |
| uncheck | Yes | Yes | Yes | Yes | Yes | – |
| hover | Yes | Yes | Yes | Yes | – | – |
| scrollIntoViewIfNeeded | Yes | – | Yes | – | – | – |
| screenshot | Yes | Yes | Yes | – | – | – |
| fill | Yes | Yes | – | – | Yes | Yes |
| selectText | Yes | Yes | – | – | – | – |
| dispatchEvent | Yes | – | – | – | – | – |
| focus | Yes | – | – | – | – | – |
| getAttribute | Yes | – | – | – | – | – |
| innerText | Yes | – | – | – | – | – |
| innerHTML | Yes | – | – | – | – | – |
Below are the points that the Playwright will ensure before taking action on any of the elements on the page.
element is bound to the DOM
element is visible
element is enabled
element receives events, as in not obscured by other elements
Below are a few examples of Playwright. In the case of the Playwright, “waits” is included by default against each action on the element.
await page.goto('https://www.bing.com')
const element = await page.$('[aria-label="Enter your search term"]')
await element.click()
await element.type('LambdaTest')
await element.press('Enter')
await page.locator('text=Log in').click();
await page.locator('article:has-text("Playwright")').click();
await page.locator('#nav-bar >> text=Contact Us').click();
Check this out: With this free online Alder32 Hash Calculator, you can provide input in plain text, Base64, or as a binary file. Hit the Calculate button above and voila! Your Adler-32 checksum will be generated in no time.
Selenium:
WebElements in Selenium are used to interact with DOM elements. We normally used different types of locators (mentioned above) to connect with the page’s elements. Sometimes when we search for the element, we assume that the element will already be loaded on the page.
In my opinion, this is one of Selenium’s biggest limitations. As a result of the test frequently failing after repeated attempts to find the element, we must start loading the element with implicit or explicit waits to avoid flakiness.
To avoid failure of the test case, here we check whether the element is available in DOM.
WebElement button = new WebDriverWait(driver, 30).until(ExpectedConditions.presenceOfElementLocated(By.id("buttonId")));
button.click();
Cypress:
In the case of Cypress, things are the opposite, as interactions of elements are done through the locators. There is an inbuilt “wait mechanism” in Cypress, which makes the execution of the tests much more stable and decreases the flakiness during the execution of the test cases.
This duration is configured by the defaultCommandTimeout option — which has a default of 4000 ms. This means that when acting on the element, Cypress will wait up to 4 seconds to move to the next command. You will get an error message if the element is not loaded within the default time. We can increase defaultCommandTimeout from cypress.config.js file
const { defineConfig } = require('cypress')
module.exports = defineConfig({
defaultCommandTimeout: 10000
})
Cypress automatically pauses for the network call to complete before moving to the next command. In the below commands of Cypress, for every command, Cypress will wait for 4 seconds.
cy.get('.btn').click() // Click on to the button
cy.focused().click() // Click on element with focus
cy.contains('Welcome').click() // Click on first el containing 'Welcome'
Check this out: HTML Escape is a simple tool that turns normal HTML into escaped HTML, making it easier to display HTML text in the pre> tag.
Language/Browser Support
Language/browser support is one of the key differentiators among Playwright vs Selenium vs Cypress.
Playwright:
It supports most languages, including Python, Java, Java Script, and .Net. Playwright has the full support of all modern browsers that include Google Chrome and Microsoft Edge (with Chromium), Safari (with WebKit), and Firefox.
Selenium:
Selenium is a suite of tools that enables automated testing of web applications. It supports various programming languages, including Java, Python, C#, Ruby, and JavaScript. You can use any of these languages to write your Selenium test scripts, and run them on various web browsers, including Google Chrome, Mozilla Firefox, Internet Explorer, Microsoft Edge, and Safari. Selenium also provides support for headless browsers, which allow you to run your tests without actually launching a web browser window.
To use Selenium with a particular programming language and web browser, you must install the appropriate language bindings and WebDriver. These will allow you to interact with the browser and perform actions such as navigating a web page, clicking on elements, and entering text.
For example, if you want to use Selenium with Python and Chrome, you must install the Python bindings for Selenium and the ChromeDriver web driver. You can then use the Python bindings to control Chrome and perform your tests.
Cypress:
Cypress is a JavaScript-based end-to-end testing tool designed to work with web applications. As a result, it is primarily designed to work with web pages and applications written in HTML, CSS, and JavaScript. It is compatible with all modern browsers, including Google Chrome, Mozilla Firefox, Microsoft Edge, and Apple Safari.
Check this out: The html unescape method helps us to convert the ascii string into html script by replacing ascii characters with special HTML characters. This tool will convert HTML entities to a string or convert plain text to HTML entities.
Parallel Execution
Parallel execution is a crucial feature for automating web browsers as it allows running multiple test cases simultaneously, thus reducing the overall execution time. When it comes to parallel execution, Playwright, Selenium, and Cypress are all popular choices among developers.
In this comparison, we will take a look at how each of these tools handles parallel execution, what kind of parallel execution options they offer, and how easy it is to implement parallel execution with these tools.
Playwright:
When it comes to comparing Playwright vs Selenium vs Cypress, Playwright supports parallel execution across multiple machines.
Let’s take an example to run the test cases in parallel.
Below is the code snippet where we are opening one eCommerce website, then clicking on the Home, Blog link, and verifying the user is redirected to the correct URL.
// @ts-check
const { test, expect } = require("@playwright/test");
test.describe("Test suite", () => {
test.beforeEach(async ({ page }) => {
await page.goto("https://ecommerce-playground.lambdatest.io/");
});
test("Open the Lambdatest Site Click on Home Link and verify user re-direct to correct url", async ({ page }) => {
await page.locator('span:has-text("Home")').click();
await expect(page).toHaveURL(
"https://ecommerce-playground.lambdatest.io/index.php?route=common/home"
);
});
test("Open the Lambdatest Site Click on Blog Link and verify user re-direct to correct url", async ({ page }) => {
await page.locator("#widget-navbar-217834 >> text=Blog").click();
await expect(page).toHaveURL(
"https://ecommerce-playground.lambdatest.io/index.php?route=extension/maza/blog/home"
);
});
});
Running the script with 1 Worker in Playwright:
In below screenshot, we can see we have 2 test cases that are running one by one [1/2].
Here we can see that when we were running test cases in a Single worker, it took 16s.
By default, test cases are run in a single worker, but if we have to run the test cases in parallel in different machines, we have to provide the number of workers either through the command line by updating the config file passing npx playwright test --workers 4 where “4” is the number of workers on which we want to execute our test cases.
Another way to run the test cases in parallel is by bypassing the worker in playwright.config.js.
Setting to update the Worker
To update the number of workers, we have to update playwright.config.js. In the below screenshot, you can see I have updated the worker to “2” in line number #33.
Running the script with 2 Workers in Playwright:
In the below screenshot, we can see both the test cases are executed parallelly [2/2].
When we run the same script with 2 workers taking 13s.
LambdaTest allows you to perform parallel testing with Playwright across 50+ browsers and OS to automate your several test cases simultaneously. You can run one test case across various browsers or multiple test case scenarios in the same browser with different browser versions.
Subscribe to LambdaTest YouTube Channel and stay updated with detailed tutorials around Playwright testing, Selenium testing, Cypress testing, and more.
Check this out: An MD4 hash calculator is a tool that calculates the MD4 hash value of a file or data.
Selenium:
Parallel execution means executing the test case of different modules in parallel instead of running them in series. In Selenium, you can do parallel execution using TestNG. Below is the structure for defining the attribute in the TestNG XML file to run the test cases in parallel.
<suite name=”ParallelTesting” parallel=”methods” thread-count=”3">
In the above example, in place of “method,” we can pass the below values.
Below pointer mentioned here is specific to Java. It might vary for other Selenium-supported languages.
Methods: All the methods with @test annotation will run in parallel.
Instances: Helps execute all methods in the same instance in the same thread.
Tests: All the test cases inside the tag of the Testing XML file will run parallel.
Classes: All the test cases inside a Java class will run in parallel.
Thread Count: No of threads you want to run in parallel.
In the above code snippet, attribute thread count helps in defining the number of threads that we want while executing the test cases in parallel.
Let’s take an example of how a thread works. Suppose we have three methods and two threads to run the test cases in parallel. So initially, two methods will run with two available threads. Once any thread between two freezes, the third method in the queue starts executing.
Let’s Take an example to run the test cases in parallel. Open the Site https://ecommerce-playground.lambdatest.io/, Verify the title, Click on the Home Page, and Verify the user is redirected to the correct URL.
Create three Classes, “ChromeParallelTest,” ”ForefoxParallelTest,” and “SafariParallelTest” to run the test cases in three browsers.
Below is the code snippet for Class “ChromeParallelTest”:
package ParallelTestCase;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class ChromeParallelTest {
WebDriver driver;
@BeforeTest
public void setUp() {
System.setProperty("webdriver.chrome.driver", "/Users/xxxxx/Desktop/Chromedriver/chromedriver");
driver = new ChromeDriver();
driver.navigate().to("https://ecommerce-playground.lambdatest.io/");
driver.manage().window().maximize();
}
@Test
public void VerifyTitle() {
System.out.println("The thread ID for Running First Test In Chrome Browser is " + Thread.currentThread().getId());
Assert.assertEquals(driver.getTitle(), "Your Store");
}
@Test
public void ClickOnHomeLink() {
System.out.println("The thread ID for Running Second Test In Chrome Browser is " + Thread.currentThread().getId());
WebElement homeLink = driver.findElement(By.xpath("//span[contains(text(),'Home')]"));
homeLink.click();
String redirectURL = driver.getCurrentUrl();
Assert.assertEquals(redirectURL, "https://ecommerce-playground.lambdatest.io/index.php?route=common/home");
}
@AfterTest
public void close() {
driver.quit();
}
}
Below is the code snippet for the Class “FirefoxParallelTest”.
package ParallelTestCase;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class FirefoxParallelTest {
WebDriver driver;
@BeforeTest
public void setUp() {
System.setProperty("webdriver.chrome.driver", "/Users/xxxxxxx/Desktop/Chromedriver/chromedriver");
driver = new FirefoxDriver();
driver.navigate().to("https://ecommerce-playground.lambdatest.io/");
driver.manage().window().maximize();
}
@Test
public void VerifyTitle() {
System.out.println("The thread ID for Running First Test In Firefox Browser is " + Thread.currentThread().getId());
Assert.assertEquals(driver.getTitle(), "Your Store");
}
@Test
public void ClickOnHomeLink() {
System.out.println("The thread ID for Running Second Test In Firefox Browser iss " + Thread.currentThread().getId());
WebElement homeLink = driver.findElement(By.xpath("//span[contains(text(),'Home')]"));
homeLink.click();
String redirectURL = driver.getCurrentUrl();
Assert.assertEquals(redirectURL, "https://ecommerce-playground.lambdatest.io/index.php?route=common/home");
}
@AfterTest
public void close() {
driver.quit();
}
}
Below is the code snippet for Class “SafariParallelTest”.
package ParallelTestCase;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class SafariParallelTest {
WebDriver driver;
@BeforeTest
public void setUp() {
System.setProperty("webdriver.chrome.driver", "/Users/xxxxxx/Desktop/Chromedriver/chromedriver");
driver = new SafariDriver();
driver.navigate().to("https://ecommerce-playground.lambdatest.io/");
driver.manage().window().maximize();
}
@Test
public void VerifyTitle() {
System.out.println("The thread ID for Running First Test In Firefox Browser is " + Thread.currentThread().getId());
Assert.assertEquals(driver.getTitle(), "Your Store");
}
@Test
public void ClickOnHomeLink() {
System.out.println("The thread ID for Running Second Test In Firefox Browser iss " + Thread.currentThread().getId());
WebElement homeLink = driver.findElement(By.xpath("//span[contains(text(),'Home')]"));
homeLink.click();
String redirectURL = driver.getCurrentUrl();
Assert.assertEquals(redirectURL, "https://ecommerce-playground.lambdatest.io/index.php?route=common/home");
}
@AfterTest
public void close() {
driver.quit();
}
}
To run the test cases in parallel, let’s create a testng.xml file with the following config.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests" thread-count="2">
<test name="Parallel Test Case Execution In Chrome">
<classes>
<class name="ParallelTestCase.ChromeParallelTest"/>
</classes>
</test> <!-- Test Case END -->
<test name="Parallel Test Case Execution In FireFox">
<classes>
<class name="ParallelTestCase.FirefoxParallelTest"/>
</classes>
</test> <!-- Test Case END -->
<test name="Parallel Test Case Execution In Safari">
<classes>
<class name="ParallelTestCase.SafariParallelTest"/>
</classes>
</test> <!-- Test Case END -->
</suite> <!-- Suite END -->
In the above screenshot, we can see we have given thread-count “2”. So the first two test cases are run in two threads. Once the threads are free, the next test case starts executing.
Let’s execute the test cases. We have to right-click on testng.xml and then “Run As” -> TestNG Suite.
You can see in the screenshot below that “FirefoxTest” and “ChromeTest” were running concurrently on Threads 15 and 16, respectively, as seen in the console logs. Later, “SafariTest” was executed on Thread 15 because it was released before 16.
Run your Selenium automation scripts across 3000+ environments. Try LambdaTest Now!
Cypress:
Cypress supports parallel execution across multiple machines from version 3.1.0. By default, the cypress run command executes the test cases serially. In Cypress, we can execute test cases in parallel, which saves your team time and good ROI when running tests in Continuous Integration (CI) tests.
To run Cypress test cases locally, we can use the command npx cypress run –record –key xxxxxx-xxx-xxxx0-xx-xxxxx. Record Keys allow you to record test results, screenshots, and videos in Cypress.
Let’s run the below Cypress test case in Cypress Cloud for the site https://ecommerce-playground.lambdatest.io/.
describe("Verify title of ecommerce-playground.lambdatest.io", function () {
it("Should display the correct title", function () {
cy.visit("https://ecommerce-playground.lambdatest.io/");
cy.title().should("include", "Your Store");
});
it("Should navigate to the Blog page", function () {
cy.visit("https://ecommerce-playground.lambdatest.io/");
cy.contains("Blog").click({ force: true });
cy.url().should("include", "/blog/");
});
});
When we run the above test case using the command npx cypress **run –record –key xxxxxx-xxx-xxxx0-xx-xxxxx. **The test case starts running in Cypress cloud.
In the below screenshot, we can see the test case running in 1 machine.
To run the Cypress test case in parallel, we must connect the Cypress cloud with CI/CD. There are numerous CI/CD solutions supported by Cypress, and they are simple to connect with Cypress cloud. To run tests with GitHub Actions, you can refer to this blog on how to run Cypress tests with GitHub Actions.
Cloud-based cross browser testing solutions like LambdaTest allow you to test your web application or website across a wide range of desktop and mobile browsers. One of the features of LambdaTest is the ability to perform parallel testing, which allows you to run multiple tests simultaneously, significantly reducing the time it takes to complete your testing.
When using LambdaTest and Cypress together, you can use the LambdaTest platform to run your Cypress tests across a wide range of browsers and devices in parallel, reducing the time it takes to complete your testing. You can read through this blog to perform Cypress testing with LambdaTest Cypress CLI
In the next section of this blog on Playwright vs Selenium vs Cypress, we will learn about how to handle flakiness in test automation frameworks like Playwright, Selenium, and Cypress.
Check this out: The MD2 Hash Generator creates a unique string of characters that functions as a password or key.
Flakiness
A test is considered flaky when it can pass and fail across multiple retry attempts without any code changes.
As one of the Google blogs says — Almost 16% of our tests have some flakiness associated with them! This is a staggering number; more than 1 in 7 of the tests written by our world-class engineers occasionally fail in a way not caused by changes to the code or tests.
In one of the research, it was found that 72 percent of test failures are false positives. Think about all the tests you have and all the test failures reported for each run. False positives consume a lot of resources that could be dedicated to much more valuable work.
Check this out: The free online line counter tool is used to count the number of lines in a document or passage of text. It allows you to count lines with or without blanks.
Playwright:
There are many ways to handle the flakiness in Playwright. Following are some best ways of handling the flakiness in Playwright.
- Auto-wait feature In Auto-wait, we make sure when you perform an action, like clicking on the element, selecting a value, and entering the data on the text field, the element should be loaded before the action is performed.
Below is a code snippet with the assertion:
test("Verify button enable or not", () => {
const submit = page.locator("button", {hasText: "Previous"});
// Will check button is enabled or not
expect(await submit.isEnabled()).toBeTruthy()
await submit.click();
})
In the above code, we don’t need the below line; you are just wasting testing time on a non-necessary assertion. The playwright has an auto-wait mechanism and automatically handles the assertion part.
expect(await submit.isEnabled()).toBeTruthy()
test("Verify button enable or not", () => {
const submit = page.locator("button", {hasText: "Previous"});
// Will check button is enabled or not
await submit.click();
})
- Use Methods like waitForLoadState waitForNavigation waitForSelector to avoid flakiness.
We can use these methods to avoid the flakiness. Using these methods, we can make sure the page is fully loaded before it interacts with the element on the page.
Method waitForLoadState is used to wait till a specific state of the page has been reached.
Method waitForNavigation is suggested when Before beginning navigation, clicking an element may cause asynchronous processing to take place.
Method waitForSelector used to hold off till the selector satisfies the state option (e.g either appear/disappear from the DOM, or become visible/hidden).
await page.waitForLoadState('domcontentloaded', { timeout: 1000 });
await page.waitForLoadState('networkidle', { timeout: 5000 });
await page.waitForLoadState('load', { timeout: 2000 });
await Promise.all([
page.waitForNavigation({ url: '**/login' }),
page.click('button'),
]);
await page.waitForSelector('.myDiv')
- Another feature using which we can avoid flakiness is Retries.
Sometimes when the page loads for the first time, the required element is not present, and test cases fail. We can avoid this situation by using the Retries option. We can provide the Retries option in the Playwright.config.js file so that when test cases fail, the system will retry again, and the probability of the test case passing is increased.
const config = {
// Give failing tests 2 retry attempts
retries: 2,
}
Selenium:
In Selenium, test cases can be flaky due to the flakiness in the test code or the flakiness in the test infrastructure. Flakiness on the infrastructure front can be avoided by moving test execution to a cloud platform like LambdaTest.
A flaky test is a test that randomly passes or fails for the same code, giving inconsistent results. One time it passes; another time, it fails; and the next time, it passes without any change being made to the build.
There are several ways to avoid flakiness in Selenium tests:
Use stable selectors: Instead of using dynamic selectors such as XPath, try to use stable selectors such as class name or Id to locate elements. Dynamic selectors can be affected by changes in the structure of the HTML, leading to test flakiness.
Use explicit waits: Instead of using implicit waits, use explicit waits to allow the page to load and stabilize before interacting with elements. This can help prevent tests from failing due to elements not being immediately available.
Add retry logic: Implement retry logic in your tests to retry failed assertions a certain number of times before giving up. This can help reduce flakiness caused by temporary network or server issues.
Use a stable testing environment: Make sure your tests run in a stable environment with consistent network and server conditions.
Debug failing tests: When tests fail, take the time to debug and understand the root cause of the failure. This can help you identify and fix underlying issues that are causing flakiness.
Use a test framework: Use a test framework such as JUnit or TestNG to structure your tests and provide built-in support for handling test failures. This can make it easier to identify and debug failing tests.
Use a reliable testing framework: Use a reliable testing framework, such as JUnit or TestNG, to manage the execution of tests and handle any issues that may arise.
Use a reliable continuous integration (CI) server: Use a reliable CI server, such as Jenkins or Azure DevOps, to run tests automatically and track the results of test runs.
One way to use a cloud platform like LambdaTest to avoid flakiness in Selenium is to run your Selenium scripts on the cloud infrastructure instead of running them locally on your machine.
Cypress:
Cypress handles flakiness very well. In Cypress, there are different ways of handling flakiness.
Following are some best ways of handling the flakiness in Cypress.
Adding .contains('element') .should('be.visible'),.should('exist') before clicking on element.
Test Retires
Use timeouts instead of waits.
Use cy.intercept() to wait for the XHR request to finish execution.
The first and very straightway method to reduce the flakiness is by using .should('exist') .should('be.visible') .contains('element'). These assertions reduce the chance of test case failure.
The second method in Cypress to reduce the flakiness is to use Test Retires. You can set the Test Retires option at Global Configuration, Individual Test, or Test Suite level.
- Another way to reduce the flakiness is to use timeouts instead of waits.
Sometimes, we have required a higher timeout for a particular element to load into the page. You can handle such cases by providing a timeout on a particular element instead of using cy.wait().
The difference between cy.wait() and set the timeout at the field level is, in the case of cy.wait() Cypress will wait for the complete time that we have provided under cy.wait(), but in case of timeout, if the element load before the specified timeout, Cypress will move to the next command to execute. This way, sometimes timeout is effective as it saves time.
cy.get(“#first-name”, {timeout: 10000}).type(“Mike”)
- We can use cy.intercept() to wait for the XHR request to finish execution. We can intercept the particular network call and wait till the call is not finished.
In the below screenshot, after clicking on the form model, we presume there is some unfinished request. cy.wait() wait for the request/response cycle to complete. By using the cy.wait("@getModel") we have access to the XHR object. Cypress will wait for the API call to end before moving forward as long as the request is not finished. So you can prevent test cases from failing in this manner.
Check this out: This online MD5 hash generator is a tool for encoding passwords, credit card numbers, and other sensitive data into MySQL, Postgress, and similar databases.
Screenshots
Screenshots play a very important role when test cases fail regularly. The screenshot gives an idea about the steps where test cases fail and help you make the test case less flaky.
Playwright:
While comparing Playwright vs Selenium vs Cypress, it is very easy to take screenshots in Playwright. There are three ways to take a screenshot.
Capture into buffer
Element screenshot
Full page screenshots
To take Full page screenshots, we have to write the below code:
await page.screenshot({ path: ‘screenshot.png’, fullPage: true });
Capture into buffer
Instead of writing into a file, you can get a buffer with the image and post-process it or pass it to a third-party pixel diff facility.
const buffer = await page.screenshot();
console.log(buffer.toString('base64'));
Element screenshot
We can take the screenshot at the element level too, using the below code.
await page.locator(‘.header’).screenshot({ path: ‘screenshot.png’ });
Selenium:
A screenshot in Selenium WebDriver is helpful for bug investigation. In Selenium, if you want to take the screenshot during execution, you can use the TakesScreenshot method, which informs the WebDriver to grab the screenshot.
Code snippet to take the screenshot of the Page attached below.
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import java.io.File;
import java.io.IOException;
public class ScreenshotCapture {
public static void main(String[] args) {
//set the location of chrome browser
System.setProperty("webdriver.chrome.driver", "C:\\chromedriver.exe");
// Initialize the browser
WebDriver driver = new ChromeDriver();
//navigate to the url
driver.get("https://www.lambdatest.com/");
//Take screenshot
File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
//Copy the file to a location
try {
FileUtils.copyFile(screenshot, new File("C:\\xxx\\xxx.png"));
} catch (IOException e) {
System.out.println(e.getMessage());
}
//closing the webdriver
driver.close();
}
}
We can take screenshots of elements in Selenium. Code Snippet to take the screenshot of the element attached below.
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.ChromeDriver;
import java.io.File;
import java.io.IOException;
public class SeleniumelementTakeScreenshot {
public static void main(String args[]) throws IOException {
WebDriver driver = new ChromeDriver();
driver.get("https://www.lambdatest.com/");
WebElement element = driver.findElement(By.cssSelector("h1"));
File scrFile = element.getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(scrFile, new File("./image.png"));
driver.quit();
}
}
Cypress:
Cypress has an in-built ability to take screenshots. You can run test cases via “cypress open” OR “cypress run”. You can use cy.screenshot() if you want to take screenshots manually.
Capturing screenshots when a test fails can be turned off completely by specifying screenshotOnRunFailure to false from within the Cypress configuration or by specifying screenshotOnRunFailure to false in the Cypress.Screenshot.defaults().
Below are different syntaxes to capture the screenshot in Cypress.
cy.screenshot()
cy.screenshot(fileName)
cy.screenshot(options)
cy.screenshot(fileName, options)
In the below Code snippet, you can see the screenshot saved by default under the screenshots folder when test cases are failing.
// cypress/e2e/login.cy.js
describe('Screenshot tests', () => {
it('takes a screenshot', () => {
// screenshot will be saved as
// cypress/screenshots/login.cy.js/Screenshot tests -- takes a screenshot.png
cy.screenshot()
})
})
By default, screenshot capture is true. However, you can disable it by the following setting under the support/index.js file.
Cypress.Screenshot.defaults({
screenshotOnRunFailure: false,
})
Check this out: RIPE MD 320 Hash Generator - The goal of using a cryptographic hash function, such as RIPEMD-320, is to take an input (or “message”) and generate a fixed-length string of characters, known as the “hash value,” that is unique to the input. This can be used for many things, including digital signatures, message authentication, and data integrity.
Video
Video features help in finding the bug and make debugging easy when there is the failure of the test case. Since we can watch what happens when a series of test steps are running, it becomes very easy to track down the exact step when some test case fails. Here we are comparing Playwright vs Selenium vs Cypress to see how we can take videos using these frameworks.
Playwright:
In the Playwright video of test cases handled by the “video” option in the config file. By default videos are off.
// @ts-check
/** @type {import('@playwright/test').PlaywrightTestConfig} */
const config = {
use: {
video: 'on-first-retry',
},
};
module.exports = config;
In option video, we can pass the below parameter
“off” — Do not record video.
“on” — Record video for each test.
“retain-on-failure” — Record video for each test, but remove all videos from successful test runs.
“on-first-retry” — Record video only when retrying a test for the first time.
Selenium:
Selenium WebDriver does not have any built-in mechanism to record videos of the test cases that are executed. In Selenium, to capture the video, we have extensions available. Using these extensions, we can record the test case execution steps, and when some of the test cases fail to record, it is helpful to know the exact reason for test step failure.
Cypress:
Cypress records the video automatically of every .spec file. When we run the command cypress run video of each .spec file is recorded. Videos are saved automatically under the folder cypress/videos. We can turn off the video entirely by making the flag video: **false **in the configuration file.
Run your Playwright automation scripts across 3000+ environments. Try LambdaTest Now!
Check this out: A sentence Counter is a tool to determine how many sentences make up a given article or document.
iFrame
iFrame is an HTML element that embeds another document within the current HTML document. Using iFrames, you can add content from another website to a web page. It allows you to engage users by supplementing your content, including sharing and embedding YouTube videos and external photos.
Below is the syntax for iFrame:
<iframe id="parent_iframe" name="demo_parent_iframe" width="700" height="450" src="https://bit.ly/38erOdQ">
</iframe>
Below screenshot is the example of Nested iFrame where we have Parent and Child both iFrame. The screenshot below shows the “Click Here” button in the Parent and Child iFrame.
Playwright:
When testing iFrames in Playwright, we can use the method FrameLocator, which allows us to retrieve the iFrame and locate elements inside that iFrame. FrameLocator captures the logic sufficient to retrieve the iFrame and locate elements in that iFrame.
Let’s take a simple example:
Let us now launch https://the-internet.herokuapp.com/iframe. We have to enter the data in the text area, which is under the iFrame.
Inspect the text area of this frame. Notice below that it is represented by the tag ‘iframe’ having the ‘id’ mentioned below.
We have a method called “frameLocator” in the Playwright, as seen below. Using this method, we can identify the above frame that we have inspected.
Here we are using the CSS Selector to identify the frame
const frame1 = page.frameLocator(‘#mce_0_ifr’).locator(‘html’)
Once we have located the frame, we have to click it before we start typing into it. Code snippet attached below
import {test,expect} from '@playwright/test'
test("frames", async ({ page }) => {
await page.goto('https://the-internet.herokuapp.com/iframe')
const frame1 = page.frameLocator('#mce_0_ifr').locator('html')
await frame1.click()
await frame1.type('Welcome to playwright')
await page.pause()
})
Selenium:
Selenium is capable of working with iFrames in a simple way. In the case of Selenium, we have to move to the iFrame window and do all activities that you want to do. Once all actions are done, then move back to the main page of the application to do further actions.
By default, Selenium has access to the parent browser driver. The driver focus must change from the main browser window to the frame to access a frame element.
There are various ways to switch to frames.
switchTo().frame(id) — The frame’s id or name is passed as input to the switchTo() method.
switchTo().frame(i) — The index of the frame is passed as an argument. The index starts from zero.
switchTo().frame(webelement n) — The webelement of the frame is passed as an argument.
switchTo().defaultContent() — Switching focus from the frame to the main page.
Let’s take a simple example:
Launch https://the-internet.herokuapp.com/iframe. We have to copy the text located under the left bottom iFrame.
Code snippet is attached below.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import java.util.concurrent.TimeUnit;
public class iFrameTest{
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver", "C:\Users\kailashpathak\Desktop\Java\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("https://the-internet.herokuapp.com/frames");
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
// Click in Nested link
driver.findElement(By.partialLinkText("Nested")).click();
// In below line we are switching to the frame in left bottom side
driver.switchTo().frame("frame-bottom");
WebElement text = driver.findElement(By.cssSelector("body"));
System.out.println("Frame text: " +text.getText());
driver.close();
}
}
Cypress:
Cypress does not handle iFrame directly; we need to install the plugin and use the plugin to perform actions on elements inside the iframe. We can install the plugin and then use the custom commands.
npm install -D cypress-iframe
Notice below that it is represented by the tag ‘iframe’ having the ‘class’ mentioned below.
The below code works with elements inside an iFrame.
import 'cypress-iframe'
describe('ifrane example', function () {
// test case
it('iframe Test cases', function (){
// launch the URL
cy.visit("https://jqueryui.com/draggable/");
// frame is loading in below line
cy.frameLoaded('.demo-frame');
//shifting the focus
cy.iframe().find("#draggable").then(function(t){
const frmtxt = t.text()
//assertion to verify text
expect(frmtxt).to.contains('Drag me around');
cy.log(frmtxt);
})
});
});
Check this out: SHA384 hash generator generates a SHA384 hash (SHA384 encoded) containing 96 hexadecimal digits. This can be used as a secure password or as a key to protecting essential data, such as personal information, money transactions, and much more.
Handling API Requests
Testing doesn’t mean to do verification of elements in UI only, but we also have to verify data that is bound with these elements from the API side as well.
API (Application Programming Interface) is the middle layer between the presentation and the database layer. APIs play a very important role in communication between one software application to another. Because of this reason, testing of API’s has become very important nowadays. Here we are comparing Playwright vs Selenium vs Cypress to see how we can do API testing using these frameworks.
Playwright:
Playwright can be used to get access to the REST API of your application. Below are examples of how we can automate the API request using the methods (GET, POST, PUT, and DELETE)
GET Method
// @ts-check
const { test, expect } = require("@playwright/test");
test.describe("API Testing with Playwright", () => {
const baseurl = "https://reqres.in/api";
test("GET API Request with -- Valid 200 Response ", async ({ request }) => {
const response = await request.get(`${baseurl}/users/2`);
expect(response.status()).toBe(200);
});
});
POST Method
// @ts-check
const { test, expect } = require("@playwright/test");
test.describe("API Testing with Playwright", () => {
const baseurl = "https://reqres.in/api";
test("POST API Request with -- Valid 201 Response ", async ({ request }) => {
const response = await request.post(`${baseurl}/users/2`, {
data: {
id: 123,
},
});
const responseBody = JSON.parse(await response.text());
expect(responseBody.id).toBe(123);
expect(response.status()).toBe(201);
});
});
PUT Method
// @ts-check
const { test, expect } = require("@playwright/test");
test.describe("API Testing with Playwright", () => {
const baseurl = "https://reqres.in/api";
test("PUT API Request with -- Valid 201 Response ", async ({ request }) => {
const response = await request.put(`${baseurl}/users/2`, {
data: {
id: 245,
},
});
const responseBody = JSON.parse(await response.text());
expect(responseBody.id).toBe(245);
expect(response.status()).toBe(200);
});
});
DELETE Method
// @ts-check
const { test, expect } = require("@playwright/test");
test.describe("API Testing with Playwright", () => {
const baseurl = "https://reqres.in/api";
test("DELETE API Request with -- Valid 201 Response ", async ({
request,
}) => {
const response = await request.delete(`${baseurl}/users/2`, {});
expect(response.status()).toBe(204);
});
});
Selenium:
Selenium is primarily a browser automation tool and is not designed for API testing. It allows you to interact with web browsers and automate user interactions on a website or web application. But, Selenium can’t be used to directly test the functionality of an API, as it doesn’t provide any functionality to send HTTP requests and assert on the API’s response.
However, LambdaTest can be used to perform API automation using Selenium. LambdaTest is a cloud-based platform that provides a Selenium Grid that allows you to perform Selenium automation testing on various browsers, operating systems, and devices.
One way to use Selenium to test an API using LambdaTest is by launching a browser on LambdaTest’s Selenium grid, navigating to the API endpoint URL, and then using Selenium to interact with the API by sending HTTP requests and asserting on the response. This is similar to how you would test a website or web application.
Cypress:
Using Cypress, we can do API automation. To make a Cypress API call, Cypress has provided the command cy.request(). GET, POST, PUT, and DELETE are some methods that are part of the API (Application Programming Interface) testing, which can be performed by Cypress.
Below are the example of different methods:
GET Method
To perform a GET operation, we shall make an HTTP request with the cy.request() and pass two parameters in cy.request() Method name and URL.
it("GET API testing Using Cypress", () => {
cy.request("GET", "https://reqres.in/api/users?page=2").should((response) => {
expect(response.status).to.eq(200);
});
});
POST Method
To perform a POST operation, we shall make an HTTP request with the cy.request() and pass three parameters in cy.request() Method name and URL and body.
it("POST API testing Using Cypress", () => {
cy.request("POST", "https://reqres.in/api/users", {
name: "morpheus",
job: "leader",
}).should((response) => {
expect(response.status).to.eq(201);
});
});
PUT Method
To perform a PUT operation, we shall make an HTTP request with the cy.request() and pass three parameters in cy.request() Method name and URL and body.
it("PUT API testing Using Cypress", () => {
cy.request("PUT", "https://reqres.in/api/users/2", {
name: "morpheus",
job: "zion resident",
}).should((response) => {
expect(response.status).to.eq(200);
});
});
DELETE Method
To perform a DELETE operation, we shall make an HTTP request with the cy.request() and pass two parameters in cy.request() Method name and URL.
it("DELETE API testing Using Cypress", () => {
cy.request("DELETE", "https://reqres.in/api/users/2").should((response) => {
expect(response.status).to.eq(204);
});
});
Check this out: SHA256 hash function generator generates a SHA256 hash (SHA256 encoded) containing 64 hexadecimal digits. This can be used as a secure password or as a key to protecting essential data, such as personal information, money transactions, and much more.
Community support
Selenium is an open-source suite of tools for automated web testing, and it has a large and active community of users and developers. There are several ways that community members can get support and help with Selenium. Here we are comparing Playwright vs Selenium vs Cypress to see their community support.
Playwright:
Playwright is a new player in the market and has a smaller but growing set of community resources. There are numerous resources available to help users get started with Playwright and troubleshoot any issues they may encounter. The Playwright team has great documentation and an active presence on GitHub, where users can submit bug reports and feature requests, and contribute to the development of the tool. The team is also responsive to questions and feedback from the community.
In addition to the official channels, there are a number of community-run forums and resources where users can get help with Playwright. These include Stack Overflow, Reddit, and the Playwright Gitter chat.
Selenium:
There are several ways that community members can get support and help with Selenium:
Selenium documentation: It is a comprehensive resource that covers all aspects of using Selenium. It includes tutorials, a reference guide, and information on installing and configuring Selenium.
Selenium forums: The Selenium forums are a good place to ask questions and get help with Selenium. There are separate forums for different programming languages, as well as a general discussion forum.
Selenium Slack channel: The Selenium Slack channel is an active community of Selenium users and developers who are available to answer questions and provide support. To join the Selenium Slack channel, you will need to request an invite by filling out a form on the Selenium website.
Stack Overflow: Stack Overflow is a popular question-and-answer website for programmers, and it has a large number of questions and answers related to Selenium. You can search for answers to your questions or ask a new question and get help from the community.
Cypress:
Cypress has a growing Community, and their documentation is excellent. In addition, there are numerous unofficial forums and communities where Cypress users can connect and share their experiences with the tool.
In addition to online forums, the Cypress community also provides support through various resources and documentation. The Cypress website features detailed documentation on installing and using the tool, a comprehensive API reference, and a library of example projects. There are also many blog posts, tutorials, and video resources available online that guide on using Cypress for various testing tasks.
Check this out: An SHA-1 Hash calculator is a tool that creates a 160-bit (20-byte) hash value for a specified input, generally referred to as a “message” or “data.”
Comparison Table: Playwright vs Selenium vs Cypress
| PLAYWRIGHT | SELENIUM | CYPRESS | ||||
| Language Support | JavaScript Java, C#, Python, Ruby | JavaScript Java, C#, Python, Ruby | JavaScript/TypeScript | |||
| Browser Support | Chrome, Edge, Firefox, Safari | Chrome, Edge, Firefox, Safari | Chrome, Edge, Firefox, Safari | |||
| Framework Support | Jest/Jasmine, AVA, Mocha, and Vitest | Mocha, Jest/Jasmine,, TestNG, JUnit, Cucumber and NUnit | Supports Mocha, Jest/Jasmine, Cucumber | |||
| Continuous Integration | Can be easily integrated with continuous integration tools like Jenkins | Can be easily integrated with continuous integration tools like Jenkins | Can be easily integrated with continuous integration tools like Jenkins | |||
| Ease of use | Playwright has a user-friendly interface and requires minimal setup | Selenium requires more setup and has a steeper learning curve | Cypress has a user-friendly interface and requires minimal setup | |||
| Test Writing Experience | Intuitive | Moderate | Intuitive | |||
| DOM manipulation | Easy | Moderate | Easy | |||
| Community Support | Growing community | Large and active community with good documentation and support resources | Active community with good documentation and support resources | |||
| Support for headless mode | YES | YES | YES | |||
| Parallel Execution | Supports parallel execution | Supports parallel execution | Supports parallel execution using CI/CD tool | |||
| Built-in network traffic control | YES | NO | YES | |||
| Setup complexity | Easy Setup | Requires some effort to build the framework | Easy Setup | |||
| Iframe Support | YES | YES | Iframe support through plugin e.g. cypress-iframe | |||
| Driver | No driver required | Each browser requires its driver | No need of driver bindings | |||
| Multi Tab Support | YES | NO | YES | |||
| Drag & Drop Support | YES | YES | YES | |||
| Test Assertions Libraries | Mocha, Chai | PyUnit, JUnit, TestNG almost any language-specific test framework can be adapted. | Mocha, Chai | |||
| Inbuilt reports | YES | NO | Default Reporter is Spec, Customizable for other supported reporters | |||
| Cross-Domain Support | YES | YES | YES | |||
| Debug features | Playwright has built-in debugging tools and a time-traveling feature for easy debugging | Selenium does not have built-in debugging tools | Cypress has built-in debugging tools and a time-traveling feature for easy debugging | |||
| Automatic Waiting | YES | NO | YES | |||
| Dashboard | NO | NO | Provided as Premium/Paid Feature | |||
| Community Support | Good | Very Good as the user base is more. | Good | |||
| Built-in Screenshot, VIDEO feature | YES | No built-in screenshot feature. Customization is required to add this feature. | YES | |||
| Pricing | Free for open-source projects, paid for commercial use | Free for all use cases | Free for open-source projects, paid for commercial use | |||
Obtain certifications in test automation frameworks like Playwright, Selenium, and Cypress that enhance your resume and increase your chances of obtaining a highly sought-after position. Obtain this certification as tangible proof of your expertise in test automation on LambdaTest’s cloud infrastructure, displaying mastery of advanced knowledge and skills.
Check this out: The SHA512 Hash Calculator lets you generate the SHA512 hash of any string.
Wrapping up
In this blog on Playwright vs Selenium vs Cypress, we have learned how tools like Playwright, Selenium, and Cypress are designed to automate web browsers for testing. While they each have distinct architectures, user bases, and objectives, it is up to testing teams to decide which tool is the most appropriate for their project and skill set. With the varying features and capabilities when comparing Playwright vs Selenium vs Cypress, it is essential to carefully consider the needs of the project and the team’s strengths before making a decision.
Top comments (0)