DEV Community

Playwright end to end Testing profile image Debbie O'Brien
Debbie O'Brien for Playwright end to end Testing

Posted on

A better global setup in Playwright reusing login with project dependencies

#testing #e2e #auth #playwright

Global setup is being used by many teams and companies to login to an application and then use this setup for tests that need to be in an authenticated state; however, it is lacking some important features. When you use global setup, you don't see a trace for the setup part of your tests and the setup doesn't appear in the HTML report. This can make debugging difficult. It's also not possible to use fixtures in global setup.

In order to fix this issue, project dependencies were created.

What are project dependencies?

Project dependencies are a better way of doing global setup. To add a dependency, so that one project depends on another project, create a separate project in the Playwright config for your setup tests, where each test in this project will be a step in the setup routine.

Every time you run a test from the basic project, it will first run the tests from the setup project.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'basic',
      dependencies: ['setup'],
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

By using project dependencies, the HTML report will show the setup tests, the trace viewer will record traces of the setup, and you can use the inspector to inspect the DOM snapshot of the trace of your setup tests and you can also use fixtures.

Running sequence

Tests in the 'setup' project will run first and then the tests in the 'chromium', 'webkit', and 'firefox' projects will run in parallel once all tests in setup have completed.

chrome, firefox and safari project that depend on a setup project

What happens if a dependency fails?

In the following example, you will see an 'e2e tests' project that depends on both the 'Browser Login' project and the 'Database' project. The 'Browser Login' project and the Database project will run in parallel. However, as the 'Database' project failed, the 'e2e tests' project will never run as it depends on both the 'Browser Login' and the 'Database' projects to pass.

e2e tests project that depends on browser login project and database project

Project Dependency example

Playwright runs tests in isolated environments called Browser Contexts. Every test runs independently from any other test. This means that each test has its own local storage, session storage, cookies, etc. However, tests can use the storage state, which contains cookies and a local storage snapshot, from other tests so as to run tests in a logged in state.

Let's create an example of how to use Project dependencies to have a global setup that logs into Wikipedia and saves the storage state so that all tests from the e2e project start running in this logged in state.

First, install Playwright using the CLI or VS Code extension. You can then modify the config file, create your login test, and an e2e test that starts from a logged in state by using storage state.

Configuring the setup project

Start by creating a basic playwright.config.ts file or modifying the one that has already been created for us. The options needed are the testDir option which is the name of the directory where you want to store your tests and the projects options which is what projects you want to run.

A project is a logical group of tests that run using the same configuration. The first project you need is one called 'setup' and by using testMatch you can filter any files that end in setup.ts and only these tests will be run when running the 'setup' project.

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

Create a login test

Next, create a login.setup.ts. To make it easier to see that this is a setup test, you can import test as setup and then instead of using the word test you can use the word setup when writing your test.

The aim is to create a test that logs into Wikipedia and ensures that the user is in a logged in state. You can use Playwright's test generator either from the VS Code extension or using the CLI to open the Playwright Inspector, and generate the code by clicking on the 'Log in' button and filling out the username and password. You can then add the assertion to ensure that once logged in you can see the 'Personal Tools' option.

If you don't already have a username or password, you can quickly create an account and then use your own credentials to see that the tests work.

// login.setup.ts
import { test as setup, expect } from '@playwright/test';

setup('do login', async ({ page }) => {
  await page.goto('https://en.wikipedia.org');
  await page.getByRole('link', { name: 'Log in' }).click();
  await page.getByPlaceholder('Enter your username').fill('your_username');
  await page.getByPlaceholder('Enter your password').fill('your_password');
  await page.getByRole('button', { name: 'Log in' }).click();

  await expect(page.getByRole('button', { name: 'Personal tools' })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

Using env variables

In order for your username and password to be secure, you can store them as .env variables and access them in your tests through process.env.USERNAME! and process.env.PASSWORD!.

// login.setup.ts
import { test as setup, expect } from '@playwright/test';

setup('do login', async ({ page }) => {
  await page.goto('https://en.wikipedia.org');
  await page.getByRole('link', { name: 'Log in' }).click();
  await page.getByPlaceholder('Enter your username').fill(process.env.USERNAME!);
  await page.getByPlaceholder('Enter your password').fill(process.env.PASSWORD!);
  await page.getByRole('button', { name: 'Log in' }).click();

  await expect(page.getByRole('button', { name: 'Personal tools' })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

Don't forget to install the dotenv package from npm.

npm i dotenv
Enter fullscreen mode Exit fullscreen mode

Once the package is installed, import it in your Playwright config with require('dotenv').config(); so you have access to the .env variables.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
require('dotenv').config();

export default defineConfig({
  testDir: './tests',

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests',
      dependencies: ['setup'],
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

Next, create a .env file and add the username and password. Make sure to add this file to the .gitignore so that your secrets don't end up on CI.

USERNAME: your_username
PASSWORD: your_password
Enter fullscreen mode Exit fullscreen mode

You can use GitHub secrets when working with CI. Create your repository secrets in the settings of your repo and then add the env variables to the GitHub actions workflow already created when installing Playwright.

env: 
  USERNAME: ${{secrets.USERNAME}}
  PASSWORD: ${{secrets.PASSWORD}}
Enter fullscreen mode Exit fullscreen mode

Create a e2e tests project

Create a project called e2e tests logged in. This project will depend on the 'setup' project and will match any test files that end in loggedin.spec.ts.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
require('dotenv').config();

export default defineConfig({
  testDir: './tests',

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests logged in',
      testMatch: '**/*loggedin.spec.ts',
      dependencies: ['setup'],
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

Adding storage state

The setup project will write storage state into an 'auth.json' file in a .auth folder inside the playwright folder. This exports a const of STORAGE_STATE to share the location of the storage file between projects.

Next, you need to tell your test to use the STORAGE_STATE variable you have created as the value of its storageStage. This returns the storage state for the browser context and contains the current cookies and a local storage snapshot.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import path from 'path';
require('dotenv').config();

export const STORAGE_STATE = path.join(__dirname, 'playwright/.auth/user.json');

export default defineConfig({
  testDir: './tests',

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests logged in',
      testMatch: '**/*loggedin.spec.ts',
      dependencies: ['setup'],
      use: {
        storageState: STORAGE_STATE,
      },
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

At the moment, there is nothing saved in the STORAGE_STATE so the next step is to populate the context with the storage state after login actions have been performed. By doing this you only have to log in once and the credentials will be stored in the STORAGE_STATE file, meaning you don't need to log in again for every test. Start by importing the STORAGE_STATE from the Playwright config file and then use this as the path to save your storage state to.

// login.setup.ts
import { test as setup, expect } from '@playwright/test';
import { STORAGE_STATE } from '../playwright.config';

setup('do login', async ({ page }) => {
  await page.goto('https://en.wikipedia.org');
  await page.getByRole('link', { name: 'Log in' }).click();
  await page.getByPlaceholder('Enter your username').fill('TestingLogin');
  await page.getByPlaceholder('Enter your password').fill('e2etests');
  await page.getByRole('button', { name: 'Log in' }).click();

  await expect(page.getByRole('button', { name: 'Personal tools' })).toBeVisible();

  await page.context().storageState({ path: STORAGE_STATE });
});
Enter fullscreen mode Exit fullscreen mode

Create a e2e test

Create some e2e tests that continue testing the application from a logged in state. When running all tests in the file the setup will only be run once and the second test will start already authenticated because of the specified storageState in the config.

// e2e-loggedin.spec.ts
import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  await page.goto('https://en.wikipedia.org');
});

test('menu', async ({ page }) => {
  await page.getByRole('link', { name: 'TestingLogin' }).click();
  await expect(page.getByRole('heading', { name: /TestingLogin/i })).toBeVisible();
  await page.getByRole('link', { name: /alerts/i  }).click();
  await page.getByText('Alerts', { exact: true }).click();
  await page.getByRole('button', { name: /notice/i  }).click();
  await page.getByText('Notices').click();
  await page.getByRole('link', { name: /watchlist/i  }).click();
})

test('logs user out', async ({ page }) => {
  await page.getByRole('button', { name: /Personal tools/i }).check();
  await page.getByRole('link', { name:  /Log out/i }).click();
  await expect(page.getByRole('heading', { name: /Log out/i })).toBeVisible();
  await expect(page.getByRole('link', { name: 'Log in', exact: true })).toBeVisible();
})
Enter fullscreen mode Exit fullscreen mode

Configure the baseURL

When using the same URL for both the 'setup' and 'e2e tests', you can configure the baseURL in the playwright.config.ts file. Setting a baseURL means you can just use page.goto('/') in your tests which is quicker to write, less prone to typos and it makes it easier to manage should the baseURL change in the future.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import path from 'path';
require('dotenv').config();

export const STORAGE_STATE = path.join(__dirname, 'playwright/.auth/user.json');

export default defineConfig({
  testDir: './tests',

  use: {
    baseURL: 'https://en.wikipedia.org',
  },

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests logged in',
      dependencies: ['setup'],
      use: {
        storageState: STORAGE_STATE,
      },
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

You can then use a / instead of 'https://en.wikipedia.org' for the page.goto in all your tests going forward including the 'setup' test that you created.

// e2e-loggedin.spec.ts
import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  await page.goto('/');
});

//...
Enter fullscreen mode Exit fullscreen mode

Configure the HTML Reporter

If you don't already have this setup in your Playwright config, then the next step is to add the HTML reporter to the playwright.config.ts file in order to setup the HTML reports for your tests. You can also add the retries for CI, set fullyParallel to true and set the trace to be recorded on the first retry of a failed test.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import path from 'path';
require('dotenv').config();

export const STORAGE_STATE = path.join(__dirname, 'playwright/.auth/user.json');

export default defineConfig({
  testDir: './tests',
  // Configure the reporter
  reporter: ['html'],
  // Retry on CI only
  retries: process.env.CI ? 2 : 0,
  // Run tests in files in parallel
  fullyParallel: true,

  use: {
    baseURL: 'https://en.wikipedia.org',
    // run traces on the first retry of a failed test
    trace: 'on-first-retry',
  },

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests logged in',
      dependencies: ['setup'],
      use: {
        storageState: STORAGE_STATE,
      },
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

You can continue to add more projects to the config to add tests that don't require the user to be logged in. Using the testing filters of testIgnore you can ignore all the setup tests and logged in tests when running the tests in this project.

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import path from 'path';
require('dotenv').config();

export const STORAGE_STATE = path.join(__dirname, 'playwright/.auth/user.json');

export default defineConfig({
  testDir: './tests',
  // Configure the reporter
  reporter: ['html'],
  // Retry on CI only
  retries: process.env.CI ? 2 : 0,
  // Run tests in files in parallel
  fullyParallel: true,

  use: {
    baseURL: 'https://en.wikipedia.org',
    // run traces on the first retry of a failed test
    trace: 'on-first-retry',
  },

  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests logged in',
      dependencies: ['setup'],
      use: {
        storageState: STORAGE_STATE,
      },
    },
    {
      name: 'e2e tests',
      testIgnore: ['**/*loggedin.spec.ts', '**/*.setup.ts'],
    },
  ],
});
Enter fullscreen mode Exit fullscreen mode

When you don't specify a browser, tests will be run on Chromium by default. You can of course run these tests on different browsers and devices and setup more projects. To learn more about Projects and Browsers check out the Playwright docs.

Viewing the HTML report and trace viewer

Now you can run your tests from the CLI with the flag --trace on in order to see a full report of your tests including the 'setup' and 'e2e tests logged in' and also see a trace of each of them.

npx playwright test --trace on
Enter fullscreen mode Exit fullscreen mode

After running the tests, you should see in the CLI that 2 tests have passed and you can now run the command to open the HTML report.

npx playwright show-report
Enter fullscreen mode Exit fullscreen mode

This command opens up the Playwright HTML report where you can see your two projects, the 'setup' project containing the login test and the 'e2e tests logged in' project containing the menu test and the log out test.

html report showing both tests

You can open the report for each of the tests including the 'setup test' and walk through each step of the test which is really helpful for debugging.

html report showing steps of login test

To enhance the debugging experience even further, when you use the --trace on flag, you have a fully recorded trace of all your tests including the setup test. You can open the trace and view the timeline or go through every action, see the network requests, the console, the test source code and use dev tools to inspect the DOM snapshots.

trace viewer showing trace of the login test

By clicking on the pop out button above the DOM snapshot you get a full view of your trace where you can easily inspect the code with Dev tools and debug your global setup should you need to.

pop out to inspect trace and use dev tools

Conclusion

Using Project dependencies makes it so much easier to have a global setup with out of the box HTML reports and traces of your setup.

Top comments (22)

Collapse
 
r0nunes profile image
Renato O. Nunes •

Hey @debs_obrien,

I tried to create a new project, using a site conduit to test here. But, for me it doesn't work.

Idk why, but the storage file is not created before the test, and when I run the command npx playwright test --headed --project=chromium the test fail.

I tried to follow the official documentation but got the same results. =(

Image description

Image description

Image description

Can you help me with this "feature"? I'm having a problem with some tests and I'm trying to find an alternative to login.

Collapse
 
r0nunes profile image
Renato O. Nunes •

PS: If I put on the storage path into the test, the storage file is created.

e.g: await page.context().storageState({ path: './auth/user.json' });

Collapse
 
debs_obrien profile image
Debbie O'Brien •

do you have a playwright-reuse-auth folder? change it to playwright and see if that works. if not can you add this a post in our discord server as much easier to get more eyes on it there for better help: aka.ms/playwright/discord

Collapse
 
victorneves profile image
Victor Neves •

Hi @debs_obrien
We are starting implementing e2e tests on our project using Playwright and we have used this approach as it's also suggested on the documentation.
We only have one QA that doesn't have so much experience with Playwright (he has mostly worked with cypress and other existing projects that contain e2e tests are also done with cypress)
So after implementing the first basic suite of tests he got blocked because he wasn't able to make the group of test that checks if the user is able to access protected routes pass.
So I decided to do some tests in order to try to understand what could be the problem and I was able to make them pass but I'm not sure if what I did is correct.
Looking to Playwright's documentation fr me it's not totally clear the order where we should add the await page.context().storageState({ path: STORAGE_STATE }); and if when executing test that are defined on a external file that requires to access the data that saved on the JSON we always need to call it not.
Probably it's not too clear what I'm saying, so I'll gonna add some of the code to see if what I did, is correct or not

setup('authenticate', async ({ page }) => {
    await page.goto(urls.login);
    await page.locator(uiIdsCookies[envConfig.brand].cookieLayerAcceptBtn).click();
    // wait for cookie layer to disappear
    await page.waitForTimeout(customTimeout);

    await page.locator(uiIds[brand].loginButton).click();
    // wait for login form to appear
    await page.waitForTimeout(customTimeout);

    await page.locator(uiIds[brand].inputEmail).type(envConfig.username);
    await page.locator(uiIds[brand].inputPassword).type(envConfig.password);

    await page.locator(uiIds[brand].sendFormButton).click();
    // reads cookies from json file to be used to test pages that require authentication
    await page.context().storageState({ path: STORAGE_STATE });
    // the browser is closed

    // new browser is open
    await page.waitForURL(envConfig.devUrl); <-- this is a protected route that requires to be authenticated to access it
    await page.context().storageState({ path: STORAGE_STATE });
});
Enter fullscreen mode Exit fullscreen mode

My question is about the last storageState
Do we need to call it on every test that is executed on a new window that requires the user to be authenticated? If yes, it's not to clear for me why is called after and not before accessing the page. Is it because after the page is open, then it will make the browser read the cookies and so from the file? I would expect that we would define from where it would read the info, from he browser as it's the default behaviour or from the JSON

Hopefully with code I was able to be clear to my question.
Thanks in advance

Collapse
 
artem_qa profile image
AnewTranduil •

There are line in the dependent project config which set the json. You don't need to call await page.context().storageState({ path: STORAGE_STATE }); again.

projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e tests logged in',
      dependencies: ['setup'],
      use: {
        storageState: STORAGE_STATE,
        ^^^^^^^^^^^^^^^^^^^^^^^
      },
    },
  ],
Enter fullscreen mode Exit fullscreen mode
Collapse
 
philipfong profile image
Phil •

I've found that making use of storage state in combination with playwright fixtures to be much more useful than coupling storage state with global setup. I've documented my experience with it in case anyone is interested.

Collapse
 
kolianas profile image
Anastasiia Kolianovska •

Sounds interesting, could you please share your experience in more detail?

Collapse
 
philipfong profile image
Phil •

Sure I posted here about it.

Collapse
 
saurav_18_ profile image
Saurav Likhar •

Hey @debs_obrien ,
I have 2 case scenarios.
First case - I have 2 type of users whose session data I want to use for different tests. Suppose I have 2 test suites A.spec.ts and B.spec.ts. First test file will use the login credentials of user1 and second test file will use the login credentials of user2. How can I store the 2 users login data using this new setup file?
In earlier version of globalSetup.ts file, I used to open two different contexts of browser and then two different pages to login the two types of users and then save the data of respective login in different state.json files.

Second case - In particular test file, I want to use the logged in data for some tests and not for all the tests. How can I do that in new version of project configurations? Earlier I used to write const context = await browser.newContext({storageState: undefined}); inside the tests in which I do not want the logged in state data.

Collapse
 
debs_obrien profile image
Debbie O'Brien •

hi, can you post this to the Q&A on our discord server. would be much easier to answer/track and get others to look at it too. thanks

Collapse
 
raducybersec profile image
raducybersec •

Hey @debs_obrien ,
I am wondering if I can use playwright test code with azure AD MFA. I have tried using TOTP instead and otpauth to generate the code for authentication but I failed dramatically with {"ResultValue": "AuthenticationThrottled",
"Message": "Authentication is throttled throttle.sas.user.authmethod.extended for tenant: 5ec97021-b760-441e-a67c-c2479291ad5d, user: b1ae5226-6cc6-4f88-a1dd-abea51a4dda4, auth method: PhoneAppOTP. Authentication limit reached: [Duration:01:00:36][Identifier:<PII-REDACTED>][Level:100][Reason:RequestCount].",
"AuthMethodId": "PhoneAppOTP",
"ErrCode": 500121,}
is there any way to save my authentication?
Thank you

Collapse
 
debs_obrien profile image
Debbie O'Brien •

hi, i havent used Azure AD personally. If you post this question in our discord server someone should be able to help you out

Collapse
 
Sloan, the sloth mascot
Comment deleted
Collapse
 
debs_obrien profile image
Debbie O'Brien •

Can you file an issue with your operating system and version number etc. here is link to my repo with it working incase it helps: github.com/debs-obrien/playwright-...

Collapse
 
aromal_mg profile image
Aromal Mg • • Edited

Hi @debs_obrien

I have implemented login once using project dependency as mentioned in above post.

I have defined the session storage in project config as in the above document.
But before each test execution using session storage, I can see a browser getting opened and closed.

Below is the actual behaviour:

  1. Dependent setup project runs and saves session storage.
  2. A empty browser is getting opened and closed.
  3. Test are executed in the new browser with login state using saved session storage.

Why is the 2nd step happening. I am not able to figure the issue by debugging.
Is it expected or can anyone help me if you have experienced similar issue before.

Collapse
 
anandtiwari1 profile image
Anand Tiwari •

Hi @debs_obrien, I'm having a problem where we want the results of the tests to be posted to our management system, and since we are sharding our tests, we are running the setup tests on each m/c, resulting in multiple test sets ids. Is there any way we can create a test set once and pass it on to all tests running on different m/c?

Collapse
 
jackbob2020110 profile image
jackbob2020110 •

Hey @debs_obrien

I followed the your github sample code (github.com/debs-obrien/playwright-...) to create M365 OWA test. the strange thing here is that when i tried to run 'npx playwright test' for all test, except login ok, other test will trigger to locate elelement timeout. but when I run each them one by one, it works without any issue. i'm curious what i was missing.

Test timeout of 30000ms exceeded.
Error: locator.click: Target closed
=========================== logs ===========================

waiting for getByPlaceholder('Search')

12 |
13 | await page.waitForLoadState();

14 | await page.getByPlaceholder('Search').click();
| ^
15 |

16 | await page.waitForTimeout(2000);
17 |

at C:\Users\vagrant\code\learn-playwright\tests\page-searchmail.spec.ts:14:41
Enter fullscreen mode Exit fullscreen mode

here is what run page-searchmail.spec.ts only, it works with out any issue.

Image description

thanks.
_Jun

Collapse
 
alice3105 profile image
Alice Huynh •

Hey @debs_obrien
I wonder if I have two projects as dependencies then would they run at the same time or would they run in the order that they are listed?

Image description

View full discussion (22 comments)

Read next

signadot profile image

Why Quick Fixes Fail: Rethinking Microservices Testing

Signadot -

pauldmooney profile image

Enhancing the Developer Experience of Testing Part 2

Paul Mooney -

szabgab profile image

Rust 🦀 version 1.83.0 came out a few days ago. It is upgrade time!

Gabor Szabo -

cryptosandy profile image

How to Debug Code Effectively: Tools and Techniques

Crypto.Andy (DEV) -