E2E testing challenges using Playwright

So you finally decided to implement e2e tests 🚀. You have a webapp with production traffic and you want to deploy safely knowing your high level features work. Unit test is not a good plan time-wise ⏱️ while e2e tests will give you that coverage you want to continue moving fast 🏃.

You choose Playwright. Playwright is currently the default e2e testing framework out there.

Initial implementation goes well 👍. Playwright is simple and straightforward. Locators are easy to declare and you add a couple tests to your CI/CD pipeline. You feel safer now and continue with fast speed 🧘‍♀️.

One month later 📅

You realize you are spending too much time fixing your e2e tests . Locators need to be updated all the time. Navigation changed. Interaction with DOM elements changed. Fixing tests start to slow you down. 😩

Next steps? Hire QA? Remove tests and pray? Go low level unit testing instead? Standardize FE elements? 🤨

Low level implementation for high level features: the GAP

The problem with Playwright or similar frameworks (Cypress, Selenium, etc) is that they work with implementation details. Locators are literal the implementation of the objects you want to interact with.

You might encounter these:

// Deeply nested CSS selector that depends on exact DOM structure
await page.click('.main-container > div.row > div.col-md-8 > form > div:nth-child(3) > button');

// Selecting elements based on their exact position in a list
await page.click('ul.dropdown-menu > li:nth-child(2)');

// Relying on internal implementation IDs from a component library
await page.click('[id="select2-dropdown-results-9-4"]');

// Using XPath that depends on exact HTML structure
await page.click('xpath=//div[contains(@class, "sidebar")]/ul/li[3]/div/button');

Devs will change these aspects of the components daily. Your tests won't be robust to those changes.

This is the result of testing high level features with low level frameworks 😳. Ideally your test shouldn't break if functionality is still there, regardless if the button was changed to a different <div>. You don't want to test how it was implemented, you want to test what your users will be doing.

Introducing Lila

If you got this far and agree with this, let me introduce Lila, a new testing framework for fast moving teams I'm developing.

Lila remains high level for doing proper high level testing. Your tests declare what your users will do, without using how it was implemented 😎. Lila has AI Agents 🤖 that will try complete the instructions with the available implementation, making it robust to your everyday dev changes.

Without more spam, you can find more information in its repo: https://github.com/lila-team/lila

Final thoughts

Would love to hear in the comments if this problem resonates with you, or if you found good engineering practices to keep Playwright stable over time or if you think a tool like Lila would work.

Thanks for reading!