Changelog 37: Test Suite Dashboard 🚀

We shipped Test Suites Dashboard for Playwright - helps you see your test runs and debug them after the fact, especially useful when you see when a test started failed and compare to passing runs. We also shared updates to Test Suite Comments, Chromium, Console Search, some bug fixes around Source Maps with photos from our Summer Offsite.
profile photo
Jason Laster
Image without caption
We’re excited to ship the Test Suite Dashboard to closed beta today! If you have a Playwright Test Suite, send us an email and we’ll set you up.
The Test Suite Dashboard helps you see your recent test runs and debug your tests after the fact. The dashboard is especially useful when you want to see when a test started failing and compare the failing and passing runs.
This past week our own react_devtools.js test started failing. The dashboard helped us see that the test started failing last Monday and begin comparing several react_devtools.js replays. When we added a print statement in the createReactDevTools function, we saw that when the test passed the function was called three times and when it failed it was called once. This helped us realize that our usage of React Offscreen was suppressing important renders (video).
Figuring out why react_devtools.js started failing without Replay, would have required switching between multiple git branches and trying to remember what was happening in each version. With Replay, we were able to debug the test runs side-by-side with the passing tests on the left, and the failing tests on the right!
Image without caption

Frontend Architecture

We are also releasing the new Console and Object Inspector today as an experimental feature. You can enable it by going to Experimental > Settings. The new Console is built with modern React APIs like Suspense Cache and designed for speed.
One of the counter-intuitive learnings we’ve had this past year is that time-travel debugging is actually simpler than live debugging because everything is in the past. This has helped us add helpful constraints that have dramatically simplified the architecture.
If you’d like to dive deeper, you can compare the new and old sources logic which we also replaced last week. The old implementation processed new sources in real time. The new implementation processes the sources up front and is far faster and simpler.
One of the hardest decisions for any team is when to invest in architecture. For us we see it as an important step for improving the frontend’s overall responsiveness and correctness.

Additional Updates

Test Suite Comments are now managed by a GitHub App which makes onboarding simpler and means that we can now update a single Replay Test Suite comment.
Chromium builds are now managed with EngFlow and under three minutes. We’re beginning to prioritize Chromium, so expect more updates on this front soon.
Console Search now uses a DOM based approach which is far closer to how native browser search works. Here’s a quick video of the new search in action (PR).

Bug Fixes

Source Maps We fixed a race condition that prevented some recordings from having source maps.
Quick Open Modal now opens and shows results far faster. You can launch the Quick Open Modal by pressing cmd+p.

Summer Offsite

We had a great time at the Cerro Pampa polo ranch last week. If you’re looking for a getaway location for your team with a clay tennis court and live polo matches that is just north of SF, you really have only one option  🏇 😜
Image without caption
Related posts
post image
Released the Cypress panel, our third framework integration, to bring local debugging into CI - join our beta. We had some fun with ChatGPT and print debugging. We released a new comments editor, stepping machine and updated sourc...
post image
We announced two big Test Suite updates - switching our default CI browser to Chromium and an update to Cypress and opening the beta to more partners. We also featured one of our beloved users, Mateusz using Replay to debug a Typs...
post image
Test Suites Comments entered into closed beta for Playwright. We published case study with Glide. We shared updates like 4X faster scope mapping, frontend architecture changes, docs for installing certificates, source maps for Nod...
Powered by Notaku