Changelog 36: Test Suite Comments 👋

Test Suites Comments entered into closed beta for Playwright. We published case study with Glide. We shared updates like 4X faster scope mapping, frontend architecture changes, docs for installing certificates, source maps for Node, and other bug fixes with source maps and data fetching. Also, a fun React bug fix with Brian Vaughn.
profile photo
Jason Laster
Image without caption
Test Suite Comments are entering closed beta today! If you have a Playwright test suite, send us an email and we’ll set you up.
We’ve always been excited about making failing tests easier to debug. Nobody enjoys being blocked by a 🔴 badge in a PR. Scrolling through CI logs is frustrating. Figuring out why a random test failed is the worst.
Test Suite Comments are one of those features that you take for granted quickly. We added comments to our DevTools repo last week and within a day, everyone was using them and sharing replays in our Discord.
If you’re as excited as we are, reach out and we’ll help set you up. We’re just getting started too so watch this space!

Glide Case Study

Image without caption
Glide is the easiest way to design data-driven applications. If you’re looking for a case study on how Replay helps you record the bug once and never worry about reproducing it again, this is for you!
🐌 “Before Replay we spent somewhere between 1–2 hours per day per dev in this reproducibility purgatory. The toll this was taking on our development velocity and our ability to respond and resolve issues was huge. We thought there must be a better way.” — Mark Probst, CTO


5X Faster Scope Mapping One of the biggest differences between Replay and Chrome DevTools is that Replay maps Files, Call Stacks, Scopes and Expressions. This means that when you open a replay, add a print statement, and jump to a console log, everything just works. Under the hood, we use Babel to parse the original and generated files and map the source locations and scopes.
Previously we were using Babel’s default traversal API which has scopes built in, but in the extreme case it took 50 seconds. We’re now using their faster traversal API which takes 9 seconds.


One of the reasons we’re focusing on the frontend’s architecture is that Replay DevTools started as a fork of Firefox DevTools. While that was a good starting point, we’ve learned that there are simpler ways to model a time-travel debugger.
This week we deleted 13k lines of unused DevTools code (PR), migrated the source logic to TS (PR), and prepared an 8k line refactor of the sources logic (1, 2). Thank god for TypeScript, Hot Reloading, and Time Travel Debugging (loom)!
The new architecture is far easier to understand and fine-tune. It will also be far easier for others to fork and experiment with their own designs 👨‍🎨

Additional Updates

  • Installing Certificates: We published docs on how to install certificates when getting setup.
  • Source Map Toggle: We fixed two issues jumping between original and generated files (1, 2)
  • Node SourceMaps: Inline source maps are now supported is standard for ts-node (PR)
  • Pause Data We fixed three issues that occurred when fetching data before it was ready (1, 2, 3)

Replays in the Wild

Last week Brian filed a React.js bug report for a potential issue in React’s suspense cache implementation (Tweet). Here’s the original Loom and Replay. It was incredible to see Mateusz jump in a couple of hours later and find the problem.
Seeing Mateusz discover the tweet, dive into React’s guts while we were asleep, and find the bug, reminded us of how magical the early GitHub days felt when you could collaborate with anyone regardless of where they were or how much context they had.
Image without caption
Related posts
post image
Released the Cypress panel, our third framework integration, to bring local debugging into CI - join our beta. We had some fun with ChatGPT and print debugging. We released a new comments editor, stepping machine and updated sourc...
post image
We announced two big Test Suite updates - switching our default CI browser to Chromium and an update to Cypress and opening the beta to more partners. We also featured one of our beloved users, Mateusz using Replay to debug a Typs...
post image
We shipped Test Suites Dashboard for Playwright - helps you see your test runs and debug them after the fact, especially useful when you see when a test started failed and compare to passing runs. We also shared updates to Test Su...
Powered by Notaku