logo

Changelog 58: Flaky tests ❄️

Today, we’re excited to share our progress helping customers debug their flaky tests.
profile photo
Jason Laster
Image without caption
Today, we’re excited to share our progress helping customers debug their flaky tests.
The best example is the story of how we helped Metabase find a race condition in their DataSelector component. Because Replay DevTools includes React DevTools, it was easy to see that the passing test was rendering the Schema Picker and the failing test was rendering the Table Picker component. And when we added console logs in the DataSelector component, it was easy to spot a race condition between when the component was being mounted and the state was being set (blog post).
This example shows how much more powerful Replay is over session replay tools. With session replay, we could see that there were some slow network calls, but with Replay we can find the React component with the race condition and fix the problem at the source.
This week we shared a similar story in our Cypress Test Replay / Replay DevTools comparison. The best compliment we’ve received recently, is that Replay is helping folks folks fix their flakes and remove cy.wait in the process!

Additional updates

This was a big week for performance updates and a step towards designing a debugging environment that’s faster than Chrome DevTools.
We improved the Network Monitor performance when handling recordings with more than a thousand requests (pull request).
Image without caption
We improved the Source Viewer scroll performance by 4X for files with more than 100k lines (pull request).
Image without caption
We improved React DevTools performance and reliability by optimistically injecting the content script. React DevTools is the only panel we have not rebuilt in a time travel context, so it still has rough edges that we’re working through (pull request).
Image without caption
Happy Debugging,
The Replay team
Related posts
We share our plans on helping AIs resolve issues fully automatically
Over the past couple of years, JS frameworks like Next.js and Remix have gone full stack. We believe it’s time for DevTools to join the party!
We discuss a project for automatically fixing test failures
Powered by Notaku