logo

Changelog 58: Flaky tests ❄️

Today, we’re excited to share our progress helping customers debug their flaky tests.
profile photo
Jason Laster
Image without caption
Today, we’re excited to share our progress helping customers debug their flaky tests.
The best example is the story of how we helped Metabase find a race condition in their DataSelector component. Because Replay DevTools includes React DevTools, it was easy to see that the passing test was rendering the Schema Picker and the failing test was rendering the Table Picker component. And when we added console logs in the DataSelector component, it was easy to spot a race condition between when the component was being mounted and the state was being set (blog post).
This example shows how much more powerful Replay is over session replay tools. With session replay, we could see that there were some slow network calls, but with Replay we can find the React component with the race condition and fix the problem at the source.
This week we shared a similar story in our Cypress Test Replay / Replay DevTools comparison. The best compliment we’ve received recently, is that Replay is helping folks folks fix their flakes and remove cy.wait in the process!

Additional updates

This was a big week for performance updates and a step towards designing a debugging environment that’s faster than Chrome DevTools.
We improved the Network Monitor performance when handling recordings with more than a thousand requests (pull request).
Image without caption
We improved the Source Viewer scroll performance by 4X for files with more than 100k lines (pull request).
Image without caption
We improved React DevTools performance and reliability by optimistically injecting the content script. React DevTools is the only panel we have not rebuilt in a time travel context, so it still has rough edges that we’re working through (pull request).
Image without caption
Happy Debugging,
The Replay team
Related posts
post image
In this failure, we look into a bug where a React component calls Math.round and gets a different value when the test passes and fails!
post image
This week we’re excited to share Elements Panel V3, which brings one second element loads.
post image
Today we’re excited to share an entirely new version of React DevTools which Brian Vaughn wrote from scratch.
Powered by Notaku