We’ve developed a prototype tool for making frontend changes with an autonomous AI developer. We are using OpenHands as our AI developer, and using Replay to provide the context needed to make changes reliably and validate the problem was fixed.
This flow greatly streamlines the process of frontend development. AI developers can make straightforward bug fixes and improvements, but have trouble understanding how large applications work, as we described in an earlier post. Using Replay to get over this hurdle allows the AI to handle even a complex new feature, by breaking it down into discrete tasks to solve with minimal guidance.
Configuration requires a couple new GitHub workflows in a project: one to ask OpenHands to generate a PR on issues containing Replay links when it gets tagged, and one to rerecord the provided Replay once the PR is created, to show the impact the PR has.
With these workflows in place, fixing a bug or making an improvement requires the following steps:
- Use the Replay browser to record the behavior which needs to change.
- Comment on the resulting recording to describe the change to make.
- File an issue with a link to the recording.
- Tag OpenHands in the issue.
- OpenHands uses Replay to get the additional context and generates a PR soon after.
- When a preview deployment is available for the PR, a new recording is generated of the same interactions to show the new behavior.
- The rerecording action comments on the PR with before/after screenshots showing the changes in the new recording.
This GitHub issue shows these steps for performing the same fix we studied in the earlier post. We asked OpenHands three times to fix the issue and each time it produced a PR which the rerecording clearly shows is a suitable fix [1] [2] [3].
This video walks through the steps above for this issue.
We’ll continue to use this to develop Replay’s frontend and will have more to share soon. If you’d like to try this out we’d love to hear from you, email us at hi@replay.io or fill out our contact form and we’ll be in touch.