Listening is a big part of a designer’s job. We’re supposed to explore, listen, accept criticism, and keep iterating. So design research is a foundational part of the job. The better you listen, the better you design.
But there is such a thing as too much. There are plenty of things you can’t know from interviews alone. Sometimes you do need to take a leap of faith. Sometimes you do need to ship something to see a real world reaction instead of trying to read the tea leaves from user interviews.
When I tell my fellow designers that we’ve interviewed 150 and counting, they look at me like I’ve grown a second head. And this morning I was asked about it by a beta tester:
That’s a mammoth amount of interviews, out of interest, if you had to do it all again, would you do that many, or is there a more optimal number?
I’m so glad he asked, because it made me realise how unique our situation is. In my opinion, 150+ interviews in a few months is too much for many kinds of apps, most websites, and entire categories of products. But there’s one big exception, in my mind: professional-grade tools that people use to do their jobs. Especially when it’s their job to design, build, and debug software. Software is hard!
At Replay, we’re making a time-traveling debugger. The harder the bug, the more our product is able to help coders and their teams. So we can’t just design for the happy path, because there’s no 80% case for all the issues people might come across. Which is a huge design challenge! It’s like that Tolstoy quote “All happy families are alike; each unhappy family is unhappy in its own way.”
I mentioned this insight to my team today, and
said “We’re more like Google Maps. We’re helping people and teams understand their code better. We’re not asking people where to place a button, or what colors to use in the UI. We’re helping them navigate their code and diagnose problems, which gets really complex!”
I hadn’t thought of it that way, but that’s exactly right. We need to understand everyone’s unique struggles with their software development process. We need to learn how to work with a vast array of different configurations, runtimes, states, and scenarios. The design research methodologies I typically use don’t scale to this level of complexity.
It’s a huge design and engineering challenge. And really fun!