How I Helped BoB's Design Team Build Its User Research Infrastructure

Design ran on principle, not people.

Hierarchy, spacing, industry patterns. It looked like design. It was guesswork.

The team was flying blind.

Rare data. Few insights. The occasional CSM suggestion coming in from the side.

There was no system to close the gap.

No steady way for design to reach users, hear them, or turn what they said into work.

Where users got stuck

I came out with a long list of insights that's since reshaped how several features work. A few, to give you the shape of it:

Impact of work

Recruiting took a long time.

Every study started with weeks of chasing the right people before a single session happened.

The Hawthorn effect crept in.

Users behave differently when they know they're being watched. The more we interviewed, the more that started to shape what we heard.

Every feature started from zero.

There was no running list of insights to draw from. Gathering fresh data for each feature would have stretched timelines I didn't have room to stretch.

I didn't want to interrupt the clients for every single feature.

Scheduling a call for every feature wasn't fair to their time, and it wouldn't scale.

1,280 tickets carried some sort of UX-actionable.

Out of 5,707 tickets read that quarter, 1,280 pointed to something the design team could act on.

The platform's biggest failure is people not knowing what to do.

Guidance (347) and Findability (234) made up 45% of all UX pain over three months. Both point to the same thing: the product wasn't guiding anyone.

The failure clusters in the builder surfaces.

The top three modules hold 35% of guidance and findability pain in the whole platform. All three are places where users have to build something: a bot flow, a campaign, a template. That's where guidance failure concentrates.

The team finally has a repository of insights.

A new feature no longer starts from zero. Echo keeps a running, searchable record of what users have struggled with.

Moderated Usability Testing

ApproachFigma prototypes, task-completion and comprehension tests with real users.

OutcomeWe could confirm a fix worked before shipping it, instead of finding out from the next ticket.

Unmoderated usability testing

ApproachPrototypes wired to Microsoft Clarity. Session recordings, rage clicks, heatmaps.

OutcomeSo we could watch real behaviour at scale without needing someone in the room.

Standardised heuristics

The audit questions I'd built for myself became a shared team practice. Any designer running an audit now runs the same one. So the diagnosis stays consistent no matter who's doing it.

The design system: split ownership with Swapneel Kar

Swapneel and I split the component library, each fully owning our own set. We also built a token-based architecture, giving the system consistent output and clear rules throughout.

Wire Echo directly to Jira.

Right now Echo gives me a ranked list; a designer still has to pull items into the backlog by hand. Auto-tagging would take that step out.

Bring unmoderated testing into the weekly rhythm.

The Clarity POC is scoped. Once it's running, every fix would ship alongside a Clarity recording that shows how real users actually behave with it.

Extend the pipeline to CSM Fireflies transcripts and WhatsApp chats.

They carry the same kind of feedback support tickets do. The pipeline doesn't care about the source; my acceptance criteria carry over unchanged.

Users tell you the product works when you're on a call with them. They tell you it doesn't in support tickets.

Both are true, and I now trust tickets more. The best user research your company runs might already be running. Nobody's reading it as research.

A good audit is a hypothesis, not a conclusion.

Mine was partly right and partly wrong. Users had to show me both.

Design ops isn't something you plan up front. It's a set of gaps you close one by one.

Every practice in this case study got built the moment I hit the gap it filled. Only later did they add up to a system.