Understanding users and UX research

tl;dr — remember humans aren’t always predictable, it’s a probability distribution curve, watch out for biases and observe what they’re thinking & feeling.

Today’s goal is to share insights on how to better understand users and do live UX research.

I want to create products that make lives better. I’m listening to Waiting for a Girl like You by Foreigner today.

Is UX research really necessary?

One of the most shocking things you’ll see in the industry is the # of product leaders and managers who don’t speak to users and customers on a regular basis, sometimes less than once a quarter! Sometimes this is justified — if you are a research PM deeply focused on bleeding edge technology rather than end user applications, or if you are a VP of a 100+ PM organization and simply have to delegate to others — just keep in mind, someone should do it, and the people closest to users will know their needs best.

If there’s one thing to remember, it’s that it’s difficult to make the right product decisions if you don’t have a deep understanding of your target users, their use cases, and needs.

This is why we do user experience research (UXR), and in my experience, it makes a huge difference when whoever is driving product decisions attends the UXR studies. UXR reports are second-hand interpretations, and the most nuanced insights are those that come from understanding a person’s emotional reactions and mental model.

from Dilbert the comic

How to think about UXR for startups/early stage products?

As usual, let’s bring this back to the goal. Depending on where you are in the product journey and lifecycle, the tactics may change, but the goals stay the same.

First, are you a user-driven or a tech-driven product team?

• User-/customer-driven product teams have an identified target market of users already. The usual approach is to understand the user needs and pain points, and offer better solutions than the current alternatives. IMO this is the standard for “product-discovery.” The pro of this approach is that you have sharper focus and often can deliver a solution on a faster timeline, but the con is you may miss adjacent market segments/opportunities and it is rarer to build something disruptive.
• Tech-driven product teams explore the capabilities and push the limits of a particular technology. This approach often means experimenting with different solutions/applications of the tech, and then trying to figure out who might find it valuable. IMO this is what some call “customer-discovery.” You’ll see some teams take a scattershot approach to the customer-discovery process, burning cash to gather as much usage and awareness as possible to identify high-value customers.

For user-driven product teams, you should do foundational UXR early on and then throughout the development process. For tech-driven product teams, you may only invest heavily in UXR after you’ve identified worthwhile customers that are worth the acquisition cost.

Either way, there’s roughly 3 categories of user research that I like to anchor around (note: this is a gross simplification):

1. Foundational research — understanding the user/market
   This can take the shape of interviews, surveys, diary studies, or market research. Ultimately it’s about understanding who these users are, what are their backgrounds, their use cases, their needs and pain points?
2. Value testing research — understanding what is valuable
   For me, this category is special. Understanding what people value, and whether a product is valuable or useful, is key to identifying product market fit. This category of research can look like concept testing, value-ranking exercises or maxdiff surveys, etc. — this category is also tricky to execute right and full of hidden pitfalls. The idea here is: if the target user understood this product and what it’s supposed to do, would they say “This is AWESOME, when can I try it??”, or would it be more like “Oh that’s pretty cool. Happy to try it.”
3. Usability testing research — understanding if a product is usable
   This is the standard “click through prototype” or “here’s a product, try to do X” type of study. This is about learning what is confusing about the UI/UX so you can fix it and ensure users can intuitively learn and use the product without a ton of training and education. This assumes the product would be valuable to the user, and is mostly about helping the user get through the journey so they can complete the funnel.

I highly recommend startups and early stage products invest time in all three categories of research. If you don’t, you risk spending a lot of time building a product that doesn’t solve for their needs, isn’t valuable, or isn’t usable — all three of these issues can make the product crash and burn.

from KC Green’s web comic strip “On Fire”

Tips & tricks for doing user research and avoiding pitfalls

I’ll try to keep this updated as new things come to mind.

Setting up for research:

• Before you do any research study or survey, write down what product/user questions you’re trying to answer and learn. Make sure all questions asked in surveys/interviews contribute to those goals.
• Understand user problems, then understand the scale/prevalence. Interview 6–10 people first, identify categories of use cases/pain points, then mass survey on those categories and collect 300–500+ responses.
• Avoid survey/response bias — word your survey/interview questions and answer options carefully! In surveys, use answer randomization when it makes sense.
• ALWAYS always always test any survey you’re about to send by filling it out yourself first (with real data, not just typing in mumbo jumbo). Always do a second pass and try to trim unnecessary or similar questions.

Interpreting responses/results/findings:

• Remember: humans aren’t always predictable, it’s a probability distribution curve. The same person doing the same study a second time may not answer and act the same way. Take them as data points, not as definitive yes/no’s.
• Watch out for biases from yourself, your team, and the research participants. If you understand where they’re coming from and their backgrounds, the easier it’ll be to spot biases. If you ran a study on a 40yr old FAANG software engineer about TikTok, you wouldn’t get the same insights or responses as a 16yr old teen.
• In live studies, observe between the lines for what the person is thinking & feeling. Just because they say something looks interesting or valuable, doesn’t mean they actually truly feel it. Sometimes people will say what they think they SHOULD say, rather than what they truly believe/value/feel. This is one of the easiest traps that PMs fall into, seeing half-hearted interest in their products in UXR studies and being convinced they have market fit and user value.
• If you’re reading a report from someone else, especially when looking at charts and statistics, make sure you understand the size of the study (how many responses), the axes on the charts, and how any percentages or percentiles were calculated. My 2c: don’t make charts or statistical comparisons for data with <10 samples. It’s more likely to mislead people into a false sense of confidence.

User experience research is crucial to helping you derisk early stage products and features so that you can understand whether they actually deliver user value before you invest too far. It’s important to build a healthy culture and muscle into the product team to do UXR on a regular basis — what you’re really doing is just getting feedback from users. Without it, it’s like driving blind.

by Francisco-G-Landa from DeviantArt