Making Decisions With Limited Data
Navigating Product Decisions In The Absence Of Quantitative Data
Startups are defined by their ability to move quickly and innovate in the face of uncertainty. When every decision can make or break a product, data can help teams understand customer behavior, measure performance, and optimize adoption. But what do you do when the data you need is just not available? This is a common challenge for early-stage products with only a few users—or none at all.
Without sufficient information, it’s hard to know what to improve. However, learning to navigate this environment is just part of the process of building something completely new. There are things you just can’t figure out until you try. The teams have to find a way to still build products that resonate with customers. In this article, I’ll dive into some simple strategies for making informed product decisions even in the absence of quantitative data.
Takeaways
Clearly state and track assumptions to mitigate risks.
Leverage qualitative insights when hard numbers are scarce.
Provide opportunities for customers to provide product feedback.
Track meaningful insights, not just metrics that look good on paper.
Make decisions based on the information available and course-correct as you go.
Identify Underlying Assumptions
Clearly state and track assumptions to mitigate risks.
Every new product or feature starts with a hypothesis. However, untested or implicit assumptions can lead to misguided decisions. Therefore, It’s important to explicitly recognize and document them thoroughly to manage this risk. Start by listing out all the assumptions that underpin your product decisions. Continue to document and revisit them regularly to assess the most critical assumptions and prioritize them for validation. As you learn more, these assumptions can be refined, adjusted, or even discarded, allowing you to develop a more effective product strategy.
Understand The Customer
Leverage qualitative insights when hard numbers are scarce.
When deciding what to build, focus on gathering as much insight as possible from customer conversations. Listening to a customer describe their situation in their own words reveals vital context that raw numbers just can't. Use interviews, observations, and anecdotal evidence to understand the customer experience. Understand how they feel about it and why. However, it’s important to not take their feedback at face value. Listen to them but also think about their underlying goals, objectives, and motivations, before you act.
Establish Robust Feedback Loops
Provide opportunities for customers to provide product feedback.
After you have built or released something, set up mechanisms to collect customer feedback. Use surveys and feedback forms to get their rating or reviews. Make it easy for them to share their thoughts, and actively encourage them to do so. Often complaints are the most useful feedback. While no one likes hearing them, they help you identify and evaluate potential improvements. If they don’t respond to any of that, send a message, an email, or schedule a meeting to get their feedback. The more feedback you receive, the better equipped you are to make informed decisions.
Focus On The Right Data
Track meaningful insights, not just metrics that look good on paper.
Product teams face a lot of pressure to quantify their success. They need a hard metric to show a tangible ROI for the time, effort, and resources committed. They might point out that 80% of users are actively using your product. If you have 1000 users, that number is great, but what if you only have 10? How have you even defined active use? You could convince 8 people to try something out. Much harder to do for 800. The context determines the value of a metric. When you have a small user base, it’s more useful to discover why people do or don’t use your product. That gives you a more powerful signal of what to focus on.
Conclusion
Make decisions based on the information available and course-correct as you go.
You can’t build a perfect product, especially not on the first try. You will never have all the information you want or need. It’s more productive to focus on what you can control instead. Make the best decision you can with the information available. Customers might hate or love your product for reasons you may or may not expect. However, every piece of information you gain gives you a better understanding of their problems, needs, and pain points.
You build up your knowledge, experience, and intuition with each attempt. It’s important to accept that you will fail or stumble as you progress. As you learn new things, don't hesitate to pivot. Maintain your momentum. Keep your team and product moving forward even when you don’t have all the answers. Each iteration will bring you closer to a product that will truly delight them.
Thanks For Reading!
If you haven’t already, please consider subscribing and sharing this newsletter with a friend.
I hope you have a great week!