#1: No impact

Research that doesn’t impact anything is a waste of time and money. Doing “research just for the sake of research” is never a good idea. The most common problems here are:

• Not having clear research goals and therefore not being able to evaluate and compare the costs and benefits of conducting research versus not doing it.
• Certain circumstances that dictate the decision, and the information obtained from the research will not outweigh them.
• By the time the research is completed, the information will no longer be relevant.
• We can’t ****obtain needed information via available research methods.
• Research is used as a tool for validating an assumptions, not for testing it.

#2: Benefits < costs

Product development is about making decisions: business, product, design, marketing, and other decisions.

Each decision has a cost of implementation, expected impact, and implied risks.

Cost of implementation — is how much we will spend on developing particular feature and not others, entering a new market, etc. Expected impact — is what we anticipate to get in the result of the decision, usually expressed in improvement of relevant metrics: active users, conversion, retention, revenue, etc. Risk — is a level of uncertainty that given actions will lead to expected result and whether the resources will be spent reasonably.

Research is a tool for reducing the risks of decisions.

The cost of implementation and expected impact can be expressed in money. Research also has a price — how many resources we spend on designing, and conducting study, interpreting, and presenting data.

Therefore, the research makes sense only when its benefits outweigh the costs. This is the case, when:

• Research costs are significantly less than potential losses due to the cost of implementation and risks. For example, spending $10k on a research to make a decision or reduce a risk that will only cost $10k is not worth it.
• The increase in confidence Provided by research offsets the costs of the study. For example, conducting a research for $10k to increase confidence in a decision that costs $100k only by 5% is not worth it. It would worth it if the predictive power would be higher, or the cost of the research would be lower, though.

I find this — “benefits should outweigh the costs” — to be a powerful mindset that helps me stay focused on the impact of the research, see the context that it exists in, and ultimately optimise the benefit-to-costs ratio of projects that I’m involved in.

More on research as a risk management tool here.