“We want to be more data driven” or “We want to create a stronger data culture” are common organizational refrains. Supposedly, having more data or data playing a larger role in the decision making process is profitable. It’s weird because I haven’t seen any research to suggest this is the case. In firms like Facebook, it’s obvious as more data improves ad personalization and thus revenue. But this is data as a engineering project rather then a tool in the decision making process. Firms want to make better decisions with data. This is a misidentification of the value chain. Data isn’t that helpful if it’s not packaged with empiricism, an epistemological way of acquiring knowledge.
To even get off the ground analyzing data, we need theory of measurement. What should we track, given limited engineering resources and raising storage costs? Claiming we should track, say payments and logins, at the exclusion of audio volume, implies a cost-benefit value ranking. Why are payment and login more value to track? The theory is that understanding payments and logins will unlock more insight then volume as volume plays a less significant role in the app. Is this true? Hopefully, the institution has the intuition or previously collected to knowledge to make an educated guess. Firms have discovered that refining this knowledge can make their bets more likely to succeed. As it so happens the West has created the best knowledge refinement process in the history of humankind: the scientific method.
Data or more broadly, empiricism, is a key part of the scientific method as it expands the sample size of a test beyond antidotal evidence. Doing this at scale, as well as the methodology of running true experiments, A/B tests, means that knowledge is more valid (less likely the result of antidotal evidence) and stable. Firms can now learn.
Arguing to be data-driven or informed misplaces the value in the supply chain. We need to more explicit in this endeavor – it’s not about data, it’s about science.