New Method: Using Multiple Outcomes (TMO) to Adjust Standard Errors for Spatial Correlation
- Maria Alice Maia
- Jun 23
- 4 min read
Updated: Jul 16
Are You Confident in Your Data? Or Just Dangerously Wrong?
Imagine you’re a leader at a major retail chain. You’ve just rolled out a new pricing strategy across hundreds of stores in different cities. Your data science team comes back with the results: "It's a resounding success! The strategy has produced a statistically significant lift in sales." You greenlight a national rollout, investing millions.
Six months later, the national numbers are flat. The projected lift never materialized. The multi-million dollar investment was a bust. What went wrong?
Your analysis was a victim of one of the most common and insidious errors in business analytics: you trusted a “statistically significant” result that was built on a lie. This is a classic case of "doing data wrong," and it stems from ignoring a fundamental truth about the real world: geography matters.
"Doing Data Wrong": The Illusion of Independence
In our retail chain example, the analysts treated each store as an independent data point. They failed to account for spatial correlation—the simple fact that stores in close proximity or in similar neighborhoods are not independent. The economic health of a neighborhood, a local marketing campaign, or even the weather can affect a whole cluster of stores at once.
When you ignore this, you dramatically underestimate the true variability in your data. Your standard errors—the measure of statistical uncertainty—are artificially small. This makes your results look far more precise and significant than they actually are. You’re not just wrong; you're confidently wrong, and that’s when you make catastrophic investment decisions.
Traditional methods to fix this are often crude. Clustering by state or region assumes that a border is a magical line where all correlation stops, which we know is false. Other methods assume that correlation is purely a function of geographic distance. But as groundbreaking new research from top economists like DellaVigna and Imbens shows, economic outcomes are often more correlated between distant but demographically similar areas (like San Francisco and Manhattan) than between neighboring but different ones. These old methods are simply not good enough.
The "Right Way": Learning the True Correlation from Your Own Data
So, how do we get a real, robust answer? We stop making flawed assumptions and start using the data we already have. This is the core idea behind a new method called
Thresholding Multiple Outcomes (TMO), detailed in a 2025 paper by DellaVigna, Imbens, Kim, and Ritzwoller.
The insight is brilliant and practical. Your company doesn't just have data on the one thing you're testing (e.g., sales of the newly priced product). You have data on dozens of other outcomes for the exact same stores: foot traffic, inventory levels, sales of other product categories, employee hours, etc.
The TMO method leverages these auxiliary outcomes to learn the true, underlying correlation structure between your stores. It assumes that this correlation structure—how the stores’ performances relate to each other—is shared across many different metrics. By analyzing the correlations across all these other outcomes, you can build a map of which stores genuinely move together and which do not. This data-driven map is infinitely more accurate than a crude state-level cluster.
The process, in essence, is:
Estimate the correlation between every pair of stores using dozens of other outcome variables you already possess.
Threshold these correlations to distinguish between a real signal (genuinely correlated stores) and noise (random fluctuations).
Adjust your standard errors based on this new, empirically-derived correlation structure, giving you a true measure of uncertainty.
This isn't just a theoretical improvement. Calibrated simulations show that standard methods can underestimate the true standard error by more than 50%, while the TMO approach can dramatically reduce this bias, leading to more accurate and reliable conclusions.
What to Do Now
Managers & Leaders: Stop accepting "statistical significance" at face value, especially for geographically distributed initiatives. Start asking your data teams: "How are you accounting for spatial correlation? Are we using crude clusters, or are we leveraging our other data to learn the true relationships between our locations?" Demand robust answers before you make a multi-million dollar decision.
Data & Tech Professionals: This is your opportunity to lead. Your company's data warehouse is a treasure trove of auxiliary outcomes. Propose using a TMO-like approach to move beyond default clustering methods. You have the power to stop your organization from making confidently wrong decisions by providing a more honest and accurate picture of uncertainty.
My career has been dedicated to using data to find the ground truth and drive real results. The gap between a brilliant strategy and a failed one often lies in the quality of the data analysis that underpins it. This knowledge is not mine to keep.
A strong, value-driven invitation to join my email list for more "no-nonsense, research-backed insights on unlocking real data value and fixing broken data practices."
If you feel like discussing your specific challenges on data realiability, schedule a 20-minutes consultation call to discuss your own, real world no-nonsense case.
