Core Concept: Potential Outcomes Framework & SUTVA

You just spent $2M on a pilot program for a new product in one city. The results look fantastic.

Before you bet the company's annual budget on a national rollout, I have a question. It’s a question that separates rigorous, value-creating data science from expensive corporate theater.

To answer it, we have to think like physicists. We must start with the Potential Outcomes Framework. It’s a simple but profound idea: for any person, city, or store, there are multiple potential realities. There’s the reality where they got your new marketing campaign (Yi​(1)), and an alternate, counterfactual reality where they didn't (Yi​(0)). The "causal effect" is the difference between these two states. The fundamental problem is that we can only ever observe one of these realities at a time.

To bridge this gap and estimate the effect, we rely on a huge, often unspoken assumption. In academia, it’s called the Stable Unit Treatment Value Assumption (SUTVA). In business, it should be called the "Don't Fool Yourself" Rule.

SUTVA has two parts, but one is particularly lethal for pilot programs:

No Interference. It assumes that the treatment applied to one person doesn't affect the outcome for another. And this is where most pilot programs become an exercise in "Doing Data Wrong."

Let's use a Consumer Goods example.

The Wrong Way (The Contaminated Pilot): A beverage company launches a new energy drink in City A (the "treatment" group), with a massive local ad campaign. They use City B, with no campaign, as their "control." After a month, sales in City A are 20% higher than in City B. The team celebrates, projects a 20% national lift, and plans a massive rollout.

They've just made a multi-million dollar mistake. They violated SUTVA.

The intense hype in City A created spillover. People who saw the ads (treated units) told their friends who hadn't (control units) about the new drink. The social buzz interfered with the local control group, artificially inflating their purchasing behavior. The "control" was contaminated by the "treatment." That 20% lift isn't the clean, direct effect of your marketing; it's a muddled mix of ad impact and localized, non-scalable social hype.

The Right Way (Designing for Reality): A rigorous approach anticipates this. It acknowledges that SUTVA will likely be violated in the real world.

For Managers & Business Leaders: Before you sign off on a pilot, you must ask your team:

• "What are the likely spillover effects? How are we designing this experiment to distinguish the direct impact of our campaign from social interference?"

• "Are we at risk of violating SUTVA? What is our plan to measure it, not just ignore it?"

  
  

For Tech & Data Professionals: Your job isn't just to run a t-test after the fact. It's to be the architect of a robust experiment. You must flag the risk of interference before the first dollar is spent. This might mean designing a cluster-randomized trial (randomizing by store or zip code, not individuals) or using more advanced models to estimate the magnitude of the spillover itself. This is your strategic contribution.

From my time launching ventures at Ambev and FALCONI to building my own companies, I’ve seen firsthand that a poorly designed pilot is worse than no pilot at all. It gives you the confidence to make a bad decision.

My mission is to drag these foundational concepts out of the lecture hall and into the boardroom where they belong. This knowledge isn't mine to keep.

If you’re ready to build your data strategies on solid rock, not on the sand of flawed assumptions, then join my movement. Subscribe to my email list for more no-nonsense, research-backed insights to fix your data practices.

And if you’re staring at pilot results right now and feeling that pit in your stomach, book a 20-minute, no-nonsense consultation with me. Let’s diagnose it together.

#CausalInference #PilotPrograms #ProductLaunch #BusinessStrategy #ExperimentalDesign #ConsumerMarketing #SUTVA #MariaAliceMaia