The 'Kindergarten Data' Trap: Why Your 'Before-and-After' Comparison is Probably Wrong.
- Maria Alice Maia
- Dec 30, 2024
- 2 min read
Your team launched a new marketing campaign in October. In November, sales were up 15%.
Success?
Maybe. But you might just be celebrating a calendar, not a campaign.
This is the most common and seductive "Kindergarten Data" trap I have seen in my career. It's the simple "Before-and-After" comparison, and it's the source of countless bad decisions and wasted budgets, especially in Marketing Departments.
The Wrong Way (The Illusion of Time): A team presents a simple, beautiful chart. The line for sales is flat, then the campaign starts, and the line goes up. It feels intuitive. It feels like a clear win. Someone probably got a bonus for it.
The problem is that this analysis operates on a dangerous, implicit assumption: that the only thing that changed in the world between the "before" and "after" periods was your campaign.
This is never true.
What else happened?
The entire market could have been trending upward.
A major competitor might have had a supply chain failure.
Your product might have been mentioned in the press.
And, if it’s November, you're likely seeing the beginning of the holiday shopping season.
The "before-and-after" analysis gives 100% of the credit for all of these factors to your campaign. It’s not just wrong; it’s an active misrepresentation of reality.
The Right Way (Asking the Counterfactual Question): To get to the truth, you have to ask a much smarter question. Not, "What happened to our sales?" but, "What happened to our sales compared to what would have happened anyway?"
To answer this, you need a parallel universe. You need a control group.
You need a set of customers, stores, or regions that were subject to the exact same seasonal and market trends but were not exposed to your campaign. The true impact of your campaign is not your "after" number. It's the difference between the "before-and-after" change in your treated group and the "before-and-after" change in your control group.
This simple, powerful logic is the foundation of the Difference-in-Differences method, which we will dive into later.
As an executive at consumer-facing companies like Ambev and as an entrepreneur who built a travel tech company from scratch, I lived through these seasonal and market headwinds. Mistaking a rising tide for your own brilliant sailing is one of the fastest ways to sink a budget. My job was often to be the person in the room asking, "This is great, but what was the rest of the market doing?"
My mission is to help you and your teams build the discipline to ask these questions yourselves. To move beyond kindergarten charts and toward real, defensible causal inference. This knowledge is not mine to keep.
If you’re ready to stop celebrating noise and start measuring true impact, join my movement. Subscribe to my email list for more no-nonsense, research-backed insights.
And if you’re looking at a "before-and-after" chart right now and feeling suspicious, book a 20-minute, no-nonsense consultation with me. Let’s find a better way to measure your success.
