The New Science of Prompting: How to Speak the Language of AI

Your new loyalty program is a roaring success… or is it?

You set a threshold: spend over €100/month and get a 10% discount. A quick analysis shows customers who just crossed that line are now spending more and buying more frequently than those just below it. The boardroom is thrilled. You’re celebrating.

I’m here to tell you that you might be celebrating a costly illusion.

This is a classic case of “Doing Data Wrong.” The common approach is a Regression Discontinuity (RD) analysis, a powerful tool for measuring the impact of this kind of threshold-based program. But a naive RD analysis, the kind most companies run, often measures self-selection, not causal impact. You're likely ignoring a critical factor: strategic manipulation.

Think about it. The customer who was going to spend €95 sees the offer. They add a €6 item to their cart to cross the €100 threshold and unlock a €10+ discount. Your model sees them as a success story—a customer whose spending increased because of the program. In reality, you just gave away margin to a savvy deal-seeker who gamed the system. Their behavior contaminates your entire analysis, making a potentially useless program look like a stroke of genius. This manipulation violates the core assumptions that make a standard RD analysis valid.

So, what do we do? Throw out the entire study? No. We get smarter.

The "Aha!" moment comes when we stop assuming everyone around the cutoff is the same. Instead, we must statistically isolate the subpopulation for whom the program was a genuine, random-like experiment. A groundbreaking paper by Forastiere et al. (2024) shows a powerful way to do this using a Bayesian approach to probabilistically classify people into distinct groups.

My analogy for this is panning for gold. A naive analysis looks at a pile of dirt and rocks and concludes it’s not very valuable. The sophisticated approach uses a statistical "sieve" to isolate the actual gold nuggets—the "Causal Core" of customers whose behavior wasn't strategic—from the worthless rock of the deal-seekers and other confounded groups . We then measure the program's impact only on the gold.

Let's revisit our €100 loyalty program. When we apply this method, we identify and effectively down-weight the individuals who likely manipulated their spending. We are left with the "Causal Core"—the customers whose spending crossed the €100 threshold due to normal, random fluctuations.

And what do we find for them? The program has zero, or maybe even a slightly negative, impact on their loyalty and spending. The glorious ROI disappears. The truth, which was hidden in plain sight, is that the program was burning cash. This rigorous approach prevents a multi-million Euro mistake by revealing the ground truth.

This isn't just theory. It's a practical, robust playbook for getting to the right answer.

For Managers: Stop accepting simple "before-and-after" charts. Start asking your teams the tough questions: "Have we isolated the group for whom this was a true experiment? Can you prove we aren't just measuring the behavior of deal-seekers? I need to see an analysis that explicitly models and corrects for self-selection."

For my fellow Tech & Data Leaders: Our value isn't just running a model; it's guaranteeing its integrity. We must move beyond picking a simple bandwidth around a cutoff. Implement probabilistic frameworks to identify the subpopulation where the "local unconfoundedness" assumption truly holds. This method allows us to build a causally valid group, no matter how far their spending is from the threshold, and formally accounts for the uncertainty in our selection. This is how we deliver bulletproof, business-critical insights.

This goes beyond one marketing campaign. It's about a fundamental shift in how we establish causality in business. For too long, we've accepted sophisticated-looking descriptions instead of demanding rigorous, causal truth.

My career has been a journey through the executive suites of Itaú, Stone, and Ambev, the entrepreneurial trenches with my own successful exit at NaHora.com, and the academic rigor of FGV, Berkeley, and HEC Paris. The knowledge I've gathered isn't mine to hoard. My purpose is to give it back—to bridge the gap between cutting-edge research and the pragmatic reality of the balance sheet.

It's time we build businesses that are not just data-driven, but truth-driven. Let's fix this together.

Are you ready to move from "doing data" to driving real, causal impact? Join my email list for no-nonsense, research-backed insights to unlock the true value in your data. It's not a newsletter; it's a community for leaders who refuse to settle for illusions.