Intro to Incrementality Series: Part 2

Real-World Example: The Basketball Test

Let’s revisit our basketball analogy from Part 1 for a quick refresher.  Imagine an NBA team with two players who both make 40% of their free throws. To improve performance, the team hires a shooting coach but assigns him to work with only one player, Player B. Both players maintain the same training, diet, and gym routines.

Player A: 40% of free throws ┃ Player B: 40% of free throws

After the 2025–26 season, Player B’s free throw percentage improved to 80%, while Player A’s percentage rose to 50% due to natural improvement. The difference between these results helps isolate the coach’s impact.

Player A: 50% of free throws ┃ Player B: 80% of free throws

The takeaway? For a valid test, the control group (Player A) and test group (Player B) must be similar in baseline behavior, with only the variable of interest (the coach) changing. Allowing additional differences—like varying diets or gym routines—would introduce noise and obscure the results.

Controlling The Control Group

Marketers face unique challenges in creating effective control groups:

• Misaligned Populations: Control groups often fail to reflect the demographics and behaviors of the test group.
• Signal Loss: As cookies and identifiers are deprecated, tracking becomes more difficult, increasing the risk of skewed analysis.
• External Noise: Uncontrollable variables (e.g., seasonality, regional events) can obscure results.

Before 2020, there were three common methods for creating control groups in digital advertising:

1. Market A/B Testing 

Pick two similar geographic markets, turn ads on in one market, and leave them off in the other. Observe and compare behavior in each market.

Challenges:

• No two markets are identical.
• Demographic and behavioral differences can skew results.
• Background noise from external factors (e.g., regional events) creates variability.

2. Random Suppressed Groups

Randomly exclude subsets of people from seeing ads using digital identifiers like cookies, IP addresses, or mobile ad IDs (MAIDs).

Challenges:

• Misaligned Targeting: Random sampling often doesn’t match campaign targeting (e.g., a men’s shoe campaign’s control group includes women, skewing results).
• Signal Loss: Privacy regulations and cookie deprecation make it harder to track users and create accurate control groups, increasing the risk of skewed analysis.

3. PSA Ads

Serve Public Service Announcement (PSA) ads to the control group instead of your brand’s ads. This ensures both groups are behaviorally and demographically similar.

Challenges:

• PSA ads are costly since impressions must be purchased.
• PSA ads prevent competitors from winning those impressions, which skews results in favor of the exposed group.

A Better Approach: Ghost Bidding

In 2020, ghost bidding revolutionized control group creation. By simulating bids without serving ads, ghost bidding eliminates many of the challenges associated with traditional methods.

How it Works:

• For every 5 eligible impressions for the campaign, 1 impression is assigned a “ghost bid” and placed in the control group.
• The ghost bid prevents the user from receiving an ad while maintaining the same targeting and optimization parameters as the exposed group.

Advantages of Ghost Bidding:

• Cost Efficiency: Ghost bids don’t incur costs since no ads are served.
• Targeting Precision: Control groups mirror the exposed group exactly, ensuring demographic and behavioral alignment.
• Real-Time Adjustments: As the exposed group’s optimization shifts (e.g., reallocating budget to high-performing segments or publishers), the ghost bids adjust in parallel.

Ghost bidding addresses the pitfalls of market tests, random samples, and PSAs while remaining inexpensive and easy to implement. That’s why Digital Remedy uses ghost bidding as the preferred method for creating control groups in the Digital Remedy Platform.

Looking Ahead

Effective control groups are the foundation of accurate incrementality analysis. By leveraging ghost bidding, brands and agencies can minimize bias and gain actionable insights into their campaigns.

In Part 3, we’ll explore how to put incrementality analysis to work in real time, driving smarter optimizations and improved ROI. Want to get started now? Speak with a member of our team today and take your marketing performance to the next level.