“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.
Today’s column is written by Dor Birnboim, VP of strategic partnerships at ironSource Aura.
Whether it’s Uber trying to eliminate media waste or Deliveroo attempting to validate investment in a new channel, brands are rethinking the way they evaluate their campaigns.
The market is attempting to evolve from a focus on conversion performance to the more objective measure of incremental growth. To do this, marketers need to correctly and systematically implement incrementality testing and evaluation.
In the simplest terms, an incrementality test is a controlled experiment that measures the number of unique new users that each media channel delivers. Importantly, this incrementality is evaluated relative to other media channels, and also against the effect of not running paid media at all.
As such, incrementality testing – when done right – allows advertisers to understand the relative values of their media channels compared to one another, as well as whether paid media is driving new users at all, or just reaching users who would have converted organically anyway.
Incrementality first
For user acquisition managers, the ability to identify and optimize towards incremental growth means they can refine the way they allocate budget across their media partners for maximum impact.
According to Appsumer’s Q3 Benchmark Report, advertisers with monthly budgets exceeding $1 million used an average of 12 different digital channels for their acquisition campaigns. Advertisers rely on their mobile measurement providers to provide attribution insights across these channels. But even when an install is properly attributed, the advertiser needs to know if that new install came from a new or an existing customer.
Traditionally, advertisers have been able to use identifiers, such as cookies, and suppression lists to exclude existing customers, thereby isolating a pool of prospects for campaigns. While imperfect, this approach served the needs of most advertisers.
But with the imminent deprecation of the IDFA and the disappearance of cookies, advertisers need an alternative approach to evaluating the effectiveness of their campaigns – one that gets as close as possible to reflecting the net impact on new customers.
What makes incrementality work
Incrementality measurement allows advertisers to compare the impact of ad exposure against control groups, and in turn to objectively measure the performance of channels against one another as well as against organic growth.
Once incrementality tests have been run properly across all their acquisition channels, advertisers can determine the right thresholds for incremental effectiveness, i.e., what percentage of incremental subscribers or return on ad spend based on purchases are satisfactory. From there, they can adjust their campaigns and partners accordingly
But if incrementality is only measured within some channels or for select campaigns, advertisers can easily misinterpret the actual impact of their user acquisition (UA) strategy.
How to test for incrementality
Well-designed incrementality tests include randomized control groups and control groups that completely mimic the campaign. This means that for the control piece of the test, the audience demographics, optimizations and definitions of a user must match the test campaign exactly.
Only when these conditions are met can advertisers confidently measure incrementality.
Most critically, though, incrementality tests do not work when evaluating single channels in a vacuum – they serve to evaluate the performance of UA channels relative to one another – so advertisers should ideally run incrementality tests across their entire portfolio of partners, or with as many as possible.
Not every channel will deliver the same percentage of incremental new users and every channel incurs a different UA cost. Advertisers are accustomed to pitting channels against one another in absolute terms, but with an incrementality mindset, the new customers delivered through each channel are additive to one another.
This means that even if a channel has a lower incidence of incremental customers, those customers would still be unreachable through the other channels in the media mix.
When to test for incrementality
A common mistake advertisers make is trying to measure incrementality on non performing channels.
Every channel should be tested for baseline performance before implementing an incrementality test. For example, does it meet minimum criteria for delivering conversions and hitting CPA targets?
Additionally, while incrementality is a key factor, advertisers also need to understand where there might be overlap in acquisition channels and where costs can be managed for non-incremental conversions. Buyers must look at incrementality on a spectrum, rather than on a pass/fail basis.
Incremental performance of channels also changes over time, and advertisers should evaluate the incrementality of their channels at a regular cadence.
As an app becomes more popular, for example, network effects may take over and in turn, certain channels become less effective. Sometimes, as new channels are added to the media plan, existing channels become less performant. Seasonality of both the product and the channel may also impact the effectiveness of different channels.
Switching to an incremental focus means that marketers have to monitor, measure and adapt their strategies at regular intervals throughout the campaign lifecycle.
Incrementality measurement works best when advertisers can use channel performance to optimize towards a more accurate return on ad spend. Developing pricing and acquisition strategies based on incremental growth can yield better results overall and ensure that advertisers are growing their customer base efficiently.
Follow ironSource (@ironSource) and AdExchanger (@adexchanger) on Twitter.