Optimizing for Success in Machine Learning-Driven Ad Auctions

Although marketers have widely adopted machine learning-driven ad systems, many continue to rely heavily on traditional metrics to gauge the success of their campaigns. But recent research from Facebook suggests that advertisers lose roughly 10% of efficiency in their campaigns when they optimize for “proxy” metrics—non-business outcomes that act as a proxy for a real business goal.

To understand how optimization choices affect ad performance, Facebook IQ’s Brittany Swanson spoke with Advertising Research Manager Neha Bhargava on the Marketing Science Research team at Facebook. In the interview, Bhargava reveals what happens behind the scenes before an ad is delivered by Facebook's machine-learning ad system, why success metrics are evolving and how to improve campaign outcomes by leveraging the auction correctly. She also delves into the findings from a study in which the Marketing Science Research team used data from 9,000 lift tests over a two-year period to determine how campaign optimization choices affect performance in a Facebook auction.

Brittany: Let’s start with the basics. What has happened behind the scenes by the time a person sees an ad on Facebook?

Neha: When you think about traditional media buying, every person gets the same ad. When a Super Bowl commercial is aired, for instance, it is the same for every viewer, regardless of where they live in the country. Obviously, the digital media landscape has changed this paradigm dramatically. Now, if I watch a viral video online, I might see an entirely different ad than someone else who watched the same video.

With machine learning-based systems like Facebook auction, marketers can determine who sees their ads or where those ads are shown, and it can all be done automatically and at scale. This means every person is delivered an ad based on a completely different series of decisions that are made by a machine and are based on a marketer’s inputs, user data and more. And since Facebook ads compete against each other, our system determines which ad will be the most successful based on the results you want, as well as other factors like an ad’s relevance to the consumer. By the time your customer finally sees an ad on one of our platforms, it has already been evaluated and ranked based on both the customer’s interests and your goals.

Brittany: How does the complexity of auction-based advertising systems differ from that of traditional ad buying? How has this affected the way marketers gauge success?

Neha: It’s really hard to measure the success of traditional media; you have to look at correlations, such as the correlation between views and sales, which isn’t exact. For example, gross ratings points (GRPs) are still the metric marketers turn to when buying TV advertising, despite the fact that the equation for calculating GRPs only takes into account 0.03% of American TV households,³ and you don’t know for certain if people are seeing your ad.

But marketers who are familiar with traditional media are used to optimizing campaigns based on proxy metrics. Proxy metrics are nuanced; video views can be proxies, but if—for instance—the goal of your campaign is to drive views for a movie trailer, video views are your desired business outcome and you should optimize for them. The issue is that advertisers sometimes optimize for clicks or video views on the assumption that they correlate with a business outcome, such as a sale. The assumption is that reach equals success and optimizing for proxies is an easy way to keep metrics consistent across all the digital channels you use. However, proxies may not be as closely correlated to your business outcome as you think. Our research looks at the impact those assumptions have in a world where marketers and advertisers are trying to adapt to a machine learning-driven ecosystem that is far from linear.

Brittany: Your team looked at 9,000 lift studies over a two-year period. Based on your research, what would you say is the best way to optimize campaigns for machine learning-based auctions?

Neha: We already know that optimizing for proxies in digital campaigns isn’t ideal; although, in some cases, it might be the only option available. Even so, we wanted to quantify just how much optimizing for proxies affects the outcome of a campaign. We trained a machine learning model to predict the lift and efficiency, or cost per incremental conversion, of these studies—and then examined how much those predictions changed if we altered how the campaigns were optimized. This way, we could control for factors like advertiser decisions and campaign settings and isolate the impact from optimization choice.

Our research shows that if advertisers are ultimately interested in driving business outcomes like sales, they will drive those sales more efficiently if they optimize for them, rather than for proxies. For example, optimizing for clicks or video views will still drive relevant business outcomes—as our research shows, click-through rates and video view rates are, in fact, correlated with conversions—but marketers will get up to 10% fewer incremental conversions and pay up to 10% more per incremental conversion than if they’re explicit about what they want.

Additionally, if they choose to evaluate the success of their campaign based on a proxy, as well as optimize for that proxy, they are doubling down on this bad practice—with significant consequences in terms of conversions. Our research shows each click is worth 25% fewer incremental conversions when a marketer optimizes and evaluates for clicks. That’s because, in this scenario, the incremental value of each click decreases as click-through rates increase—you need more clicks for a conversion. Ultimately, you are telling the auction you care more about clicks than you do business outcomes. That can hurt your campaign.

It’s important for marketers to understand that proxies like clicks or video views don’t have a one-to-one relationship with business outcomes. Sure, you can optimize for a proxy and the auction will deliver those proxies, but be cautious in thinking that your campaign is seeing optimal performance just because clicks went up. If you optimize your campaign based on the actual business outcome you want, you see 10% more lift—that is, an increase in your desired business outcome as a direct result of your campaign—on average.

Brittany: So it sounds like optimizing for actual business outcomes is the most efficient way to run a campaign in an auction setting. Why would advertisers still try to optimize for proxies?

Neha: The answer to this is twofold: Sometimes, advertisers want to have a consistent view of performance across multiple channels. If they’re used to reporting the performance of their ad campaigns based on clicks or video views in other channels, they may measure performance the same way with Facebook’s platforms for the sake of convenience.

Secondly, outside of the Facebook auction, it’s possible advertisers simply don’t have the capability to optimize for their desired business outcome. In cases like these, we encourage advertisers to make use of what they have—but to be aware that they might not be driving the impact they could be.

Brittany: Beyond optimizing campaigns for a desired business outcome, did your team identify any other considerations for advertisers using the auction?

Neha: In addition to examining lift studies, we also looked at over a million ads on our platform and found that, when it comes to advertising in the auction, creating ads that are high quality and relevant to your customers is key. More ads don’t equal better performance in the auction—in fact, running too many ads with too little budget or low quality ads with a large budget can result in less-than-optimal performance. Furthermore, ads with lower bids often win if our system predicts a person is more likely to respond to them or finds that they’re higher quality.

Our research shows that the cost of an ad tends to decrease as its likelihood of inspiring a customer to convert increases. Similarly, cost to the advertiser decreases as the quality of an ad increases. In other words, it's best to focus on creating highly relevant and high quality ads if you want the best results from the auction.

Ultimately, what advertisers should take away from our findings is that the auction is a powerful tool for getting you the outcomes you want. Advertisers should focus their energy on making better creative choices and simply optimize their campaigns according to their preferred business outcome. Doing so will get them the best results.