Causal Model of Facebook Ad Performance

Introduction

20 years ago, tracking ad visitor performance was pretty easy. Then 95% of users were on desktop computers with only 5% smart phone users. Now privacy features in browsers prevent tracking. Apple iPhones are a great example of very strict privacy. 90% of my client’s web visitors are smart phones and 90% of these are iPhones! This makes measuring ad performance very difficult. But smart phone browsers can click on phone numbers in web pages and ads. Smart phones offer better conductance than desktop computers.

Thankfully, causal science, over last 20 years, has also evolved and provides a solution for calculating the Conditional Average Total Effect of ads on performance metrics like sales and new customer acquisition. Direct measures of Facebook ad performance look very poor. Direct measurement of new customers suggests new customers cost $250. But the real CPA is $41. We also found that for every ad dollar spent $9.50 sales is produced.

My client’s business is open Tuesday to Friday. When a lead is produced, an appointment time is set and sales occur on this latter date. The average time between the lead and sale was 36 hours. Consumers can make appointments in 4 ways: Web Form, Call, In-office, Walk-in.

Model Variables

Data for our model came from several source. Meta provided Ad Costs an Ad Clicks. HubSpot provided New Customers, Site Visitors, Other Clicks. Google Analytics provided Form Leads and Click Calls. The POS, Square, provided Order count and Gross Sales. Site Visitors – Ad Clicks = Other Clicks. Leads = Calls + Form Leads.

One engineered feature is made to transform Lead date to Appointment date. If a lead is produced on Monday (office closed), the transformer, D, spreads out the appointment to future work days using a probability table. So that Appointments = D x Leads and average Delay was 36 hours.

Daily totals for each variable was collected over 75 days. Missing data of note are Direct Calls (not measured), Walk-ins, and in business appointments (when customer makes a new appointment at checkout). Variables were discretized with K-Means (zeros isolated) into 4 bins. BayesiaLab was used to learn a good model.

Ad Cost Effect on Gross Sales and New Customers

BayesiaLab calculated the CATE curves shown. BayesiaLab calls this Total Effect. Notice that, as with most media, as costs grow gross sales growth slows.

New customer acquisition is vital to every company. So the CATE of Ad Cost on New Customers is very helpful.

Notice when Ad Cost = 0, New Customers = 2. These are New Customers caused by Other Traffic, Direct Calls and Walk-ins.

Conclusions

My old company SureHits built ad tracking software, AdProve, that used direct measurement using cookies. It worked well 20 years ago. Now it would fail badly. Today most analytics services like Google Analytics use several ad attribution models like First Ad Touch, Last Ad Touch, and Data driven (which uses all touches). But these deliver very poor performance metrics for my client.

Making a Causal model of Ad performance offers much better insight on the mechanisms between variables. A single Bayesian Network model can measure CATE for all performance metrics (here we showed only two: CPA and Profit Margin.

Model Improvements:
1. Include ad impressions and Audiences in model
2. Break out sources in Other Traffic