Lead Gen Revenue, impossible to estimate?

Part 1 – the Problem

It was years ago, about 2001, when we met with Todd Chapple (now at My Agent Solution, Inc.), who at the time was pioneering auto insurance leads online at Netquote. He is a very smart guy who’s company was generating internet leads for auto insurance. They sold their leads to agents and directly to insurance carriers. On their web site consumers looking for a policy can complete a form, detailing their cars, their driving records, coverage desired and contact information which is sold as a lead.

 

In the meeting Todd said that it was very difficult to estimate the value of visitors to his web site until the user completed the form and were matched with orders. Since there are so many variables and so many possible lead matches that the possibilities seemed innumerable.

 

Big Numbers

Netquote sold leads to agents who serve local areas. So it makes sense to sell leads by Zip Code. There are about 40,000 zip codes in the US. So our numbers start big. Agents also have preferences for leads that match well with the carriers they sell. Age and number of cars and number of recent traffic violations are important. Here are a few that I remember.

Zip Code = 40,000
Age = 4 Buckets {18-21, 21-25, 26-55, 55+}
Number of cars = 3 buckets {1,2,3+}Traffic Citations = 3 buckets {0,1,2+}
Currently Insured = 2 buckets {yes,no}
Gender = 2 buckets {M,F}
Marital Status = 4 buckets {Single, Married, Separated, Widowed}
Homeowner = 2 buckets {Y,N}
Credit Rating = 4 buckets {Excellent, Good, Fair, Poor}

Possible Combinations = 40,000 x 4 x 3 x 3 x 2 x 2 x 4 x 2 x 4 = 184,320,000 !

 

More Complexity

Yet the problem grows even bigger when you consider the lead orders that Netquote had to manage. Imagine that they had lead orders from 3,500 agents. Each agent might select from the above variables for leads that were suitable. Normally, for each selection the agent makes, Netquote charged more for the lead. To keep costs down, Agents likely picked a just few of the most important variables to them.

 

I like to use the the metaphor of a blanket. The agent places a blanket over their selection space. We have a bed that has 184,000,000 spaces to cover! Yikes! We either need very ‘large’ blankets or many, many small blankets to cover our space.

 

Now consider all of the agents laying down blankets across the huge selection space. Some spaces will have no orders and have no blanket coverage, while other more competitive spaces might be several blankets thick. Some carriers might have very large blankets, while small agencies might cover a single Zip Code. Prices for each order varies as it varies in each layer of the order blanket. I like to think about this hole riddled blanket. I call it the Book of Orders or Book for short.

 

So imagine my simpleton question to Todd Chapple: “When a visitor lands on your web site, how do you estimate the revenue they will produce?” Remember, too, that when a visitor just lands on Netquote’s site most user variables are unknown. Todd has one easy answer with an easy calculation: the last 100 visitors produced $37 in lead sales, so each new visitor is worth $0.37. This is the marginal value per visitor.

 

When a user lands on our site, what can we DO to increase sales? What can we DO to increase sales if the user provides their age, gender and Zip Code? With so many possible combinations can we make these conditional “DO” decisions in 40 milliseconds (fast enough for a good user experience and fast enough for new ad serving systems)?

 

One solution to this very big problem is to ‘compress’ the problem. Take advantage of the obvious dependencies among the user variables. Regression analysis would have these dependencies eliminated by combining variables. Another way is to use a Bayesian Network to describe the distribution of user variables. These models might compress the space of 184,000,000 combinations to a manageable number.

 

Information Science meets Do Calculus

Bayesian Networks are an interesting mix of information science and probabilistic Do calculus. The information science comes from Claude Shannon, who applied principles of entropy and mutual information to communication problems. Do calculus, invented by Judea Pearl, is the calculus of prediction and inference based on very compact probabilistic graphical models.

 

The picture to the right is a Bayesian Network learned from about 50,000 auto insurance leads. This representation is very compact. In this model ZipCode was discretized into 18 contiguous buckets. I suspect only 200 parameters are required to describe the entire distribution of leads!

 

An advantage to the BN is that if we know the value of any of the variables (if we have hard evidence), then calculating (thanks to Do calculus) the conditional probabilities of all other variables is very swift and easy to program. This model is comprised of 12 small tables of conditional probabilities.

 

I think this is a good first step in solving the lead gen problem. In the next few blogs I will describe the two other parts of the problem and offer a solution to the optimization problem.

1. Adding Do variables and Response variables to the model – what can we DO to improve response?
2. Adding a compact form to describe the Book of Orders – this opens up a way to calculate revenue and costs.