Valuing Conversions: How You Count Matters

What exactly are you trying to optimize? This might seem like a simple question, but it deserves a second look. How you measure and record data during any landing page optimization test will have a profound impact on the results (and your test’s basic validity).

You may also have to use different math in order to analyze the data that you have collected.

Fixed Value vs. Variable Value

In many cases, your conversion actions all have the same estimated or actual revenue value (e.g., form-fills, downloads, registrations, click-throughs). For tests involving a single action with the same value, you can simply count the number of conversions, and use basic statistical tests to analyze the significance of your results.

However, if your conversion actions have a variable value, you must take this into account. For example, if you have an e-commerce catalog, you may sell items at widely varying prices. If your test variables have an effect on the average sales price, you must take this into account along with the conversion rate. If you do not, it is possible that any improvements in conversion rate might be diminished or actually canceled out by decreases in your average sales price. In such cases, you should measure the revenue per visitor. This will provide you with a normalized measure that takes into account the conversion rate and the average value of the transaction. By using revenue per visitor, you can determine whether a shift to a higher or lower average transaction value is actually a net benefit to your business.

Depending on your profit margin on different items (or categories of products), you may also have to consider the profit margin on each item. For example, it is very easy to shift your product mix toward selling low margin or “loss leader” items in the hopes that your clients will eventually buy more from you (either during the same transaction or in subsequent ones). Such sales can increase your conversion rate, and even improve your overall sales (i.e., they may increase your revenue per visitor).

However, this may devastate your profitability. In this case, use available information about the wholesale price of the individual product. You can then calculate the gross margin contribution by subtracting the cost of the product or service from the sales price. Although gross margin contribution is not technically your profit, it is closely related and I will use the two terms interchangeably.

Instead of using revenue per visitor, you can use the more accurate profit per visitor measure for your test. If individual product margins are not available, you can often estimate them at the category level. For example, if you know that your cost of goods sold for a particular product category is 60 percent, you can assign a value corresponding to 40 percent of the sales revenue to the relevant transactions. If all of your product categories have similar profit margins, you can bypass this complexity and continue to record the simpler revenue per visitor.

Single Goal vs. Multiple Goals

If you have a single conversion goal, and it has a fixed value, you should be able to use simple counting as described earlier. If you have multiple conversion goals, you must use revenue per visitor (or profit per visitor) even if each type of conversion action has a fixed value.

For example, imagine that you run a lead-generation campaign. Visitors have the option of completing your online form (a $20 value), or calling your toll-free number and providing the same information over the telephone (a $40 value based on the higher conversion rate of this more-motivated, self-selecting audience segment). One of the variables that you are considering testing is the prominence of the toll-free number on the landing page. A smaller one will presumably lower the proportion of phone leads, while a more prominent one would increase it. To properly handle this tradeoff, you should record each conversion action and its accompanying value for each landing page recipe. You can then use a revenue-per-visitor-based analysis to find the best recipe.

You also need to be clear about whether you are dealing with saturating goals versus accumulating goals. A saturating goal is one for which you receive no additional credit after it has been completed. For instance, in a lead-generation business, once the lead is generated and you get paid for it, you can’t get credit for generating the same lead a second time even if the same person fills out the form again. Product sales is an example of an accumulating goal, where selling more product to the same person adds up additional revenues (and usually profit). These need to be handled differently.

Note that all of the metrics that you are trying to optimize are normalized. In other words, they are a ratio that divides one quantity by another. In all of my previous examples, the item that you normalize by has been the unique visitor (e.g., conversion rate per visitor, revenue per visitor, and profit per visitor). In reality, things are more complicated. You will need to decide whether to normalize on a per-view, per-visit, or per-visitor basis. The correct choice is very important.

This article originally appeared in Tim’s ClickZ column April 25, 2011