Common Landing Page Testing Terms
While members of the landing page testing community may use different terminology, you'll find the following definitions useful in understanding the basic elements of testing.
Input and Output Variables
A landing page test has two basic components: a set of input variables (also called “independent variables”) that you can control and manipulate, and one or more output variables (or “dependent variables”) that you measure and observe. Independent variables are simply the tuning elements that you have chosen for your test.
The word "variable" (when used by itself) means a tuning element that you have selected. Variables can be of any granularity or coarseness, for example a headline on your landing page or a whole-page redesign. In multivariate testing, a variable is also commonly referred to as a factor. Not surprisingly, in a multivariate test you will have more than one variable. Examples of variables include headline, sales copy, button text and button color.
A value is a particular state that a variable can take on. For example, a button color might be green, blue, or red. Measuring the effect of one value does not provide any information about the other possible values. In other words, if you measured the average conversion rates with the green button and the blue button, you would still not not have any information about the performance of the red one.
The total number of possible values for a discrete variable is called its branching factor. For discrete variables, the branching factor must be at least 2 (the original version and one alternative). Using the button color example, the button color variable has a branching factor of 3, because it can take on the values signified by green, blue, or red).
A recipe is a unique combination of variable values in your test. It is a sequential listing of the specific values that each variable takes on in the specific version of the landing page.
Search Space Size
The number of unique recipes in your test is your search space size. It can generally be computed by multiplying together all of the branching factors of the variables in your test.