This comprehensive 30-page whitepaper educates you in non-technical language about all important aspects of fractional factorial landing page testing approaches such as the Taguchi method (see Executive Summary below).

Our privacy policy: We will not share your information with anyone else. Period.

Other downloads

Executive Summary

Landing page optimization and testing is a powerful way to improve the profitability of your online marketing programs. By testing alternative presentations of information in your mission-critical online processes on your audience you can dramatically improve the conversion rate of these desired actions (such as sales, registrations, form-fills, or downloads).

The most basic method of landing page testing is a simple head-to-head test of your original landing page against an alternative version. Beyond such "A-B Split testing" there are a number of more-powerful multivariate testing methods that allow you to consider multiple changes to the landing page at once.

With the growing popularity of landing page optimization, certain approaches have been canonized and have taken on an almost mythical reverence among the ranks of online marketers. It is almost as if the buzzwords themselves confer some special power on the practitioner (e.g., "Design of Experiments", "fractional factorial," and the "Taguchi method").

In reality there is a huge mismatch between the original environment in which such fractional factorial testing was developed and how it is usually applied to landing page optimization. It was basically transplanted to online marketing because it is relatively easy for a nonmathematical audience to understand, and not because of its appropriateness or fitness for the task.

The principal drawbacks of fractional factorial methods are:

• Very small test sizes
• Restrictive & inflexible test designs
• Less accurate estimation of individual variable contributions
• Drawing the wrong conclusions
• Inability to consider context and variable interactions

Despite misinformation to the contrary, fractional factorial methods do not offer any data collection speed advantage over similar full factorial data-collection approaches (such as those available in the Google Website Optimizer tool). If you plan on using parametric (i.e. "model building") approaches for landing page testing you should use full factorial data collection regardless of the subsequent analysis you plan to do.

All parametric methods (including fractional factorial) are also outclassed by newer non-parametric testing methods, which have the following advantages:

• Very large test sizes (1000-10,000 times larger with the same data rate)
• Much faster data collection (on the same data rate)
• More accurate results (consider variable interactions)
• Flexible test construction

Landing page testing consists of two main parts: deciding what to test, and finding the best solution among your available options. Those who still insist that fractional factorial methods are terrific because they have produced significant conversion rate improvements are confused. The improvements in these tests are a direct result of the quality of the ideas tested, and not of the inaccurate fractional factorial methods used to find the best version of the landing page.