Limitations of Personas in Landing Page Testing

One of the tools commonly used in usability testing and user-centered design is a persona. A persona is a fictional person who acts as a representative stand-in for a large class of representative visitors to your site.

Personas should be developed by observing usability tests, conducting contextual interviews (working in their actual typical environment), online surveys, focus groups, and regular interviews (where you don’t observe people working).

Typical elements of a persona include:

• • A name and picture

• • Demographics (age, education, ethnicity, family status)

• • Job title and major responsibilities

• • Goals and tasks in relation to your site

• • Environment (physical, social, technological)

• • A quote that sums up what matters most to the persona with relevance for your site

The following persona is an example from a usability test for a consumer technology comparison website:

Ned Adams (“Ned the Nerd”)

Ned Adams is a “thought leader” for people who are interested in emerging consumer technologies.

• • 24-year-old college drop-out; studied computer science

• • Single, not in a relationship; lives at home with his parents

• • Clerk at the local computer game arcade and Internet café

• • Influencer for new gadgets among his peers

• • Spends a lot of time on the Internet

• • Has his own consumer technology blog

• • Always upgrades his high-tech gear to have the “latest and greatest”

• • Quote: “If it’s cool and new, I want to know about it!”

The advantages of personas in user-centered designs are:

• • They help everyone have a consistent view of major audience groups.

• • They are easy to communicate.

• • They increase empathy toward the audience and help focus on their needs.

• • They help prioritize possible new product features (based on how well they would meet the needs of a particular persona).

However, personas are generally not appropriate for large-scale statistical landing page optimization tests. In this setting, personas have several significant drawbacks and limitations:

Basis for construction. The proper basis for constructing personas are usability tests, contextual interviews (seeing users work in their actual native environment and not a controlled usability lab setting), focus groups, and surveys (telephone, in-person, or online). Personas should always be representative of key audience classes and grounded in experiences with real people. Unfortunately, some Web conversion companies have begun fabricating personas based primarily on their personality types or behavioral styles, with no reference to actual classes of visitors to a website. Without access to real people, these personas basically become window dressing around the various cognitive styles and temperament types (such as Myers Briggs).

Difficult to understand other cultures. Many website audiences are worldwide in scope and include a significant percentage of international visitors. It’s difficult to emotionally connect with people from other cultures and backgrounds and to even begin to understand their motivations. In such settings, attempting to construct international personas would be difficult, while modeling them with standard local counterparts is bound to be inaccurate.

Diverse visitor populations. Most consumer-oriented websites have a diverse audience. Large websites have visitors whose demographics and psychographics closely mirror the population mix of Internet users as a whole. Since a given temperament or behavioral type represents only a small fraction of the overall population, any attempt to span the whole audience with a small number of personas is bound to fail. Whole segments will be missed.

Of course, this is less of an issue for some niche and special-purpose sites. For example, I’m pretty confident that returning visitors to a bungee jumping coupon site (I’m pretty sure that such a thing must exist somewhere on the Web…) can be effectively modeled by a young thrill-seeking male persona. But in most cases, the visitors to your website are a pretty bland mix of standard people.

Competing needs. You’ll find that each persona has differing and conflicting needs and desires. This isn’t so much of an issue when one persona can be used as a stand-in for a whole visitor class – then the landing page and conversion task is designed around them.

However, as discussed earlier, many websites have diverse populations, thus requiring multiple personas to effectively capture the characteristics of a single visitor class. Unfortunately, this implies compromise and suboptimal landing page designs. Instead of a consistent user experience based on a single persona, the landing page becomes a hodge-podge of features and trade-offs designed to appeal somewhat to each relevant persona. The clarity and integrity of the design is bound to be destroyed via the resulting additional clutter. In many cases, what helps one persona will actually undermine the needs of another.

Addressing the wrong question. Personas allow you to ask, “What is best for this persona?” Even if your answer to that question produced the best results possible, you would still not be addressing the needs of all visitors who don’t match the persona. Since you don’t know whether a particular visitor to your site is well-represented by the persona, you must deal with the audience as a whole. The appropriate question then becomes, “On average, which of the tested alternatives does this audience prefer?” This is exactly the question that large-scale statistics-based testing can answer.

At odds with the spirit of testing. Personas, as often applied to landing page testing, include some hidden assumptions: that users can be divided into a small number of key classes and that if you create detailed personas you can come up with the perfect design to address their needs. The only real validation conducted is to determine if the recommended new solution is better than the original.

The persona-based design may well perform better than the original (often simply because the original was so poorly designed in the first place). But why would you restrict yourself to a single design alternative? Remember, in user-centered design, personas are used to run a series of usability tests. It’s rare that the first proposed solution is accepted as is.

Unfortunately, personas are commonly used on a one-shot basis in landing page optimization to create a new landing page design. There is no iterative redesign and testing process. When practiced this way, you will get one alternative design choice that is the best educated guess of an expert or outside consultant. This is radically different than the large-scale statistical testing perspective. Statistical testing looks for the needle in a haystack and can identify which of a million possible page versions is the best performer.

No human expert can guess at the answer under such circumstances and hope to consistently predict the winner. In my experience, I am often wrong about which final landing page version (or even individual test variable setting within it) will end up as the winner. This sobering and humbling realization is the reason for trying many alternatives in the first place, and antithetical to the notion that any one person can come up with the right solution. Your audience must vote through their actions on the page to guide you to the solution.

This article originally appeared in Tim’s ClickZ column August 30, 2010