When to A/B Test: Website Redesign vs Split Testing

Summary: There are instances when it makes sense to run an A/B test and cases when it’s better to redesign the website.

Sometimes, you have to tweak, edit, and optimize – maximize the conversions your existing website has. Other times, you have to think about burning the whole thing down and doing a website redesign.

The thing is, it can be real pain understanding when to do an A/B test and when to do a much larger overhaul.

And the stakes are high.

If you try optimizing while you have a host of technology issues and structural problems, you’ll be plateauing instead of pushing the business forward. If you redesign a website that just requires tweaks, you can waste a lot of time, money, and energy before things get better.

Thankfully, there are a few things you can observe to make sure you’re doing the right thing. Here’s how you know when you need a redesign versus when to do an A/B test:

When to Redesign Your Website

There are a few indicators that a website redesign makes much more sense for you:

1. You have information architecture issues

No matter how much split or multivariate testing you do, you can’t solve issues when you have bad product categorization. That is, when you have too many groups of products with no clear delineation between the categories, or some other problem deeply rooted in structure, you can’t A/B test your way out of them.

The challenge becomes how you detect those types of issues. Here are a few signs you have an information architecture problem:

• Bounce rate concerns are not localized to a few high-traffic areas
• The category pages tend to have high bounce and exit rates. Visitors seem to have difficulty drilling down to the right product details
• The top-level areas get significant traffic, while the deeper pages don’t get much traffic. Visitors appear to have problems with your menus
• Satisfaction on the website is low

If you have information architecture issues, you need to follow these steps:

• run card sorting or tree testing exercise,
• figure out what the correct groupings are, and
• then build your site with the new groupings in mind

You’ll need to have some proficiency around redirects and URL structures to minimize the SEO risks of moving things around. But you’ll solve more problems than you’ll create.

2. Your content management system (CMS) does not support important things

There are a lot of things you can do even if your CMS does not cost you hundreds of thousands to millions of dollars per year. That said, your CMS does need to support the basic online marketing functions of the day:

Responsive web design

If your CMS is preventing you from rendering good content for both desktops and mobile devices, it may be worth considering moving to a CMS that supports responsive web design. Then, you can rethink your site design, so you serve visitors of both device types content that renders properly.

Location-based personalization

If you have a global site and your CMS doesn’t allow you to render different content for different regions, it could be time for an upgrade, and for structured thinking about what you can  and cannot afford to regionalize.

Incompatibility with certain tools and platforms

It’s okay if your CMS does not have native web analytics, survey, tag management or testing platforms. You can use market tools like those offered by Google if that’s the case. That said, if you have compatibility issues with the tools you need, or no way to render scripts across the entire site, it could be time for a CMS upgrade and a look at your tool set.

3. Split tests are showing diminishing returns

After you’ve run your site for a while on the same rough designs (assuming you’ve been running split tests to optimize the key pages), you’re likely to see the gains from A/B testing get smaller and smaller.

That can mean the next set of major conversion improvements requires a design leap. But you can lose out on conversions first, and you’ll have to spend on the actual site redesign. If this is a path you’re taking, you should consider hiring a conversion rate optimization agency.

When to Run an A/B Test

Here are situations when an A/B test can bring incremental benefits that will move the needle over time:

1. A few product areas are underperforming

Let’s say you have a few key conversion points, and about a third of them are underperforming. That basically indicates you have prime candidates for split testing. You can find the underperforming pages to test by going to your web analytics tool and looking for these groups of pages:

High-traffic, high bounce rate areas

If you have Google Analytics on your site, there’s a pretty cool feature called weighted sort that can find all this stuff for you. If you’re using another tool, prepare a list of high-traffic pages, and then sort through which ones have the highest bounce rate manually. Either way, this will tell you where moving the needle a little will amount to making significant headway.

High-traffic, low-conversion areas

If you have conversions set up as a goal, look at where you have enough traffic to run tests, but low enough conversions to warrant the tests.

2. The site has localized navigation issues

If your bounce rate on the site is decent except for a few areas, you can test the category-level area where you are having issues. Try different words or grouping schema on the challenger page, and see if you can solve the issue with basic tweaks.

Conclusion: When to Redesign and When to Do an A/B Test

Redesigns can be tricky and expensive, so you shouldn’t treat them lightly. That said, a website redesign is sometimes less expensive than not doing anything and experiencing limited conversions as a result.

Walking the fine line means understanding exactly what is wrong with your site, going through the data, and understanding whether you need to keep running smaller optimization efforts or clearing the field and starting anew.

If your …

• site has serious information architecture issues
• CMS has limited capabilities and cannot handle responsive web design, personalization for locations, or running the right tools
• tests have started to give you smaller and smaller returns on investment

…  it may be time to bite the bullet and consider a website redesign.