This article covers a Hippo CMS version 11. There's an updated version available that covers our most recent release.

Experiments

This feature is only available in Hippo DX.

Experiments allow you to determine if changes you make to a page component lead to more visitor conversions to a specific goal. You can experiment with the content a component displays and with the audience a component is targeted to. A running experiment serves visitors one of the variants of a page component, tracks conversions for each variant and optimizes variant serving based on visitor analysis. After the experiment has been running for a while you can complete the experiment. Usually you complete an experiment if one variant is a clear winner, i.e. realizes a significantly higher conversion than the other variants.

Tasks

Concepts

Experiment

An experiment tests whether a change to a page component's configuration - creating a variant of the page component - leads to better conversion. Multiple variants can be tested in the same experiment. An experiment is either a content experiment or an audience experiment.

Content Experiment

A content experiment tests whether a variation in a page component's configuration (i.e. which content it contains, and how it is presented) leads to better conversion, compared to the current default configuration. A content experiment is similar to a traditional  A/B test, although Hippo allows testing of more than two variants and uses a different, more refined algorithm.

Audience Experiment

An audience experiment tests whether a page component leads to better conversion when it is targeted at a specific audience, compared to when it is not.

Page Component

A page component is a part of a web page containing functionality which can be configured in the CMS Channel Manager.

Variant

A variant is an alternative configuration of a page component. It either has different configuration parameters such as the content it displays (in case of a content experiment), or is targeted at a different (specific) audience (in case of an audience experiment).

Goal

page goal is defined as a visitor reaching a pre-determined page after viewing the page the experiment runs on. A goal determines when a conversion takes place.

Conversion

A conversion takes place when a visitor reaches a pre-determined goal.

Winner

The winner of an experiment is the variant tested by the experiment that leads to most conversions.

How do experiments work?

  • In one experiment:
    • you can have only one goal.
    • you can test from 2 up to 26 variants of a single component.
    • you can test any number of components on the same page.
    • you can't add any more variants or components while the experiment is running.
  • On one page:
    • you can run multiple experiments (each with a different goal).
    • you can run an experiment with the same goal as a different experiment on a different page.
  • The chance a visitor is served a particular variant:
    • is equal for each variant at the start of an experiment, when no data has been gathered yet.
    • converges towards the 'winning' variant as the experiment progresses and more data on conversions is gathered.