Measuring Web experiences with multivariate testing
NOTE: The Multivariate Testing feature is part of Ektron's marketing suite.
Multivariate testing lets you offer several variations of the same page, then measure site visitors' reactions to each variation. The reaction that generates the most conversions during the testing period is automatically promoted to the design for that page, thereby maximizing your website's ROI.
For example, the marketing manager wants to know which of 3 ads produces the most visits to a registration page.
Conversions are tracked, and when the target number of conversions occurs, the variation achieving the best results automatically becomes the permanent page on the site, without intervention by the website administrator.
You can create variations for a number of page elements (images, copy, buttons, form fields, and so on) to find the combination that most effectively compels site visitors to take the next step. You can use actual, timely data to drive decisions.
Multivariate testing is easy to implement, requiring no help from a developer or designer.
IMPORTANT: The Multivariate Testing feature requires the use of PageBuilder templates. For more information, see Developing wireframe templates.
Prerequisite
To set up and run a Multivariate experiment, you must be an Administrator or have the Multivariate Tester Role assigned. See also: Defining roles.
To set up a Multivariate experiment:
Use these PageBuilder widgetA mini-application that you place on a Web page using PageBuilder; a widget provides either specific functionality (calculators, search, social bars, and so on) or areas into which you can add content (content blocks, list summaries, collections, and so on).s for Multivariate Testing.
NOTE: The Multivariate Experiment widget temporarily stores data in server memory before writing to the database in batches of 100, by default. If you want to change the batch size, insert the following key into the siteroot\web.config
file, within the <appsettings>
tags: <add key="ek_MultivariateThreshold" value="100"/>
.
Change the value
to adjust the batch size.
If logging is enabled and the key is missing from web.config
, Ektron creates a warning and provides the key in the event viewer.
The target page is the Web page you want your visitors to view. It contains the Multivariate Target Wizard. Visitors arrive here by clicking links on the experiment page.
To create the target page:
Whether you arrive at the target page by browsing to the variations page first or by browsing to the target page directly, the views are counted with the following exceptions:
NOTE: Logged in users are counted as a page view of the experiment page, but not as a conversion on the target page.
Next, create the Multivariate Experiment Page.
The Experiment page contains:
The following illustration shows steps for creating an experiment page with Multivariate Test widgets.
To create a multivariate experiment page:
NOTE: During an experiment, you can choose a specific variation to be the page that all visitors see and end the experiment.
The experiment widget also indicates how each variation has performed during the experiment.
Use the slider control to select content used in each variation. The following sample shows the slider control for variations 1/2 and 2/2.
You can create a PageBuilder page with more than one Multivariate Section widget. This arrangement helps identify content and images that provide the best conversion results. The following page contains 2 Multivariate Section widgets.
The content inside the Section widgets are part of the variations shown randomly during the experiment. The number of variations in a Multivariate Experiment grows exponentially as you add sections and variations. For example, if you have 2 sections with 3 variations in each section, there are 8 (23) pages to be tested. If you increase this to 4 variations each, there are 16 (24) pages.
After you create the Target Page and Experiment Page:
When an experiment is running, visitors see a randomly-chosen page variation. When visitors view the experiment page, a browser cookie is set, which indicates the variation seen by the visitor. From this point forward, if the visitor next visits the target page, this cookie is read and the proper variation counter is increased. The visitor continues to see this variation until the experiment is ended.
A cookie persists forever or until the cookiesData related to a user and Web browsing activity, such as login information, shopping cart data, and other data. are removed from the browser cache. The cookie gives a visitor a consistent page variation each time they revisit during the experiment. If a visitor's browser blocks cookies, that visitor sees random variations each time they visit the page; they are not counted in conversions.
NOTE: During an experiment, testers can modify variations and content within the Multivariate Section widget.
Click Show Reports to see the experiment’s progress. The reports shows:
The bar graph shows the conversion percentage. The numbers to the right of the bar graph show 2 values.
The following list describes the buttons.
At any time during the experiment, you can disable any variation. Do this by clicking Enable (). The variation becomes disabled and cannot be viewed by site visitors. When a variation is disabled, visitors who have seen this page see another variation instead.
You can re-enable a variation by pressing Stop (), which currently shows the variation as disabled. The variation then becomes enabled.
The PageBuilder page has the same properties as other content types, including the Schedule property. To set the beginning and ending date of the page, edit the PageBuilder page and follow the procedure for setting content schedules as described in Scheduling content.
Be sure to set the Schedule End Date to be long enough past the time you believe the experiment will automatically end.
At any time during the experiment, click Stop on the Experiment widget. This suspends the experiment. Counters stop incrementing, even though visitors continue to see the page variations.
While the experiment is stopped, cookiesData related to a user and Web browsing activity, such as login information, shopping cart data, and other data. are neither set nor read. For more information about cookies, see When an experiment is running, visitors see a randomly-chosen page variation. When visitors view the experiment page, a browser cookie is set, which indicates the variation seen by the visitor. From this point forward, if the visitor next visits the target page, this cookie is read and the proper variation counter is increased. The visitor continues to see this variation until the experiment is ended.. When you stop an experiment, the Multivariate Section widget remains on the experiment page.
You can restart the experiment by pressing Start. Counters are reset to 0 and restarted.
When one variation reaches the target number of conversions, it automatically becomes the page variation that all visitors see; the other variations are disabled.
You can stop the experiment by promoting a specific variation. When you click Promote (), the selected variation becomes the page variation all visitors see; the other variations are disabled.
When the experiment automatically or manually ends, the Multivariate Section widget is removed from the experiment page.
To repeat an experiment after it has ended, first restore the history of this page by following steps in Restoring a previous version. After restoring the previous version, you can edit the parameters of the experiment, modify the content, and run the experiment again.
If the target page is outside your website, you must create an intermediate page that contains the Target widget, and redirect the visitor to the outside address.
For example, to test visitors from your page to http://www.NFL.com
:
www.NFL.com
.While the experiment is running, the experiment page links the visitor to the intermediate page, and the conversion is counted. Then, the redirect takes the visitor to the external page.