At Selesti we follow a lean, agile approach with both website design and conversion rate optimisation. We believe in responding to change rather than creating concrete, unchanging plans. One way in which we do this is to follow the following process:
A big mistake when designing any website is to spend months researching and creating the ‘perfect’ design and never making changes. A better approach is to create a minimum viable product based on a set of assumptions and hypotheses, launching this early and using insights from the use of this to feed into research and learning for the next stage. There are three key approaches for testing:
User testing
User testing is a powerful method of measuring a website’s usability, design and structure. This process involves observing real users as they interact with a website, and asking these users to complete a set of tasks while describing their thoughts and feelings.
This type of testing can take place at any point in the design phase, from testing prototypes or wireframes before they are built to fine tuning existing site journeys; to test a journey or a specific area (such as the checkout); or to compare your website with a competitor’s site.
User testing can be approached in two ways:
Laboratory testing
Participants are observed completing tasks on a webpage in a ‘lab’ environment. To conduct lab user testing a computer is set up with screen recording software, and participants are asked by an interviewer to complete a set of tasks. While completing the tasks, the user will be asked to say out loud how they are completing each task, and the interviewer will observe their responses.
In advanced lab settings, eye tracking can be used to see where users are looking on the screen in response to the questions being asked. This can be extremely helpful when understanding where things should be placed on a site, as this may not be something the user actively verbalises. For example, if when asked to navigate to an account section the majority of users tend to look at the top right of the screen, it would make sense to place this element there.
The main advantage of lab testing is the breadth of data gained from these studies. With lab testing it is easier to monitor eye movement, user stumbling blocks and satisfaction.
Two of the main disadvantages to lab based user testing are the cost and the artificial setting. Lab user testing requires specialist equipment, which will likely mean that a specialist room will need to be hired out. Additionally, the lab environment can skew results since participants aren’t in their natural work or living environment, they may not behave as they would at home.
Remote testing
Remote user testing works in a similar way to lab based testing, but instead of an interviewer being in the room with the tester, a test script is sent to them to read through and complete.
There are different solutions for this, but one of the best remote user testing solutions available is UserTesting. This program allows you to target either your actual customers or to use a test panel that can be filtered using a set of demographic filters and screener questions.
Once users have completed your test script you will receive recordings of their screen with audio of their experience.
The advantages to remote user testing are the cost and the speed at which you can complete your test. Once your test has been sent out participants, it can take as little as 24 hours to have a full set of results from multiple users.
However, there are drawbacks to this form of testing. One of the biggest issues is with the test panel itself. Unless you have decided to source your own users, the test panel will be used to completing these tests, as they may complete multiple tests per week, meaning their responses may be less natural.
Whichever way you chose to complete user testing on your eCommerce site, you should always have a clear outcome in mind. It is impossible to test your whole website and highlight all issues in one session. Use data to understand which areas of the site or journey require your focus and start there.
Heat mapping and screen recording
Heat mapping and screen recording is the process of looking at how customers are engaging with the website without them being aware. This process is entirely anonymous, and looks at how your actual customers are interacting with your site.
Heat mapping
Heat maps are used to see where users are interacting on a webpage. There are three main types of heat maps: scroll depth, click tracking and mouse tracking.
Scroll depth looks at how far users are scrolling on a page.
Click tracking looks at where users are clicking on a page.
Mouse tracking shows where users tend to move their mouse on a page.
Screen recording
Screen recordings offer a more granular view of your customers habits. These recordings show actual customers as they look through the pages of your site. These observations are instrumental in understanding how engaged users are on your site, where customers tend to dwell and what element of a page causes them to leave the site.
This form of testing, when used alongside user testing, can give a much better insight into how real customers are engaging with your site. With user testing there will be an element of inaccuracy due to being observed, whereas with screen recordings the customer is unaware that they are being watched.
A/B testing
A/B testing is an experimental process whereby two or more variants of a page are shown to users at random, and statistical analysis is used to determine which variation performs better for a given conversion or goal. In its simplest form, A/B tests create two designs for the same webpage and sends 50% of users to version 1 and 50% of users to version 2.
Remote testing
The data and insights gained through advanced analytics tracking, user research, journey mapping, user testing and heat mapping will give you a good basis to begin testing the best solutions for your site.
Identify your goal
Although there may be multiple outcomes from your experiment, you should always determine a main goal for your experiment. Goals can be anything from clicking a button or link to product purchases and email signups. The number of goal completions for each will determine whether or not the variation is more successful than the original version.
Create your hypothesis
Your test hypothesis should explain what results you expect to return and why. A test hypothesis should follow this simple structure:
“If ____, then ____, because ____.”
For example:
“If the ‘buy now’ button is moved to be above the fold on mobile then there will be more conversions because user recordings suggests customers find this button hard to find and tend to exit the site”
Decide how significant your results need to be
Making sure the results from A/B testing are reliable requires a statistically significant result. To test whether the changes to a page show a true improvement a success level needs to be decided. This will vary between experiments depending on what is being tested. For example, if testing for conversion rate increase, a successful outcome could be if the page’s conversion rate has improved by 20%.
Pick one single variable to test
Making sure the results from A/B testing are reliable requires a statistically significant result. To test whether the changes to a page show a true improvement a success level needs to be decided. This will vary between experiments depending on what is being tested. For example, if testing for conversion rate increase, a successful outcome could be if the page’s conversion rate has improved by 20%.
Run experiment
With everything in place it’s time to run your experiment. During this time there is no need to change anything within your test. However, it is important to observe the results on a regular basis to ensure that the changes you have made aren’t having a drastically negative impact on your results (which is another reason why having solid research and a hypothesis in place is so important).
Analyse results
Once your test has ended you can analyse the results. There is no wrong answer when it comes to A/B testing. If your hypothesis has been proven incorrect, or you do not have a statistically significant difference, this result can be included in your research data for following tests.
Draw up a new test plan
s mentioned, A/B testing is an ongoing process. Once your test has completed, it’s time to write up your next plan and do the whole thing again.
Much like every aspect of marketing your eCommerce store, A/B testing is a never ending task. It’s an iterative process with each test building upon the results of the previous tests.
Here are some basic test that can be carried out to give you an idea:
- Content layout on product pages
- Changes to navigation layout or structure
- Changes to the body copy on category or product pages — Testing different headlines, banners or USPs
- CTA colours and placement
