Which? – Smart Recommendations tool

An MVP of  new tool launched for Black Friday had proved initially successful. Before more variations were built we wanted to do some qualitative testing of  the tools with new user

Link to live tool Washing Machines  Laptops

Hypothesis: If we better inform prospects of the paywall and improve onward journeys we will see an increase in usage and Sign Ups

Issues identified from remote usability testing:

  • Results: Users want to see their shortlist quickly but some
  • Results: User weren’t sure why they have been shown certain products
  • Feedback/choices made: This was too technical and confusing, it also features too prominently that some users missed their shortlist
  • Paywall: It was not clear why the products were locked, repeated sign up buttons were frequently missed

We also wanted to understand more from the quantitate date:

  • the start rate was higher on washing machines
  • why conversion rate was higher on laptops compared to washing machines
  • apart from signup there was little interaction with the page and the rest of the site

How might we:

  • better demonstrate the value of subscription to non-members
  • include useful and relevant information on why they are seeing shortlisted products
  • make it clearer that some content is locked

Sketching/ideas

After a short worksop session to gather ideas and possible solutions a new results page was proposed.

  • Paywall design: add padlock into design to show locked content
  • Page layout: move shortlist higher up page
  • Needs based advice: add contextual advice content generated from the choices made in the questions
  • Understanding of Which?: add in test score scale and information about impartial product testing
Quick session to record ideas and examples of desk research
Quick sketches of ideas from the team

Prototype

After mapping out the final user flow, I created a high -fidelity prototype to replace the live tools and test a new results page with users. Due to availability of the lab there was only one week to design, built and initially test the prototype.

A questionnaire was also designed to try to gauge users response and feeling toward the visual design and overall quality of the tool

Mapping out the finalised user flow and high fidelity screens synced to InVision

 

Screens created for the prototype (built with Sketch and InVision)

Testing

Day 1: Laptops / Day 2 : washing machines

5 participants each day (non-members)

User who were “in the moment” looking for to purchase a laptop or washing machine. 

Results:

  • There was a positive response to the new content around buying advice with 80% of participants reading it and commenting that it helped informed their decision.
  • About half the participant said it would convince them to sign up for a subscription trial. This was included in the live tool and would be added to the existing tools on the site.
  • All users felt the design and layout was clean, clear and easy to use.

Phase 2

As many users missed the start button for tool a new design will be A/B/C tested to include imagery and colour

The placement of the tool will be tested, adding into various page types and position on the page