User Testing - Awesome Lessons

Since the end of 2011 I've been working on the business' internal product for online learning. Our portfolio of about 170 courses has a range of well-understood problems, and some less well-understood difficulties and aspirations. I was brought on board to deliver a new set of tools to author and and publish courses. It's a fantastic role, with a huge amount of trust and autonomy from my management, and involving me in Product Management almost as much as Project Management.

Near the start of the year, my Product Manager and Senior Stakeholder asked me to arrange user testing of the existing product, to find out what users thought were its best and worst points.

My previous experience of user testing hadn't been brilliant. It was run by an agency's lead designer with a highly suggestive questioning style, and it resolved a disagreement about site navigation between the client and the agency. So this time round I told my managers they they should have the courage of their convictions and specify the product that they believed in.

I was dead wrong. User testing turned out to be a fantastic experience, full of lessons for the project and for me.

In about 10 years of publishing online courses we'd simply never asked our users what they thought. We'd user-tested individual courses in development, but never the portfolio as a whole. So even if everything our users said had been entirely expected (it wasn't), their feedback would have moved us from specification based on gut feeling and sentiment to known user requirements.

Lessons for the project? Loads - not least the fact that our content specialists and technical specialists, whose aspirations sometimes seem to be at odds, are both right in their own domains. It's nice to find out that we all know our own fields! Beyond that, there are dozens of points to extract from the report and we'll end up with a much better product because of it.

Lessons for me? User testing can be a really valuable investment of time and money, provided you're in a position to be as receptive to criticism as you are to praise. Of course that's a matter of mindset and leadership, and of planning and execution.

Guy