Sunday, 7 October 2012

e-Learning industry, get your act together!

I've lately had cause to examine e-Learning authoring tools on the market. I've found that the entire market - the e-Learning culture - is broken. It's stuck around the turn of the century, and badly needs to catch up to the lessons of the last decade.

Hey, chill out Guy! Are you really taking issue with the entire industry?

Well it’s not just me but kind of. Yeah.

A challenge for you

If you are of the e-Learning faithful, I expect none of my arguments below will persuade you. So here’s an uncomfortable fact about my own department, that I expect is true of yours too.

Our colleagues in the business come to us for online learning only when they have to. We have dozens and dozens of e-Learning modules, built to a very high standard of learning content and immersive interactivity. But the only ones that get high user numbers are the mandatory ones.

If this is true in your company, then there’s something wrong with the resources that you’re providing. The industry has sold you more and more ways to improve your content – PowerPoint conversion, synchronised narration, templates, enhanced interactions – and elective users stay away.

If I am wrong, if you organisation produces non-mandatory e-Learning modules that your colleagues proactively and enthusiastically seek out, then please get in touch. My boss will pay you for your secret sauce!

If not, then something is broken.

How did I get here?

I've been an internet professional for about six years now. I joined a large media corporation as a software engineer and gradually became a technical project manager. Turns out, this particular corporation puts a huge emphasis on things like information architecture and useability - I honestly believe that many of our engineers understand user experience better than a lot of user experience professionals out there.

A few of the things I've learnt:
  • The site your users spend the most of their time on is somebody else’s, so your navigation and user interaction have to be simple and obvious, fitting the broadest best practice on the web
  • Standards-compliant sites win - you want as many people as possible to use your site, right?
  • Links and URLs are core elements of the web - sites that freely link to other sites provide a richer and more valuable experience for the user
  • All the great sites respect these rule of the web - Google, Wikipedia, YouTube, Amazon - even (to a degree) Facebook
And of course...
  • Content is King

e-Learning software - what do we need?

I'm hard at work in our e-Learning department, not commissioning or authoring courses, but devising an approach to update our course commissioning workflow and technology.

Back in June I blogged about some of the user research we commissioned. In the first part of this year, I also did a fair bit of research online into what the most effective learning material looks like. So I've recently been able to complete a vision document for the new product.

Next question: build-or-buy? Obviously it would be fun to build our own authoring tool, that does just exactly what we want. But if I'm honest, buying something off-the-shelf will be cheaper, much quicker and a great deal less risky. So lately I've investigated a large number of e-Learning authoring tools.

Oh dear.

Page-oriented or Slide-oriented

A few product features are available over and over again:
  • Outputting a Flash presentation
  • Converting from PowerPoint
  • The ability to play on iPads and Android tablets through special software or...
  • Playout on iPads and Androids with HTML5
These characteristics are common across almost all the authoring systems available, from niche players to the biggest names in e-Learning. Next to these, the features that distinguish one product from another are scarcely relevant (desktop vs cloud, stock photos of 'actors' with hundreds of poses and thousands of emotions, etc). This is the industry paradigm.

These features push what I call a slide-oriented view of e-Learning content.
  • Page-oriented: Consecutive pieces of content in the course load as separate web pages
  • Slide-oriented: Consecutive pieces of content load as consecutive slides in a player, within a single page
Page-oriented content is how the majority of the web works. All those web hero sites I mentioned earlier: Google, Wikipedia, YouTube, Amazon and Facebook. The top 10 websites in the US and in the UK work this way. That's a pretty strong recommendation.

Slide-oriented content is my description of content that emulates PowerPoint. I'm particularly concerned with e-Learning here, but it includes some other experience-based websites and even some corporate sites.

A deep-seated mindset

From my research online, and discussions with my e-Learning production colleagues, I think this reflects some deeply held beliefs in the e-Learning community:

  • You have to control the user's learning journey - you can't allow them to visit any slide (or page) in any order.
  • Keeping all your content within a slide container is somehow 'immersive' - re-loading a web page would break the spell.
  • e-Learning content is not web content - it's something different, with different rules.
  • PowerPoint is a really great model for online learning.

The first two of these are exploded by all the best websites out there. Think of any website you love - it allows you to browse from page to page, unconstrained. And those pages re-load in your web browser - they don't seamlessly re-load in some container within the page. (OK I'm sure someone will identify a determined exception.)

Why do you keep coming back? Because you can find what you want easily, go where you please, and because someone emailed you a great link to some really great content - just that little bit there that's really interesting/funny/relevant, not the start of a 10-minute presentation to wade through. Maybe it piqued your interest and you checked out the rest of the site. Maybe it didn't.

That's how the internet works. It's been pretty successful.

But e-Learning content isn't a website, and it follows different rules. Really? Here are the best and biggest e-Learning sites I know: Google, Wikipedia, YouTube, Khan Academy. All of them follow page-oriented, not slide-oriented design.

Do you use Wikipedia? Do you find it off-putting when the page re-loads - does it break your learning spell? Would it be more powerful if it forced you to read the whole page in order, before moving on to the next page of its, not your, choosing? Those seem to be the priorities of the e-Learning industry, in contravention of everything we know about the web.

As to PowerPoint, I simply can't fathom why the industry considers it a great pedagogical model. Death by PowerPoint isn't just a saying - it was a key contributor to the Columbia shuttle disaster, killing seven astronauts on re-entry.

Flash is great, except when it sucks

Flash is great at two things: games and video.

There are JavaScript and HTML5 games out there, but there's no doubt that Flash games dominate the online arena (with apologies for time lost wasted enjoyed from those links). Flash provides a really powerful toolkit where graphic design, interactivity, music and speed can come together to create woefully addictive games.

And Flash is still the only mainstream way to deliver video online. YouTube has an HTML5 trial, but due to a format war between the major browsers, Flash is still more compatible all-round.

Flash sucks when it's used for navigation and for presenting content. I can be quite specific about this...

Flash sucks for site navigation

Custom navigation tools: Your internet browser comes with a variety of built-in tools for navigation. These have been developed over 20 years (wow!), with various browsers vying to invent new tools and keep one step ahead of each other. The best of these have stuck, and you use them daily without thinking about it.

Flash breaks all of these. Instead, e-learning vendors have to create their own ‘players’ with their proprietary navigation: play buttons, forwards, backwards, menus etc. None of them are quite alike, so it’s an extra layer that sits between the learner and the course content.

Two broken tools are particularly egregious: the URL and the back button.

The URL: By encapsulating the entire 'experience', Flash stops me from bookmarking a page, or copying a link and sending it to a friend. To find a page that I’ve visited before, I can generally just start typing something like the title in my browser’s address bar - the browser remembers which pages match and help me find them again. The Flash course stops me from doing this.

If I want to get to a specific piece of information in the course, I have to wade through from the start. If I’m very lucky, the courseware might provide some kind of table of contents...hidden away somewhere in the custom player. Using HTML, it would be an intuitive design pattern.

The back button: Study after study has found that the browser’s back button is a core tool for web users. If I click it in a Flash course, I'm taken out of the course altogether. Clicking the player’s own back button is a mug’s gamble. How far back it will take me - the previous slide or a whole chapter?

Flash sucks for web content

Accessibility: Assistive technology tools have been built around HTML websites. This provides for a wide range of assistive needs, from browser text zooming to screen-readers and more. Flash can be made to work with some of these, but more often isn't.

Copying: I can easily copy-and-paste from a page of HTML, eg to take notes or to drop a quick quote into an email. Flash breaks this.

I hope you understand that these are profound weaknesses, even if some of the detail might be unclear. You might consider these to be benefits in e-Learning. You consider it vital to control the user journey, rather than letting the user skip through. And since the course is your intellectual property, you'd rather the user didn't copy it.

This isn't how I learn best. I don't believe it's how anyone does. If Wikipedia imposed these restrictions, would you ever go back again? Unless you’re the one with the e-Learning secret sauce, it’s not how your colleagues learn best either.

...and Flash is dead - so it kinda sucks for the future

Over the last year, Adobe has started retreating from Flash on Android (November 2011, June 2012, August 2012). Having been kept off the iPhone and iPad, this signals their demise in the mobile market.

That’s why the authoring tools are jumping through hoops to play through proprietary apps on the iPad or output HTML5. If they used HTML5 to create true page-oriented content, it would be an incredible revolution. Unfortunately it looks like they're simply using to reproduce the slide-oriented experience without Flash.

Empower the user

No software or website can compete in a free market on this basis. Users hate it and will go somewhere else, or nowhere at all. e-Learning only gets away with this kind of quality because it’s built for a captive corporate audience. But poor-quality content is still poor-quality content.

Good quality e-Learning will be e-Learning that users would choose to use, of their own volition. The experience of the internet shows that this means relinquishing control and putting it in our users’ hands - treating them as adults.
  1. Ditch the sausage-machine courses that force users to sit through every page, every video, every narration. It’s useless at ensuring the users take in all the material - our user research found that they simply turn off the volume and get on with their email. And it alienates casual users from trying the course at all.
  2. Need to ensure your users know the content? Test them at the end instead, with a challenging pass-mark. It doesn’t matter if they can pass the test without reading the course material - the point is to ensure they understand the content, not force them to sit through a lesson they already know.
  3. Drop Flash, and embrace the full vocabulary of the internet.
  4. And remember that Content is King. If you provide great learning resources, and pull down the barriers to accessing it, you users will come.

One intriguing alternative, and Google

There’s a glimmer of something new. eXe is an authoring tool that produces SCORM-compliant content in pure HTML and JavaScript. It’s an open-source product, so you can pick it up and try it for free. It installs pretty easily too.

eXe comes from New Zealand academia. Combined with its open-source heritage, that means it doesn’t have the slick marketing budget of all the commercial players. Its website speaks to internet technologists rather than eLearning specialists or corporate procurement departments. I think that’s a real shame, because if packaged and marketed well I think eXe has the potential bring the real change that e-Learning needs.

Of course the other potential disruptive player is Google Learn. Learn is if anything rougher round the edges than eXe. I’m formerly a software engineer, and I couldn’t install it easily. So unfortunately I can’t give any feedback about its features or its quality. But of course being Google, watch this space...

What’s the bottom line?

To recap:
  1. Flash e-Learning derived from PowerPoint alienates users…
  2. ...but it’s pretty much all the market provides
So what’s a corporate learning coordinator to do?

What’s your budget? Whatever you’re spending on the big authoring tools, it’s not worth it. Why not spend it instead on developing fantastic content in eXe? Maybe it needs a bit of CSS work, or a new interaction. That’s a great place to put your budget, instead of into online PowerPoint! At least give eXe a shot - it’s free so you have absolutely nothing to lose.

Create a couple of courses that allow your users to jump in and navigate around however they please. Throw in a pass/fail test at the end if you like. Then do some user testing - did they get more or less out of it than an expensive Flash job? Would more courses like this encourage them to come back of their own accord?

I truly hope we can shake up this industry, and if enough people learn that e-Learning is just one form of web content maybe we can. Of course if you’re an e-Learning vendor I’m offering you the chance to steal a march on your competitors. If anyone built an enterprise tool that did this, my corporation would procure it in a shot. Build it and we will come!

As for me, I think I’m about to have the opportunity to build my own authoring and publishing tool, and do it right...

…just one more thing

If you really are that e-Learning author whose non-mandatory modules are as popular as your mandatory ones, please do get in touch!


Saturday, 14 July 2012

Pair-PM part 2

It's almost a month since blogged about Pair-PM, and promised to report back on how it went. So...

Our PM community was really enthused by the idea, and about a dozen signed up to the session. Then one by one they cancelled (mostly for annual appraisals) until there were 4 of us left.  Well that’s enough for 2 pairs!

I had a great chat with A - we worked through some difficulties on both of our projects. I had worries that I simply hadn’t vocalised before, and just talking about them made the answers pretty obvious. A had some concerns about collaboration with a colleague, and we discussed ways to work through those. It was great just to have a chat with another practitioner - I'm in a tech-light corner of the organisation and from that one conversation I felt more plugged-in to a community of practice than I’ve felt in a while. For me, the session was everything I’d hoped, and I think the others all found it equally positive.

Everyone remains really enthusiastic about the idea, and I've booked another session in a couple of weeks - just before the Olympics. Well some folks might drop out. No matter - I'm making it a regular opportunity to connect with my colleagues and become better at what I do. ANyone who wants to take part is welcome!


Wednesday, 20 June 2012

Courage and integrity - a heartfelt thankyou

A true story about my colleagues Olivia and Bob. Not their real names.

We launched a new website yesterday. It's a huge improvement on its predecessor in every way - congratulations everyone involved! Olivia noticed that the homepage inadvertently promoted a problematic message about the organisation's aspirations and views on ethnic diversity (sadly it's accurate about our success tackling internal diversity issues). She sent the site's senior editor, Bob, an email outlining her concerns. Bob invited Olivia round for a chat.

Whereupon he subjected her to a tantrum pressurising her to recant. "The tail wagging the dog" is the most printable part of his tirade.

My point here is not that Olivia was right (she was). It's that in the face of immense pressure she stuck to her point calmly, firmly, politely and consistently. She was exemplary, and it clearly took an emotional toll.

So yesterday Olivia, for genuine integrity and courage, you were my hero. Thankyou.


Tuesday, 19 June 2012


Hey why do engineers get all the fun!

Back when I was an engineer I learnt to pair programme. Now I'm a project manager, I don't pair any more.

Let's recap some of the benefits of pair programming:
  • Sharing knowledge - anyone can work in any part of the project
  • Sharing experience - everyone becomes a better coder
  • Acting as one anothers' conscience - ensuring that the tests (or specs) are written first, the code is properly factored, and whatever other tasks a programmer might be tempted to slide under the carpet
  • Working together is more fun than working on your own!
I think all this works for PMs too:
  • Sharing knowledge - everyone understands each others' projects and the broader business context, and can switch project if necessary
  • Sharing experience - everyone becomes a better PM
  • Acting as one anothers' conscience - ensuring that the release management has been considered, the training needs have been addressed, and whatever other tasks a PM might be tempted to slide under the carpet
  • Working together is still more fun than working on your own!
I don't care whether you call yourself a PM, a TPM or a Scrum Master, and which breed of Agile, Waterfall or anything else you apply. Learning from each other can only be a good thing.

So what am I doing about it?

So later this week I'm running a Pair-PM session. We're not going to shadow each other and we're not going to play chicken at each others' Scrums (though I think that would be a great next step).

We're going to get together, pair off and just chat about our projects. What's going great?  What's vexing us?  Where could we do with another brain? It doesn't matter if a novice pairs with a past-master and much of the learning's one-way - next time the past-master will pair with someone else. And we'll be breaking out of our usual boxes and hierarchies.

I'll post again to tell you how it went.


Monday, 11 June 2012

User Testing - Awesome Lessons

Since the end of 2011 I've been working on the business' internal product for online learning. Our portfolio of about 170 courses has a range of well-understood problems, and some less well-understood difficulties and aspirations. I was brought on board to deliver a new set of tools to author and and publish courses. It's a fantastic role, with a huge amount of trust and autonomy from my management, and involving me in Product Management almost as much as Project Management.

Near the start of the year, my Product Manager and Senior Stakeholder asked me to arrange user testing of the existing product, to find out what users thought were its best and worst points.

My previous experience of user testing hadn't been brilliant. It was run by an agency's lead designer with a highly suggestive questioning style, and it resolved a disagreement about site navigation between the client and the agency. So this time round I told my managers they they should have the courage of their convictions and specify the product that they believed in.

I was dead wrong. User testing turned out to be a fantastic experience, full of lessons for the project and for me.

In about 10 years of publishing online courses we'd simply never asked our users what they thought. We'd user-tested individual courses in development, but never the portfolio as a whole. So even if everything our users said had been entirely expected (it wasn't), their feedback would have moved us from specification based on gut feeling and sentiment to known user requirements.

Lessons for the project? Loads - not least the fact that our content specialists and technical specialists, whose aspirations sometimes seem to be at odds, are both right in their own domains. It's nice to find out that we all know our own fields! Beyond that, there are dozens of points to extract from the report and we'll end up with a much better product because of it.

Lessons for me? User testing can be a really valuable investment of time and money, provided you're in a position to be as receptive to criticism as you are to praise. Of course that's a matter of mindset and leadership, and of planning and execution.


Thursday, 16 February 2012

Non-Testable Acceptance Criteria

Non-Testable Acceptance Criteria

I recently discovered the INVEST criteria for user stories. A quick recap - user stories should be:


Setting aside the interesting tension between Independent and Small, I have a difficulty with Testable.

Non-testable stories

A canonical example:

The software should be easy to use.
thanks to Vabihav

To make it testable, this might get broken down:

A novice user can is able to complete common workflows without training.

For testability that might become: "80% of novice users can use the core functionality without instruction in 5 minutes." OK that's not automatable, but at least it's objectively testable. But does it capture the business value?

A real-world example

The promo area on our homepage contains beautifully designed links to 6 articles in our CMS. We have to select those 6 articles in the CMS. Consider three ways to do this:

  1. We open each of the 6 most important articles and click the "make homepage promo" link.
  2. The CMS contains a checkbox list of the articles, and we select 6 of them.
  3. The CMS contains 6 empty slots, each of which we populate with an article title.

1. is plainly impractical, and almost certainly fails our test.

2. is a clear UX and can certainly be used by novice users. It passes the test. And it looks nice and easy to use in development. But we have 2000 articles and more every week. Selecting 6 of them in an alphabetical list is anything but easy to use.

3. might be nice and easy, if coupled with a decent search facility (eg auto-complete). The development team implemented 2.

Tightening up the acceptance criterion

80% of novice users can set the right number of homepage promos without instruction in 5 minutes, for any set of 6 top articles, for a CMS load of up to 10000 articles.

Advantages of this approach:

  • It's testable
  • It's specific in a dispute (the legal department will love it)

Disadvantages of this approach:

  • I've no idea whether I've missed any ways the statement needs tightening up
  • It's plainly absurd

Frankly I think we're better off with "The software must be easy to use" and an adult conversation about what that means for the user group.

Some business needs just aren't necessarily black-and-white testable.