Digging my own Ditch

Archive for the ‘testing’ Category

What went wrong with Google Wave

Wednesday, November 4th, 2009

Like thousands of people like me a few weeks ago I got my Google Wave invite, I logged in clicked about, played with a few Waves, had some conversations and signed out.

Overall I was a combination of slightly bermused, impressed by the technology bowled over by the real-time goodness and the slick UI, but fundamentally unsure of what I was supposed to use Google Wave for.

But what actually went wrong here? I’ve got a theory and it’s not what you might expect – valid though those criticisms are. Google Wave went wrong, not because of what it is, but because it didn’t harness any viral power:

  1. Invites are not sent fast enough
  2. Wave is closed
  3. Wave has no pointers

Google Wave is first first and foremost a communications technology, which means for people to really see the benefits they need to be communicating with people they know.

When you join Google Wave you receive 8 invites, when you’re through those you get another 12. But when you send the invite they don’t arrive for several days. Meaning you’ve got practically no-one of significance to communicate with when you’re first into it*.

So your first experience of Wave is with strangers.

The second problem with Wave is that you don’t know when you’ve got an unread Wave. You signed in first time around, played around a bit with a few people you vaguely know online, and sign out. If someone now communicates with you whilst you’re signed out you’re not informed.

No email, no RSS feed, no Tweet, nada.

So you don’t sign in and respond and you don’t slowly fall away from being a Wave user.

The final thing which they got wrong is that there are no easy to follow pointers to get your started, clues for intentional uses, a quick “first action”.

When people first use a technology they are in a state of mild stress, in short they don’t want to publicly demonstrate that they don’t know what they’re doing and therefore look stupid. So they want something easy to do, figuring out the complex stuff afterwards. This is as true of techie geeks as it is of our non-techie friends.

Google Wave doesn’t supply these pointers.

What’s important about what I’ve said here is that all of this is entirely fixable, and there are plenty more people who desperately want to try out Google Wave. With any luck the feedback from the 2nd and 3rd round of adopters will be a lot more positive!

* I totally acknowledge that there maybe good technical scalability reasons for this, but it doesn’t change the affect.

How to test your system with real users

Thursday, February 14th, 2008

Aroxo recently turned a major corner in its development, we moved from closed functional testing to using real live alpha testers, people who’d never seen Aroxo before.

Without doubt, this is one of the most revealing, painful, and valuable stages in creating your start-up, so I thought I’d blog about it.

When launching anything you want ensure that not only does it work, but people find it easy and natural to use. Your own functional testing should cover the first objective, your alpha testing should cover the UI.

At Aroxo we’ve employed four different techniques to get the system ready for launch:

Test type

Description

How many people

Over-the-shoulder The main sticking points in the system. Where the system confuses users.
Start when functional testing at 80% readiness. Earlier with mock-ups also possible.
10-15
Task-driven testing How well the system stands up on its own.
Start when the major usability holes uncovered in OTS  testing have been fixed.
Start with 20-30 keep growing invites to 100 or so
Goal-driven testing End-to-end flaws across the system.
Start when functional testing at 95% system readiness with a slick UI.
150-200
Beta testing The marketing points for the site, highlights future developments. If there are enough users it may also reveal performance issues
Start when the system is 99% ready.
250+ including members of the public

Not only is each stage different but you get different learnings from it. I discuss each stage below.

(more…)

Get Adobe Flash playerPlugin by wpburn.com wordpress themes