Without doubt, this is one of the most revealing, painful, and valuable stages in creating your start-up, so I thought I’d blog about it.

When launching anything you want ensure that not only does it work, but people find it easy and natural to use. Your own functional testing should cover the first objective, your alpha testing should cover the UI.

At Aroxo we’ve employed four different techniques to get the system ready for launch:

Not only is each stage different but you get different learnings from it. I discuss each stage below.

Over-the-shoulder tests

This test phase is the most fascinating for the founders, as you get to watch people actually interacting with your system. This test should be done independently, as those close to the system are inclined to guide the user towards positive responses, rather than getting genuine feedback.

But if you are bootstrapping or operating on a tight budget, you may well be doing it. So, here’s what you do:

1. Write out a list of scenarios or tasks which you are going to as people to do
2. Find 5-15 people who represent your expected customer base
3. Sit them in front of a machine and get going

Your first task should be to get them to look at the homepage for a few seconds (no more than 30 seconds), then ask them what they think the site does. People should find it easy to understand the site from the homepage. If not, probe a little to find out what you can change to make it clearer. Then get started on the scenarios.

Your job is to generally keep quiet and take notes as they use the system. You are particularly looking for times when something confuses them, you may notice the person lean forward and start to look confused. At these points feel free to interrupt and probe, ask them what they are looking for and what’s missing.

When probing, be very careful to use words like “what” (open) rather than “why” (implies fault). Try not to reveal any emotion, you’re not asking people to tell you how great your system is, you’re after honest feedback. Therefore it is also important not to make people think they are stupid for not being able to use the system, frame it as “we want to get some open and honest feedback”.

Expect some excellent and easy improvements to come out of this to really make the system substantially easier to use.

Task-driven testing

Task-driven is quite different. For a start, you won’t actually be watching people, they’ll be running through tasks on their own. This is particularly interesting because for the first time people will be using the system without anyone to ask for help. This means you can get more people involved.

To perform this sort of testing there are a few things you need:

1. around 20 particpants to start with (expect about half to actually use it and 2-3 who give you really good feedback)
2. a well defined list of tasks for them to perform
3. a way to communicate the tasks to them and
4. a way for them to communicate their findings back to you.

As you push these changes through to the alpha platform you should start adding more and more people onto the trial. You want fresh eyes to test your UI improvements, people who’ve already seen it, won’t test it as well, if at all.

Your objective here is the test how well the system performs with a small number of people performing defined tasks, so each task you ask people to do should come with some prompts to help them give you relevant feedback.

Goal-driven testing

Goal-driven testing differs from task-driven testing in two ways. Firstly there are a lot more people involved in the trial, and secondly instead of having tasks which people have to perform on the system (e.g Register on the site). You set wide goals for the participants. In context of Aroxo this is something like “buy something” or “sell something” the UI and the website itself should then guide the users through the system, rather than the tasks.

Start your goal-driven testing when you’ve reached the point that you believe the UI of the system is strong enough that people know what they should be doing and how to use the system.

Beta testing

Beta testing isn’t really testing in the same way as the alpha testing which you’ve already performed is. You want a large number of people get see the system and many of them will have registered on your site to do it, so you don’t know them.

This makes beta testing public. If it is public, it’s part of your marketing. Therefore it is essential that it shows your site running well and easy to use. What you may get is some useful learnings about how the system scales, or you may also be able to learn something from some clickstream work.

However, what is most important about beta testing is that it should not be part of your functional and UI testing, it is too late to make any significant changes to the system as you’ll be moving towards a full launch within weeks of starting your beta.

Closing thoughts

One thing which I’ve not discussed in this article is the emotional impact which testing has on the founders. If you’ve been careful with your IP whilst you’ve been building your system it’ll be the first time which you’ve had large numbers of people looking at your system. This means you are going to get a lot of feedback.

It is important not to take this feedback personally. It should help make the system stronger.

You need to remember that you are asking people to look at your system and tell you the flaws and problems with it. You are not asking them to look at it and tell you how clever it is! This means you are going to get a lot of (hopefully) constructive criticism. You need to take this on the nose and think of creative ways to improve the system.

Finally, in the case of Aroxo something which we learned is that it just isn’t possible to test the UI of your own stuff: you are way too close to it. So it is important to have lots of fresh eyes to test each new release.

Good luck!

Over the next 5-6 posts I’ll outline the process which I’ve now followed at several corporates and which I’ve honed to work with my own start-up, Aroxo. I’ll discuss what skills you’ll need, how to write your requirements, how to source developers and designers, how much to budget, how to agree a development contract, how to manage your vendors, how plan your release, all the documentation which you need and much more.

The Aroxo plug

I’ll use Aroxo as my primary reference for how to run the process so it is useful if you know a little about it.

Aroxo is a person-to-person trading exchange for consumer electronics, computer gear, whitegoods, and more. It uses new way to match buyers and sellers that has several benefits over auctions and fixed price sales models. Interestingly it isn’t constrained by the critical mass of other marketplaces – you cannot compete with an existing auction with another auction, no matter what your marketing budget, hence we’ve got a different model which is currently under-wraps.

Marketplaces are inherently complicated systems to build, with lots of workflow which needs building and testing. Especially in our case where we’ve built something which hasn’t been done in the consumer space before. If you’d like to know more may I suggest you sign-up for our beta trial using the sign-up code “bootstrap“.

At 400,00 lines of code, we’ve been up a steep learning curve whilst we’ve bootstrapped Aroxo and we’re passing on the benefits of our learnings here.

What is bootstrapping?

So, what does it mean to bootstrap a company? Bootstrapping involves launching a business on a low budget. Practically this means that you’ll out-source (most likely off-shore) your design and development, you’ll rent your servers, you won’t have an office and you’ll have no salary. Prior to launch, the only expensive professional services which you’ll buy will be your legal advice and accountancy services. Everything else, you’ll have to pick up yourself and learn as you go along.

Why bootstrap? There are a couple of good reasons a company should consider bootstrapping its market entry. The founders may believe that they are onto such a good idea that they don’t want to give up any equity. Or the founders have taken on a small amount of seed financing, just enough to get them into the market. Either way, bootstrapping is a viable model.

There one very bad reason to bootstrap and a major pitfall to be avoided. If no-one would give you any money to build your business, but you’re still convinced you’ve got a great idea then for the sake of my conscience, please think long and hard before spending any money or giving up a source of income. The information I’ll impart here will give you some clear instructions for launching a business, make sure it’s a good way to spend your money before you do.

OK, intros over. On with how to do it.

Overview of the bootstrapping process

I’m starting this series of weekly posts with an overview of the whole process. In the upcoming posts I’ll go into much more detail about each one.

Bootstrapping is a potentially very exciting prospect to an entrepreneur. It enables you get keep a much greater slice of the equity, getting you in to the market so you can raise financing (if you’re going to need it) with a trading history in hand, rather than some PowerPoint. But as I said before, it is fraught with risks and the primary risk which you need to guard against is the failure of the developer to deliver. Two thirds of out-sourced projects fail, whilst this varies from developer to developer, there is a high likelihood of failure – especially when you consider the general start-up failure rate.

Much of the process which I’ll outline is designed to prevent this from happening, so it is worth understanding the main causes:

1. Client fails to precisely express their requirements to the vendor
2. Client continuously changes their mind during the development process
3. A poorly constructed development contract

I could go into detail here with some warning signs and observations about these, but if you faithfully follow the process which I’ll describe here, then you’ve done much to protect yourself.

With this in mind, the bootstrapping process is as follows:

1. Strategy – build your idea into a documented strategy, write down the market you want to serve, what is wrong with how it is currently being served and how you are going to fix it, also think pointedly about how you are going to get the word out to your potential users, where they are online and how you’ll approach them. Test that your idea makes sense
2. Mock-up – build a mock-up of the system you want, each screen in the mock-up must contain the information and navigation you want the final system to contain, you’ll need several goes to get this right, it is not simple
3. Functional Specification – write a document called a functional specification, this explains to the developer what each screen in your mock-up does and what every button and link on the mock-up should do
4. Vendor long-list – build a “long-list” of possible software vendors and NDA them all, but don’t rely on it (more on this later)
5. Request of Information – Write and run an RFI process to sift your long-list of vendors down to a short-list of between 4 – 10 vendors
6. Request for Quotation – Write and run an RFQ process to determine approach, costs, timescales and conditions under which each vendor would deliver, select a preferred vendor and a spare in case of any problems during contract negotiations
7. Award contract – Negotiate a software development contract. There are some essentials to ensure that you’ve got covered, which I’ll go through
8. First trip – if you’ve off-shored then you’ll need to go over there and see the developers to get them started, take them through the entire mock-up and all the documentation, to ensure that they know how the system hangs together. Building a real face-to-face relationship with your developers is not only immensely valuable for you, but also for them too
9. Managing the development – running the development as a client has many responsibilities, you’ll need to know how to deal with delays, systems problems and you’ll need to know how to think like a developer so you can communicate with them. You also need to know when you’ll need to be out there. During this phase you’ll need to be there for the developers to answer all their detailed queries which they’ll have as they start to architect and design the system
10. Alpha, Beta, launch - Aroxo is currently in the process of recruiting beta testers (feel free to sign up using the sign-up code “readwrite”). Getting a large number of triallists and acting on their feedback is essential to building a strong business and having a successful launch, I’ll go through the best way to achieve this

Clearly, whilst I’ve presented this list linearly, it is entirely possible to parallel run some components, e.g. you can start building up a vendor long-list whilst you are doing the mock-up and you can easily be running your RFI and RFQ process when writing your functional specification. But each stage needs to be completed if you are to reasonably expect a quality product by the time the process delivers.

Running through a process such as this is what it means to be an “entrepreneur”, so I’ll be going through each of these stages in plenty of detail in the coming posts.

How much does might this cost?

It is often said that bootstrapping is cheap, and whilst it is certainly cheaper, it is not cheap. Producing a professional and slick product costs money. Our development was a particularly complicated build requiring 5 developers, a systems architect, an html coder, a SQL specialist, 4 testers, 2 designers, a graphic designer, a project manager and a security specialist. If our aim was to raise financing and employ these people directly we’d have burned through $5-7 million before launch. So yes, it is considerably cheaper, but it is not cheap.

With that in mind a rough budget which should be sufficient to bring a system of reasonable complexity to market would be (note that this doesn’t consider any marketing or PR spend):

Project complexity is the main driver of cost, delivery risk and timescale, therefore it is exceptionally hard to be accurate with this, but you are in for a minimum spend of $50k, systems as complex as a trading exchange will involve much bigger numbers. If you have less than this, then you’ll have to do the development yourself, nothing else is going to shift the needle. By doing the development yourself you’re competing with an off-shore company which has specialists in all the skill areas I mentioned above, and there’s a practical limit to the size of the project you can accomplish.

When you consider the question of whether to off-shore your development, if you keep things local, you’ll save on your travel budget, but you development cost will be anything from 5-10 times higher than off-shoring to India or Eastern Europe.

Similarly you’ll need at least one fulltime founder during the development phase to handle the developers and manage any other business development issues which arise during set-up. You’ll notice that there is no budget above for any salary for this individual, so the real cost much higher if you consider lost earnings during the build.

This should give you a good overview of the boot-strapping process, in my next post I’ll outline how to document your requirements in a “functional specification”, how to build a mock-up of the service using free development tools and how to structure your strategy. And I’ll be giving away plenty of examples.

Key takeaways

• If you are out-sourcing design and development, then you’ll want a budget of around USD 50k – 100k, more if you’re developing a complex solution. Anything less and you’ll need to develop the system yourself
• You are not going to be able to manage professional developers part-time
• You absolutely must thoroughly think through how to implement your idea
• Be 100% sure that you have the drive and finances in place to see it through to delivery

In this post I’m deliberately looking at the impact which Web 2.0 has on users which makes it so appealing to a certain segment of the population. This information will help entrepreneurs know what to build into their propositions in order to take advantage of the opportunity today to steal a march on older style Web applications.

To bring the post to life I’ll be demonstrating how the principles I’ve uncovered can be applied to a classic Internet application – corporate email. I’ve arbitrarily chosen email as it is a example of an application which has not substantially changed in the last decade, by applying some of the Web 2.0 principles outlined below we’ll see how its feature set can expanded to make it more inclusive, fun and sticky.

My research into Web 2.0 has lead me to take a particular interest in how people are affected by “Web 2.0″, in short, rather than worrying about what it is, or how it is defined, I’ve tried to look at what impact it has on users. I believe that Web 2.0 affects users in 3 ways:

1. Personal expression – It gives users a greater ability to express their personalities to others
2. Efficient connections – It makes meeting new like-minded people more efficient
3. Information discovery – It changes how people discover information

None of these dynamics are new or unique to Web 2.0 – they represent some of the core motivators of most humans – all Web 2.0 has done is make achieving these human objectives online easier.

This is much more than “AJAX and RSS”, and other technologies frequently associated with Web 2.0.

1. Personal expression

Web 2.0 gives people many more option to express themselves. We now have increasingly widespread adoption of capabilities such as blogs, wikis, Q&As (as well as technologies which have been around for much longer, including personal webpages, forums, etc). These enable people to communicate and craft their personalities more effectively. Practically, Web 2.0 style applications frequently make use of:

• Personal icons – From Windows Live Messenger, to the mybloglog link on this webpage, Web 2.0 expression often start with including an image to represent yourself
• Profiles – People now commonly include detailed information about themselves in the form of profiles, publicly available information about their interests and habits. Personal websites such as myspace and facebook extend these “profiles” to the point at which they are one of the core reasons for using the site
• Snippets – It is more acceptable than ever to express only a snippet of information. To add something small to a conversation or stream of thoughts, blogs, Q&As, forums, etc are all ways to achieve this
• Comfort – People are becoming more comfortable at expressing themselves online, the reticence people used to feel has been blown away, with people becoming increasingly happy to share very personal information with the community

All these changes give lessons to entrepreneurs. New propositions under consideration need to ensure that they give users ample opportunity to express their personalities, whether that be through personal icons, chat rooms, their own web-pages, etc.

Applying these effects to corporate email would add in:

1. Simple personal icons which people select and appear in every email sent
2. Give people control over their corporate directory entry, enabling them to include information about what they are an expert it, enabling others to leave comments for them against their directory entry

2. Efficient connections

This is one of the most fascinating areas of development for Web 2.0. We are now forming more relationships with more people at an increasingly faster rate. People actually seem to compete to build networks of friends (e.g. Facebook, Myspace, Bebo, LinkedIn and the rather exciting VisiblePath).

These services demonstrate the importance of enabling people to form connections between each other. Whilst this isn’t particularly new in the context of the web – discussion forums have been popular for years, there is one important change. These personal networks are being structured. Traditionally online forums where people used to meet were effectively like stepping into an enormous room of people and shouting your question out – Web 2.0 now enables people to build networks and therefore filter that communication down to people they know and trust:

Web 2.0 enables users to structure their network – rather than posting to a discussion forum, users can find information out from within their personal network. This does two things: it effectively filters the amount of information they receive, but because their network contains people like themselves, it filters the information leaving data which should only be relevant to them. For entrepreneurs it means we should design our propositions such that:

• Users can find other users on the basis of shared characteristics (be that location, interests, sex, goals, etc)
• Enable users can create structured networks

In applying this to corporate email, we’d make the following changes:

• The mail server would capture the structured network data, enable people to see who they email, and who they email the most (note this is effectively what VisiblePath does).
• Users could then search their networks for subject matter experts within their organisation – breaking down the barriers formed when working in large corporates.

3. Information discovery

Information discovery is an absolutely key area of development for Web 2.0. Traditionally information discovery on the Internet was performed through search, now Web 2.0 has augmented this with:

• The personal network
• Tagging of data
• Social bookmarking
• Trackbacks and cross-linking

Of these, the first – information discovery through your personal network – is a very underexploited area for entrepreneurs. Discovering information through your personal network is like applying a highly effective filter to the massive pool of information on the Internet. Amazon’s “customers which bought this, also bought this” is a prime example of this working. People who we choose to have in our personal networks are by definition like us in some way, and therefore their preferences will be similar – but different enough to be interesting – to act as a powerful filtering system.

In effect, this dynamic enables entrepreneurs to create new propositions which start to release the information which people currently hold in their heads. A service such as WAYN performs this function for travellers.

It is interesting to note that classic social networking sites really only pull together the first two Web 2.0 dynamics. Services such as Facebook, LinkedIn and MySpace do not do much to capture the information which a user has in his or her head and share it with others.

Applying this principle to corporate email gives us:

• Enable mail users to selectively make corporate emails indexable by the server. The mail client would then enable users to search for, say, a prospect’s name and get back a list of other employees in the company who’ve already contacted them.

To summarise

Entrepreneurs wishing find new ways of approaching old propositions can shake up and old school “web 1.0″ businesses easily. To achieve this they need to:

1. enable users to express their personalities through profiles
2. enable users to build structured relationships with others
3. enable users to discover information through the network structured by the system

Taking an existing business and applying these principles gives a unique Web 2.0 proposition which should significantly help drive traction, stickiness, usage and therefore value.

I confess that when we were starting Aroxo I was working fulltime and so I only had evenings and weekends to spend on the business, but this does not explain the increased efficiency. This extra speed comes directly from the experience of launching Aroxo, quite simply for the new business I followed a tried and tested process which I’d already put through its paces for Aroxo (and other projects).

The process which I am about to describe is directly targetted at launching a business which is “bootstrapped” (i.e. self-funded) and where as much of the work creating the business is out-sourced. The process rough is:

Over the next series of posts I will detail precisely what we did at each stage, focusing in much more detail on what information was needed to get each stage started and what to look for from the output. I’ll provide examples for each of the stages and make it clear where it is possible to skip certain stages or where the process might differ for different project structures.

I’ve no doubt that others adopt different processes and I’m happy to carry comments to capture these differences and would appreciate everyone’s feedback.

Having been through a vendor selection process which encompassed 32 different companies, some off-shore and some on-shore I’ve seen a massive variation in quality, costs and skills. I found the principle of caveat emptor very relevant.

My key learnings were firstly that off-shoring is not cheap, it is cheaper, but it isn’t cheap. Secondly there is a significant variation in quality which a properly run RFI and RFQ process can help select out. Many of the companies we included in our process I would use again, several I would not.

In my experience, there are two pitfalls to avoid when out-sourcing, both of which a professional client can avoid. Firstly the client must prepare a clear specification of what they want – in order to communicate precisely what needs to be developed across timezones and continents the client needs to know exactly what they want. My understanding from talking to off-shore developers is that they frequently receive little information about what needs to be developed.

Secondly, out-sourcing still requires time on the ground with your development team, all the technology in the world won’t replace the accuracy of face to face communication. In addition careful and close management is necessary – speaking to your developer on a daily basis helps to prevent costly errors.

I am also of the view that, assuming the development company has the basic technical skillset to deliver your project then any failure to deliver will be as much in the hands of the client as the developer.

In the end my company we went with a development partner based in India and so far we’ve been very impressed with their professionalism and skills.

See my LinkedIn profile to link to the full range of answers: http://www.linkedin.com/in/mrogers

Yesterday we reached a major milestone and I wanted to share the news with you. We have signed a development contract with an off-shore development company and we are leaving for India tonight to begin the development process. The functional specification is at the printers.

I am personally really excited to be starting the development process and can’t wait to be able to show you more of what we’ve been working on. Our next steps when we’re back we will be to start the patent drafting process, select our site designers and begin work on our detailed marketing plan.

We can’t reveal too much yet as the patent hasn’t been filed, but to give you a little more detail – we’re launching a new online trading exchange, behind it will be an entirely new way of buying and selling – something which has not been done before in the consumer space. You’ll be able to use Aroxo to buy and sell a range of tens of thousands consumer products at extremely competitive prices.

There’s much more to come, watch this space…