How search engine spam created Web 2.0 and drove the social revolution
Tuesday, February 19th, 2008A friend of mine mentioned something to me recently. He commented that a site he works with recently changed the text on their registration button from “Register here” to “Register for free” and, as a result, registrations shot up.
That really blew me away.
Clearly pretty small decisions can have an enormous and disproportionate impact.
This got me thinking and something else occurred to me, namely how a similarly small decision caused by search engine spam created the Web 2.0 phenomenon.
Back in January 1996, Page and Brin were busy building a new search engine. One that we now know as Google. Their starting point for this adventure was fixing the problem with the other search engines: that their search results were not very accurate.
There were two problems, firstly the search engines were happy to take payment from companies who wanted to appear in their search results and secondly the order of the search results was both arbitrary and easy for spammers to fool.
Companies which wanted to appear in the search results would just stock up their pages with the search terms they were interested in and, voila, in came the traffic. The problem was that the people using the search engine didn’t find what they were looking for. The existing search engines basically weren’t working.
This is where Google came in, they calculated the search engine positions by looking at how many other sites linked to the page, the text which the user clicked on, and how popular the referring page was. They built Google like this because (at the time) this had not been spammed.
This technique worked, web users were suddenly able to access the pages they were looking for and Google became massively popular. What also then happened is fascinating.
This decision encouraged genuine content producers to generate inbound links in order to get access to Google’s traffic. It drove more and more linking on the web, and instead of people getting annoyed about losing control of their content, they loved it. It drove greater sharing,
communication, and integration – the bedrock of Web 2.0.
This, small architectural decision has had a massive impact on the web, both its utility and its social impact.
I wonder whether Page and Brin considered this when designing Google?
