In the beginning there was the web and to find things on the web you need a

search tool Webmasters built their sites based on what they thought people

wanted to see with a minimal of SEO.

Search was pretty easy at the start but as the internet evolves and grows into a

viable commercial avenue for business as well as a source of the worlds

information search gets harder to manage and the stakes get much much higher. As

the commercial potential of the web is now being harnessed,  money is a

primary motivation for an online presence for so many companies this has fuelled

and driven search technology into uncharted territories in terms of indexing and

storing the worlds information.

Be it commercial motivation or the information freely available over the

internet itself  the battle over the top places in the "search engines" had

begun in a very big way. SEO became

a marketable skill and somewhere in the battle for the top spots the people were

quickly forgotten and sites were and still are being built to suit the search

engines - machines. As a result of this a whole new set of problems has evolved

with the ongoing battle against "search engine spam" spiraling out of control as

millions of webmasters try and second guess every measure google takes to curb

the over optimization of websites. The message is quite clear that trying to

cheat the new search technology will get your site banned or panelized - maybe.

With the advent of super sites, massive database driven auto generated sites

designed to capture as much free search traffic as possible things have only

heated up further, Google creates a revenue model - Adwords - and allows

webmasters to sell the adwords product in the form of adsense via their

websites. This in turn  motivates webmasters to get as much free traffic as

possible and as Google is the single best source of free web traffic these same

webmasters are building more SE spam! So in effect Google is now funding part of

it's own problem. We now see quite a volatile search index where google is

concerned, they are constantly updating their technology to counter the spam

problem but as they change things with their algorithms, storage and crawl

technology it would seem to many legitimate webmasters  things just become

more unstable and less predictable. Google makes a change and whole industries

spring up - Link building as an example is an industry based on getting inbound

links to websites to help your SE rankings in particular. This applies more to

Google because it's key in their search algorithm.

Moving forward search relevance is still and will always be the most

important factor in the survival of any search engine including Google. Their

ability to overcome storage and crawling issues on such a large scale may be a

telling factor, recent volatility in the Google serps indicates things are not

perfect,  they may be a giant but they also face a giant challenge.