A Single Search Index Would Speed Up The Entire Internet - A Zero Carbon Speed Boost
By Tom Foremski - November 24, 2009
Yesterday I argued that a single search index administered by a non-profit could address issues around de-listing and indexing out-of-print books.
Google's founders supported the idea when they were at Stanford university.
A single index would allow Google and others to apply their analysis and their algorithms to the same data set creating a level playing field.
A single index would also speed up the entire Internet. For example, looking at my server logs for SVW, it shows 16 robots/spiders visiting the site. In total they are responsible for 37% of the hits and 45% of the bandwidth.
These spiderbots are creating their own index of the Internet and they are taking up nearly one-half of my bandwidth and slowing down my server. SVW has fresh content everyday so it gets more attention from spiderbots, but this is happening to tens of millions of other web sites every day.
Spiderbots are a huge drain on Internet resources
If there were a single search index that could be held in the public domain, administered by a non-profit, it would cut down on the spiderbots and speed up the entire Internet -- all without having to install any new routers, servers, or lay new fiber optic lines!
Google would still be Google as would Microsoft's Bing because their value lies in their analysis and ranking of the search index. But it could also spur innovation because startups wouldn't need to spider their own index. We could see all sorts of innovation in algorithms and applications.
Larry Page and Sergey Brin were once strong supporters of the idea that the search index should be run by a non-profit.
On page 39 "Inside Larry and Sergey's Brain" by Richard Brandt (referral link).
Brian Lent, now CEO at Medio Systems:
In one fell swoop Google can:
- Overcome many of its problems with indexing content
- Dramatically speed up the entire Internet at almost no-cost
- Plus the Internet gets a zero carbon speed boost
GOOG's founders have a chance to live up to their ideals, fulfill their mission to index all the world's information, and keep all their money.
It's win-win all around.
- - -
Please see: GOOG Founders Originally Believed Search Should Be Non-Profit - Why Not Do It Now?
« Adify: Food CPMs Jump In Q3 And Brand Advertising Is Recovering | Main | Guest Post: Social Media Marketing is Swiss Cheese »
Posted to A Top Story
November 24, 2009 | Permalink | Comment | Subscribe to SVW
- Top Stories:
- Rob Glaser: Pioneer Of Badware Leaves Real Networks
- New Info: Chinese Hackers Targeted GOOG's Internal Spy System
- Analysis: GOOG China Pull Out Might Be An Extension Of Its 'Borderless' Korean Policy
- Google Finds Its Backbone - Won't Censor China Search, May Close China Operations
- EAST MEETS WEST - 5 Observations on Silicon Valley from an MIT Sloan Perspective
- Additional Thoughts On The Killer Pitch . . .
- The Killer Pitch? - When PR Agencies Can Do This - Look Out . . .
- CES Diary: Media Woes... When Foresight Is No Better Than Hindsight
- Analysis: Here's What's Next After Nexus - GOOG's Cunning Mobile Strategy...
- Arrington Defends Zynga Scam Story From Top New York VC
- Gawker Media Focuses On Uniques As Key Metric
- Analysis: GOOG Needs To Have Its Own Telco Service More Than It Needs A Phone...
Comments (1)
I like this idea more and more. How do we get this idea some traction?
Posted: November 24, 2009 7:16 PM