The above comment is from Aaron Wall's excellent SEOBook.com -- my go-to source for understanding search engine optimization and the algorithmic games that Google [$GOOG] plays with hundreds of thousands of businesses the world over.
Long time Google [$GOOG] watcher Scott Cleland has floated an interesting strategy that Microsoft [$MSFT] could adopt in the wake of its failed attempt to have the FTC pursue anti-trust prosecution of the search giant: exit the search business.
Microsoft has never made a profit from its search business, which loses about $2 billion a year. If it abandoned search, Mr Cleland says that the action could harm Google in several ways:
- It would show that no one in search can make money because of Google's massive presence globally.
- It would take away Google's claim that Microsoft competition is just one click away.
- It would allow Microsoft to file a private antitrust lawsuit that could bring triple penalties if it won.
Unlike the FTC, the experts at the DOJ and EU have long understood the stark reality of the search advertising business because they approved the highly-unusual combination of #2 Yahoo and #3 Microsoft search advertising competitors in a three competitor market in a last ditch effort to salvage a competitive search market.
This is shocking and it's a direct result of Google's algorithm changes. Here's a case study of a web site being prepared for launch and how a competitor managed to get it flagged as a scam site -- before it was even launched.
From SEOBook: "A case study in being PRE negatively seo'ed."
SearchEngineLand has a very good, long look at the recent fake press release announcing Google's $400m acquisition of WiFi company ICOA.
PRWeb distributed the press release and said it slipped through its internal tests for "integrity."
Danny Sullivan explains how PRWeb has become a popular distribution network for a lot of content, some of it shady, and how it ends up on well respected newspaper sites.
My recent post about Google's battle with spammers seems to have confused some people in the Search Engine Optimization industry.
There's a perception that I'm advising web sites not to improve the quality of their content. I'm not advising anything, I'm reporting that a little known Google patent appears to be playing a major role in how a post-Panda Google now ranks web sites and web pages. It also explains some of the bizarre, random changes in rankings that have bedeviled SEO efforts by webmasters.
If Google detects that a web page has been changed between visits from its spider, it will check to see if the changes are designed to improve the rank of that page in its search index. This will flag the site as a potential spammer and trigger a reassessment of the site's rank in Google's index. In between the reassessment the site's rankings will fluctuate randomly.
This creates the situation where if a site owner tries to improve the quality of a page, by rewriting passages to make them clearer, adding additional information, links, video, etc, this could result in a spammer flag from Google, and a period of randomly fluctuating index ranking. Thus, trying to improve the quality of your site could sink your rank.
Yet Google is constantly telling web sites to improve the quality of their content to gain a better ranking.
That's a seriously messed up situation.
I've always advised people not to worry about search engine optimization (SEO) with the explanation that it's the job of the search engine to optimize its performance -- not yours.
Over on SEOBook, there's a great article pointing out how Google is now measuring any attempt at raising the rank of a web page as the work of a spammer -- no matter the quality of the content -- and it will penalize the site.
Any attempt to modify the rank of a web page, after it's been ranked, could spell disaster for the site owner.
Sergey Brin, co-founder of Google, wrote that his comments about Internet freedom made to The Guardian Newspaper needed some clarification.
He begins by re-stating his original premise: "I believe the internet has been one of the greatest forces for good in the world over the past quarter century."
He then goes on to say:
Foremski's Take: The problem with the Internet is that the same methods that a company such as Google uses to monitor its users for clues about what they might purchase next, so it can show relevant ads, can just as easily be used by governments to monitor its citizens for political and oppressive purposes.
"Big Brother" is already here, it's just masquerading as "Big Sales Assistant."
Google made 79 acquisitions last year, mostly small companies and mostly for their engineering talent. The largest acquisition was Motorola Mobility, a $12.5 billion deal.
This year, the search giant expects to focus on smaller numbers of deals in strategic areas such as mobile and video, said David Lawee, VP of Corporate Development.
I always pay attention to Aaron Wall, who runs SEOBook, because he is always on the money. He is one of the very few Google Watchers that is consistently insightful and isn't afraid of writing hard hitting articles critical of Google -- if it's called for.
Here is one of his latest infographics, and again, he is spot on about the changing trends in how Google views the web. It's shocking how much valuable advice he gives away:
Which industries and companies were Google's top spenders in 2011? Did you know that "Self Employed Health Insurance" cost $43 per click? Here's a fascinating infographic that breaks down the $38 billion that Google made in 2011:
Larry Page, CEO of Google, said at the analyst call on Thursday that G+ has 90 million users and that 60% are active daily.
Last week Google introduced search results heavily influenced by relevant signals from people's social circles and promoting G+ content in its search results.
The move is an attempt to boost its nascent G+ social network, which has had a rocky start.
This looks like a way to drive traffic to G+, which has lost some traction.
Google is trying to push pout affiliates in key verticals. After all, why let affiliates make money on Google traffic?
SEOBook has produced an interesting infographic explaining Google's affiliate strategy.
Here's an excellent infographic from SEOBook explaining the world of search engine optimization, a somewhat "dark" art...
Aaron Wall over at SEOBook, has put together an infographic explaining how Google has made the organic link irrelevant.
Google has been focusing search results on brand names as a quick and easy way of trying to distinguish between "quality" content and not. The idea behind this is that brands are owned by large organizations who produce the best, or at least the most reliable content around that brand keyword.
It also allows Google to essentially become the affiliate marketer for large brands because that's where they get most of their traffic. This is at the expense of smaller companies trying to make money online and who don't own major brands.
New services, such as Google Instant, make it easier to focus user searches on brands rather than the massive long tail of keywords that small marketers use to attract traffic. Google Instant can direct searchers to keyword terms they weren't going to type in but that Google knows it can make money from.
Aaron Wall, at SEOBook has produced an infographic that explains how Google has managed to chop the long tail off and concentrate on the thick, juicy, short tail of keywords, which is far more lucrative than what it can make from long tail keywords. (Embed code.)
Google has been boosting large brands in its search rankings as part of a deliberate strategy to take business away from thousands of third-party affiliates -- small businesses that make money selling larger brands.
This strategy has managed to generate billions of dollars in extra revenue for Google this year. It is a war against small businesses, a major source of jobs in the US. And it's largely a secret war with very few people following, or able to understand what is happening. Google doesn't want attention on this strategy because in today's tough economy and high unemployment, Google is destroying jobs at countless small companies -- a PR nightmare.
Google has adopted an aggressive program of stuffing its search pages with paid listings as the financial quarter draws to a close — a risky strategy as the US Senate holds hearings on its industry dominance.
Google's search pages are being flooded with paid search listings from its AdWords advertising network in the final weeks of the Q3 financial quarter, possibly to boost revenues to meet, or exceed Wall Street analyst estimates.
US internet users spend more time on Facebook than any other website. That's according to the latest findings by media research house Nielsen.
By Stuart Thomas
Nielsen's "State of the Media" report for the third quarter of 2011 tries to capture a snapshot of the social media landscape in the US.
Among the key findings is the fact that social networks and blogs account for nearly a quarter of the time that Americans spend online.
I've been writing about Google's Panda algorithm change and the huge amount of pain that it's caused for content websites since it was introduced in late February.
Before the launch of Panda Google spent months bad-mouthing "content farms" and how the new algorithm would weed out the bad content. The first version was called "Farmer" as a reference to "content farms" -- sites that scrape or produce low quality content just to game search rankings.
Who could argue with such a noble goal?
By Graeme Lipschitz
The meteoric launch of Google+ (20-million users in less than 3 weeks) can be seen as the proverbial phoenix rising from the ashes of such failures as Buzz and Wave - the word 'Circles' immediately comes to mind here.
Steven Levy, a senior editor at Wired, spent a lot of time at Google researching his new book, "In the Plex." And he spent a lot of time with the new CEO Larry Page.
Mr Levy says Google is now in the hands of "a true corporate radical."
In an article for Wired, "Larry Page Wants to Return Google to Its Startup Roots" he reports that Mr Page has said many times, that he has always "wanted to change the world."
Google [$GOOG] reported stellar 2011 Q2 earnings but Google's partner sites, which used to account for half of its revenues, showed a massive lag in growth.
The partner sites are part of Google's AdSense network and include large media companies such as The New York Times.
By Nur Bremmen, MemeBurn
It's not easy to run an online content business in an age where everyone is a publisher. Everyone is doing it and there's never been competition in the content business like this before.
If we cut through the hyperbole and the evangelism, we're left with the simple fact that online advertising, while promising, is not making the same bucks as its traditional media counterparts.
The model is still evolving and can be confusing at times. So what is the future of content and online advertising?
Memeburn got the answers from Google's content Tzar, Peter Barron. He spoke about paywalls, Google News, filtering news by social recommendations, and the future of online advertising.
Douwe Osinga, a software engineer, recently left Google after seven years. He's written a series of blog posts explaining why he left and also describing what it was like working there, and he dispels some of the many myths about Google.
For example: The 20% time myth.
I've been helping Murray Newlands with his San Francisco Blog Club meetups, which are great for helping to educate a mostly newbie group about blogging.
Last night the featured speaker was Chang Kim, product manager at Google's Blogger.com. Trouble is he was a no show. He didn't call to say he was running late. Nothing.
Google regularly changes its algorithm and it's a smart move because all the companies that were trying to game Google -- and were succeeding in taking advantage of the some 200 rules that make up the algorithm -- get shaken out of the results.
It's a quick way of finding the most egregious gamers of the system.
And Google's recent release of its "Panda" algorithm update was designed to find quality sites and raise them in the search rankings.
But Panda is causing a lot of pandaemonium for all types of businesses, those that played by Google's "white hat" SEO rules, and those that didn't.
Google's Q1 2011 financial results show a significant shift in revenues away from its partner sites in its AdSense advertising network.
Google [$GOOG] typically relies on partner sites for about 30% of total revenues but over the past year it has managed to reduce that to 28%.
Larry Dignan at ZDNet reports:
With the move, Google would get about 6,000 patents covering wired, wireless and digital communications technologies.
Google's top lawyer, Kent Walker wrote:
Google is a relatively young company, and although we have a growing number of patents, many of our competitors have larger portfolios given their longer histories.
On the eve of Google's launch of an experimental feature that allows users to recommend search results, Blekko, the curated search engine, hosted a lively dinner in San Francisco with journalists to discuss the future of search.
Vivek Wadhwa, a visiting scholar at UC Berkeley (photo) kicked things off by saying that Google's results were polluted and that Google itself was financing the "pollution" through its AdSense advertising network.
While many geeks are heading to SXSW Interactive in Austin, Texas, others are probably looking for a break somewhere else. Bing, the Microsoft search engine, is carving out a specialty in travel and has published some interesting results:
- Flights from Seattle to Salt Lake City are down about 7% from last year, as are flights from Dallas to Jacksonville. Deals can also be found to Cancun, Mexico's spring break hotspot, with fares from Philadelphia down 8% and fares from Atlanta down 16%.
- Lowest average fares: San Francisco to Tampa, $300
- Lower airfares this year: San Francisco to Paris (-9%)
- Highest price increase this year: New York to Vancouver, Canada, $596 (+64%); Philadelphia to San Juan, Puerto Rico, $939 (+54%).
You can save $25 to $125 per ticket by travelling from a Monday to Monday or Tuesday to Tuesday instead of from Saturday to Saturday. It's best to get a head start on travel and schedule your trip for the first week of March rather than waiting until mid-April to escape the grind. Traveling during the first or last week of March as opposed to mid-March can save travelers $25 per ticket. Traveling in early April as opposed to the middle of April will save travelers an average of $30 to $50 per ticket. Use the Bing Travel Flexible Travel graph to help determine the best days to travel.
Google's decision to give prominence to links in search results that come from your social network might help to improve some search results but it will not improve overall search quality.
...if you're thinking about climbing Mt. Kilimanjaro and your colleague Matt has written a blog post about his own experience, then we'll bump up that post with a note and a picture.
The departure of Eric Schmidt as CEO of Google has led to a lot of speculation about changes at the company as Larry Page moves back into that job in April.
But there will be no changes because there is essentially no change in the management of Google.
People forget that Mr Schmidt held the title of CEO in name only. The decision making was shared between the founders: Larry Page and Sergey Brin; and Mr Schmidt in what they termed a "triumvirate."
Here it is explained in extracts from Larry Page's 2004 Founders' IPO Letter:
Eric has the legal responsibilities of the CEO...
We run Google as a triumvirate. Sergey and I have worked closely together for the last eight years, five at Google. Eric, our CEO, joined Google three years ago. The three of us run the company collaboratively with Sergey and me as Presidents.
We hired Eric as a more experienced complement to Sergey and me to help us run the business.
Mr Schmidt always held the minority vote against Messrs Page and Brin. And there is nothing to suggest that they have dissolved their management structure beyond removing Mr Schmidt.
Mr Page gets to hold the CEO title for legal reasons but the power sharing with Mr Brin is still there. Both founders will continue to make the key decisions.
Why would we expect anything to change at Google if there is no new management?
The only thing that's changed is that the founders have regained some of their lost dignity. Mr Schmidt was brought in as CEO to provide the founders with "adult supervision" as he termed it. Google's chief investors believed that the coming IPO would do better if a veteran Silicon Valley executive was seen to be at the helm.
Continuing to have "adult supervision" ten years later when you are nearly 38 years old must be embarrassing for the founders. Especially since wonder kid Mark Zuckerberg has been doing great without having any official adult supervision. He's tripled the value of Facebook in less than a year and he's just 26.
This is the real reason Mr Schmidt was shifted, imho. And it's all plain to see in Mr Schmidt's tweeted announcement:
It's taken Google co-founders Larry Page and Sergey Brin many years to end their "adult supervision" in the form of Google CEO Eric Schmidt, who leaves in April.
Mr Page will become CEO and Mr Schmidt becomes Executive Chairman.
Mr Schmidt Tweeted:
Mr Schmidt was brought in as CEO in August of 2001 because of pressure from Kleiner Perkins, one of Google's chief investors. His goal was to provide "adult supervision" to the young founders as Google was preparing for an IPO.
The VCs believed the IPO would be more successful if investors saw that Google was headed by a veteran Silicon Valley executive. The company created a "triumvirate" where the founders and Mr Schmidt shared responsibility for executive decisions.
Many years later, Google still maintained its "adult supervision" which must have rankled the founders, now nearing 38 years old. The situation must have become especially galling in recent months since Mark Zuckerberg, the 26 year old founder of Facebook is also CEO. He has managed to more than triple the value of Facebook in less than a year--without the need for any adult supervision.
One year ago I asked: Nine years later does Google still need 'adult supervision?' | ZDNet
The Google triumvirate has worked reasonably smoothly but there have been large differences of opinion between the founders and Mr Schmidt, especially over China. Mr Brin was in opposition to Google's entry into China and he advocated an end to its China operations, against Mr Schmidt's position that Google should overlook China's human rights record.
Mr Schmidt has served as a useful "lightning rod" at Google and he has consistently distracted the media from paying much attention to the founders. Sometimes Mr Schmidt's less carefully considered remarks have gotten him into trouble.
His unusual leadership style has attracted a number of critics. For example, Elizabeth Corcoran, senior editor at Forbes, once asked "Who's Really Running Google? - Forbes.com
...as charming as he is, Schmidt runs Google about as much as much as the Dalai Lama runs the world's spiritual life.
...he has defined his job not so much as leading Google but as running interference for it--placating the investment community, soothing nervous regulators and policymakers and doing whatever it takes to create a magical force field protecting Googleteers...
He hasn't had much success at "running interference" with Wall Street and policy makers.
- Google is facing increasing scrutiny from the US administration over possible anti-trust business practices. That’s despite Mr Schmidt’s very public support for President Obama.
- Google has run into big problems internationally with its books scanning project and with European anti-trust authorities.
- Its relationship with Microsoft is very bad, because Google actively opposed Microsoft's acquisition of Yahoo and other deals.
- Mr Schmidt has managed to upset newspaper companies both in the US and internationally. He has failed portray Google as an ally rather than as an adversary.
- Google’s relationship with Apple soured badly, especially after Google introduced its Nexus phone, and then dropped it. Mr Schmidt was forced to resign from Apple’s board.
- Mr Schmidt admitted that he persuaded the Google board to pay $1 billion more than YouTube was worth. Critics said that was to reward VCs who were investors in both Google and YouTube at the cost of GOOG's shareholders. Google is still trying to figure out how make YouTube profitable.
- Google has failed to diversify its business away from search and online advertising despite many acquisitions.
Mr Page will now have to step out of the shadows and assume a much more vocal and public role. That will be a challenge for a shy software engineer who is much happier working on engineering problems than schmoozing Wall Street analysts and building media partnerships with other companies.
Much has been written lately about the lower quality of Google's search results. A poll by Lifehacker found that 77% agreed that Google search results had "become less useful to you lately."
If search is becoming less useful then how will we navigate the web?
The answer is by curation, by using the recommendations of others acting as curators.
Paul Kedrosky writing on Infectious Greed:
... the re-rise of curation is partly about crowd curation -- not one people, but lots of people, whether consciously (lists, etc.) or unconsciously (tweets, etc) -- and partly about hand curation (JetSetter, etc.). We are going to increasingly see nichey services that sell curation as a primary feature, with the primary advantage of being mostly unsullied by content farms, SEO spam, and nonsensical Q&A sites intended to create low-rent versions of Borges' Library of Babylon. The result will be a subset of curated sites that will re-seed a new generation of algorithmic search sites, and the cycle will continue, over and over.
UPDATED: Faith Merino at VatorNews reported Monday that Google has purchased Groupon for $2.5 billion: "
... according to an unnamed insider who spoke with VatorNews. Neither Google nor Groupon could be reached for comment to confirm the report, but Vator's source is reliable and the report falls in line with the recent string of Groupon acquisition rumors.
Tuesday Kara Swisher at All Things D reported that her sources said that Google will pay $5.3 billion plus a $700 million earnout if Groupon meets sales targets.
The New York Times reported that its sources said Google will pay $6 billion for Groupon.
Google made an initial bid of $3 billion to $4 billion, these people said. But in the face of Groupon's resistance, Google raised its offer to $5 billion to $6 billion. The company is unlikely to offer more than that, according to one of the people with knowledge of the situation.
The deal doesn't make sense for Google. Groupon relies on a large number of people to make the sales for Groupon to work.
Google is a business that relies on machines and algorithms -- not on managing large numbers of sales people.
Caroline McCarthy at CNet News makes a similar point:
But she writes that Google needs to move beyond engineers and beef up its direct sales force. She quotes David Ambrose, co-founder of Scoop St, a Groupon clone, that Google might pay more than $2.5 billion because "Google has never really been able to do direct sales well at all."
I don't believe that Google can buy a direct sales force capability. If it does, that effort will fail. Google's culture is engineering based and the sales force will never have the clout of engineering.
Just because it makes sense to have a strong direct sales force doesn't mean that it makes sense for a company to acquire one. Company culture always trumps logic and reason. And company culture is the least agile part of any organization.
And Google's engineering culture is deeply wired. For example, Google has been encouraged by vocal observers to buy a newspaper, such as the New York Times. But again, something like that would never happen because Google doesn't want to manage editors, journalists, foreign news bureaus, etc. It knows how to manage servers and software.
At the bottom of every Google news page you see the following:
"The selection and placement of stories on this page were determined automatically by a computer program."
Placement was not determined by a person but by an algorithm.
Algorithms and machines are a far more scalable and profitable business than a people based business such as a newspaper, or Groupon.
In addition, the higher costs of doing business with a large sales force will lower Google's profitability and that will affect its stock price. In mid-morning trading Tuesday Google [GOOG] was down 4% or $23.45 to $558.66 in reaction to the news.
The latest US search engine rankings from comScore show that only Google and Microsoft grew their market share while Yahoo!, Ask and AOL lost share.
Microsoft's Bing outpaced Google in October, adding 0.3 percentage points to 11.5% market share.
Google increased its lead by 0.2 percentage points to 66.3%.
Second placed Yahoo! slipped by 0.2% percentage points to 16.5%.
Ask was in 4th place with 3.6% share, losing 0.1 percentage points.
AOL was in fifth place losing 0.2 percentage points to 2.1%.
Microsoft increased its number of searches in October by 7% compared with a 4% increase by Google.
More details are here.
Ask recently pulled out of the search market; it will be interesting to see which company will pick up its 3.6% market share. Are Ask users deliberately not using Google? That would mean a big boost to Microsoft in the November numbers.
The introduction of Google Instant has forced comScore to change the way it analyzes its results. This is explained here: comScore September 2010 Search Reporting Enhancements (comScore Voices)
The Google "Instant" team showed off one-touch search. Othar Hansson, a senior Google engineer, demonstrated (twice) how typing the letter "W" gave "weather in San Francisco" as the first result.
Being a brand that is called up by Google Instant on the very first letter is now the height of search engine optimization (SEO). I took a look at what came up for each letter. Results will vary somewhat by your location.
A is for ... Amazon, AOL, Apple, ATT.
B is for ... BART, Bank of America, Best Buy, Bing.
C is for ... Craigslist, Costco, CNN, Chase.
D is for ... DMV, Dictionary, Droid X, Disneyland.
E is for ... Ebay, ESPN, Expedia, Evite.
F is for ... Facebook, Facebook login, Fandango, FIFA.
G is for ... Gmail, Google maps, Google, Great America.
H is for ... Hotmail, Hulu, Home Depot, HP.
I is for ... Iphone, Ikea, Inception, IMDB.
J is for ... Jet Blue, Jeremy Lin, Jamba Juice, Java.
K is for ... Kaiser, KTUV, Kayak, Kohls.
L is for ... Lowes, Lost, LinkedIn, Limewire.
M is for ... Mapquest, Myspace, MSN, Maps.
N is for ... Netflix, Nordstrom, NBA, News.
O is for ...Outsidelands, Orbitz, OSH, Office Depot.
P is for ... Pandora, PG&E, Paypal, Petco.
Q is for ... Quotes, QVC, Quicktime, Quinoa.
R is for ... REI, Ross, Redfin, Rosh Hashanah.
S is for ... Skype, SFGate, Sears, Southwest.
T is for ... Target, Twitter, Thesaurus, Travelocity.
U is for ... USPS, UPS, United Airlines, Utube.
V is for ... Verizon, Virgin America, VLC, Verizon Wireless.
W is for ... Weather, Walmart, Wells Fargo, Wikipedia.
X is for ... Xbox, XKCD, XM Radio, Xfinity.
Y is for ... Yahoo, Youtube, Yahoo Mail, Yelp.
Z is for ... Zillow, Zappos, Zipcodes, Zynga.
If you can get your brand into the one-letter list you are golden. This is now the holy grail of SEO.
The launch of "Google Instant" represents the single largest change to Google's search services in its history. "For the first time search is interactive," said Othar Hansson, a senior Google engineer.
The company said it performed extensive studies of users and how they interact with Google Instant plus extensive eye-tracking studies. They all showed a much improved user experience and a lot less time spent searching for the right page.
Google says it now has 1 billion users per week and that the time saved by users worldwide from using Google Instant is 11 hours per second. That's the equivalent of 350 million hours of user time per year, said Marissa Mayer, VP of Search Products and User Experience.
During the question and answer part of the launch, the Google team could offer no data on how the change would impact its advertising business beyond saying that what's best for the user is usually best for the advertiser.
This is shocking. Google makes a major change to the search service and the user experience -- yet has no measurement on how that impacts click rates on its adverts!
Let me say this again: Google makes a major change to the search service and the user experience -- yet has no measurement on how that impacts click rates on its adverts!
Those text ads on the side of its results page represent about two-thirds of its revenues. Any change to the structure of the search page has to impact the click through rate of its advertising for better or worse. Yet Google did not measure this impact. Wow.
It seems that the company is divided between those working on improving the search experience and the rest of the company that is trying to sell advertising. And there appears to be no communication between the two sides.
If I were a GOOG shareholder I'd be very concerned that the effectiveness of Google's advertising has been compromised. At the very least, as a shareholder, I would want to know that Google's advertising business has not been compromised by this major change in serving search results.
There must be an effect on advertising.
Think of it this way: If Google could guide me to my intended destination in the shortest time possible then the amount of time I spend staring at Google ads decreases. In this case, it decreases by 11 hours per second.
How is it possible that this will not affect advertising revenues? If I spend less time looking at a page with ads on it then surely that means less time spent clicking on ads?
And if Google has indeed studied the effect on advertising from using "instant" then why did it not offer supporting data instead of just saying that what's good for the user is usually good for the advertiser.
Google may have shot itself in the foot.
- - -
I went to the launch of "Google Instant" this morning. It's a major overhaul of Google search in that it seeks to lessen the time people spend typing in their search queries by trying to guess what the user wants. It's a glorified "I'm feeling lucky" button without requiring a click.
But will this interactive search function distract users from Google ads?
During the 90 minute launch Google executives showed off the result of an eye-tracking study of users interacting with Google Instant. The dynamic nature of the search page forces users to look at the suggestions offered as they type, and user's eyes are drawn the area just below the search box towards the best choice in the results.
The eye tracker did not show any activity on the right side of the page where Google text ads are placed. These Google Adwords text links provide about two-thirds of Google's revenue and the majority of its profits. Google Adsense ads, which appear on third party web sites provide the rest of Google's revenues and far less profit.
The dynamic nature of the page, constantly changing as users type, forces users to concentrate on the search results as they are being presented.
This is far different from typing in a search term and then receiving a static page of results where the eyes are not drawn to any movement, and they can take in a much larger section of the page, including the text ads on the right.
Google executives said they expected SEO and advertising to continue as before and that a focus on improving the search experience usually helps its advertising.
However, if Google improves search to the point where people can get to the site they want with just typing in a few letters, there is less opportunity for Google to serve up ads on its search pages and receive clicks. Google does not make money from impressions but from clicks on its ads.
Google estimates that users worldwide are saving 11 hours per second by using the "Instant" version of its search service.
That means there is 11 hours every second that is not being spent looking at search ads.
It shouldn't take long for Google to see if the change to the new search format impacts its advertising. The Google Instant service is being rolled out throughout the day on four major browser platforms and the company will be able to collect a large amount of usage data fairly quickly.
It would not surprise me if the user data on this new service shows a big change in how users interact with its advertising.
Also: The "I'm feeling lucky" button, which has been on the Google search page since the first days of Google is on the way out. That's what Udi Manber, Vice President of Core Search told me. "There's a long answer but the short answer is yes, the button will be gone. The entire page is all about 'I'm feeling lucky.'"
The Washington, DC organization Consumer Watchdog is making a big publicity push for its "Do not track me" campaign, which asks Congress to create a list for people that don't want to be monitored on the Internet.
If the campaign succeeds it will be a massive blow to Internet advertising and e-commerce companies because they will lose the ability to tailor targeted commercial messages and discovering user behaviors across a range of web destinations.
Consumer Watchdog has purchased a 540 sq. Ft. Jumbotron digital billboard on Times Square and is showing a video showing an animation of Eric Schmidt, Google's CEO, driving an ice cream truck so that he can spy on kids.
Consumer Watchdog said:
Google has collected massive amounts of personal data from Wi-Fi networks through its Street View cars, made private Gmail contacts publicly available on Buzz, and done a complete about-face on net neutrality, joining with Verizon in calling for toll lanes on the Internet.
Schmidt has appeared clueless regarding privacy himself, Consumer Watchdog said. When questioned about privacy, he has said, “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.” Recently, he suggested children could change their names when they got older if they wanted to escape what was embarrassing and public in their online lives.
- About 80% of Americans support a national "Do not track me" list according to a poll by Grove Insight.
- 90% said that it is important to “have more laws that protect privacy of your personal information” online.
The poll indicated strong support for:
Has Google lost the battle against the companies that game its system of ranking search results? It certainly seems that way from my anecdotal usage.
John Byrne, the former BusinessWeek editor seems to feel the same. He recently launched a site that looks at business schools: Poets and Quants.
He published a post detailing his frustration at Google's listing of the site, or rather its failure to list the site. C-Change Media Inc.: Google? Where are you?
One of the most fascinating aspects of our debut is what Google has been able to discover, or fail to find, about the site.
...So what do you find when you Google "poetsandquants?" Not a single mention of the actual website.
...As you go through the first five pages of Google results, there are all kinds of websites that have essentially highjacked Google, rendering its search product less useful and helpful to users. There's a so-called weblog that is little more than a place to advertise Viagra and Cialis. There's links to TweetMeme, Interceder, tweetcepts, twapperkeeper, rallyclips, and whotechpunditstweet, among many others. Most of them are search traps that have gamed Google.
Part of Mr. Byrne's frustration is related to the fact that his site is a young site -- Google trusts sites that have been around for a longer time. However, I've been noticing a similar fall in the quality of the rankings on general searches.
Often I have to look at second and third pages of listings when I used to be able to find what I needed on the first page.
Also: there are plenty more companies that make a business gaming Google results. For example: Demand Media, which has built a large business based on the fact that it publishes content that is gamed to attract Google rankings and Google AdSense advertising.
Demand Media is so confident in its business model based on continued gaming of Google that it has filed for an IPO. And there are other companies whose entire business model is based on their ability to game Google.
Google changes its algorithm on a regular basis and that serves to shakeout all those sites that have tried to game Google based on the characteristics of the prior algorithm. Has Google ran out of ways of shaking out the spam sites?
I agree with Mr. Byrne's conclusion:
...This goes to the quality of Google's primary product: search. If Google can't find PoetsandQuants or any of the stories published on the site, I wonder how many other legitimate, substantive efforts are also going undiscovered because Google's algorithms have been so effectively gamed.
Why is Vint Cerf, Google's chief Internet Evangelist and father of the Internet silent on this issue of Internet neutrality?
It's an issue that has blown up over the past few days yet Mr. Cerf has been absent.
Just over a year ago, Mr. Cerf wrote this on the Google public policy blog:
Allowing a handful of broadband carriers to determine what people see and do online would fundamentally undermine the features that have made the Internet such a success, and could permanently compromise the Internet as a platform for the free exchange of information, commerce, and ideas.
...we believe that providers should have the flexibility to manage traffic congestion and malware on their networks in non-discriminatory ways. They should not, however, be in the anti-competitive business of picking winners and losers.
Today on the same blog, Google and Verizon have thrown out these principles when applied to wireless broadband!
In recognition of the still-nascent nature of the wireless broadband marketplace, under this proposal we would not now apply most of the wireline principles to wireless, except for the transparency requirement. In addition, the Government Accountability Office would be required to report to Congress annually on developments in the wireless broadband marketplace, and whether or not current policies are working to protect consumers.
This is a bullshit excuse. Traffic has to be prioritized on the backbone and wired connections before it reaches the wireless transmitters. That means prioritized traffic will be available for wireline too!
Here is a short video interview I shot with Vint Cerf at the Fortune Brainstorm conference two years ago. In it he says the Telcos are harming the US national interest.
(The lighting was bad so I turned it into a high contrast video.)
Australia's IT wire wrote this story with a partial transcript.
Vint Cerf, who is widely regarded as the 'father of the Internet' for his contribution to the original TCP/IP specification, has lashed out at carriers accusing them of behaving like young children throwing tantrums. In an especially strongly worded attack, Cerf called for structural separation between the wholesale and retail broadband arms of carriers among other changes.
In a brief interview with SiliconValleyWatcher, Cerf said carriers were effectively saying "I'm not going to build this system unless you give me three scoops of ice cream and a pony", and provided a laundry list of changes in the regulatory environment that he'd like to see to improve the situation. These include:
The reintroduction of common carrier status;
Structural or accounting separation, with a requirement that carriers wholesale broadband at the same prices that they charge themselves; and
No interference with other providers' applications (i.e., net neutrality).
The current behavior of carriers is harmful to the national interest, he said - an observation that wouldn't only apply to the US.
"[Deregulation] is crap, especially where you have a set of incumbents," he said. Instead, we "need a set of rules that makes sense."
Consequently, carriers need to be given incentives to behave differently or (Cerf's emphasis) incentives should be provided for competitors to compete with incumbents.
"In places where there is strong regulatory control, it seems to be working," he said.
You need to keep in mind that Cerf is a vice president at Google (a strong proponent of network neutrality), but there's no reason to assume that his opinions are not genuinely held.
If Google wanted to calm the waters around this issue of Internet neutrality, they would wheel out their Chief Internet Evangelist who also happens to be Father of the Internet: Vint Cerf.
Yet he is absent. And by his absence in this discussion -- one that has involved him on past occasions -- can we assume he is in disagreement?
It looks that way to me.
My recent post about speeding up the Internet by creating an open index of web sites to cut down on robot traffic, received a lot of attention and a lot of great responses.
Several people wrote that it would be important to be able to verify the information on a web site. That's true but maybe if there was just one "openbot" verifying the data and punishing those web sites that tried to cheat we could have a fairly clean index.
Also, there could be a business model here. If there was one open index then all the bots could query just that database because it would have the best information.
The open index could even charge for access especially if it offered a variety of analysis tools or offering to run a custom algorithm. It would be far cheaper than a company crawling the web, collecting the masses of data and then trying to analyze it.
The value is in the analysis and not in the index.
For most web sites, nearly 50 percent of their traffic goes to serving robots that are all crawling exactly the same data. A central repository updated as soon as anything changes would go a long way to setting free a substantial amount of broadband resources for far less cost than adding more servers, more network capacity and the energy required to run it all.
A central shared index would save a tremendous amount of energy and carbon dioxide plus we'd have a faster Internet for a fraction of the cost of building a faster Internet.
When I was in Brazil recently, I met with Berthier Ribeiro-Neto, head of engineering at Google Brazil. During our conversation I mentioned an idea I had about making the Google index into an open database that anyone could access, I said that this could dramatically speed up the Internet.
He said it was a good idea and that I "should write a position paper" on this subject.
(As a further thought, maybe it could also serve to take away some of the heat Google is feeling lately, in terms of its index rankings potentially favoring its own business interests.)
Here is my logic:
Looking at my server logs shows that 20 different robots visit my site, one of the more frequent ones is the Googlebot. Each of these robots is trying to create an index of my site.
Each of these robots takes up a considerable amount of my resources. For June, the Googlebot ate up 4.9 gigabytes of bandwidth, Yahoo used 4.8 gigabytes, while an unknown robot used 11.27 gigabytes of bandwidth. Together, they used up 45% of my bandwidth just to create an index of my site.
These robots are all seeking the same information and they use nearly one-half of my bandwidth, slowing the site for all my readers. This is also the same for tens of millions of web sites.
What if there was a single index that anyone could access?
You would get an immediate speed increase in the Internet for no additional investment in infrastructure.
Google and others, could perform their own analysis of the index using their secret algorithms. After all, the value is not in the index it is in the analysis of that index.
Mr. Ribeiro-Neto said, "That's a good idea. You probably wouldn't even need to spider the web sites."
Each web site could update the central index automatically each time something changed. This would result in a massive savings in bandwidth used by dozens of robots scouring the Internet for new information.
What if Google opened up its index to the world as a goodwill gesture because it has the best index? It could still maintain the privacy of its algorithm but everyone would have the same information on which to perform their analysis.
It would show that there was nothing unusual or unethical in how Google collects information for its index. This might relieve some of the pressure it has come under this week to reveal more about how its search service is presented.
Also, Google founders were once strong advocates that the search index should be run as a non-profit.
On page 39 "Inside Larry and Sergey's Brain" by Richard Brandt (referral link).
Andrei Broder, who led the team that created the AltaVista search engine, the best of its time, talks about meeting Larry and Sergey. "When the discussion turned to the topic of making money from the technology, Broder found that Page had a profound difference of philosophy on the subject. "It was a very funny thing about Larry," Broder recalls. "He was very adamant about search engines not being owned by commercial entities. He said it should all be done by a nonprofit. I guess Larry has changed his mind about that."
Brian Lent, now CEO at Medio Systems:
"The problem with the Google search engine at the time, Lent recalls, is that Larry and Sergey didn't want to commercialize it, and Lent was anxious to become an entrepreneur. Their mantra at the time was more socialistic than entrepreneurial. "Originally, 'Don't be evil' was 'Don't go commercial,'" says Lent.
- - -
Don MacAskill, CEO of SmugMug writes:
... I would estimate close to 50% of our web server CPU resources (and related data access layers) go to serving crawler robots. Stop and think about that for a minute. SmugMug is a Top 300 website with tens of millions of visitors, more than half a billion page views, and billions of HTTP / AJAX requests (we're very dynamic) each month. As measured by both Google and Alexa, we're extremely fast (faster than 84% of sites) despite being very media heavy. We invest heavily in performance.
And maybe 50% of that is wasted on crawler robots. We have billions of 'unique' URLs since we have galleries, timelines, keywords, feeds, etc. Tons of ways to slice and dice our data. Every second of every day, we're being crawled by Google, Yahoo, Microsoft, etc. And those are the well-behaved robots. The startups who think nothing of just hammering us with crazy requests all day long are even worse. And if you think about it, the robots are much harder to optimize for - they're crawling the long tail, which totally annihilates your caching layers. Humans are much easier to predict and optimize for.
Worst part about the whole thing, though? We're serving the exact same data to Google. And to Yahoo. And to Microsoft. And to Billy Bob's Startup. You get the idea. For every new crawler, our costs go up.
We spend significant effort attempting to serve the robots quickly and well, but the duplicated effort is getting pretty insane. I wouldn't be surprised if that was part of the reason Facebook revised their robots.txt policy, and I wouldn't be surprised to see us do something similar in the near future, which would allow us to devote our resources to the crawlers that really matter.
Anyway, if a vote were held to decide whether the world needs an open-to-all index, rather than all this duplicated crawling, I'd vote YES! And SmugMug would get even faster than it is today.
- The NYTimes: The Google Algorithm
- FT.com / Comment / Opinion - Do not neutralize the web's endless search (Subscription required.)
Richard Waters in an article at FT.com (Subscription required):
Groups magnify chances of Google hits
Companies with a high page rank are in a strong position to move into new markets. By "pointing" to this new information from their existing sites they can pass on some of their existing search engine aura, guaranteeing them more prominence.
This helps companies such as AOL and Yahoo as they move into the low-cost content business, says Mr Bonnie. "They can use their Google page rank to make sure their content floats to the top," he says.
Google's Mr Singhal calls this the problem of "brand recognition": where companies whose standing is based on their success in one area use this to "venture out into another class of information which they may not be as rich at". Google uses human raters to assess the quality of individual sites in order to counter this effect, he adds.
I've known about this for several years but wasn't able to get anyone from Google on the record. These Google employees have the power to promote or even completely erase a site from the Google index.
This admission is potentially a very large problem for Google because it has maintained that its index rankings are unbiased and are computed from a natural pecking order derived from how other sites find a specific site important.
The Google algorithm is a mathematical expression drawing on the PageRank patented method (named after Larry Page, co-founder). It counts how many links to a web site come from other web sites and determines the importance of that web site for millions of search terms. These rankings are worth huge amounts of money to many web sites and changes in rankings can put companies out of business.
Google is currently being sued by several companies claiming bias in Google results.
Scott Cleland, whose blog "The Precursor" has been critical of Google, writes:
"... this first-ever disclosure by Google that "human raters" manually discriminate in the "quality scores" that determine a website's supposed neutral and unbiased search ranking, exposes a rats nest of conflicts of interest that Google has in its "black box" business model."
He says that antitrust authorities are bound to ask key questions such as:
"If links are a factor in determining the rank of content, and Google's advertising revenue is derived from sites' search rankings, how does Google ensure the human raters of the SDB are not influenced to reward Google-owned content or Google partners' content that Google revenue shares with?"
It's a huge can of worms.
Google has had a long string of failures. It encourages its engineers to spend 20% of their time developing side projects but when those projects reach launch stage, their take-off is nearly always very disappointing.
Take a look at some of Google's failures.
Colin Gibbs reporting on GigaOM:
I can easily add a lot more to this list. Google Video; Google's acquisition of Jot; Google Wave; Knol; Checkout; Catalogs; Base; Squared; and Google Buzz could be the latest.
Google has tremendous scale so it is puzzling to some why so many of its services should have been such failures. But, it isn't that surprising if you consider its culture because Google believes that good products will find their users based on their own merits.
What Google fails to recognize is that it needs to assign marketing support. Without marketing support it is wasting the cream of its engineering talent.
Have you seen any marketing for Google services beyond an occasional text ad?
I've never been contacted by any PR companies, or Google corporate comms people to talk about a new Google service or product. Yet I receive countless such invitations from smaller companies trying to get media attention.
Google's failure to recognize the need for effective marketing is deep rooted within its engineering culture. Engineers don't believe in marketing. Many software engineers will deride a company's success (e.g Apple) as "it's just marketing." It makes it seems as if "marketing" is something that can be easily acquired and put to good use.
But marketing is not easy, and successful marketing is not a commodity (it's interesting that software engineers are (becoming) a commodity...)
Google's own success grew out of a non-marketing approach; Google search was simply a better product. Google is proud that it didn't use marketing to become a success.
But times are different today. There is a tremendous amount of media already on the Internet and this level will rise to a media tsunami as companies and individuals make full use of their media publishing capabilities. The media tsunami will drown less able companies, products, and services.
Effective marketing is going to become ever more important, and more expensive, simply because the media tsunami is raising the bar for everyone to stand out.
Company culture is very difficult to change and it changes slowly and that's why Google will continue to launch new services, and it will continue to fail because it doesn't understand the need for follow up marketing and PR.
And that means it will continue to remain a one-trick pony.
Google today made a big deal out of disclosing how much money it shares with web site publishers hosting its AdSense advertisements.
"In the spirit of greater transparency," wrote Neal Mohan, Vice President, Product Management, he revealed that it pays publishers 68% of advertising revenues around content, and 51% for ads related to search.
However, some large publishers, such as the New York Times receive a larger share, some have received 100%. So what does this revenue share percentage represent?
As one commenter wrote:
I didn't get from the article how they came to that 68% figure. Is it an average, mean or mode?
... it may mean that a handful of huge publishers are getting a revenue share of 90%, a fair number of medium publishers are getting 50% and the vast majority of small publishers are getting 20%.
If we don't get more information about how the 68% and the 51% figures were calculated and about whether all publishers get the same or similar rates, then I will still consider that I have no idea what my effective rev share is.
Anticipating the meaningless nature of the disclosure, Mr Mohan urged Adsense partners to "focus on the total revenue generated from your site, rather than just revenue share, which can be misleading." Misleading it certainly is. How this improves transparency is a mystery -- It's as clear as mud. However, Google watchers applauded.
Jeff Jarvis from BuzzMachine took credit for pressuring Google to reveal the numbers. And John Battelle indicated it was a good split since "as recently as two years ago, sources I know to be extremely reliable were actively negotiating with Google to get a 65% cut."
Both are authors of books about Google.
Because the accounting isn't clear, a key question is if Google can vary payouts in order to meet Wall Street expectations, a form of cookie jar accounting. Small variations in publisher payouts could have a large effect on its share price if GOOG exceeds Wall Street quarterly estimates.
Mr Mohan says the percentage Google keeps for itself hasn't changed since 2005 for search, and 2003 for content. However, Google has considerable lee way within those figures.
The revenue share it keeps for itself is used to cover costs comprising of expenses and investments in AdSense. Mr Mohan admits that these costs "can vary significantly," which means Google could choose to reduce investments in AdSense over the short term in order to boost income and meet or beat financial targets.
Also, Google has been actively trying to shift revenues to its own web properties so that it can keep 100% of ad revenues.
When Google went public in 2004 its revenues were equally split 50/50 between it's own online properties and third party sites. Google's own sites now account for 66% of total revenues while third-party sites generated just 30% of total revenues -- a 40% reduction.
The quickest way for Google to boost profits is by further reducing the percentage of total revenues paid out to its AdSense partners.
It's a strange business when you can make more money by competing against your publishing partners. Where's the incentive to improve payouts and the AdSense platform?
- - -
Occasionally, I Google myself to see who is referencing my work. A couple of years ago Google returned 160,000 references to my name.
In March 2009 Google returned 135,000 references to my name.
In August 2009 Google found 102,000 references.
Today, Google had 94,100 references for "Tom Foremski."
I write a lot here on SVW but also on ZDNet and often people will reference my work in their posts. Surely, my search results should be growing and not shrinking?
Plus, now there is Twitter too, and my handle is tomforemski. Google found 98,500 references to "tomforemski" -- yet I've been publishing online for more than 20 years, ten times longer than on Twitter. Surely, I should have far more references than my Twitter handle?
Clearly, something is not right. Clearly, the number of results that Google claims is bogus.
Here's another example. I found this post by Debbie Weil, a veteran author and speaker: Redux: Have You Googled Yourself Lately?
At the time of writing the post, May 2009, she had 143,000 results. Five years prior to that she had 77,900 results. That's a decent increase.
Today, if you Google Debbie Weil you'll find just 33,200 results.
What's going on? Why are there fewer results?
Also, have you ever tried to see the very last result of your name? I tried to see the very last result in March 2009, I could only get as far as 552. Today, I could only get as far as the 304th reference.
What's going on?
We are warned that Google will remember everything about us, all our youthful transgressions, everything we did online is searchable.
That's plainly not true. In fact, the longer the time between now and then, the fewer results.
At this rate I'll be practically invisible online within a few years...
Updated: Several commenters have said that Google publishes estimates of the number of citations. But what is the accuracy of those numbers? Is it within 10 percent, 15 percent of the true number or is it a completely meaningless number?
Simon Firth makes an excellent point, he notes that journalists often use those numbers.
...Foremski’s observation makes clear that Google search statistics have dubious validity as measures of scale. And yet journalists, myself included, use them all the time as a shorthand way of measuring impact.
Also, Google publishes trend data that clearly, must use those very same estimates. How reliable is that data? Shouldn't Google reveal the margin of accuracy in its estimates?
Google has always been about getting people connected to the content they are looking for as quickly as possible. They even publish how quickly, in milliseconds, they perform a search.
This has been a much different strategy from that employed in the dotcom boom days of the late 1990s when "stickiness" was a prized metric and mantra. The more eyeballs stuck to a site the longer the better.
Google chose quick over sticky but was that a good business strategy?
Facebook likes sticky over quick and it seems to be working. Nielsen estimates that US Internet users spent nearly 2 hours on Google in March 2010.
They spent nearly 7 hours, or 3.5 times longer on Facebook.
Although Facebook revenues are secret, you could argue that Google makes more money per user/minute than Facebook. However, look at all the future opportunities to monetize all that extra time that people spend on Facebook versus Google. There's a lot of low hanging fruit that Facebook can pluck.
The Facebook experience is also much richer -- you get to see content from your family, friends, and contacts -- versus Google's plain vanilla page with a blinking cursor inside a search box.
I appreciate Google's quick in-and-out policy but if I were an investor I'd prefer a much stickier strategy.
Google's strategy with its Nexus One mobile phone is in trouble as Verizon drops plans for support. Bloomberg's Amy Thomson reports:
Without a Verizon partnership, Google loses access to the carrier's more than 90 million customers, potentially blocking the phone from gaining more widespread popularity and hurting its competition with Apple Inc.'s iPhone. The breakdown of the deal signals Verizon may view Google as a competitor rather than a partner when it comes to Nexus One sales, said analyst Colin Gillis at BGC Partners LP in New York.
"It's really a flop for Google," said Gillis, who advises investors to hold Google shares and doesn't own any. "They paid a price to roll out their own branded phone -- it's a price of trust and relationship with some of the other players in the space."
Google's insistence on bringing out its own phone has been a pricey strategy in that it has alienated telcos and mobile phone makers. It also angered former ally Apple, which quickly moved to cut ties with Google, asking CEo Eric Schmidt to leave its board of directors.
Google could have maintained a fairly neutral position, licensing its hardware and software technologies to others, instead it chose to directly enter the mobile phone market, without any of the support infrastructure such as customer service.
This also shows that Google is at the mercy of the Telcos when it comes to what phones and services the Telcos choose to carry.
Will Google choose to become a Telco so it is able to introduce the phones and services it wants to rather than be at the mercy of what the Telcos choose to support?
As expected, Google has stopped censoring its China searches and directed queries to its Hong Kong site.
David Drummond, Chief Legal Officer wrote about the redirection "it's entirely legal and will meaningfully increase access to information for people in China. We very much hope that the Chinese government respects our decision, though we are well aware that it could at any time block access to our services."
Google said it would retain its R&D centers and its sales teams.
But can it remain in China given the rising animosity to its opposition of the Chinese government? Can its employees in China feel safe from repercussions?
Google is trying to protect its Chinese employees, it stated:
...we would like to make clear that all these decisions have been driven and implemented by our executives in the United States, and that none of our employees in China can, or should, be held responsible for them.
But feelings towards Google are running high in China. Over the past weekend, the Chinese government and allied media organizations, stepped up a publicity campaign against Google.
Sam Gustin, writing in the Daily Finance, reported: Chinese Media: Google Is Tied to U.S. Intelligence - DailyFinance
Chinese media organs lashed out at Google (GOOG) in an apparently coordinated assault on Saturday -- with one paper suggesting Google is linked to the U.S. Intelligence Agencies...
Reuters reported: China state media accuses Google of political agenda | Reuters
China's state media on Sunday accused Google Inc of pushing a political agenda by "groundlessly accusing the Chinese government" of supporting hacker attacks and by trying to export its own culture, values and ideas.
The negotiations with the Chinese government have failed badly. Google has demonstrated a shocking lack of historical knowledge and lack of understanding of Chinese culture in its dealings with the Chinese government.
For a foreign organization to give the Chinese government an ultimatum on changing its laws is like poking a sharp stick into an old wound. Google should have Googled "Opium Wars" before it issued its ultimatum.
The British forced the Chinese to make opium legal, which led to huge amounts of instability in Chinese society, and resulted in two brutal wars, the second one included the French.
The Chinese government is concerned that without Internet censorship, there will be instability in its society. Yet Google made demands that its Internet censorship laws be changed, after it was angered by a "sophisticated" hacker attack it said originated in China.
- Google has not provided the Chinese government with any of its evidence that sophisticated hackers were acting as agents of the government. Those attacks now appear to be amateurish rather than "sophisticated" as it originally claimed.
- Google has allowed the National Security Agency (NSA), the world's largest spying organization, to help it with its security -- a move that has badly backfired. The Electronic Frontier Foundation, ACLU, and Electronic Privacy Information Center have all warned about Google and the NSA getting together. [Was China An Excuse For GOOG + NSA Collaboration?]
- Google users in China have not had any explanation for its position, claims an open letter by Chinese "netizens" to Google and Chinese government ministers. Please see: RConversation: Chinese netizens' open letter to the Chinese Government and Google
I like Google's stand against Internet censorship but its dealings with China have become very messy and could have repercussions beyond China.
- - -
Here is a PearlTree about Google and China - a visual way to see related web content:
The Federal Trade Commission (FTC) appears to be stepping up its scrutiny of Google and anti-trust issues related to its business. Bloomberg reports that the FTC is asking for sworn statements from competitors and advertisers.
This action usually precedes the filing of a lawsuit seeking to block the $750 million AdMob purchase announced in November.
Agency officials typically collect declarations "when they think there is some significant chance" the agency will ask a court to block a merger, or seek to modify a deal, said Stephen Calkins, a former general counsel at the FTC who is now a professor of law at Wayne State University's law school in Detroit.
You can bet that Microsoft is happy to testify about this deal. Microsoft has a lot of experience with anti-trust and can craft its complaints in a language that FTC's lawyers can understand.
Scott Cleland, writing on The Precursor Blog, predicts that Google is likely to withdraw from the deal before the FTC files a complaint.
The last thing Google wants to do is have its own actions or recalcitrance be the catalyst for a much broader and more serious antitrust crackdown on Google's rapidly proliferating anti-competitive behavior.
For Google, it means it has to rethink not only its mobile ad strategy but also its acquisition strategy. It will be very difficult for Google to make large acquisitions in the future. But trying to grow its businesses organically, will reduce its ability move quickly, and dominate new markets. Which is precisely what the FTC means to do.
At some point, Google may decide to make a stand and challenge an anti-trust complaint. But it will have to choose its battleground carefully, because it will lay itself open to anti-trust lawsuits from other businesses, it will need solid footing.
Google's difficulties in making acquisitions bad news for the VC community. With very few IPOs around, acquisitions have become the main exit strategy for many venture-backed companies.
- - -
Wayne Rosing, when he was VP of Engineering at Google, once told me that Google saves every bit of data from people's searches and puts it onto tapes and ship it off to a storage facility.
Why does Google collect all that data I asked? We don't know, but we collect it all, he said.
Search data is mined to "learn from the good guys," in Google's parlance, by watching how users correct their own spelling mistakes, how they write in their native language, and what sites they visit after searches. That information has been crucial to Google's famously algorithm-driven approach to problems like spell check, machine language translation, and improving its main search engine. Without the algorithms, Google Translate wouldn't be able to support less-used languages like Catalan and Welsh.
Data is also mined to watch how the "bad guys" run link farms and other Web irritants so that Google can take countermeasures.
Google eventually anonymizes the data:
The last octet of the IP address is wiped after nine months, which means there are 256 possibilities for the IP address in question. After 18 months, Google anonymizes the unique cookie data stored in these logs.
This isn't especially ambitious; Europe's data protection supervisors have called for IP anonymization after six months and competing search engines like Bing do just that (and Bing removes the entire IP address, not just the last octet). Yahoo scrubs its data after 90 days.
But this data could still be traced to individual users.
This is what happened when AOL released search data on 685,000 search users in August 2007. The data had been anonymized but it was easy for reporters to find the actual users from clues in their searches, such as zip codes and town names.
You can search for what AOL users searched at these sites:
The AOL searches revealed a glimpse into the unguarded thoughts of the digital haves.
In one instance, it looks as if a wife and a husband are using the same computer, each hiding their extramarital affairs from the other, then later looking for help online to deal with the pain of a failed relationship.
And there are real soap operas, tracked over a period of months... from the excitement of first meetings:
"how to get rid of nervousness of meeting a blind date 23 Apr, 12:27"
"if your spouse has an affair should you contact the other person's spouse and let them know : 07 May, 09:58"
And the same user account asks:
"i had sex with my best friend and now he treats me differently :26 May, 13:58"
There are also "how to kill your wife" searches and more.
All this data was anonymized but all the searches from a single computer were kept together and that means they can eventually be traced. A New York Times reporter quickly managed to find one of the searchers.
Welcome to the future transparency of your past.
In the future, there will be vast databases of anonymized data from a variety of sources: search engines, credit card companies, cell phones, geo-location data, etc. It will be possible to triangulate that data, and if one piece of that data is linked to a user, it will unlock everything else.
Yes, it would take quite a bit of data mining but we have the technologies to do it today.
While each silo of information, technically might be anonymous, in aggregate, it would help identify users from their behavior. Each digital interaction throughout your day, whether through mobile, or desktop, or bank, leaves a trace and that can eventually be tracked and matched with an identifiable person.
And Google, with its dominance of your life, search, email, docs, buzz, photos, video, etc, is collecting huge amounts of your behavioral data, and it will be one of the main keys in unlocking your privacy.
Welcome to the future transparency of your past.
I'm a big fan of Gabe Rivera's Techmeme news aggregator, (and I'm not the only one as my photo shows), I became even more of an admirer when he hired five editors to help with story placement, because news sites stopped linking to each other, and his usual search algorithm didn't work anymore (Google take note).
Earlier today, Intel, the world's largest chipmaker, reported that one of its senior execs, Sean Maloney, number two at Intel, suffered a major stroke.
I wrote about it around 4pm. Others have too. Nine hours later, nothing has shown up on Techmeme. Top story at this time (10pm Monday) on Techmeme is an old story: Picnik Acquired by Google. The most recent Sean Maloney story is Intel Risks It All (Again) Oct 14, 2009.
I speak as a fan, Gabe, fix that algorithm :)
UPDATED: A story finally appeared on Techmeme at 6.55am Eastern Time the next day - linked to a Guardian story in the UK - NOT to the source statement from Intel the previous day.
- - -
"Death might be too strong a word but this is definitely a watershed moment. Techmeme, which used to rely completely on Gabe Rivera's secret algorithm to pick tech news and blog posts, now has six editors.
This is significant because Techmeme shows that human aided algorithms are more effective than just software and server. Techmeme is a microcosm of the rest of the search-enabled world of services, from news aggregators to basic search.
If Techmeme can't be Techmeme just by using its algorithms, and now needs lots of editors, then that means much larger news aggregators and search companies will likely have to add human editors too."
- - -Sir Martin Sorrell: Rupert Murdoch's pay wall plan is right) from the Daily Telegraph shows one link, from a Daily Telegraph blog.
The demise of linking seems as if we have gone back to where things stood five years ago, when newspapers hated linking out to other news sites, or even acknowledging that other news sites existed.
Bloggers loved linking and many still do it -- but not as much, especially if they've transitioned into online news magazines such as GigaOm, orReadWriteWeb, blogs only in name because they use blogging software (Wordpress) to publish but act like and look like the traditional old media.
This all has a more serious implication: Google PageRank. The whole bedrock of Google's search is in links. That was its founders' great insight: pages with more links to them are more important than those with fewer links. There is a PageRank patent.
Not everyone has reduced their linking. Spammers and publishers of commerce sites are continuing to use links a lot because that's what Google pays attention to in determining the quality of a page. This means that if non-spammers are linking less then Google's first page of results is going to get flooded by spam."
Following an Italian court ruling earlier this week, Google is facing the prospect of having to check Italian sourced videos before they are posted, to make sure they don't violate Italian privacy laws.
That's a daunting task.
One potential way around this problem is to do what it did in South Korea last year. A new law forced Google to collect the real names of Koreans uploading videos or commenting online.
On the day the law came into effect, Google simply switched off the comments and blocked the ability for people to upload videos to its Korean YouTube site. Koreans were still allowed to upload video to YouTube sites in neighboring countries.
It was neat sidestep of its legal obligations.
Run for the border...
Courts only have jurisdiction within their country. But web sites and data, can be located anywhere. In the future, Iceland, might become a favored destination for Internet data because it is debating passing strong laws that protect freedom of speech.
Google could use the international nature of the Internet to thumb its nose at any government seeking to control what it hosts.
Such a strategy however, is a risky one. If it chooses the wrong issue, it would be seen as an international pariah, which would harm its brand. After all, a competitor is always just one click away.
Google needs to decide whether its claim to "Internet freedom," as its right to host and distribute a video of a disabled boy being beaten and insulted, is one that would justify disregarding a country's laws.
There might be more important battles to be waged in the future and it would do well to keep its powder dry.
- - -
It seems that the New York Times agrees with my view that the Italian court ruling on Google hosting a video of a disabled boy being bullied, [Italian Decision Could Help Traditional Media Orgs], potentially benefits Italian prime minister and media magnate Silvio Berlusconi:
In Italy, where Prime Minister Silvio Berlusconi owns most private media and indirectly controls public media, there is a strong push to regulate the Internet more assertively than it is controlled elsewhere in Europe. Several measures are pending in Parliament here that seek to impose various controls on the Internet.
Critics of Mr. Berlusconi say the measures go beyond routine copyright questions and are a way to stave off competition from the Web to public television stations and his own private channels -- and to keep a tighter grip on public debate.
The Google ruling comes amid other proposed legislation that would seek to bureaucratize the Internet in Italy, including the highly contested Italian version of a European directive that would compel online broadcasters to seek the same licensing agreements as broadcast television. Google lobbied for changes to the proposal.
I've said this before and I don't mind saying this again: Google is a media company -- it publishes pages of content with advertising around it. Just like a newspaper. What's not a media company about that?
Google is not a technology company.
You cannot buy any technology from Google. You can buy technology from Oracle (databases), or from Intel (microprocessors).
(OK. You can buy a search appliance from Google but that is a tiny business that barely registers in its financial results.)
Google is a technology-enabled media company. Therefore, it will come under similar regulations that apply to media companies.
As the New York Times reports:
The verdict, though subject to appeal, could have sweeping implications worldwide for Internet freedom: It suggests that Google is not simply a tool for its users, as it contends, but is effectively no different from any other media company, like newspapers or television, that provides content and could be regulated.
I've been saying that Google is a media company for about five years. Today, more people understand this point, and this is what the Italian ruling is about.
And this is the issue that will chase Google in other countries too.
[This is a very important point: This issue is one that can help traditional media companies. If Google's business costs rise because it has to hire people to monitor its content, like other media companies have to do, then that establishes a more even playing field. The cost of online advertising will rise and that will be good for media companies struggling to make the transition to online. It will increase their potential revenues from online media.]
Google can't continue to turn a blind eye to its social responsibilities. It has to face them or it will be forced to face them.
If media companies such as newspapers have to shoulder social responsibilities then Google, and other Internet companies, need to do the same.
"Do no evil" is passive "Do some good" is what Google needs to do.
Yes, it is tough to do that, but I know Google can do it. I'm constantly amazed at how it manages to deal with some of the most difficult problems around. It can do this and still prosper.
It's better that turning a blind eye to its social responsibilities and hiding behind "Internet freedom" as an excuse for the right to publish a video of a disabled boy being beaten and insulted.
Google's position on this issue is risky.
The risk is not from public opinion but from within its ranks. If its own people, its employees, start to lose their respect for the company, then it is in trouble.
That's what happened at Yahoo, with its horrible policy of working with Chinese authorities to finger and imprison dissidents. I know from personal experience, that people that worked at Yahoo lost a lot of respect for their management, and that damaged Yahoo tremendously.
You can trace Yahoo's problems directly from the start of its China policy.
Take a look at the stock market value of Yahoo at the time it was revealed by Reporters without Frontiers, to have acted as a "police informant for the Chinese."
I'm not saying that this Italian issue is anywhere as bad as Yahoo's actions in China. But I am saying that this is a good time for Google to step up and consider its responsibilities -- before it is forced to do so.
And before people, especially its own, lose respect for the company.
This is an issue that will come up time and again. It's something that Google cannot continue to evade. I think it's best to deal with this sooner rather than later.
Google is the world's single largest user of Internet bandwidth. But it doesn't have to pay for most of the bandwidth it consumes, in the form of its spiderbots copying web pages, and YouTube video streams.
A study in December 2008 estimated Google's 'free' bandwidth use to be about $6.9 billion, today it could be double that amount. [Precursor study- December 2008]
Scott Cleland, head of Precursor, is a leading Google critic especially of its position supporting net neutrality. Google has been lobbying the Federal Communications Commission (FCC) to maintain net neutrality regulations, which prevent Telcos from charging companies for bandwidth based on usage. He claims that Google is pushing net neutrality because it is the biggest beneficiary of such rules.
This is from his post: How much should Google be subsidized? | The Precursor Blog by Scott Cleland
- Google is world's largest Internet user and uses 21 times more bandwidth than it pays for.
- Google's lobbying for open access restrictions on auction of 700 Mhz spectrum resulted in $7 billion less money for the US treasury.
- Google received $89 million in taxpayer subsidies from the state of North Carolina to build a data center. It does not disclose subsidies received from other data center sites.
- Google's recent push for 1 gigabit networks could force other Telcos to also offer similar networks. Yet Google would be the biggest beneficiary of such high speed networks.
- Google is lobbying the FCC over what broadband providers can charge consumers because it takes attention away from its own massive use of Internet bandwidth.
Google's massive hidden implicit subsidy in the FCC's proposed open Internet regulations is that it is acceptable for broadband providers to charge consumers more for faster service, but it is unacceptable for broadband providers to charge web companies like Google more for more speed.
He makes the point:
Why should a consumer have to pay more for more broadband speed and capacity, but Google should not?
Mr Cleland's criticism of Google comes from his long standing position representing the Telcos and their ability to charge companies such as Google for bandwidth use.
Both Google and the Telcos are spending a lot of money trying to influence the FCC. [Google Spends Millions To Influence Washington]
While Google might be protected in the US from such charges, at least for now, that might change in other countries. Recently, Spanish Telco Telefonica spoke about charging Google.
Google has called Mr Cleland a "payola pundit" because of his Telco funding.
It's clear that Google has made an enemy of the Telcos, both in its net neutrality lobbying and because of its mobile strategy.
- - -
It is the most important conference for the wireless Telco industry. And this year, there was a lot of concern about the growing power of Apple and also Google.
The wireless Telco companies are banding together to provide apps stores to counter Apple. But also, they are increasingly concerned about Google:
Vittorio Colao [Vodafone CEO] warned the telecommunications industry that companies controlling 70 percent to 80 percent of a market, such as Google in mobile search, should raise the attention of regulators.
Here is a list of things that Google has done that worry the wireless Telcos:
- Its NexusOne handset.
- Its Android handset OS.
- Its investment in Clearwire, the WiMAX telco.
- Its plans to build a high speed network for select communities.
- Its large holdings of 'dark' optical fiber.
- Google Voice.
- Its position supporting network neutrality, which limits what Telcos can charge.
- It has placed competing bids for US wireless spectrum.
- Its Android apps.
It's looking like Google is assembling many of the components it needs for a wireless Telco operator, but with the extra muscle that comes from dominating the search business.
At the conference, Google was accused of trying to turn wireless operators into "dumb pipe providers."
Google CEO Eric Schmidt tried to calm the operators:
"We feel very strongly that we depend on the success of the carrier business," Schmidt said. "We need a sophisticated network for security and load balancing." ...Google is not looking to compete with wireless operators.
You can read the rest of the article here:
Good luck to Eric Schmidt in trying to reassure the Telcos that Google is their friend and partner. Just look at the list above.
It seems as if Google has not considered the strategic implications of its actions on wireless operators, and its other partners.
Nearly all of its actions in the list above do nothing to increase revenues for Google.
They are all outside of its core business: selling advertising related to search.
Yet those actions have put the wireless operators on guard. They can now take steps that can impact Google's revenues much sooner than Google's actions can improve revenues for Google.
This is a strategic blunder on Google's part because the wireless operators are in a much stronger position than Google.
- They have control over what network services they will support.
- They control which features they will support on phones on their networks.
- They can make alliances that are designed to work against Google's dominance.
The mobile market is very important to Google's future but so far, it has managed to upset a lot of strategic companies, Apple included.
Google forgets that the mobile market is not like the desktop Internet market.
The wireless operators saw what happened in the PC market where the dominance of Intel in hardware, and Microsoft in software, aggregated most of the value and profits.
PC makers were left with razor thin operating profits in single digits while Intel and Microsoft produced 60% plus profit margins.
That's why the wireless Telcos have resisted attempts by Intel to standardize the handset hardware, and from Microsoft to extend its dominance in software to the mobile phone market. They don't want a repeat of what happened in the PC market.
That's why there are a myriad operating systems and hardware platforms in the mobile space. They didn't let Intel and Microsoft turn them into "dumb pipe providers" and they won't allow Google and Apple to do it either.
Google will have to change tactics but it might be too late, it has put the entire wireless industry on guard.
- - -
Mr. Cowie is trying something new: for a flat fee of $25 a month, he is making his listings on Google stand out. Whenever his shop comes up in a search page or on a Google map, it is adorned with a bright yellow tag.
It's working pretty well:
Just type "skateboards in Houston" on a search engine, and his store will be among the first listed.
Mr Cowie loves the new service. And it's no wonder:
... buying search ads on Google -- got to be pretty expensive. Mr. Cowie, whose shop is just 1,000 square feet, found himself bidding for placement against deep-pocketed national chains, and having to spend $1,500 to $2,000 a month just to keep up.
What a great new business for Google. It replaced revenues of $1500 to $2,000 a month from Mr Cowie, with $25 a month.
But there is another aspect to this story. Google has begun to mix organic search results with 'sponsored' search results. Prior to this, the results were separated. Sponsored links were always on the right, and the main search results were 'organic,' they were listed on how relevant they were to the query -- not because of payment.
Take a look:
Google said it is collecting requests from communities interested in being among the first to try out an experimental high-speed network it is planning to build.
We're planning to build and test ultra high-speed broadband networks in a small number of trial locations across the United States. We'll deliver Internet speeds more than 100 times faster than what most Americans have access to today with 1 gigabit per second, fiber-to-the-home connections. We plan to offer service at a competitive price to at least 50,000 and potentially up to 500,000 people.
Google believes this will encourage developers to create new types of applications that aren't limited by current bandwidth speeds. But where else are there high concentrations of software developers? San Francisco/ Bay Area has the highest concentration and this is where Google should build one of these networks.
App development has to take place where there is also access to culture. Apps developed outside of a culture won't succeed. If Google builds its network somewhere remote, say a small town in Nevada, what types of high speed apps would they need there?
Better to build the network, and the apps, within this community, in San Francisco. This is where the apps can be best tested, within the Bay Area, a community with a very diverse culture.
I would think that diversity would be key for this endeavor. App development that is not connected to its culture will not succeed. (Could Foursquare have been developed in a rural community?)
You can vote for your community here. Google Fiber for Communities: Get involved
- - -
Also, isn't this showing that Google is inching towards becoming a Telco?
The American Civil Liberties Union says that a proposed alliance between Google and the National Security Agency is a frightening and dangerous combination.
The NSA--a component of the Department of Defense--is an intelligence collection agency with few effective checks against abuse and no public oversight of its activities. In the last decade, the NSA's vast dragnet of suspicionless surveillance has targeted everyday Americans, in violation of the law and the Constitution.
...The specific terms of the proposed deal are very unsettling. Google announced that they're asking the NSA to find "vulnerabilities."
What assurances do we have that a spy organization like the NSA would fix the holes they find and won't instead use them to tap into our personal data? Cybersecurity for the American people should not be handed over to a military spy agency that has a history of secretly exploiting vulnerabilities, not fixing them.
The ACLU makes some very good points.
What is interesting about this proposed alliance is that the two sides are still negotiating the terms. This must indicate that Google has not been happy about terms that the NSA has specified.
The NSA has a long history of spying on US citizens. And Google has more information on US citizens than anyone...
This can't be a good mix.
The ACLU asks that people send a message to Google that strongly opposes a deal with the NSA.
Here is a template for a message to Eric Schmidt, CEO of Google.
Associated Press reports: Spain's Telefonica considers charging Google - Yahoo! News
Speaking Monday at a press conference in the northern city of Bilbao, President Cesar Alierta said companies like Google use a lot of network bandwidth for free, something which was lucky for them but not for Telefonica.
...He says things had to change.
Foremski's Take: If ISPs, such as Telefonica, start charging Google they will also charge other companies providing web services. This will be bad news for thousands of startups worldwide.
Because Google can afford to pay the fees while startups cannot. This will be a huge tax on innovation.
Google, Yahoo, Microsoft, etc -- can afford to pay the fees. This will help their businesses because it will keep competition at bay.
Those companies will then be able to acquire smaller companies that don't have the means to scale their businesses, and eliminate large numbers of potential competitors. While they can do this already, the fees will speed up the process.
The Telcos are in a strong position because in every country they are government regulated entities, which provides a strong barrier to competition. How many new Telcos have arisen in the last five, or ten years? Consolidation has strengthened the current Telcos.
Here is another reason why GOOG should look at the strategic benefits of becoming a Telco or acquiring a Telco.
- - -
On BusinessWeek, Peter Burrows and Cliff Edwards report:
Apple (AAPL) is in talks with Microsoft (MSFT) to replace Google (GOOG) as the default search engine on its iPhone, according to two people familiar with the matter. The talks have been under way for weeks, say the people, who asked not to be named because the details have not been made public.
Foremski's Take: The story says that the discussions are due to " accelerating rivalry between Apple and Google."
I don't buy it. Yes, there is some increased rivalry but on the Internet, that's increasingly true of many companies. This is more likely a financial decision.
Apple receives tens of millions of dollars a year from Google for search traffic from the iPhone and also from users of the Apple Safari web browser. Microsoft is most probably using its cash hoard to muscle its way onto the iPhone, and maybe even become the default search engine in Safari.
We might see Microsoft do the same with Mozilla and bid to become the default search engine on Firefox.
The rivalry explanation doesn't hold water.
MSFT a larger business rival to APPL than GOOG. MSFT has a rival OS, it has Zune a potent rival to the iPod; it has a mobile OS that is on many cell phones; it has many competing applications.
Apple is paid by Google to provide search traffic and to provide prime placement for features on the iPhone such as the YouTube app (which cannot be erased by the user). Industry sources have said that Apple receives enough money from Google to fund most of its OS development.
Whether it gets those funds from Google or Microsoft -- it doesn't matter to Apple. This is purely a business decision.
Google's new found willingness to pull out of China because of hacking attempts by agents of the Chinese government doesn't make sense.
Why would the Chinese government seek information through hacking? It could give Google a legal order to divulge specific information and Google would comply, as it has done countless times in the past, not only in China but in other countries including the US.
Time and again Google's top executives have said they comply with all laws within each country.
Also, the hacking attempts were largely rebuffed. Google's security technology prevented the attackers from gaining much useful data.
David Drummond, SVP, Corporate Development and Chief Legal Officer reported:
Only two Gmail accounts appear to have been accessed, and that activity was limited to account information (such as the date the account was created) and subject line, rather than the content of emails themselves.
...accounts have not been accessed through any security breach at Google, but most likely via phishing scams or malware placed on the users' computers.
Leaving China does nothing to improve overall security, Google faces such attacks all the time, from every region.
The question remains is why now? Google could have taken a stand on human rights in China on many occasions in the past.
In a Tweet yesterday, Bobbie Johnson, a reporter for the UK based Guardian newspaper, pointed out:"Goog acted after its rights were infringed, not the rights of its users."
But even this doesn't explain why Google has found its backbone and made a stand now. It has to be because it has adopted a new policy, one that it tested last year in Korea.
And it's a policy in which Google is able to take advantage of the global nature of the Internet to thwart any national ambitions -- no matter how large.
We first saw this new policy at work last year in South Korea where the government has passed a new law that requires users posting video, or comments on large web sites to use their real names. Web site operators must collect and verify users names. The "real-name" law is designed to clamp down on government critics.
What did Google do? When the deadline arrived for the new law to be implemented, it blocked Korean users uploading videos or leaving comments on its Korean YouTube site. It told them to use YouTube sites in other countries.
[You can read my news report here: Google Is Facing Wrath Of Korean Government Over Snub Of Internet Law]
It neatly sidestepped the law by making use of the global nature of the Internet. It showed the limits of government power and its jurisdiction -- the Internet is global and beyond the reach of any one country.
Based on its Korean experience, Google is gaining in confidence that it can stick to its principles and stay clear of any repressive government policies, anywhere, by taking advantage of the global Internet.
It can stay beyond the reach of any government, including the US if it has to, by moving its HQ and its operations around the world. It has more than 50 data centers all over the world.
This is a watershed moment and one that governments everywhere must be paying close attention to because it demonstrates that when its citizens engage with the Internet, it is a world without borders and where building borders is futile. [China's "Great Firewall" is very porous.]
That's a frightening scenario for any government especially a Chinese government wary of any competition for the hearts and minds of its citizens. And the Internet is its largest source of competitive ideas.
With China, Google is making a bet that there is a business and moral value in an unfiltered Internet, an Internet that is larger than the borders of any one country. I'm pretty sure we already know who's right.
The larger picture here is that Google has realised it has an important advantage: it's not constrained by borders but governments are.
- - -
Hilary Clinton: Statement on Google Operations in China
Techcrunch reports that Google [GOOG] is about to acquire Yelp, the local restaurant and store reviews site for as much as $500 million.
Comscore puts worldwide traffic at nearly 9 million monthly unique visitors, and it has been growing fast - the company says its real numbers are more like 25 million monthly uniques.
Yelp has whispered that 2009 revenues will be around $30 million and are expecting $50 million or so in 2010.
Foremski's Take: If Techcrunch's figures are correct this is a very high valuation for Yelp. More importantly, it could signal a new strategy for Google, which so far, has avoided acquiring content sites.
Yelp, which depends upon user generated reviews, has done well in San Francisco and a few other large metropolitan areas. But with hundreds of reviews per restaurant, the reviews become less valuable because of their large number and because large numbers of people have different tastes.
The way to deal with this growing mass of information would be to use a search engine to sort through this huge pool of data. Google, which prides itself on its search technology, could simply unleash its search algorithms against Yelp and surface the best (or worst) reviews and help people make a decision. It doesn't need to own Yelp to do this.
Google is more likely going after the advertising relationship that Yelp has with local merchants. The local advertising market is the next big battle ground for e-commerce and the many billions of dollars spent on Yellow Pages type directories.
Google could use Yelp's advertising relationships to offer local merchants a range of other services such as their own web sites and buying local adwords to attract traffic.
This would also push Google closer towards eventually having to compete with Craigslist, which is by far the largest platform for local advertisers. Craigslist is a much larger site than Yelp, it is the eighth largest English language web site in the world.
Craigslist offers local store listings and reviews. And its services are free compared with Google, which offers an auction of local keywords.
If Google does acquire Yelp, the trend will be in place for it to eventually have to compete with Craigslist, a privately held company with about 30 employees but with tremendous amount of loyalty among local users and businesses, that Yelp cannot match.
All the recent debate about the Google index and newspapers, and Google's interest (and problems) in indexing out-of-print books, would all be a much different discussion if the Google index were run as a non-profit.
And it is rather ironic to note that Larry Page and Sergey Brin, the founders of Google, strongly believed that search engines should not be commercial enterprises.
Take a look at page 39 "Inside Larry and Sergey's Brain" by Richard Brandt (referral link).
Andrei Broder, who led the team that created the AltaVista search engine, the best of its time, talks about meeting Larry and Sergey. "When the discussion turned to the topic of making money from the technology, Broder found that Page had a profound difference of philosophy on the subject. "It was a very funny thing about Larry," Broder recalls. "He was very adamant about search engines not being owned by commercial entities. He said it should all be done by a nonprofit. I guess Larry has changed his mind about that."
Brian Lent, now CEO at Medio Systems, also ran into this obstacle when he met with the Google duo.
"The problem with the Google search engine at the time, Lent recalls, is that Larry and Sergey didn't want to commercialize it, and Lent was anxious to become an entrepreneur. Their mantra at the time was more socialistic than entrepreneurial. "Originally, 'Don't be evil' was 'Don't go commercial,'" says Lent.
When he was at Stanford University, Sergey Brin wrote a paper: "The Anatomy of a Large-Scale Hypertextual Web Search Engine."
In that paper, he argued against an ad-supported service as a corrupting influence. "Advertising-funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers," he wrote.
Wow. What if Google were a shared resource, a non-profit, with everyone having equal access to its index?! That would solve a lot of problems that Google is currently facing and that would be avoided if search was a shared resource -- instead of the Internet's dominant index being controlled and exploited by one company.
Could this be a possible future outcome? Could a non-profit search engine "out Google Google?"
If site owners blocked all commercial search with a robots.txt file but allowed a non-profit search engine, that would build a vendor-neutral index very quickly.
A non-profit search engine could support itself by licensing the index to various companies -- even Google, so that they can then apply their algorithms to rank the results according to their specialty.
After all, the value isn't in the index it's in how you present results from the index.
Maybe Google could gain everything if it were to spin-off its index into a non-profit? It would fulfill its founders' original ideals and Google could still be Google.
This would also block any moves by Microsoft to "buy" parts of an index and prevent access by Google or other search engines. Microsoft would have to compete on its ability to analyze the index.
Also, if the index were a commonly-owned resource, every site owner would be able to grant permission to different groups, to access its part of the index.
That's what's missing with the current system. You can either allow a search engine to index your site or not. You have little control over who can access your part of the Google or Bing index.
If the index were held in common, newspaper publishers and others, would have more options, and we could avoid search engine wars, and problems with incomplete indexes.
When the first iPhone came out two years ago I was astounded that there was no way to search its contents: Searching For Search On the iPhone
One year later, in 2008, after many updates to the software and new hardware: still no search function! It's taken another year to add search in the most recent update iPhone 3.0 in June. It's taken two years to add search.
That's not accidental it has to be deliberate. A year ago I wrote: Is Apple About To Launch Apple Search?
And it makes sense. It has its own browser and having its own search site makes perfect sense.
The browser and search service are essentially the operating system for our modern times. Apple knows all about owning the OS and that has been a winning strategy for Apple.
Take a look at how Apple would be able to quickly build a large search engine business:
1: The Apple fanboy market would leave Google in a heartbeat. No question. That's about 5 per cent of the computer market, and that's not counting the iPod, iPhone users.
2) All the users of Safari, the Apple web browser on Macs and Windows systems would be default users of Apple search.
3) Search is all about brand. Tests have shown little difference in the quality of search results between search engines. Microsoft rebranded its search service Bing and it now has traction. Apple knows "brand" very well and it knows how to parlay that expertise into new businesses.
Study: Good Brand Can Make Search Seem More Relevant
The study showed that when a searcher was given an identical result set across Google, Yahoo, Windows Live Search and an in house search engine, Google and Yahoo came out as more relevant. Why? Because of the brand of the search engine.
Despite the results pages being identical in content and presentation, participants indicated that Yahoo! and Google outperformed MSN Live Search and the in-house search engine.
One drawback for Apple is that GOOG pays Apple for placing its search engine in the search bar in Safari. Industry whispers report that this payment is enough to fund development of Mac OS X.
However, that revenue is jeopardized by Google's announcement of its Chrome browser, and also there is competition from Google's ChromeOS, and its Android cell phone software.
While the industry focuses on Microsoft and Google competition, Google and Apple are becoming ever more competitive with each other. That's why Eric Schmidt is leaving Apple's board of directors.
- - -
I went to Google's biennial Searchology event this morning. This is where top execs unveil their latest advances in search.
This year there was a lot of new stuff, and a lot of it revolved around understanding the semantics of search, to understand the meaning of what users want, rather than what they typed. If you misspelled something, or used a seemingly cryptic search term, GOOG execs said it was their responsibility to figure out what you meant by that.
For example, a search for "sfll playoffs" could mean a misspelling of "nfl playoffs" or, more likely, "San Francisco Little League playoffs." Google is using location data and other factors as part of what it calls "rocket science" search technology to try to determine what the user really meant -- not what they typed.
Udi Manber, vice president of core search, spoke about how difficult it was to determine the semantics of user queries, how to understand what was meant.
The demos they showed were impressive. But, how will Google sell advertising against an increasingly semantic set of results? Currently, Google sells keywords to advertisers, and those keywords trigger advertising served up on the results page.
Keywords are very specific. How will this work when the results that people want, aren't related directly to a keyword they typed? Is there such a thing as a "semantic keyword?"
If Google tries to guess what the advertiser wants and what the user wants, it creates more opportunities for a mismatch. Since Google gets paid on clicks, that could be a problem if it misses the intentions of both user and advertiser.
How do you sell "semantic" advertising?
[This is a guest post by Todd Hogan founder of Surchur, which specializes in real-time search. Please see http://blog.surchur.com/. If you'd like to offer a guest post please contact tom(at)siliconvalleywatcher.com.]
- - -
By Todd Hogan
This past week you've seen the mighty Google show its interest in the upstart Twitter. Now, whether those discussions really were as serious as many at first believed, the takeaway is the same: what is happening "right now" is becoming increasingly important to many internet users. One important trend growing from this is the need to search through all of the "right now" content that is being generated every moment and make it accessible. Our team at surchur.com is one group trying to make that happen.
But who cares about what's happening "right now?" Here are 5 profiles of users that are very interested in real time search:
Foremski's Take: What would happen if somebody developed a better search engine than Google? Would it be able to unseat Google? And what would happen to Google?
Nothing would happen to Google. Because if you could build a better search engine GOOG could monetize it way better than you or anyone else.
GOOG would partner with the better search engine because it has the monetization platform, it has the ad delivery network, and ad sales representatives in nearly every country.
The market has moved on--it's not enough to be better than Google in search, you also have to have be better in all aspects of the business.
That's what Microsoft needs to focus on, building a great monetization platform for search - search alone is not good enough even if it's great.
Yahoo said it had agreed to let Google put search ads on its site in what it called an $800 million annual revenue opportunity that would boost cash flow by $250 million to $450 million in the first 12 months.
Yahoo's ads and Google's would be pitted against each other in an auction style process that could make a deal easier to pass regulatory approval.
"Yahoo is being a reseller of Google whenever it makes sense and that is likely to be a lot of the time given how much more effective Google Web search ads have proven to be," Global Crown Capital analyst Martin Pyykkonen said.
Monday morning I was down in Mountain View at the Googleplex, GOOG's HQ for a briefing on new search initiatives and to find out about Google Health - a potentially wonderful and problematic service.
[For the first time, I was using Twitter to "live blog" from the event. Twitter is particularly well suited to such an activity, plus my "Twits" served as a decent note taking device for this post. I'm tomforemski on Twitter if you'd like to follow me.]
A planet sized wiki...
On search there was a lot said about local search and how difficult it is to do because place names are not standardized and some facilities share the same address and phone number. GOOG is very excited by image search because there are about 100 billion photos taken every year and Google loves new content (after all why go back to the Internet if there is nothing new?)
Google is trying to create a 3-D virtual earth. Its satellite and street level imagery are lacking. What it wants is a 3D rendition such as the one for San Francisco, which is very impressive. But I think that Google will have trouble getting users to continue to create that 3-D content because the novelty will subside so it'll have to think up some games and incentives.
Google likes images and the geo-tags that can go with them, but again, this relies on the continuing goodwill of users to upload and tag images, etc.
What this adds up to is a wiki-earth that users create using Google tools, or other tools, and also annotate and tag the locations. It would be a massive project and on top of that it would require fresh content - good luck with that. I just don't see legions of users willing to update Goog's database time and again, and again. Businesses will-- but that's advertising and that doesn't require payments to Google, (there could be a shot in the foot here ...).
Google Health very important...
Google Health is interesting and I would say it is Google's most important business launch since its AdSense/AdWords text-link ads. Not only is this a great way to integrate health data from multiple sources by having the individual do it but it also enables Google to access billions of dollars in pharma and medical services marketing.
Government regulations on health data make it very difficult for medical practitioners to pull an individual's data from many sources. But a Google Health user can consolidate their medical records in a highly secure Google database then authorize sharing that data with their doctor or other third party, and even allow that data to cross-borders. How about MyIndianMD.com...? (Hold on one second while I visit GoDaddy...)
Google Health could reduce health care costs because tests won't have to be repeated, and individuals could choose not to have expensive procedures just because their doctors are trying to cover their legal liabilities.
The problems with this approach is that the individual is being asked to make choices which their doctors would be best at making but because of legal and other issues, they cannot. Similarly, Google has to be careful not be providing medical advice. It gets around that - it provides users with "choices and options."
I spoke with Martin Harris, M.D and Chief Information Officer at Cleveland Clinic, one of the beta test partners for Google Health. He said that one problem was that computer literate users would be the first to benefit. "It's part of what I call the Internet divide," he said. "Ideally we should be able to let people access their Google Health accounts through their TV set top box or through the phone. That will come soon."
I spoke with a couple of people on the Google Health team and they said users would in the future be able to designate "delegates" to administer their Google Health records on their behalf so that the elderly could have their families help them.
Building businesses on top of Google Health . . .
There is also a great opportunity for a range of third party services to be built on top of Google Health and it's open API. Google for example, is offering an application that allows people to monitor how much they walk and will give $100k to charity.
I can think of linking gym machines to my Google Health record, what I ate that day, even what I ate at a restaurant automatically uploaded with calories and vitamin info. How much I slept, etc. This could become many people's home page.
There are some obvious potential problems if insurance companies or employers seek permission to view health records. What about potential spouses opening their hearts . . . and their medical records to each other?
How to monetize search, Google Health . . .
Nothing was said about how Google would make money from the improved search or from Google Health.
I asked about Google's search and if it improves then fewer pages are served. . . and therefore the performance of Google ads has to improve just as fast as search improves.
I was told that the advertising teams are ahead of search, and with better search there will be more search, therefore more pages will be served. I don't see that as being true.
It would have been good to have a presentation about the business side of Google's search and Google Health.
Here is further discussion of the business opportunities: The Google Health Problem...And The Quest For Pharma Gold
- - -
Notes: Google did a nice job pulling together top media and allowing good access to its executives and also to Google Health partners. Its PR people didn't get in the way but facilitated interviews etc. This meant that journalists could each have a chance in coming away with original quotes and material that wasn't part of the general presentation. I always appreciate the opportunity to come away with some unique content.
I recently visited Powerset to get a look at the company's much buzzed-about search technology based on the semantic understanding of language. Powerset is not the next Google but it does have an interesting technology thanks to its dozens of noted academics in lingustics and artificial intelligence.
The first product is a hand-optimized use of its technology to search Wikipedia along with Freebase, an open-source database of information. The results are fascinating in that the search results provide a new way to navigate through subjects. You can see what Powerset calls "factz" similar to facts, which are terms drawn from the subject matter dispersed throughout the two databases, that highlight and summarize complex subjects.
It takes a lot of computing power and time for Powerset to process a web page and it generates a lot of metadata. Thanks to Moore's law and Powerset's computing architecture, this processing time will continue to shrink but right now it cannot be used to process the entire web in a similar way to that of the major search engines because of those constraints.
There are about 2.5m pages in Wikipedia which is a lot of information and a good size to demonstrate Powerset's technology. Wikipedia was chosen because of its open license.
I can think of many uses for Powerset's approach such as improved ad serving based on meaning rather than contextual keywords.
What will be Powerset's next product? I wonder how many types of searches require a semantic understanding? It'll be interesting to find out. In the meantime I encourage you to take Powerset for a spin.
Here is Powerset's news release:
. . . An end to Enron and other corporate scandals?
Friday I met with Michael Lynch, CEO of Autonomy [AUTN], the second largest European software company with a market capitalization of more than $4 billion.
I wrote about its new software that can identify illegal content in corporate communications such as blogs, emails, any document, and phone recording, and even in video files.
The Autonomy Information Governance (AIG) software makes sure that a corporation complies with huge numbers or regulations, it also monitors any document being created by an employee, including a blog entry, an email, even audio and video. It checks in real-time to see if any laws or company policies are being violated.
AIG also makes it quicker and cheaper to find documents related to a lawsuit.
Mr Lynch said that technology has made it easier for people to engage in illegal activities and do it on a much larger scale.
When a corporation is sued, it often has just 90 days to produce huge quantities of documents, emails, etc, related to the lawsuit. It faces huge fines if it doesn't meet the deadline. The software understands the meaning of documents and can reveal which documents should be reviewed by lawyers.
Companies being sued or investigated, must lock-down all their documents so that they can be searched for evidence. This has to be done even if a company has not been sued or investigated but has a reasonable chance of that happening.
"With 14,000 separate records retention regulations out there and the complexities and costs being incurred just trying to comply with legal hold requests, a company doesn't have the capability to manage this without advanced technology," said Browning Marean, a top lawyer, and partner at DLA Piper US LLP.
"I was on a panel recently and one of the panelists said that the Internet is different and people will behave better. I don't think it will change human nature at all. Human nature hasn't changed in hundreds of years, if you read Shakespeare. And if you look at Roman and Greek times, human nature is pretty much the same thousands of years ago."
"It's seems really arrogant for people to say that in this one generation human nature will be different," Mr Lynch said.
The AIG software also carries out document retention policies, which in the corporate world of double-speak refers to: how soon can we shred or delete this document just in case it comes back to bite us.
Autonomy is trying to bolster its position in the multi-billion dollar e-discovery market, which is lead by Recommind, based in San Francisco. You can read more about the software here.
Foremski's Take:There are some good and bad aspects to this software. The bad is a big brother type use for it. I can imagine the Chinese government using it to monitor Internet use and communications in real-time. You'd get a message pop up: "Report to your local police station immediately - you just committed an illegal act." (The Chinese police sent out text messages to Tibetan demonstrator's cell phones to report to the police.)
It could be used to restrict blogging. A lot of people tell me that large corporations are scared of blogs violating a regulation and so every corporate blog entry has to be run through lawyers-- it has to be "lawyered." This can take time, days, even weeks.
Paradoxically, I think AIG could be used to clear a blog post in real-time and could thus increase the amount of good, legal information that company workers can share in public. Either way, it automates some of the tasks of a lawyer, and that means less need for lawyers, and smaller legal bills for corporations.
Less lawyering, means lower operating costs, which maximize share holder value, and that's what corporate officers are required to do. It's the ultimate SEC regulation.
Please also see: Autonomy CEO says tags don't work
News.com's Charles Cooper: What's interesting is what we say it is. Really?
Last week I attended a briefing by Autonomy, a company based in the United Kingdom and San Francisco. On Monday, Autonomy will announce a product designed to assist companies with governance compliance. This likely will be a big deal for IT administrators and law firms that are scrambling to enact internal information management policies in the wake of the subprime mortgage and credit crisis.
Please see: Mike Lynch on the meaning of meaning based computing.
About a year ago, Internet mogul Barry Diller appointed Johnny C. Taylor Jr. president and CEO of a new business group at his IAC corporation called Black Web Enterprises Inc. Mr Taylor was previously head of human resources at IAC, which owns large Internet properties such as Ask.com, Match.com, LendingTree, Ticketmaster and 54 other leading Internet brands.
Black Web Enterprises is the first business to be created by IAC rather than acquired. I met with Mr Taylor recently and he explained the background to tomorrow's launch of a new search site: Rushmore Drive targeted at the Black Internet user.
"We felt that the Black community could be better served by the Internet and so we carried out a lot of focus groups. We found that Black people in the US do a lot of search, they look at news, and they look for jobs. Out of that work we decided to launch a search site called Rushmore Drive that offers mainstream search but also something extra for the Black community." It also has its own editorial department staffed by journalists, and a jobs site where people can also network similar to LinkedIn."
Is "Black" considered a good designation? "Yes, Black designates a large number of people. Barry Diller asked why we didn't use the term "Afro-American. First of all, "afro" is a hair style and secondly there are many people such as those from the Caribbean that don't identify themselves as African-American."
IAC estimates that there are 40 million African-Americans and an additional 20 million people who fall under the designation "Black."
Suspicious of a Black web...
I asked if there was a suspicion by the Black community about a search site focused on them, because there have been many cases where minorities paid more for cars, loans, etc.
"It's interesting. When we ran focus groups I was surprised that there was a lot of suspicion from people about a large corporation targeting the Black community, because I don't really pay much attention to being Black myself. I was watching a lot of hostility through a two-way mirror. But when I stepped into the room and people could see me, they immediately changed their attitude and were much more welcoming of the idea. They also told us that they want great search, they don't want just a bunch of Black targeted search results."
Not a Black Ask...
Mr Taylor said that the site is not a Black version of Ask.com. It provides high quality search results plus more. The term Black is hardly used on the site and it won't be used in marketing or advertising.
Radio ad spots will be targeted at Black audiences only in the sense that a radio station has a Black audience, and not in the advertising message itself. "It's important to be inclusive," Mr Taylor says.
The search site's name Rushmore Drive is based on the address of the business group, located on 1115 Rushmore Drive, in Charlotte, North Carolina. "It's like Rodeo Drive, it is a classy name." Classy is an important quality, Mr Taylor says, which is why the right type of advertiser is important.
"I could have tripled my first year numbers from just one advertiser that wanted to be part of the site but I turned down the business because I did not like the association with the product." Mr Taylor declined to name the advertiser (I would guess it could be a malt liquor brand) and said he wanted top advertisers such as Gucci and Ferragamo.
A Black Linkedin...
Part of the job search at Rushmore Drive is a network feature similar to Linkedin. "One of the first things Black people ask each other is which church do you go to, and what professional associations do you belong to," Mr Taylor says. Those two answers are key to the networking feature on the site.
News on the site will come from mainstream sources but there is also an editorial department with Black writers that will engage users in discussions around topics that are of importance to that community.
"For example, the recent death of R&B singer Sean Levert was a big deal in the Black community but it didn't make much news elsewhere. We would promote that news and also discuss why he was in prison for failing to pay child support. We would look at the issue of paying child support in the Black community and not shy away from uncomfortable issues," Mr Taylor says.
IAC is investing many millions of dollars in this first home grown business. "I can't say how much we are investing. Barry Diller told me to make sure that we build a quality product. I can tell you that it's not cheap setting up a search site, I've got a large group of engineers and also a large group of marketing people."
It is certainly new territory for a search site. And maybe it will spur the development of other search sites focused on specific types of users. For example, I recently wrote about the possibility of an Apple branded search site.
Mr Taylor's challenge will be to demonstrate how Rushmore Drive can provide a better search experience that is inclusive while at the same time exclusive in its strong focus on the 60m strong Black community.
- - -
(Thanks to Steve Gillmor for the headline inspiration.)
Latest results from Comscore show a slight drop of 0.2 percentage points in Google's share of search in December 2007 compared with November 2007 to 58.4 percent.
Second place Yahoo gained 0.5 percentage points to 22.9 per cent over the same period.
Third place Microsoft remained at 9.8 per cent.
Fourth place Times Warner gained 0.1 per cent to 4.6 per cent.
Ask lost 0.3 percentage points to 4.3 per cent.
Americans conducted 9.6 billion searches at the core search engines, representing a 3.9-percent decline versus November. With many Americans traveling and spending time away from home during the holidays, search activity typically experiences a seasonal decline during December. Google Sites saw 5.6 billion core searches during the month, while Yahoo! Sites recorded 2.2 billion.
Additional info: www.comscore.com
I recently met with Aggregate Knowledge and Proximic, both companies have technologies that serve up relevant content to web site visitors, especially on media sites. Relevant content means visitors are likely to stay on the site and that provides another chance to try and monetize that visit.
- Aggregate Knowledge, based in Silicon Valley, could be called "Aggregate Web Analytics" because it watches the behavior of web visitors and then guides future visitors to the same pages that most other visitors visited. It can also guide visitors to higher margin pages that the web site owner is seeking to monetize.
"Most sites have click through rates of abut 2 percent, with some of our clients we can get 22 per cent click through rates," said Paul Martino, CEO of Aggregate Knowledge.
Kleiner Perkins is the main investor in the company. Randy Komisar, a partner at Kleiner Perkins Caufield & Byers, said that the web is changing. "It is not about search it is about discovery. Search is only good if you know what you are looking for, 'discovery' means serving up content that is relevant and that you didn't know was there," said Mr Komisar. (Kleiner Perkins was an early investor in Google.)
- Proximic, founded in Munich, Germany tells a very similar story of being able to serve up relevant content on media sites. It claims a radical search technology that indexes pages by "patterns" not by links or keywords. Pages with similar "patterns" represent similar and relevant content.
Philip Pieper, CEO of Proximic, says that the beauty of their approach is that their index is easily scalable and very compact. "We have even written our own disk operating system, to speed up searches. We believe that our technology can help media companies better monetize their online operations because we can provide relevant content," said Mr Pieper. The index can be held in just a few gigabytes of memory, it could be loaded onto a flash drive. Proximic says that its search technology could be run on a bank of iPods.
Proximic is working with Nature magazine to boost its online revenues, and also with the British newspaper "The Independent."
Foremski's Take: They may get there in different ways but Aggregate Knowledge and Proximic serve up similar messages to web site owners. They promise to improve online revenues by making the content on those web sites more relevant to visitors, including the advertising.
And neither of them collects personal data on visitors. This is a key advantage because privacy issues are only going to get bigger and messier--you don't want to be in a business that relies on collecting and using personal user data.
Media sites in particular, need all the help they can get to monetize their online operations. However, making sites more "sticky" produces value only if the site publisher can monetize those extra clicks. And there aren't many media sites that are good at that. Even if media sites doubled their online revenues, it is still not enough to support their businesses as they transition from traditional media to digital media.
The success of both companies is tied to the ability of their customers to figure out their own online business models. Yes, Aggregate Knowledge and Proximic can help in that regard, by serving up advertising that pays better, and is more relevant to visitors, but that will likely not be enough to help media sites make money, and survive.
Working with online retailers to help optimize their sites is likely to produce longer lasting customers than in the media sector.
- - -
Please See Information Week: Aggregate Knowledge Aims To Be A Mind Reader On The Web
About a year ago I met with Bob Chandra, founder of GrayBoxx. At the time, GrayBoxx was still in very stages of development around the idea of getting recommendations for local businesses from the contact info in people's address books and telephone logs.
Mr Chandra says,"I already know who my favorite businesses are because they are the ones I call and they are the ones whose phone numbers I already have." That's a great start to finding out the recommendations of other people, without any tagging or reviews or community action to create that content.
GrayBoxx also uses other sopurces of data to compile an impressive database of listings. It has 185m recomendations compared with 3m for Yahoo Local.
It is still in stealth mode but is building up its user base with plans for a broad launch in early 2008. The Sierra Ventures backed company says it has developed its own version of GOOG's PageRank, but one that is tuned to the recomendation of local services.
The local search market is extremely competitive but it is an area that is poorly served. Yes, some sites are good for say, restaurant reviews, while others for automotive repairs. None offer a comprehensive service, at least none so far...
GrayBoxx just announced a board of advisors that includes Rajeev Motwani, one of the top experts in search and co-developer of PageRank. It's all about the algorithm...
Terry Semel, the embattled head of Yahoo! has been ousted and replaced by co-founder Jerry Yang. The move comes in the wake of shareholder criticism of Yahoo! in its bid to keep up with Google (GOOG).
Mr Semel will remain as a non-executive chairman and advisor.
Susan Decker has been appointed president.
Here are details from Yahoo! and SVW analysis will soon follow:
Google said searches will now combine all other types of results such as video, images, books as well as web pages.
It is a move away from silo search engines that focused on video, images, etc. Marissa Mayer, VP Search Products & User Experience, said that it was confusing for users to pick from search engines. "You almost needed a search engine for search engines."
Foremski's Take: This is bad news for the huge SEO industry because there is less space to get web pages into the first page, new opps for video placement. The focus now is on the first two results...
Over on New Rules Communications I was writing about Google's ad networks and the bad economics for media companies.
I noticed that there is an interesting conflict of interest emerging at the heart of Google's business model.
Please check my reasoning:
Google makes almost all of its revenues from two ad networks:
-AdWords: Customers advertise on Google web sites.
-AdSense: Customers advertise on Google partner sites, which includes many media companies.
Google's revenue in 2004 was about evenly split between AdWords and AdSense.
Since then, Google's revenue from its own sites has grown by 24 percent to 62 percent of total revenues. AdSense has fallen.
And this makes sense because Google makes far more money from its own sites than from partner sites. It is better for Google's shareholders that it channel more of its revenues through its own sites because:
-Google gives back about 80 per cent of AdSense revenues to its partners.
-It keeps all of its AdWords money.
And this is where it has a mammoth conflict of interest:
Which advertising network should Google invest in?
-A dollar invested in its AdWords produces far more profit than invested in AdSense, its partner network. Management has a fiduciary duty to its shareholders to maximize profits.
-Google can boost overall profits by undercutting AdSense at anytime it wants, say by offering a discount on AdWords compared with AdSense. A 20 percent discount on AdWords would make more money for Google than the corresponding loss of business through AdSense partners.
-Google can undercut AdSense in other ways, and is already doing it, by investing in technology that improves AdWords conversions over AdSense. Google can apply technologies to its own sites that make them more efficient at selling ads. It can't do that with partner sites. And partner sites don't have the resources to improve their advertising conversions at a similar pace.
Which means AdSense revenues for media companies will continue to fall because AdWords is more efficient.
Media companies that partner with Google in its AdSense program do it because they don't know what else to do. The economics of partnering with Google are poor and the relationship is unsustainable because of the inherent conflict of interest.
Why Would GOOG Maintain AdSense?
-There are strategic purposes, it forces media companies in its network to compete with its far more profitable business model which weakens them as potential competitors.
-Also, it keeps third-party sites out of rival ad networks.
-AdSense is a great "cookie jar" because if GOOG ever needs to meet its numbers for its quarter, it can push more ads through its own sites rather than through partners.
UPDATE: Independent Advertising Network Advantage
Independent advertising networks, which don't compete with their publisher partners, such as Blue Lithium, Federated Media, and others, will be able to attract partner sites away from Google because they can invest in technologies to improve revenues for the entire network.
But how much freedom do large publishers have in leaving the Google network? Some have contracts with Google that could tie their hands for years.
FAST Search and Transfer, the leading European search firm, introduced a search based business platform for media companies and said that search traffic from its customers has surpassed Yahoo! and will overtake Google in 2 years.
The Norwegian based company said its FASTMedia software platform is the first to match content to individual users, in addition to supporting all types of online advertising models. The suite of applications is designed specifically for media companies and enables them to run their own ad networks.
FAST said that its top 35 media customers already generate search traffic that exceeds that of Yahoo!.
The market research firm IDC estimates that 70 per cent of search queries do not come from search engines.
Instead, people are choosing to bypass search engines and are going
directly to their preferred information sources, retailers, and other
websites and searching there.
FAST recently completed a survey of its top 35 media customers:
Together this "FAST Media Network" is generating as much search traffic as Yahoo!, with search traffic growing more than twice as quickly. At the current pace, searches within the "FAST Media Network" collectively will surpass Google in two years...
FAST wants media companies to "turn search from the disrupter of their worlds" and use the technology to grow their businesses.
For example, newspaper publishers have stood by and watched Google and Yahoo sell massive amounts of advertising based on their content without any direct benefit--unless a reader clicks through to their news site, and then happens to click on a Google or Yahoo text ad.
FAST wants to cut out the middle-men by selling search-based business software to media companies to run within their own data centers and operate their own adverting businesses.
Global Media Warming
The competition is for a very lucrative business--running massive advertising networks for global media companies. The search giants keep about 20 per cent of total advertising revenues.
But this revenue share is skewed towards a handful of large media companies with the clout to negotiate better deals. Most publishers receive far less, between 60 and 40 per cent of ad revenues.
The FASTMedia software supports pay-per-click, contextual, classifieds and all other types of online advertising services, plus self-service, and unique features. It's "Featured Content" technology matches content on-the-fly with online users' searches. Publishers can use this for targeted premium content services.
FAST Search and Transfer, a leading European enterprise search company, said it will build an advertising exchange for its media company customers, who between them have an online audience that rivals Yahoo! in size.
The move will help to establish it as serious contender in the battle for large media advertising accounts. Yahoo and Google have recently signed big ad distribution deals with some of the largest newspaper, radio, and TV companies.
The difference with FAST's approach is that media companies use FAST software to run their own advertising networks within their own data centers. And they get to keep all the revenues instead of paying advertising networks such as Google and Yahoo as much as a 40 percent share.
FAST's advertising exchange would link together media companies that use its advertising network software. The value of the ad exchange is in providing publishers with opportunities to sell excess inventory and gain the liquidity of a much larger network.
There would also be opportunities for advertising alliances and new types of ad services between publishers.
The FAST software can be configured to run many types of advertising models, such as pay per click, impressions, classified, and auction based models. It includes self-service features. Customers install and run the software within their own data centers rather than using third party networks.
From its founding in 1997 in Norway, FAST has been steadily building a strong name in enterprise search within global corporations, It has about 800 staff in the US, and about 200 in Norway, with offices around the world.
Autonomy in the UK, is the market leader in this sector but FAST has managed to win some large customers and also large media companies. IBM and the New York Times are among its search customers.
It's expansion into advertising software builds on its core search technologies, which are used to match advertising with content and users--the same as Google, Yahoo, and Microsoft advertising services.
Don't expose the customer
But using Google AdSense or Yahoo Publisher Network, not only reduces revenues, it exposes customer relationships that the ad network giants can target with additional services. Their publishing partners get none of this extra revenue.
"We think that newspapers and magazine publishers should keep more of their advertising revenues, and their customers, " John Lervik, CEO and co-founder of FAST, told SVW during a recent visit to San Francisco.
Mr Lervik said that FAST will also partner with some of its media customers in joint ventures around online advertising. One of these is a joint venture with Softbank in Japan.
The details of the FAST media enterprise software suite are under embargo. It builds on its FAST AdMomentum product.
Recently I wrote "Is Search Broken" about the amount of help search engines need to find and index web pages. Now Matt Cutts, GOOG's chief evangelist in the blogging community, has asked people to report web sites which are selling paid text links.
Those web sites would then be given a lower pagerank and might even be banished from GOOG's index.
It seems a bit rich for Google, the world's biggest seller of paid text links, to ask for help in identifying sites that sell paid links.
Threadwatch.org, a leading site about search, weighs in:
If Google doesn't CLEARLY mark their own paid links, encourages publishers to blend them into content, and doesn't police their own network, why do they think they have the right to police other sites?
Paid links can confuse Google's pagerank system which pays attention to links in web pages to help find relevant search results. Mr Cutts says that Google has some technology in the works that will identify paid links.
In the meantime, he wants webmasters to clearly identify paid links, and also content that has been paid for.
You could put a badge on your site to disclose that some links, posts, or reviews are paid, but including the disclosure on a per-post level would better. Even something as simple as “This is a paid review” fulfills the human-readable aspect of disclosing a paid article.
Does this mean if a magazine publisher pays a freelancer to write a review and to mention specific companies this has to be clearly marked? Does it make the review less trustworthy or less worthy of being indexed? There's a potential Pandora's box of issues here if you start applying this to mainstream media...
Over at Threadwatch.org, the paid links issue continues to rankle:
The more I think about it the more I realize why Google doesn't like the various flavors of paid links. It has nothing to do with organic search relevancy. The problem is that Google wants to broker all ad deals, and many forms of paid links are more efficient than AdWords is. If that news gets out, AdWords and Google crumble.
Mike Lynch, the CEO of Autonomy, the UK enterprise search company, was in town this week (video is coming). I used to meet with Mr Lynch regularly when I was at the Financial Times, and when the dotcom boom was in full swing.
The dotbomb affected Autonomy along with tens of thousands of other companies, and Mr Lynch's visibility suffered too. So it was good to catch up with Mr Lynch and we joked that the "bubble" was back (it's not, of course).
We chatted about some of the trends in Autonomy's space, which is how to deal with unstructured data--estimated at about 87 per cent of all data.
Autonomy's technology uses statistics to find correlations between data, documents, video, and audio. It doesn't need artificial intelligence to understand the connection between data--it understands the probability of that data being related.
Autonomy uses Bayesian probability developed by Thomas Bayes, an 18th century British Presbyterian minister. The advantage of this approach is that Autonomy can find stuff without knowing any keywords or tags or taxonomy--it can determine the taxonomy on the fly.
During a presentation, Mr Lynch slammed the popular practice of tagging web content and says that it won't help to organize information. Mr Lynch quoted an essay by Cory Doctorow, the science fiction writer, titled Metacrap. "Tags don't work because people lie, they are lazy, and they use different tags. And there is a huge amount of information that will never be tagged."
I agree, and I resent the work in having to tag everything (See: Is search broken?) But, tagging does work to some extent, and in some applications.
Autonomy's technology could be used to improve "tagging." Often, it is not clear what tags to apply--technology such as Autonomy's could help identify the appropriate tags. For example, are the Technorati tags at the end of this post the right ones to use to associate this post with other, similar posts?
Earlier this week, Autonomy introduced a new product called Virage ACID (Automatic Copyright Infringement Detection) which uses its technology to search through video images. It can automatically detect copyrighted videos.
Being totally independent of media format, ACID can not only detect whether distributed video infringes copyright, but also whether audio content ripped from a copyrighted video or audio track that is overlaid on legitimate video has been uploaded.
Seems like it was designed for YouTube...
Under revisions announced late Wednesday, Google promised to wrap a cloak of anonymity around the vast amounts of information that the Mountain View-based company regularly collects about its millions of users around the world.
Google believes it can provide more assurances of privacy by removing key pieces of identifying information from its system every 18 to 24 months. The timetable is designed to comply with a hodgepodge of laws around the world that dictate how long search engines are supposed to retain user information.
Google to Adopt New Privacy Measures By Michael Liedtke, AP Business Writer
Authorities still could demand to review personal information before Google purges it or take legal action seeking to force the company to keep the data beyond the new time limits.
Nevertheless, Google's additional safeguards mark the first time it has spelled out precisely how long it will hold onto data that can reveal intimate details about a person's Web surfing habits.
I hope it is not the same type of unidentifiable data that AOL released last year, which didn't take much work to identify some users.
The most compelling content on the Internet, by far, is AOL's release of search terms linked to individual users. This is a glimpse into the human condition that goes way beyond anything else we have seen, beyond Dostoevsky, Dickens,...
TOKYO (Reuters) - The online collaboration responsible for Wikipedia plans to build a search engine to rival those of Google Inc. (Nasdaq:GOOG - news) and Yahoo Inc. (Nasdaq:YHOO - news), the founder of the popular Internet encyclopaedia said on Thursday.
Wikia Inc., the commercial counterpart to the non-profit Wikipedia, is aiming to take as much as 5 percent of the lucrative Internet search market, Jimmy Wales said at a news conference in Tokyo.
"The idea that Google has some edge because they've got super-duper rocket scientists may be a little antiquated now," he said.
Describing the two Internet firms as "black boxes" that won't disclose how they rank search results, Wales said collaborative search technology could transform the power structure of the Internet.
Interesting. I've been writing about people-powered search compared with machines and algorithms:
Eurekster seems to be well positioned in this area already. I like what they are doing with Swicki, which uses people-powered search. It has a very easy to interface and a ton of management tools. I set up this "Search Silicon Valley" Swicki in just a few minutes:check out the Search Silicon Valley swicki at eurekster.com
Microsoft thinks that it is - at least, it concedes that MSN Search is not doing the trick (Google market share: 53.7%; MSN marketshare: 8.9%). So at TechFest - Redmond's three-day chance to show off to reporters and employees the neat tricks MS Research is working on - the word of the week is "search," says the Times' John Markoff.
Lili Cheng, a user-interface designer for the Windows Vista operating system, showed off a new service called Mix that will allow Web surfers to organize search results and easily share them. She said Mix would be released in six to nine months.
A second tool demonstrated, called Web Assistant, is intended to improve the relevance of search results and help resolve ambiguities in results that, for example, would give a user sites for both Reggie Bush and George Bush.
Microsoft is not the spiffiest when it comes to product names, but these widgets are not products yet.
Personalized Search compares web-search results with an index of content on the user's hard drive (that's generated by Desktop Search, quite similar apparently to Google Desktop). It does a neat trick, though.
Susan Dumais, a veteran Microsoft search expert, demonstrated the effectiveness of the program by searching for Michael Jordan. By culling through local information on her hard drive, the program was able to discern that she was interested in finding the Michael Jordan who is the machine-learning expert at the University of California, Berkeley, not the basketball player.
Search in the future will look nothing like today’s simple search engine interfaces, she said, adding, “If in 10 years we are still using a rectangular box and a list of results, I should be fired.”
About 7,000 employees participate in the event as well as reporters and business partners. Rick Rashid, head of MS Research, originally opposed the festival idea but he's now a fan.
“We realized we were clearly tapping an underserved community,” he said. Now the company uses the event to move technology from its research division into products.
That includes monitoring which employees visit which lectures and booths and looking for patterns, he said.
(Thank you all for your comments on this topic of search...)
The holy grail within search is to be able to serve up the right search result from one user click.
The "I'm Feeling Lucky" button next to the "Google Search" button constantly tests this ability. One day it might very well be the only button found there because the underlying search technologies will be hugely improved.
And researchers within the search engine communities will be widely recognized for their groundbreaking work--I'm certain that there will be future Nobel prize winners among them.
But improvements in search, through the development of better algorithms and IT architectures, will very likely be of lesser value over the next few years compared with what can be achieved developing "people-powered" search technologies.
It can be seen in sites such as Digg, which is a better news aggregator using people-powered technologies, than machine-powered Google News, which states near the bottom of its page:
"The selection and placement of stories on this page were determined automatically by a computer program."
It is not just at Digg, there are "people-powered" search efforts all across the globe, involving tens of millions of people, laboring every day to help improve the search experience. This is done by adding tags, site maps, headlines, etc, -- they are creating ever larger amounts of valuable search metadata about content.
And we are just at the beginning of this trend. It is a markedly new phase in the development of search, one best described as "people-powered and technology-enabled." As opposed to mostly "machine-powered" through algorithms and servers.
What will be interesting to see if the people-powered component outweighs the value the machine-powered component of a search service. Because that would determine where the competitive edge is to be found. Is it in the algorithms or in the people?
And it begs questions such as: Is GOOG's lead in machine-powered search technologies large enough to keep it ahead of competitors harnessing people-powered search?
Search engines say they use complex algorithms to help users find exactly what they want. Google's "I'm feeling lucky" button (btw, does anybody use it?), right below the search box implies that very thing.
The legions of top Ph.Ds working for the search engines publish oodles of scientific papers on complex mathematical concepts related to search.
It all looks very impressive but it seems to have more to do with contributing to the mythology surrounding search--that it is very complex and scientific--than to the actual reality of how search is done.
From my vantage point as an online publisher, it is clear that search is increasingly "people-powered" rather than machine-powered. There are millions of people helping the searchbots find information.
Here are some examples and gripes:
- There are many publishers that try to make sure their headlines catch the attention of the search engines rather than catch the attention of readers. The same is true for content, editors increasingly optimize it for the search engines rather than the readers.
- Why should I have to tag my content, and tag it according to the specific formats that Technorati, and other search engines recommend? Aren't they supposed to do that?
- Google relies on a tremendous amount of user-helped search. Websites are encouraged to create site maps and leave the XML file on their server so that the GOOGbot can find its way around.
- The search engines ask web site owners to mask-off parts of their sites that are not relevant, such as the comment sections with no-follow and no-index tags.
- Web sites are encouraged to upload their content into the Googlebase database. Nice--it doesn't even need to send out a robot to index the site.
- Every time I publish something, I send out notification "pings" to dozens of search engines and aggregators. Again, they don't have to send out their robots to check if there is new content.
- Google asks users to create collections of sites within specific topics so that other users can use them to find specific types of information.
- The popularity of blogs is partly based on the fact that they find lots of relevant links around a particular subject. Blogs are clear examples of people-powered search services.
And there are many more examples. If the search engines are so great at doing what they do, then how come we have to do all of the above?
I resent the fact that I have to create all this content describing my content--the search engines should be creating this "metadata."
I just want to write stuff, and leave it up to the search engines to find it, classify it, index it, and do all the other things their mythology suggests that they do.
In the world of enterprise search, companies such as FAST, Vivisimo, Autonomy, etc, have to find information without the benefit of aids. Corporate documents have no pagerank or tags or much metadata of any kind.
Yet in consumer search it seems as if nothing would be found without a huge amount of help from millions of people every day. Why is it that we have to help the search engines do a job they are supposed to be doing by themselves?
I wonder about the productivity cost to society from all this human labor--work that is supposed to be done by robots.
It's as if these searchbots are blind, and we have to lead them patiently along the street and point things out to them, while they tap away at the world with white canes.
Part 2: Search seems to be broken...
FAST Search and Transfer has suddenly popped into my sphere of attention and I mean really popped. I got to spend some time at FAST's user conference at the end of last week, and it was an educational experience that got me interested in search again.
This Norwegian based enterprise software search company has made the subject of search compelling again. For too long GOOG has made it appear as if it had already won the search wars--anything better would be an incremental improvement.
Yet enterprise search--which is where FAST has staked its expertise--is a much more interesting subject than I imagined, and much more interesting than consumer search. Enterprise search is much more difficult problem, and one of the most challenging problems in IT.
Consumer search can be vague and still be successful. It can bring up a list of nearly relevant sites or documents, and usually that is all that is needed. But in the enterprise, search is usually needed to find something very specific, a contract, a purchase order, a memo.
And there are all sorts of conditions associated with access to data, some security based, others are regulatory. Search quickly becomes quite a complex process and one that can lead to other things.
Enterprises use a lot of structured data, but there is also a massive amount of unstructured data too. Search in the enterprise could potentially bring the two data world's together.
You might even be able to create enterprise applications by using modified search algorithms.
This type of scenario gets very interesting: enterprise applications by algorithm. This is already happening in business intelligence, I wonder how far, in theory, such an approach could be taken. Could you create CRM applications through search algorithms?
I'll be writing more about this subject, and FAST, over the next few days.
Susan Feldman, a senior analyst at IDC, will release on Friday the results of a groundbreaking study that shoots down one of the largest myths in search engine marketing: that the majority of traffic to web sites comes from the top ten search engines.
By comparing publicly available traffic data from companies such as Nielsen Research, with research of its own, IDC found a big discrepancy in terms of the number of search queries tracked. Ms Feldman said, "Our model showed that there were seven to ten times more search queries being made and that the large search engines had only about 30 per cent of the search query traffic."
Ms Feldman said that the missing search queries, the dark matter of the search engine world, were coming from direct queries. People would go to a web site such as Amazon.com and type in a search query.
"This means that there is a massive business opportunity still to be had. The top search engines do not own the web, at least yet," said Ms Feldman.
The IDC results will be released at a Friday session at a conference organized by Fast Search & Transfer, a client of IDC and a vendor of search enterprise software. Earlier this week Fast introduced its AdMomentum product which allows online publishers to set up their own advertising networks instead of sharing revenues with Google, Yahoo or other ad networks.
. . .
Foremski's Take: The IDC findings are not a revelation for any online publisher. Peeking into the server logs reveals where traffic is coming from. For example, SVW gets less than 5 per cent of its traffic from search sites, and that is great because my traffic is not "surfer" it knows where I live and comes in direct. Yet, the bandwidth used by the search engine robots is one third of my total--to serve less than five per cent of my visitors.
The IDC numbers will help to dispel one of the big myths about the Internet, that the search engines drive substantial amounts of traffic therefore sites need to optimize for the search engines.
I've long said: optimize for your customers/readers not for the search engines. SEO, beyond basic principles, is not worth it, yet many companies spend a lot of money making their sites searchbot friendly rather than user friendly.
- - -
Posted by Tom Foremski on October 26, 2005 5:51 AM
By Tom Foremski
ComScore Networks' latest analysis of search engine activity was for December 2006, it showed Google gaining share, and at a faster pace than second placed Yahoo!
Google increased its lead by 0.4 share points to 47.4 percent of the total US market, compared with November 2006. Yahoo added 0.3 share points with 28.5 percent of the total. Microsoft sites were third with 10.5 percent, followed by Ask Network with 5.4 percent, and Time Warner with 4.9 percent.
Google and Yahoo gains were at the expense of Microsoft, which lost 0.5 share points, Ask, which fell by 0.1 share points, and Time Warner which lost 0.2 share points.
ComScore assembles its data from monitoring the Internet activities of more than 2m consumers.
Foremski's Take: Google's lead shows no signs of flagging. Yahoo is doing a decent job in gaining share but not enough to catch Google. Google continues to grow much faster.
Third, fourth and fifth places in search rankings are all declining. Is there a place for these and specialized search engines on the Internet? Or is it that only the top two Internet businesses in each category have the best chances to succeed?
From ComScore Networks:
* Americans conducted 6.7 billion searches online in December, up 1 percent versus November. Annual growth rates in search query volume remained strong with a 30-percent increase since the same month a year ago.
For more information, please visit www.comscore.com.
* Google Sites led the pack with 3.2 billion search queries performed, followed by Yahoo Sites (1.9 billion), MSN-Microsoft (713 million), Ask Network (363 million), and Time Warner Network (335 million).
Local search and local online commerce are the next battlegrounds for the giants such as Google and Yahoo, but also for many startup companies. The lure is the billions of dollars spent on Yellow Pages advertising by local businesses.
We have Yahoo Local and Google Local, plus Max Levchin's Yelp, CitySearch, Ingenio, plus local newspapers and other companies--all trying to grab a piece of the local search and local commerce ad spend.
But tapping into the local businesses market through online services is hard. The same factors that make scaling a global online service easy on the Internet become reversed when applied to local businesses.
For example, a local pizza parlor gets nearly all its business from within two miles of where it operates. Reaching China or even a neighborhood five miles away is something the Internet does well for many companies. But it doesn't make much sense for a local pizza parlour, and the same is true for most local businesses--they all nned to reach customers in their neighborhoods.
I recently spoke with two startups, Smalltown and Grayboxx, with two different approaches.
Smalltown targets neighborhoods and small communities
I was very impressed with Smalltown's approach to local search because of the elegant design and simplicity of the site. Simplicity is not easy to do but it is extremely vital in the online space.
Last week I spoke with Smalltown CEO Hal Rucker. "To succeed in local markets we belive that you have to be local," said Mr Rucker. "So we've created 'webcards' which are a type of virtual index cards with their own web address that can be created by anyone local, by a business or an individual."
An example of a webcard:
Webcards can be attached to each other, and also sent to people. They provide an easy to use template that allows busy, small business owners to create a web page about their business without having to have a website.
And the process is simple enough that any customer of a local business can create a webcard that acts as a recommendation, or a warning, by filling in text and adding photos.
"With webcards we are helping to build what we call the 'local web.' And it creates 'social advertising,'" said Mr Rucker.
If they can get the formula right, then they can reproduce it in other small towns and city neighborhoods.
Each Smalltown would have a local person/agent to help evangelize the service and also aid local businesses and people to produce webcards. Smalltown is =offering cash bounties to people of $5 per five webcards, a smart way to create a large inventory of webcards, which creates a potentially useful library of local content.
SVW's take: Smalltown has a great user interface and I love the concept of webcards, each with their own web address, each discoverable through the search engines, as well as the local Smalltown portal.
What I'd like to see with such approaches is a compelling front end, by which I mean a reason to go to the site even if I'm not actively searching for something local. What would draw me to it if I did not have a need to find something? I've got a couple of ideas...
Coming up next in SVW: Interview with Bob Chandra, founder and CEO of Grayboxx (still in stealth mode). This local search startup taps unseen databases to create ranked lists of top local businesses favored by their communities. . .
Baynote came out of stealth mode today with a software as a service product it calls "content guidance." It helps users find what they want on web sites with thousands of documents by studying the behavior of prior visitors.
Type in a search term and up pops a selection of relevant documents that are "useranked" according to how others found what they wanted.
"Our technology works as a silent observer and we can tell by a user's behavior if they found what they were looking for. When you consider the fact that studies show only 17 per cent of web site visitors find what they were looking for, that means you are wasting 83 per cent of your money spent on online marketing campaigns to drive users to your site," says Jack Jia, CEO of Baynote.
Typically, it can take six clicks to find relevant content and today's visitors are increasingly less patient and more likely to give up. Mr Jia claims Baynote's technology provides a 20 times increase in users finding the content they want, usually in just one click.
Mr Jia is a former CTO of Interwoven, so he has a good understanding of web site design and content plus his team makes use of the latest discoveries in artificial intelligence. The Baynote technology tracks 12 key behavioral characteristics displayed by web site visitors in order to determine whether they found what they wanted or left frustrated.
The service also offers a variety of usage reports and is available for $995 per month for large businesses and $95 per month for small businesses.
SVW's take: The price seems a bit steep for small businesses but for large web sites serving many different types of user groups, Baynote potentially provides a way to avoid leaving lots of frustrated visitors.
The service is also a good way to test web site designs to make sure the most popular content is easily accessible and where to tweak navigational elements.
One problem is that search on large web sites is notoriously bad because you can't apply the traditional page rank methods to determine the importance of a specific web page or document. And so people's bad experiences with site search on large corporate sites means that they will often avoid the search box and thus might skip the little icon that signifies Baynote's search technology is being used.
Baynote does offer other ways of connecting users with relevant content by clever use of AJAX and mouse-over techniques that popup pages in a Browster-like way without requiring users to commit to a click. The background loading of pages into cache however, will skew resident analytical software. But Baynote provides an analytics component as part of the service, which provides detailed usage reports.
Interestingly, Baynote does not use its technology on its own web site - there is no search box!!!!
I recently met with Raul Valdes-Perez, the CEO and co-founder of search engine company Vivisimo. It is not a well known company but it is one that is doing very well, and doing it very profitably--all without any venture capital money.
I first met Mr Valdes-Perez nearly two years ago. At the time, this Pittsburgh, PA based company was ready to take on Google and the others with its unique technology that can index information into subject groups or clusters, by automatically detecting the underlying taxonomy within a set of information, on-the-fly.
Founded in June 20002 by Carnegie Mellon University researchers, Vivisimo offered an new approach to search. I became intrigued by the promises of Vivisimo's technology. I wondered if it could be used in other ways too; if it might reveal emerging social trends before they become visible in traditional ways.
And the clustering approach always seemed much more useful for researching various topics. For example, by seeing the various groupings of a term, i.e.. "bond" which could be James Bond, or a chemical bond, or a family bond--you could find unexpected, serendipitous connections to a term without requiring serendipity to reveal them.
I would sometimes fantasize about using Vivisimo's technology for local search sites. I own the domain names SearchSiliconValley.com and SearchBayArea.com, and the same "searchxyz.com" format for the San Francisco Bay Area counties and towns. Vivisimo's spider is not designed to crawl the entire web, but it is very good at crawling thousands of web sites, perfect for specialized applications, such as indexing a specific region. Clustering the results would be a great way to quickly find all the local Mexican restaurants, or public libraries, and many other uses.
One of these days I might get to try out Vivisimo's technology for such an application, but in the meantime the company has been doing very well in the enterprise search market, especially in government applications. Earlier this year it won a prestigious contract, following stiff competition, to provide the search technology for FirstGov.gov, the official portal for the US government.
Search results are from government web sites and also from related web sites outside of the government, and Vivisimo has been winning considerable praise for a much improved search experience.
I was glad to hear that Vivisimo was doing well. And Mr Valdes-Perez was kind enough to remind me of some advice I gave him a while ago, when Vivisimo was launching Clusty.com, the consumer version of its technology, and dreaming of taking on and winning against Google and the others.
The advice I gave was that the noise level in the consumer search space would soon become very intense and would require massive amounts of money to stay visible. For a small company such as Vivisimo, it would not be a game it could play without having to raise considerable capital. For Vivisimo, the opportunities would be in more specialized approaches where it could offer unique advantages, and fly under the radar while the big companies battled for the prized consumer markets.
And that's what the company has been doing. Apart from its FirstGov and other government related business, it is also doing well in the Life Sciences arena, where finding clusters of information is exceedingly important. And there are other such opportunities in other industries.
Clusty.com did not usurp Google, but it is a very good business for Vivisimo. "Clusty gave us experience in creating simple user interfaces. Too often, our competitors in the enterprise search space pay very little attention to the user interface, which makes their products difficult to use." says Mr Valdes-Perez.
Specialized search applications such as the FirstGov contract require specialized approaches unique to each implementation. FirstGov for example, requires at least 58 boolean terms for each search query. And the algorithm has to distinguish between different parts of the web page--not all dot-gov web pages are created or formatted in the same way.
The Google, Yahoo, Microsoft approach to search is far more general than the tuned, specialized approach taken by Vivisimo. And that means the search giants won't find it that easy to expand into the enterprise search markets.
Mr Valdes-Perez is also a critic of the behavioral technologies that the large search engine companies use to try and improve the search experience by collecting personal data. Clusty.com does not collect any user data which means that there can be no privacy breaches, accidental or subpoenaed.
"Users search based on their whims at the time, and not on past behavior. It is much better to provide a user with several options on what they are searching for and allow them to choose," he says.
That certainly makes sense to me. I hardly ever run the same search twice (except for ego surfing :-) so why collect my personalized search data?
Here is an excellent story on the subject of click fraud from the consistently good CIO Today magazine.
Click Fraud Gets Smarter
Does Google and the others have the technology to filter out click fraud? Greg Boser is going to put that to the test:
Web consultant Greg Boser has an ingenious method for sending loads of traffic to clients' Internet sites. Last month he began using a software program known as a clickbot to create the impression that users from around the world were visiting sites by way of ads strategically placed alongside Google search results. The trouble is, all the clicks are fake. And because Google charges advertisers on a per-click basis, the extra traffic could mean sky-high bills for Boser's clients.
But Boser's no fraudster. He cleared the procedure with clients beforehand and plans to reimburse any resulting charges. What's he up to? Boser wants to get to the bottom of a blight that's creating growing concern for online advertisers and threatens to wreak havoc across the Internet: click fraud.
The article makes a good case for the need for an independent third party to monitor this problem.
There's no doubt about it, Google's patina of goodwill is dissolving. I'm hearing a fair amount of anti-GOOG chatter all over the place.
Personally, I still like Google. I still feel that I get a whole lot more out of Google than Google gets out of me as an internet user.
In many ways, Google exemplifies what I call one of the new rules of the new economy.
Old Rule: Success is the ten-bagger company--returning to its VCs/investors more than ten times their investment.
New Rule: Success is the reverse ten-bagger--the company that monetizes ten per cent or less of the business opportunities it has.
Otherwise it is not providing enough value to its customers and it will be considered as trying to fleece its users.
It's the latest round in the battle between content creators versus the index creators. Human versus machine-produced content.
"By HELENA SPONGENBERG, Associated Press Writer
BRUSSELS, Belgium - European publishers warned Tuesday that they cannot keep allowing Internet search engines such as Google Inc. to make money from their content.
"The new models of Google and others reverse the traditional permission-based copyright model of content trading that we have built up over the years," said Francisco Pinto Balsemao, the head of the European Publishers Council, in prepared remarks for a speech at a Brussels conference."
Google, Yahoo and the Oodles of this world will shrug and say, that's fine, we'll stop indexing you and I guess you don't want all the traffic we send you.
But maybe the traffic isn't that great? The traffic that news sites get from search engines isn't high quality traffic. It is the web surfer, happy-go-lucky, a click here, and one there, then gone.
News is not consumed through a search box. You cannot search for news because you wouldn't know what to search for. It's new. That's why there are products such as Google News, so you can see what is news.
But advertising on news pages is not very efficient. Conversions are the highest on search pages.
So, if Google and others publish headlines and extracts of news content on their pages, it takes away traffic because that is all the content most people need for news. Fewer visitors means it makes it more difficult for the news organizations to pay their journalists--and that must affect the quality of journalism.
Let's also mention the devastating effect Google et al are having on newspaper/ news sites and their classified ads and small business advertising.
This is a double whammy against the professional media sector, imho.
. . .drug makers seeking drug takers
By Tom Foremski, Silicon Valley Watcher
Googling for healthcare info can make you sick, says Healthline's CEO West Shell. What he means is that you get millions of pages and you don't know which ones to pick. That's where he hopes Healthline, a recently launched beta health care search engine, can help.
It uses doctors, rather than just servers, to help create a dictionary that translates medical terms into common words, and doctors rank the quality of the medical information the search engine links to.
Healthline also offers "health maps" that graphically represent information, such as treatment and symptoms, that's linked to a specific condition. And it has quite a few other ways to slide through--and understand--a lot of important medical information.
"These days, the patient has to educate themselves about treatment options because the doctor patient load has increased tremendously. And doctors love to have an educated patient that understands their condition," says Mr Shell.
My recent post on Battling the online bot armies of the search engine giants and startups brought out some interesting discussions.
Over on the VeriSign Infrablog, Michael Graves, 'techno-evangelist' wrote:
. .there are problems with crawling. Over time, the number of crawlers grows. There used to be just a handful that got broad coverage, now there’s a dozen or more. In the future, there may be a hundred or more. At some point, Tom Foremski’s argument will become undisputable, and crawlers will have to be managed much more carefully than they are now through the use of Robots.txt or some other means.
Michael goes on to propose full content pings as a solution to the crawler problem:
For example, if this post were submitted in full as part of the ping, Googlebot and the gang wouldn’t need to come fetch this post to analyze it for inclusion in their databases and indices. It would be available from the ping server directly. Search engines could maintain a high-bandwidth, always-on connection to the ping server, and have the full content of newly published articles in hand, without having to do any fetching at all from the origin server.
Full post: I, Robot
Veteran media entrepreneur Mitch Ratcliffe, at Ratcliffeblog notes:
What I see happening is what happens with any medium as it matures—people will stop looking for new content as much as they do at first and start settling into relationships with trusted sources. This conforms to the conclusion Tom Foremski arrived at, but I believe search will play a bigger role in the Web (hell, let's call it "Web 2.0" to separate it from the first decade's worth of Web) because so many more sources are introducing new content.
Long term, though, we're going to see the value of relationships, which are largely built on content. If, to reach the people I want to have relationships with I need to allocate a lot of bandwidth, I'm happy to do it.
And Niki Scevak over at Bronte Media checks my math: You Say Toe-mato I say Tomar-to
. . . bloggers are big in Japan
I popped down to Yahoo on Monday and Linda Kozlowski (great name btw :-) from Text 100 introduced me to Christine Castro, chief comms officer and snr VP at Yahoo, and some of her comms team, Mary Osako, senior director, and Linda Du, who manages international PR.
Yahoo is a lot different now from my first experience with the company. That was when Yahoo was located in a small building in Palo Alto that had clearly seen better days. I interviewed co-founder Jerry Yang as we sat on old plastic chairs and the Yahoo server sat in an adjoining room.
Now Yahoo has a thoroughly modern campus that is overflowing with hordes of attractive twenty to thirtysomethings. There are cute cafes and eating places throughout the campus, and the distinctive purple and yellow Yahoo colors are seen in the flower beds, and in the Memphis style furniture inside the building.
The visit to Yahoo was an opportunity for me to continue an exploration of the different approaches taken by Yahoo and Google. The two companies have similar roots and similar products, but there is a big difference in their approach to business.
Google is a media company yet it is run by engineers from head to toe, there is not a single media professional in its top echelons.
Contrast that with Yahoo, which has seasoned media professionals in nearly all of its senior ranks. Terry Semel, ex-Warner Bros, John Marcom, ex-Financial Times US, and Christine Castro, ex-Disney, to name just three.
My question about Google has been: can engineers build a media company?
AP reports that Google CEO Eric Schmidt has denied that Google is launching a competitor to PayPal, even as he confirmed that Google is in fact readying some manner of payment system.
"The payment services we are working on are a natural evolution of Google's existing online products and advertising programs which today connect millions of consumers and advertisers," he said. That's not much to go on but I would guess that Google's payment system would allow them to manage a cost-per-sale advertising model, in which they can integrate advertising placements, clickthroughs, sellthroughs, and user behavior.
The Become.com shopping search site has come out of beta and has gone beyond 10,000 registered users. No registration is now required to use the site. Company cofounder Michael Yang recently clarified for me some of the anti-spam measures that Become.com has in place around its AIR (Affinity Index Ranking) technology, which is a different approach than Google’s PageRank method.
While it is true that AIR technology is superior to Google's PageRank for preventing spammed pages from showing up on the search result (and in theory it is spam-proof) we can not completely eliminate spams 100% in implementation. Also on the issue of patents for anticipated spam techniques … our patent lawyer recommended that we do [file] but we have not filed yet because we have not identified any spam techniques that would work against AIR.cd1225
Google this week announced a cell phone based search service using short text messaging, also known as SMS. UpSnap, privately held, is also in this space, offers a free directory assistance (411) mobile SMS service, and says that Google's entry has helped to validate the SMS market, but that only UpSnap! has a working business model.
Tony Philipp, chief executive of UpSnap, said that SMS is growing fast, especially among the 18- to 30-year-old demographic. He also points out that UpSnap has a business model already in place, in which merchants pay for listings and use a free phone call feature, based on UpSnap's VOIP infrastructure, to talk with customers.
We had another scorcher of a week on the Watcher with coverage of two competing geek conferences. Nick Aster hit the ground running in Austin with this report on the SXSWi conference. And in San Diego, Richard Koman produced a veritable bit torrent of great copy from the O’Reilly Emerging Tech conference.
While folks like Odeo's Evan Williams made the SF-Austin-SF-San Diego circuit, the two cons neatly bifurcate geekspace. SXSW is the hipster and chickster web/UI/interaction design space, while ETech boasts a interesting mix of alpha geeks, A-list press (not counting SVW ;)), and unbuttoned biz types. One borg victim (black-clad, earpiece installed in head) paced the hallway near the pressroom yelling into his cell, "You would not believe the movers that are at this conference." And of course it was true.
One alphageek was heard to say, "This is the conference for business people who want to feel like they're at a hacker conference." But at $1250 a pop, all the geeks had speaking gigs and free passes.
The purity of search results, free from commercial influence, is likely the only way a new search engine company can challenge Google and win over its users.
That's because Google is struggling to purify its search results in the face of a large industry that actively tries to manipulate its searches for commercial gain, so that a client website link appears on the highly-desired first page of Google’s keyword search results.
When Flickr CEO Stewart Butterfield took the stage at O'Reilly's ETech conference Tuesday he joked he had an important announcement to make about rumors that the photo-sharing company would be acquired: "The next person to ask me about it gets punched," he said, displaying a slide of a cartoon POW!
As previously reported here, Yahoo! is about to acquire Flickr, the popular photo sharing community web site. It now seems likely that the announcement will be made March 29, along with the official invitation-only beta launch of Yahoo 360, which Yahoo announced this morning after press reports revealed the project's existence.
SiliconValleyWatcher has confirmed that Yahoo, through its advertising network Overture, is testing "YPN" a competitor to Google Adsense--the hugely profitable advertising network.
The confirmation came from a highly informed source at Yahoo. And it follows the recent discovery of strange ads on the blog of Yahoo program manager Ken Rudman.
In his blog, Andy Baio drew attention to the fact that Rudman’s blog was running text ads that were called from a "ypn.overture.com" server.
While the YPN acronym is set, what it stands for is still up in the air. Andy Baio suggested Yahoo! Publisher Network but we like Yahoo! Profit Network. Any other ideas out there? Email rkoman (at) gmail.com or use our Anonymous Tip box to the right.
Via the Register:
Click fraud consultants Clickrisk.com found one of their clients click-through rates suddenly dropped to zero. Turns out out they were the victims of "keyword hijacking" -- automated searches that cause a company's ads to be displayed like crazy. Here's how it works:
By running searches against particular keywords from compromised hosts, attackers can cause click-through percentage rates to fall through the floor.
This, in turn, causes Google Adwords to automatically disable the affected campaign keywords and prevent ads from being displayed. By disabling campaign keywords using the technique, cybercrimals could give their preferred parties higher ad positions at reduced costs, according to click fraud prevention specialists Clickrisk."
GuruNet, the Israeli company behind Answers.com, bolted more than six points in trading today to 22.98. The stock price ended last week at 12.50, so it has almost doubled this week. What did Guru announce this week to make investors swoon so hard? Nothing. Here's the Motley Fool's take:
Investors would be wise to go back to the IPO prospectus and read through the risks this company faces. One significant one is that of IPO funding. While it is adequate for the next 12 months, the company is going to be producing operating losses. More funding, and probably share dilution, is coming. Oh, and the company has more debt than cash. Yikes.
The company's content is licensed from year to year. That's hardly comforting when it is the content, as well as the search, that counts.
Why is GuruNet soaring? Investors may be looking at Google's $55.6 billion market capitalization and thinking that GuruNet's Lilliputian $93 million capitalization leaves a lot of room for growth. Maybe, but Answers.com has just been released, and its future, and the company's too, are far from certain.
Google reported stunning financial results yesterday, with profits of 71 cents per share, far in excess of analysts' expectations of 54 cents per share, and seven times their results of a year ago. The Washington Post reported that Google will roll out significant upgrades to the service in 2005.
"I'm very excited about how Google will look to the end user in six to 12 months, lots of new and interesting ways," CEO Eric Schmidt told analysts.
In the conference call with analysts, Larry Page made it clear that it's all about advertising. Google even certified ad execs in how to use search and how to track sales from Google ads, Page said.
Growth internationally looks especially bright, Google execs said.
Google yesterday posted a job listing for a strategic partner development manager on craigslist. Of interest is this line: "This high-visibility role requires someone who will be responsible for identifying, structuring and negotiating licensing relationships with some of Google’s largest and most strategic partners to acquire and monetize a wide range of video and audio content."