The Death Of The Search Algorithm? Techmeme Has Six Editors
By Tom Foremski - November 18, 2009
Death might be too strong a word but this is definitely a watershed moment. Techmeme, which used to rely completely on Gabe Rivera's secret algorithm to pick tech news and blog posts, now has six editors.
This is significant because Techmeme shows that human aided algorithms are more effective than just software and server. Techmeme is a microcosm of the rest of the search-enabled world of services, from news aggregators to basic search.
If Techmeme can't be Techmeme just by using its algorithms, and now needs lots of editors, then that means much larger news aggregators and search companies will likely have to add human editors too.
Here is Gabe Rivera announcing the additional editors:
Ever since we first plugged human editors into the automation behind Techmeme, we've been improving how we work to make the human/machine combo more and more effective. While technological innovations have contributed to these improvements, smarter and more continuous human editing has played a critical role as well.
Over the past few months three more editors have joined Techmeme's editorial team. They are (with links to their Twitter profiles) Rich DeMuro, formerly of CNET and various TV news outlets, Lidija Davis of The Drill Down podcast, formerly of ReadWriteWeb, and Mahendra Palsule, writer for MakeUseOf.com and a former IT project manager. At this point we're now staffed 24 hours a day most weekdays.
We've created a list on Techmeme's Twitter profile to let you follow the whole team, which also includes Megan McCarthy, Omer Horvitz, and myself.
Six editors at Techmeme. Wow. That's got to signal a huge loss of faith in search algorithms. But I think Gabe is doing the right thing.
Also, Techmeme has Atul. He is a one-man Techmeme. He sends great article links to Techeme all day long. If you want to know what is in tomorrow's Techmeme just subscribe to @atul on Twitter -- he is tireless and usually spot-on.
Gabe didn't have choice bringing in human editors because there has been a big drop in the number of sites linking to each other's news stories.
For a long time you could see items on Techmeme that had no links to them at all. They were chosen by hand. Without sufficient numbers of links Techmeme's algorithm doesn't work.
What does this mean for other search engines, what does this mean for Google PageRank which lies at the heart of Google's search algorithm?
Surely, this is GOOG's Achilles' Heel!?
And surely this is a very signifiant watershed moment. I've always said that humans bring value but most geeks believe in the superiority of the algorithm. Things have changed. This is a very big change.
{The Demise Of Linking ... And Its Effect On Google Pagerank]
New Media Increasingly Looks Like Old Media Says Techmeme Founder
« TEDxSF - Little TED Just Like The Big TED | Main | Technology In The Service Of Humanity »
Posted to A Top Story
November 18, 2009 | Permalink | Comment | Subscribe to SVW
- Top Stories:
- Rob Glaser: Pioneer Of Badware Leaves Real Networks
- New Info: Chinese Hackers Targeted GOOG's Internal Spy System
- Analysis: GOOG China Pull Out Might Be An Extension Of Its 'Borderless' Korean Policy
- Google Finds Its Backbone - Won't Censor China Search, May Close China Operations
- EAST MEETS WEST - 5 Observations on Silicon Valley from an MIT Sloan Perspective
- Additional Thoughts On The Killer Pitch . . .
- The Killer Pitch? - When PR Agencies Can Do This - Look Out . . .
- CES Diary: Media Woes... When Foresight Is No Better Than Hindsight
- Analysis: Here's What's Next After Nexus - GOOG's Cunning Mobile Strategy...
- Arrington Defends Zynga Scam Story From Top New York VC
- Gawker Media Focuses On Uniques As Key Metric
- Analysis: GOOG Needs To Have Its Own Telco Service More Than It Needs A Phone...
Comments (9)
I think human editors can improve quality and help frustrated editors (myself included) complaining about why their blogs aren't making it onto TM, but I am curious to see how Techmeme's new setup can be optimized for speedy updates when humans take a lunch break or sign off for the night. That was one area in which the old Techmeme setup and the current Google News setup have excelled.
Posted: November 19, 2009 7:49 AM
Ian: I agree that human editors can make Techmeme better. But it won't neccesarily help list other blogs because Techmeme monitors a core set of blogs/news sites and if you are not in it your chances of being mentioned are slim. You make a good point about what happens when they go home for the night - I guess the machine takes over...
Posted: November 19, 2009 11:16 AM
In many ways Google is a human-aided algorithm as well.
PageRank leverages human editorial decisions by measuring linking patterns between sites.
Similarly the Hilltop algorithm, developed by Krishna Bharat creator of Google News, uses a list of expert documents to refine search rankings.
Clearly there's no shame in using human intelligence to refine search results. I suppose the real trick is using others' human intelligence instead of hiring your own.
Posted: November 19, 2009 11:28 AM
Tom - A small correction. I believe the # of editors is 4 (or 4.5 depends if you count Gabe is an editor or tweaks the algorithm) and not 6. I believe Megan was the first editor back in Dec 2008 and then Techmeme announced the addition of three more yesterday
Posted: November 19, 2009 1:00 PM
Thanks Atul. Gabe said that he and Omer are engaged in editorial duties so it still adds up to 6 but I take your point about them not doing it full time. But I guess this also means that Techmeme more than doubled their editors...
Posted: November 19, 2009 1:08 PM
Marshall: Yes, you are right, PageRank is a human-aided system but the harvesting of that knowledge was done by machine, but is this is no loger good enough and requires direct supervision by humans? That seems to be what's happening...
Posted: November 19, 2009 1:13 PM
Personally, I think we're moments away from a paradigm shift in how we rank and filter the web.
Integration of social media annotation data into search algorithms will effectively turn every one of us into an editor of the web. Eliminating the the need for manual tweaks and human editors.
This concept is the focus of my earlier PageRank for People article (http://bit.ly/128U9V) and features prominently in the amazing algorithms that OneRiot (http://bit.ly/1845IQ) is currently building.
Posted: November 19, 2009 2:27 PM
Tom - you are right - I shouldn't be so pedantic about it. Gabe seems to online 24x7 so one could also argue he counts for more than one person. And thanks for you flattering comments about me in the article. Cheers.
Posted: November 19, 2009 3:36 PM
I appreciate your tireless work Atul.
Posted: November 20, 2009 12:58 PM