Posted by Richard Koman - August 23, 2005
Like a pledge who hits on every woman at a frat party in the first hour, only to be taken aside by a brother and told to chill, MSN Search is apologizing for its overly aggressive RSS bot, which was hitting feeds every 10 minutes. According to SEW there is a robots.txt command to slow things down. Says Eytan Seidman, MSN Search program manager, (and kudos to him for knowing about it, linking to it, and responding quickly):
Tweet this story Follow @tomforemski
At this point we are just experimenting and suffice to say that in our short experiment we have learned a lot. We have been a bit more zealous in our crawling then we would have liked and we are going to fix that. Thanks for the feedback!
Permalink | Subscribe free | Categories:
Contact: editors@SiliconValleyWatcher.com
If urgent: send text or call 415 336 7547
Bacon's names Silicon Valley Watcher one of the
most influential blogs in the US.
SF Publicity Club's ninth annual awards
celebrating excellence in media.