Hewlett-Packard recently published a research paper: Predicting the Future With Social Media by Sitaram Asur and Bernardo Huberman.

The study analyzed 2.89 million tweets from 1.2 million users referencing 24 movies released over a three month period. The researchers discovered that the rate of Tweets could predict the success of movies prior to their release, and also spot sleeper movies that grew successful over time.

The quality of the predictions was significantly better than any other measure such as the Hollywood Stock Exchange.

HP believes the same methods could be used to predict product success and election results. [Please see: HP Study Shows Twitter Predicts Success Of Movies]

I spoke with Bernardo Huberman, head of HP's Social Computing Lab, about the Twitter research project. Here are some notes from our conversation:

- We're very interested in social attention and how attention is allocated, especially in a very fragmented media world that we now have.

- Also, the study is about the wisdom of crowds and collective intelligence. I'm not sure about the 'wisdom' part, but on average, there does seem to be a collective intelligence.

- We are interested in how agendas can be set when the influence of traditional media is waning. In the past, for example, the editors at the Wall Street Journal or Financial Times, or CNN, would decide which stories they would publish and that's what we would see but now the world is fragmented.

- The Twitter movie study helps to show how agendas can be established and how it could be applied elsewhere.

- The Twitter movie research was very difficult, we looked at a lot Tweets. We made use of Mechanical Turk for sentiment analysis, hiring thousands of workers.

- The Twitter movie study was interesting, we have some departments within HP that want to use the same methods for predicting the success of products. We have applied for a patent on the process.

- I don't think the movie tweets could be gamed because we looked at a very large number from a large number of people.

- We have thought about looking at financial information such as stock market tweets and also at where other online conversations are happening.

- I'm very interested in the problem facing media companies, how to monetize content. In a world where everything is free, information rapidly becomes commoditized and when everyone has the same information, it has little value. Our studies have shown that if people pay for information they value it more highly, they have something others don't have. It's a very hard problem. I'm not yet sure how to tackle it.

- - -
Please see:

Thought Leader - HP's Bernardo Huberman: Studies Of Mass Social Behavior On The Internet

HP Labs: A Gold Mine Of Original Research Into Online Social Behavior

HP Labs : Research: Social Computing Lab

Video: Social Dynamics In The Age Of The Web