Using online analytics and social media trends to predict real-world events is nothing new. Twitter’s been used to predict box-office sales (story link, detailed paper) and Google search data has been telling us about future flu epidemics for a while now.

Even I got in the act, demonstrating back in 2009 that Google Insights could anticipate changes in UK unemployment figures.

UK unemployment rate charted against search volumes for 24 related keywords, from January 2004 to April 2009 Sources: Office for National Statistics, Google Insights

Maybe I should have followed through with that idea, because there’s now a hedge fund that bases its investment decisions on data from Twitter. It’s called Derwent Capital Markets, it opened for business last week, and if its managers end up making a mint there might well be a new bandwagon in town.

So how do you run a hedge fund based on tweets? From what I understand of Derwent’s methodology, their algorithms measure the “calmness” of the Twittersphere – presumably based on sentiment analysis, which I’m a bit skeptical about. This is used to estimate the volatility of the Dow Jones Industrial Average index, with a three-day time lag.

This leaves a lot of unanswered questions. Does a non-calm day of Twitter conversations always correspond to a drop in the DJIA, or just volatility? Are they trying to predict metrics like trade volume and so on as well as broader day-to-day movements in the overall index? And are they ranking Twitter users based on credibility, or are spam bots equal to financial journalists, economists, and prominent investors?

Obviously algorithmic hedge funds aren’t about to disclose their inner workings so questions like this will have to remain unanswered for now. But what of the other, larger, question – isn’t the whole idea just, well, a bit… silly?

I can see why people might react in this way, and even I feel a bit skeptical about something describing itself as a “social media-based hedge fund” and that apparently pulls data only from Twitter, when there are lots of other sources that could be tapped. But it would be wrong to dismiss the basic concept.

Our everyday activities – web searches, page views, purchases, things we say on open social networks – leave a trail of data behind, which we tend to see as ephemeral or throwaway. We severely underestimate the value of this data but Google doesn’t, Facebook doesn’t, and we shouldn’t either. This data becomes even more valuable when aggregated across entire countries, continents, or the planet as a whole. In fact, it could be argued that the predictive potential of aggregated global real-time data has yet to be fully imagined, let alone realised.

The biggest problem with this resource is that we don’t really know how to exploit it yet. Things like Google Flu Trends or this Twitter-based hedge fund may be crude and experimental, and will definitely look even more so in five years time. Along the way there will be hype, bandwagonism, maybe even a stock market bubble, resulting from the application of real-time data to real-world problems.

But we need to make a start somewhere, and as silly as a Twitter-based hedge fund might sound, it’s as good a place to begin as any.