Since its 2006 launch, Twitter has been, in many respects, an unbridled success story. It is now one of the 10 most visited websites in the world and is used by 13% of the US population. Its users generate an average 200 million posts every day.
And it is not all banal chatter; many a news story has been broken in 140 characters or less, long before being picked up by traditional media outlets. In Egypt, Tunisia, Iran and Moldova it even played a central role in organising large-scale protest and revolution.
Editor's choice
One thing the social networking behemoth has arguably failed to do is develop a sustainable business model. Despite numerous lofty valuations and plans to launch paid advertising, it ran at a net loss in 2010. But just because Twitter cannot currently make money for itself does not mean it cannot make money for others. In fact, some academics, and even a hedge fund or two, are beginning to suggest that data garnered from the site’s legions of users might be used as the basis for lucrative trading strategies.
Unexpected results
Johan Bollen, associate professor of informatics and computing at Indiana University, did not intend to involve himself in the financial markets when he embarked on a project designed to measure the collective emotional state of Twitter users. He, along with colleague Huina Mao, and Xiao-Jun Zeng of the University of Manchester, hypothesised that this information might be used to quantify sociological trends. Accordingly, the trio attempted to analyse the daily content of Twitter feeds using a sentiment analysis tool designed to assign a positive or negative mood to the site’s users, as well as an algorithm that reports back on six emotional states: calm, sure, alert, vital, kind and happy.
The resulting data, which was collected over a three-month period at the end of 2008, looked promising. However, when considered in isolation, there was no way to prove its validity, so the team attempted to correlate it with other known socioeconomic values.
Market predictor
The team struggled to find a match, until they eventually noted a striking correspondence with the Dow Jones Industrial Average. Given that the research was conducted during the onset of the financial crisis it seems a natural enough fit; one might well expect public sentiment to nosedive as markets crash and retirement funds evaporate.
But to Mr Bollen’s surprise, instead of following the markets, movements in the Twitter data generally occurred three or four days before movements in the Dow, apparently predicting daily up and down changes in closing values with 87.6% accuracy. It was, he thought, “an interesting scientific finding”, which was duly published in a widely reported October 2010 paper 'Twitter mood predicts the stock market'. Some parts of the financial community thought the research was rather more than an idle curiosity, however, and “jumped on it like their lives depended on it”, says Mr Bollen.
Foremost among these was Paul Hawtin, founder of Derwent Capital Markets, which describes itself as Europe’s first social media-based hedge fund. He recruited Mr Bollen to develop his research for commercial deployment. Derwent has been running a demo operation since February, which is now up 7.2%, clearing the way for the July 1 launch of a £25m ($40m) fund proper, expected to deliver between 10% and 15% annual returns, Mr Hawtin says.
Too good to be true?
Consistent returns of that magnitude would not be insignificant, but for some, the entire concept sounds rather too good to be true. Concerns range from methodology – criticism has been levelled at the comparatively short length of testing, as well as the specific period in which the data was collected – to sheer disbelief that any meaningful connection could exist between a disparate group of social media addicts and the performance of 30 publicly owned US firms.
Certainly, it is quite possible to backdate data and find correlations without any certainty of a causative link or guarantee that those parallels will continue. In his book Nerds on Wall Street, David Leinweber, portfolio manager and head of the Centre for Innovative Financial Technology at Lawrence Berkley lab in the US, found that over a decade, butter production in Bangladesh predicted the movement of the S&P 500 with 75% accuracy. When additional variables were added, such as US cheese production and sheep population in both countries, this increased to 99%.
Causal concern
Mr Leinweber’s example may be intentionally ridiculous, but he voices more solemn concerns about the relationship proposed by Mr Bollen’s research. “I’d be really surprised if there is a link between Twitter and markets,” he says. “The Casey Anthony murder trial has been a leading Twitter topic recently and made lots of people very unhappy. Did that cause markets to move?”
Perhaps surprisingly, if you ask Mr Hawtin, the answer is yes. He is, in fact, at pains to point out that Derwent is not scouring Twitter for news, or attempting to measure the emotion or sentiment for any particular demographic or geographical group. “We’re looking to gauge global sentiment in real time at any one moment,” he says. “We take every single tweet into consideration, whether it is someone talking about [teen heartthrob] Justin Bieber or the US federal funds rate.”
It is an unarguably unorthodox approach, but the one thing that critics and advocates agree on is that the only real test of its worth is long-term performance history. And that is something Mr Hawtin is keen to provide.
“People tend to think this is fantastic or absolutely rubbish. And the calibre of people making these proclamations range from all levels and segments of the industry,” he says. “There are a lot of sceptics out there, which I totally understand, but we just want to get our heads down, get some numbers on the board and let the results speak for themselves.”
Alternative approaches
While the prudence of Derwent’s strategy may have been called into question by some, many more believe social media analysis does indeed have a solid future as the basis for, or at least a constituent part of, a successful trading strategy. Twitter has had more than its fair share of usage in this kind of research due to the volumes of information it makes available.
Timm Sprenger, for example, a PhD student at the Technical University of Munich in Germany, analysed 250,000 tweets gathered over a six-month period, concluding that investors that took advantage of the information could have achieved average returns of 15%. Using similar methods, he had already used specific terms related to German federal elections posted on Twitter to predict results for each party within a 2% margin of error, an approach which was at least as accurate as more conventional (and significantly more expensive) opinion polls.
This research led to the foundation of the website TweetTrader, which allows users to access sentiment on individual stocks in real time and, Mr Sprenger told The Banker, is currently the subject of discussions with data vendors and hedge funds.
The science of sentiment
As unlikely as it may seem, this approach is actually rather more traditional. The science of sentiment has been around for decades, used by companies to track public perception of their brand and products. Applying this to financial markets is tough, however, when an average analyst armed with red and green pens might only process six or 10 articles per hour.
Replace the dawdling humans with a computer capable of processing the same volume of work every second, and you have something that will look far more attractive to traders.
Electronic analysis of news first appeared some years ago, but was often the exclusive preserve of high-frequency trading shops. These relatively crude systems were designed to buy or sell as swiftly as possible following automatic analysis of structured, numeric data, such as interest rate changes or earning reports.
Things are changing, however, and analysis of the actual sentiment of a news piece is becoming increasingly important, says Rich Brown, global business manager for machine-readable news with Thomson Reuters. “It’s still relatively niche but becoming more mainstream among the sophisticated quants.”
The New York-based firm offers a product that measures how positive or negative the tone of an article is for a particular company, based on a dictionary of about 20,000 words and phrases. As standard, the product bases analysis on about 60 sources, typically news wires. However, it can be adapted to draw from that other social media stalwart, the blogosphere.
Risky business
Of course, both blogs and social media sources are not subject to the same standards of accuracy as a news article, so more care may be required, warns Mr Brown. As Mr Hawtin is keen to point out, however, it is certainly not unheard of for rumour to move markets regardless of underlying fact, so it may not prove to be an insurmountable barrier to the effectiveness of such tools.
A bigger concern could be the perils of fraudulent activity. “There are additional risks such as market manipulation, including past examples of 'pump-and-dump' strategies, which need to be monitored,” cautions Mr Sprenger. Jonathan Lebed, for example, famously netted hundreds of thousands thanks to promoting penny stocks in social media’s spiritual predecessor – the internet chat room. And a single post really can move markets. Twitter has already seen rapper 50 Cent net $10m after encouraging his 3.8 million followers to buy stock in a business he part owned.
Ancient concepts
The suggestion of philosopher Epictetus that it is not events which matter, but rather a person’s reaction to them, applies equally well to financial markets as it does ancient Greek self-help. It is hardly controversial to suggest that the same news would affect a company's stock price differently in bullish rather than bearish conditions, and what better way to gauge overall sentiment than through the avalanche of content churned out by hundreds of millions of social media users?
There may well come a time when tools to analyse public mood and emotional state are as common as a Bloomberg terminal in a trader’s arsenal. But despite some early promise, we are not quite there yet.
