Predicting break-points in trading strategies with Twitter
Arnaud Vincent & Margaret Armstrong
Abstract
The importance of being able to identify changepoints in financial time series has been stressed by many authors, both for econometric forecasting and for enhancing the performance of trading strategies. Strategies which work well in one type of context may lose money under different circumstances so it is crucial for traders to be able to identify break-points as they occur.
Our basic hypothesis is that these breakpoints are usually linked to factors outside the markets such as breaking news or the changes in the socio-political context. The buzz on the internet especially on social networks like Twitter provides an advance indicator of these changes.
In this paper we propose a novel approach for identifying micro-breakpoints based on Twitter. In order to measure the impact on trading strategies, of knowing when changes occur, we used a simple genetic algorithm using Forex data as the reference benchmark. It makes a decision every two minutes whether to hold US dollars or euro. We compare its performance to a hybrid algorithm which stops trading and goes through a relearning phase after each Twitter alert. In tests over a 5 month period, the hybrid algorithm performed significantly better than the benchmark algorithm. One unexpected result was the discovery of a wave-like relationship between the time to react to each alert and the performance of the algorithmic trader. We call this the Twitter wave. As our results have only been validated over a 5 month period, longer tests are clearly required but the preliminary results are very promising. If confirmed they open up new perspectives for identifying micro-breakpoints in real-time, for high-frequency trading.
Keywords: Twitter, breakpoints, Forex, genetic algorithms, high frequency trading
Introduction
The literature on trading strategies and models for US equities, forex, derivatives etc, is vast. In most cases the authors demonstrate that particular strategies have performed well in certain markets over a given period of time. Pesaran & Timmermann (2002) puts these results in perspective because their performance depends on which market is chosen and when. The primary difficulty for algorithmic traders is to identify events or changes in the context which make some strategies obsolete. Pesaran & Timmermann (2002) listed a series of exogenous factors (market feeling, speculative bubbles, and political changes) that could cause breakpoints, and developed several methods for identifying them.
As traders rapidly become aware of developments in financial markets, we have focussed our research on the impact of factors that are exogenous to the markets. A priori many types of exchanges of information (news, email, the internet) could contain a signal that could be exploited by a monitoring system. The buzz on the web where millions of messages are exchanged in an informal way may be a suitable medium for our research. The social network Twitter makes it possible to follow the buzz and measure changes throughout the world in real-time and to generate alerts when the context changes.
The aim of this research project is to show that the breakpoints in trading strategies correspond to Twitter “alerts” (or to a subset of them). We chose to work with genetic algorithms because they are capable of adapting to gradual changes but are very sensitive to sudden changes in context and should therefore allow us to test our hypothesis. We decided to work with foreign exchange data because it is such a highly liquid market.
Why Twitter rather news or emails? Breaking news would seem to be the natural choice for early warnings of changes in context that might influence financial markets. However, using news has its limits:
- The information arrives after the event has occurred and markets only take a few milliseconds after the news breaks to react.
- While it is easy to keep abreast of news that occurs at predictable dates or that contains certain keywords, it is difficult to detect political or social events of an unexpected nature. This is why we decided not to use news in our research.
Another possibility was email which provides a real-time source of information that is precise and exhaustive. However, using emails to detect breakpoints raises many problems: the sheer quantity of information to process, getting access to the information, and concerns about privacy. Consequently this possibility was dropped.
Twitter turned out to have several advantages over traditional means of exchanging information. The buzz on Twitter is uncontrolled and uncontrollable, as well as being exhaustive and real-time. Several research projects have shown the predictive ability of this vast mass of informal information. Twitter allows measuring the importance of each message in real-time, which makes it a better tool for identifying context changes.
The first attempts to measure the internet buzz were semantic, aiming at identifying the content of the messages or results of their queries. Instead, we focused on the concept of “change of context” in a broad sense, independent of specific words, intending to develop a measure of the volatility of Twitter along the lines of the volatility index VIX that measures the state of financial markets in the USA.
Twitter volatility: a high frequency measure of context change
One difficulty with constructing an indicator of the volatility of the buzz is that Twitter is so new. We chose to work with high-frequency data using data every 2 minutes over a 5-month period. The start-up, Lollicode, has developed an end-user application called Twitscoop, which watches the activity on Twitter all the time. We have set up a procedure for extracting the top 70 new words on Twitter once per minute.
Our first attempt at measuring the volatility on Twitter consisted of counting the number of new words that appeared in the list from one instant to the next. Our second attempt was more radical. We counted the number of words that appeared for the first time in our Twitter dictionary. A change of context was defined as being 2 or more new words.
Testing the Twitter alerts experimentally
Typically there are between 2 and 10 alerts per day. To test whether these alerts could be useful for traders, we developed a trading strategy using a genetic algorithm because these algorithms adapt themselves to gradual changes but are sensitive to sudden changes. By working with high frequency foreign exchange data, we can evaluate the effect of stopping trading after each Twitter alert.
The genetic algorithm functions by generating random changes in the algorithm and allows for comparison of results through large parallel runs. We decided to compare the performance of a basic genetic algorithm with a hybrid one that takes into account the Twitter alerts.
Results from the empirical tests on the basic algorithm
Our experimental procedure was as follows: starting from a randomly chosen set of individuals with randomly chosen genes, we carried out 450 runs, all based on the same data and under the same conditions. The results were averaged for sets of 10 runs. Over all runs, the genetic algorithm made a slight profit of 0.56% per month.
Results from the hybrid algorithm incorporating the Twitter alerts
In 62% of the cases the hybrid algorithm performed better than the basic algorithm. The average gain rose from 0.56% per month to 1.27%.
Twitter wave To test potential improvements with better IT systems we recalculated the results of the hybrid algorithm for time differences ranging from -10 minutes to +10 minutes. The results showed the percentage gain or loss (Fig 6) if the Twitter alert had been detected earlier or later. The maximum gain occurs if the information had been known 4 minutes late.
Conclusions
The preliminary results from these experimental tests are encouraging. They open up new perspectives for detecting breakpoints in the performance of high-frequency trading strategies using Twitter. A correlation exists between the buzz on Twitter and variations in the €/$ exchange rate, especially noted between 4 and 6 minutes before break-points occur. Further testing is required to confirm these results.