Technical Analysis does not work for predicting price movements of financial assets

In countless examples people claim to be successfull using technical analysis of financial instruments. Technical analysis refers to investment decisions that rely on analyzing the historic price movement of assets e.g. via moving averages and "support" and "resistance" in order to predict future price movements.

The efficient-market hypothesis

Despite widespread use of technical analysis, there is also the efficient-market hypothesis, which states that there is no use in technical analysis in order to predict future price movements. Unfortunately, it is only a hypothesis and it has not been proven. Some of the most famous investors claim the efficient-market hypothesis does not hold.

I am convinced that a buy-and-hold strategy is the best way most investors can maximize profits – given that the investor chooses the right assets. So, how can we find out the truth, i.e. can we prove that technical analysis works in the era of machine learning (ML) and artificial intelligence? The strenght of ML is to find patterns in large amounts of data and to get even more successfull in doing so, as the amount of data gets larger. Why should a human be better at detecting patterns in technical analysis than a modern ML algorithm?

Using Machine Learning

Assuming that ML can yield superhuman performance in this task, we need to use a very large dataset so that ML can really show it's strengths. Stocks are typically only traded during the working hours of a day but the foreign exchange FOREX is open around the clock. FOREX leads to more data than stocks.

The following experiment is about predicting the direction of the next minute of a FOREX price. This means we only want to predict, whether the price will go up or down in the next minute. If the ML algorithm would come up with useful predictions in that basic scenario, this would count as strong evidence that technical analysis does work.

The data

Somewhat randomly, we use the CHF.USD price pair. Through a prorpietary source we obtain a dataset of approximately seven million datapoints.

The code

You find the complete code of the experiment in the notebook below.

Results

In the experiment, the next minute direction of the price movement could not be predicted. Looking at ROC AUC, one might conclude, that the model can distinguish between upward and downward price movements in the next minute. But, looking at the precision values in the recall-precision curve, there is no significant portion of the curve above 50% precision. As an additional evaluation, the scatter plot of scores vs. gain/loss does not indicate any correlation between the score value and the gain/loss classification. Interestingly, the scaterplot shows, that the strength of the price movement can be predicted, but that is not what we are interested in.