Creating a prediction machine for the financial markets

Artificial intelligence and machine-learning technologies have evolved a lot over the past decade and have been useful to many people and businesses, especially in the realm of finance, banking, investment and trading.

In these industries, there are many activities that machines can perform better and faster than humans, such as calculations and financial reporting, as long as the machines are given the complete data.

The AI tools being built by humans today are becoming another level more robust in their ability to predict trends, provide complex analysis, and execute automations faster and cheaper than humans. However, there has not been an AI-powered machine built yet that can trade on its own.

There are many activities that machines can perform better and faster than humans, such as calculations and financial reporting, as long as the machines are given the complete data.

Even if it was possible to train such a system that could replace human judgment, there would still be a margin of error, as well as some things that are only understandable by human beings. Humans are still ultimately responsible for the design of AI-based prediction machines, and progress can only happen with their input.

Data is the backbone of any prediction machine

Building an AI-based prediction machine initially requires an understanding of the problem being solved and the requirements of the user. After that, it’s important to select the machine-learning technique that will be implemented, based on what the machine will do.

There are three techniques: supervised learning (learning from examples), unsupervised learning (learning to identify common patterns), and reinforcement learning (learning based on the concept of gamification).

After the technique is identified, it’s time to implement a machine-learning model. For “time series forecasting” — which involves making predictions about the future — long short-term memory (LSTM) with sequence to sequence (Seq2Seq) models can be used.

LSTM networks are especially suited to making predictions based on a series of data points indexed in time order. Even simple convolutional neural networks, applicable to image and video recognition, or recurrent neural networks, applicable to handwriting and speech recognition, can be used.

The most important ingredient in this whole process is the data, which is key to everything and serves as the backbone of the whole prediction machine. It’s important to have enough data while building an AI-based prediction machine.

However, it’s crucial to first “clean” the data. Explore it and study it, because in large datasets, there is sometimes a lot of junk data that is not at all useful for the purpose at hand. Junk data can result in inaccurate predictions — negating any usefulness of the machine as a financial tool.

A prediction machine must also be tested thoroughly. High-accuracy measurements may be the result of a model that is performing well, but they could also be a sign of overfitting, biases or other factors. Data should be checked to confirm that it is balanced so that the machine can make predictions based on a neutral scale. For example, in a trading-prediction machine, the data shouldn’t all be derived from just a few industries, or only from high-performing assets.

Financial decisions should never be 100% reliant on AI

There should be a red flag whenever anyone says they want to be able to rely 100% on predictions made by machines. A prediction machine may be trained using a lot of historical data and takes into consideration all the important factors, but it’s still not a good decision to rely solely on machine predictions, especially when finances are at stake. For example, in automated trading systems, the signals predicted by a machine can sometimes lead to a huge loss when a machine’s prediction is wrong.

Financial markets are very unpredictable. As one might expect, it’s impossible to build machines that can predict the unpredictable. An AI-based financial tool might be able to predict an upcoming trend in investments, but it’s up to the user to think the recommendation through and then decide whether they want to act on the prediction. Any AI model should include an assessment of risk, which would also help the user in making decisions.

AI simply cannot replace human judgment when it comes to financial decisions. Why? Because a predictive machine that makes decisions like a human — and mimics the problem-solving approaches of people who are masters of their domain — would have to have a touch of both rationality and irrationality.

Human traders or investors include both when making a decision, and this is part of what makes the markets difficult (if not impossible) to predict with 100% accuracy — hence we see the occasional, unexpected trading frenzy, such as what happened with GameStop and AMC.

The human touch is a necessary part of prediction machines

Smart investing these days requires AI-powered machine learning in addition to the human insights and knowledge we’re used to. However, humans will not be completely replaced by prediction machines unless we desire the stock market to be a playground for machines, rather than for human-focused investments.

It is important for us to keep AI on a short leash that remains in our own hands, particularly for financial businesses.

Algorithms are powerful and automated, but they cannot yet manage everything on their own. We still have a long way to go before we get to the point where we can stop providing inputs and can say that an AI system is intelligent enough to manage on its own without human intervention. Certainly, that time is not very far off — if the current rate of technological progress is maintained. Yet still, that time is not now.