How Machine Learning in Finance is different from other fields?

Aug 8, 2018 | 858 Views

How Machine Learning in finance is different from other fields?

Answered by Igor Halperin on Quora:
Machine Learning plays an integral role in many areas of financial services like from approving the loans, managing assests, minimizing the risk and many more. Machine Learning plays a vital role in fraud detection and protects and thus protect the consumer from the fraudlent activities.

The main differences stem from differences in data. In finance, data are very noisy, and often non-stationary. "Signals" cannot be split from "noise" in any unique way, as a matter of principle. This is very different from, say, image processing, where the level of noise can be controlled, at least in principle. Also, the notion of non-stationary data is non-existent for image processing. Because of a pronounced role of noise, some ML models, for example non-probabilistic models, are not very useful in finance.

The other difference is the amount of data. Many interesting problems of finance are problems with small-to-medium datasets, which makes applications of data-hungry methods such as deep learning problematic. Therefore, in finance enforcing some prior knowledge is often necessary, via depending on a method used choices of regularization, Bayesian priors, or other general principles such as analysis of symmetries.

One more important difference is that the "true" state space in finance is not well defined. There are so it is called black swan events-things that are outside of financial models, for example political risk, that nevertheless might have severe impact on security prices. There is a difference between uncertainty and probability risk. Most ML models as well as most of classical financial models deal with probabilistic systems with a well defined state space they do not admit black swans. They are models of risk but not models of uncertainty.
  • Low signal: noise ratio 
  • Nonstationarity
  • Limited data
Consider that any one of these creates challenges for standard ML practice, and each compounds the problems the others create or the overfitting, sample bias, data snooping.
There are other issues depending on the application. Hence machine learning is different in finance from the machine learning in other fields. Using machine learning, systems can detect unique activities or behaviors and flag them for security teams. The challenge for these systems is to avoid false-positives situations where "risks" are flagged that were never risks in the first place. 

Source: HOB