AI-Driven Detection of Financial Market Anomalies
Keywords:
Big Data, AI, anomaly detection, flash crash, liquidity droughts, arbitrage windows, qualitative trading, market microstructure, Novozhilov test, attention sub-network, temporal anomalies, finance market alerting.Abstract
The past decade has witnessed a proliferation of Big Data in the Finance industry. Several technologies are being developed in parallel, and a consolidated assortment of AI-based analytical products is emerging. The bulk of these Big Data-based projects within the finance domain are concerned with Investment Management. These services deal with the application of Machine Learning or Natural Language Processing on alternative datasets (e.g. Unstructured Data, Sentiment Analysis, Web traffic, etc.) with the aim of predicting Price Movements or even Stock Prices of Stocks, commodities, or different Asset Classes. Another set of companies are providing data-driven strategies that predict Financial Market Anomalies. But these innovative products do not appear to provide reliable and robust results for Early Detection of Financial Anomalies.
One technology that aims to contribute in this area is AI-Powered Anomaly Detection. Early detection of Financial anomalies (e.g. Flash Crashes, High Liquidity Drought, Opening Price Dislocation, Price Dislocation in the presence of Large Imbalance) has been attempted through traditional machine learning-based models. A natural extension of this effort is to employ AI-Powered Big Data Analytics for Early Detection of Financial AAnomalies. An end-to-end solution capable of ingesting Big Data in real time, processing with AI-Powered Technology, and signalling detections with Data Governance is developed and presented. Real-time Scalability and versioning of the data has also been examined.
References
1. Azevedo, V., & Hoegner, C. (2023). Enhancing stock market anomalies with machine learning. Review of Quantitative Finance and Accounting, 60, 195–230.
2. Azevedo, V., Kaiser, G. S., & Mueller, S. (2023). Stock market anomalies and machine learning across the globe. Journal of Asset Management, 24, 419–441.
3. Bakumenko, A., & Elragal, A. (2022). Detecting anomalies in financial data using machine learning algorithms. Systems, 10(5), 130.
4. Chullamonthon, P., & Tangamchit, P. (2023). Ensemble of supervised and unsupervised deep neural networks for stock price manipulation detection. Expert Systems with Applications, 220, 119698.
5. Kolla, S. H., & Loganathan, R. (2023). Cloud-Native Deep Learning Architectures For Secure Generative AI Deployment In Enterprise Workflow Platforms. Journal of International Crisis and Risk Communication Research, 603-618.
6. Du, Q., Wang, Y., Wei, C., & Wei, K. C. J. (2023). Machine learning, anomalies, and the expected market return: Evidence from China. Pacific-Basin Finance Journal, 82, 102391.
7. Giglio, S., Kelly, B., & Xiu, D. (2022). Factor models, machine learning, and asset pricing. Annual Review of Financial Economics, 14, 337–368.
8. Hilal, W., Gadsden, S. A., & Yawney, J. (2022). Financial fraud: A review of anomaly detection techniques and recent advances. Expert Systems with Applications, 193, 116429.
9. Hsu, T.-Y. (2021). Machine learning applied to stock index performance enhancement. Journal of Banking and Financial Technology, 5, 21–33.
10. Jiang, W. (2021). Applications of deep learning in stock market prediction: Recent progress. Expert Systems with Applications, 184, 115537.
11. Kumbure, M. M., Lohrmann, C., Luukka, P., & Porras, J. (2022). Machine learning techniques and data for stock market forecasting: A literature review. Expert Systems with Applications, 197, 116659.
12. Leangarun, T., Tangamchit, P., & Thajchayapong, S. (2021). Stock price manipulation detection using deep unsupervised learning: The case of Thailand. IEEE Access, 9, 106824–106838.
13. Li, G., & Jung, J. J. (2023). Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges. Information Fusion, 91, 93–102.
14. Liu, Q., Wang, C., Zhang, P., & Zheng, K. (2021). Detecting stock market manipulation via machine learning: Evidence from China Securities Regulatory Commission punishment cases. International Review of Financial Analysis, 78, 101887.
15. Mailagaha Kumbure, M., Lohrmann, C., Luukka, P., & Porras, J. (2022). Machine learning techniques and data for stock market forecasting: A literature review. Expert Systems with Applications, 197, 116659.
16. Mattaparthi, R. (2022). Engineering Predictive Industrial Systems Through IoT-Driven Asset Monitoring and Machine Learning Prognostics. International Journal of Future Innovative Science and Technology (IJFIST), 5(1), 7790.
17. Mirza, N., Elhoseny, M., Umar, M., & Metawa, N. (2023). Safeguarding FinTech innovations with machine learning: Comparative assessment of various approaches. Research in International Business and Finance, 66, 102009.
18. Naidoo, V., & Du, S. (2022). A deep learning method for the detection and compensation of outlier events in stock data. Electronics, 11(21), 3465.
19. Olorunnimbe, K., & Viktor, H. (2023). Deep learning in the stock market—a systematic survey of practice, backtesting, and applications. Artificial Intelligence Review, 56, 2057–2109.
20. Thakkar, A., & Chaudhari, K. (2021). A comprehensive survey on deep neural networks for stock market: The need, challenges, and future directions. Expert Systems with Applications, 177, 114800.
21. Tobek, O., & Hronec, M. (2021). Does it pay to follow anomalies research? Machine learning approach with international evidence. Journal of Financial Markets, 56, 100588.
22. Reddy, V. A. R. (2022). Designing Fault-Tolerant Data Ingestion Pipelines for High-Volume Healthcare Transactions. Frontiers in Health Informatics, 11, 861-889.
23. Uslu, N. C., & Akal, F. (2022). A machine learning approach to detection of trade-based manipulations in Borsa Istanbul. Computational Economics, 60(1), 25–45.
24. Khodabandehlou, S., & Hashemi Golpayegani, S. A. (2022). Market manipulation detection: A systematic literature review. Expert Systems with Applications, 210, 118330.
25. Li, G., & Jung, J. J. (2023). Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges. Information Fusion, 91, 93–102.
26. Sönksen, J. (2022). Machine learning for asset pricing. In F. Chan & L. Mátyás (Eds.), Econometrics with machine learning (Vol. 53, pp. 337–366). Springer.
27. Iqbal, A., & Amin, R. (2023). Time series forecasting and anomaly detection using deep learning. Computers & Chemical Engineering, 178, 108560.
28. Aravind, R., & Surabhii, S. N. R. D. (2023). Harnessing Artificial Intelligence for Enhanced Vehicle Control and Diagnostics.
29. Atypical stock transactions with possible misuse of insider information and market manipulation: Methods and models. (2022). Procedia Computer Science, 213, 1
Additional Files
Published
Data Availability Statement
None
Issue
Section
License
Copyright (c) 2024 Hiroshi Tanaka (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Authors retain full copyright of their published work. Under this license, others are free to share, copy, distribute, transmit, remix, transform, and build upon the published work for any purpose, including commercial use, provided that appropriate credit is given to the original authors, a link to the license is provided, and any changes made are clearly indicated. No additional restrictions may be applied that limit others from doing anything the license permits. All published articles are freely and permanently accessible online to readers worldwide without any subscription or access fees.
License URL: https://creativecommons.org/licenses/by/4.0/
