Real-Time Data Backbones for AI-Driven Finance Governance
Keywords:
Cloud-Native Data Engineering, Financial Risk Management, Regulatory Compliance Frameworks, Enterprise Data Pipelines, Risk Taxonomy Models, Data Governance Gates, Audit-Ready Architectures, Quality-Driven Data Systems, AI-Driven Financial Pipelines, Scalable Data Workflows.Abstract
The financial services industry has gradually embraced AI to automate business processes and augment decision-making. Yet cloud-native data engineering — a critical prerequisite for AI — remains nascent, particularly in mission-critical areas such as accounting and regulatory compliance, where years of regulatory scrutiny have made CEOs and Boards wary of data-quality lapses and compliance risk. Cloud-native data pipelines for risk management are typically built, operated, and maintained in isolation, with quality and governance applied only after the fact, rather than by design. This paper presents an end-to-end, enterprise-grade cloud-native data-engineering framework built specifically to meet the requirements of a regulated financial institution.
The framework is anchored by a risk taxonomy and a regulatory compliance library, with cloud-native principles applied consistently across every layer. Governance gates enforce quality, testing, validation, traceability, and audit-readiness without slowing development velocity. Through an open, extensible architecture and a quality-first implementation of its core components and data workflows, the framework provides a solid foundation for accelerating the development and live operation of use-case-specific, AI-driven data pipelines.
References
1. Aros, L. H., Bustamante Molano, L. X., Gutierrez-Portela, F., Moreno Hernandez, J. J., & Rodríguez Barrero, M. S. (2024). Financial fraud detection through the application of machine learning techniques: A literature review. Humanities and Social Sciences Communications, 11, 1130.
2. Al-Hashedi, K. G., & Magalingam, P. (2021). Financial fraud detection applying data mining techniques: A comprehensive review from 2009 to 2019. Computer Science Review, 40, 100402.
3. Zhu, X., Ao, X., Qin, Z., Chang, Y., Liu, Y., He, Q., & Li, J. (2021). Intelligent financial fraud detection practices in post-pandemic era. The Innovation, 2(4), 100176.
4. A, R., Maheswari, N. U., Umamaheswari, P., Josephson, P. J., Mangalampalli, B. M., & Gudada, P. V. (2026). Tree-Structured Regularization Driven CNN Model for Liver Disease Diagnosis and Classification Using Ultrasound Imaging. In 2026 International Conference on Intelligent and Sustainable Electronics & Computing Technologies (INSECT) (pp. 1–6). IEEE. 2026 International Conference on Intelligent and Sustainable Electronics & Computing Technologies (INSECT). https://doi.org/10.1109/insect68872.2026.11663742
5. Li, Y., Tam, D. S. H., Xie, S., Liu, X., Ying, Q. F., Lau, W. C., Chiu, D. M., & Chen, S. Z. (2021). Temporal graph representation learning for detecting anomalies in e-payment systems. In Proceedings of the 21st IEEE International Conference on Data Mining Workshops (pp. 983–990). IEEE.
6. Li, Z., Huang, M., Liu, G., & Jiang, C. (2021). A hybrid method with dynamic weighted entropy for handling the problem of class imbalance with overlap in credit card fraud detection. Expert Systems with Applications, 175, 114750.
7. Błaszczyński, J., de Almeida Filho, A. T., Matuszyk, A., Szeląg, M., & Słowiński, R. (2021). Auto loan fraud detection using dominance-based rough set approach versus machine learning methods. Expert Systems with Applications, 175, 113740.
8. Gupta, S., & Mehta, S. K. (2021). Data mining-based financial statement fraud detection: Systematic literature review and meta-analysis to estimate data sample mapping of fraudulent companies against non-fraudulent companies. Global Business Review, 25(5).
9. Cheng, D., Wang, X., Zhang, Y., & Zhang, L. (2022). Graph neural network for fraud detection via spatial-temporal attention. IEEE Transactions on Knowledge and Data Engineering, 34(8), 3800–3813.
10. Huang, M., Liu, Y., Ao, X., Li, K., Chi, J., Feng, J., Yang, H., & He, Q. (2022). AUC-oriented graph neural network for fraud detection. In Proceedings of the ACM Web Conference 2022. Association for Computing Machinery.
11. Bhandari, P., Prithviraj, J., Saxena, R., Kumar, D., Mangala, N., & Chourasiya, P. (2026). Harris Hawks Optimization–Based Deep Learning for Resource-Aware Cloud–Fog Computing. In 2026 5th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0 (pp. 1–6). IEEE. 2026 5th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0. https://doi.org/10.1109/otcon69332.2026.11629982
12. Fritz-Morgenthal, S., Hein, B., & Papenbrock, J. (2022). Financial risk management and explainable, trustworthy, responsible AI. Frontiers in Artificial Intelligence, 5, 779799.
13. Daníelsson, J., Macrae, R., & Uthemann, A. (2022). Artificial intelligence and systemic risk. Journal of Banking & Finance, 140, 106290.
14. Wang, K. (2022). Internet financial risk management in the context of big data and artificial intelligence. Mathematical Problems in Engineering, 2022, 6219489.
15. Ma, X., Wu, J., Xue, S., Yang, J., Zhou, C., Sheng, Q. Z., Xiong, H., & Akoglu, L. (2023). A comprehensive survey on graph anomaly detection with deep learning. IEEE Transactions on Knowledge and Data Engineering, 35(12), 12012–12038.
16. Weber, P., Carl, K. V., & Hinz, O. (2024). Applications of explainable artificial intelligence in finance—A systematic review of finance, information systems, and computer science literature. Management Review Quarterly, 74, 867–907.
17. Motie, S., & Raahemi, B. (2024). Financial fraud detection using graph neural networks: A systematic review. Expert Systems with Applications, 240, 122156.
18. Jogi, N. L., Kumar, K. R., Mangala, N., M.Govindarajan, Gamini, P., & Bundhe, S. (2026). Machine Learning-Based Random Forest Model for Sustainable Supply Chain Sales Forecasting. In 2026 6th International Conference on Intelligent Technologies (CONIT) (pp. 1–6). IEEE. 2026 6th International Conference on Intelligent Technologies (CONIT). https://doi.org/10.1109/conit69683.2026.11620788
19. Khan Achakzai, M. A., & Peng, J. (2023). Detecting financial statement fraud using dynamic ensemble machine learning. International Review of Financial Analysis, 89, 102827.
20. Leevy, J. L., Hancock, J., Khoshgoftaar, T. M., & Abdollah Zadeh, A. (2023). Investigating the effectiveness of one-class and binary classification for fraud detection. Journal of Big Data, 10, 157.
21. Zhang, T., Zhu, W., Wu, Y., Wu, Z., Zhang, C., & Hu, X. (2023). An explainable financial risk early warning model based on the DS-XGBoost model. Finance Research Letters, 56, 104489.
22. Weber, P., Carl, K. V., & Hinz, O. (2023). Applications of explainable artificial intelligence in finance—A systematic review of finance, information systems, and computer science literature. Management Review Quarterly.
23. Usman, A. U., Abdullahi, S. B., Yu, L., Alghofaily, B., Almasoud, A. S., & Rehman, A. (2024). Financial fraud detection using value-at-risk with machine learning in skewed data. IEEE Access, 12, 64285–64299.
24. Reite, E. J. (2025). Improving client risk classification with machine learning to increase anti-money laundering detection efficiency. Journal of Money Laundering Control, 28(1), 93–107.
25. Shih, Y.-C., Dai, T.-S., Chen, Y.-P., Ti, Y.-W., Wang, W.-H., & Kuo, Y. (2025). Fund transfer fraud detection: Analyzing irregular transactions and customer relationships with self-attention and graph neural networks. Expert Systems with Applications, 259, 125211.
26. Lou, C., Wang, Y., Li, J., Qian, Y., & Li, X. (2025). Graph neural network for fraud detection via context encoding and adaptive aggregation. Expert Systems with Applications, 261, 125473.
27. Dong, X. (2025). Financial risk identification and accounting analysis method of listed companies based on machine learning. Discover Computing, 28, 258.
28. Yoshida Junior, V. T., Schiozer, R., de Genaro, A., & dos Santos, T. R. E. (2025). A novel credit model risk measure: Do more data lead to lower model risk? The Quarterly Review of Economics and Finance, 101960.
29. Kumar, S., Rao, A., & Dhochak, M. (2025). Hybrid ML models for volatility prediction in financial risk management. International Review of Economics & Finance, 98, 103915.
30. Reite, E. J. (2025). Developing a scoring model for managing money laundering transactions using machine learning. Journal of Money Laundering Control, 28(7), 30–49.
31. Ajagbe, S. A., Majola, S., & Mudali, P. (2025). Comparative analysis of machine learning algorithms for money laundering detection. Discover Artificial Intelligence, 5, 144.
32. Malhotra, A., Niharika, D. S., Mangalampalli, B. M., Nema, S., Gupta, Z., & Kriplani, K. (2026, June). Distributed and Edge AI for Continuous and Context-Aware Healthcare Intelligence. In 2026 5th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0 (pp. 1-6). IEEE.
33. Andreescu, A. I., Oprea, S.-V., Văduva, A. G., & Bâra, A. (2025). Anti-money laundering compliance using feature engineering with SQL analytics, TF-IDF and oversampling: Conditional tabular generative adversarial networks. Informatica, 36(3), 491–524.
34. Amoako, D., Obodai, T. N., Amoako, E. K., Mensah, N., & Adukpo, T. K. (2025). Leveraging machine learning, deep learning and 6G technologies in anti-money laundering strategies: A systematic review of implementation, effectiveness and challenges in the U.S. financial industry. Asian Journal of Economics, Business and Accounting, 25(5), 85–101.
Additional Files
Published
Data Availability Statement
All data supporting the findings of this study are included within the article.
Issue
Section
License
Copyright (c) 2026 Katarzyna Nowak (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Authors retain full copyright of their published work. Under this license, others are free to share, copy, distribute, transmit, remix, transform, and build upon the published work for any purpose, including commercial use, provided that appropriate credit is given to the original authors, a link to the license is provided, and any changes made are clearly indicated. No additional restrictions may be applied that limit others from doing anything the license permits. All published articles are freely and permanently accessible online to readers worldwide without any subscription or access fees.
License URL: https://creativecommons.org/licenses/by/4.0/
