Accelerating AI-powered Finance with VMware, NVIDIA and Domino Data Lab

By Paul Nothard, Solutions Architect - Financial Services Industry Solutions, VMware; Dr. Jochen Papenbrock, Head of Financial Technology EMEA, NVIDIA; Yuval Zukerman, Senior Director, Content Strategy, Domino Data Lab

While many industries are investing in AI/ML, with a majority seeing cost savings and increasing revenues, the financial services industry has not always been as fast at adoption, having faced some industry-specific headwinds. With several repeated projects not graduating from the testing and prototyping phases into production, we must ask why this is.

The rapid adoption of AI/ML within financial services has led to disparate groups forming, mirroring the tensions previously seen between more traditional developers and the operations teams. This often leads to a separate center of excellence dictating technical direction and policy for AI/ML-driven initiatives, much like a form of Shadow IT. This is exacerbated with the 'buy button' ability of a cloud consumption model.

Exploring the state of AI/ML within the financial services industry (FSI), demands an examination of some of the specific challenges faced by financial services customers, and the consideration of a potential solution. Any financial services solution must give customers an optimized, validated, and unified data science and infrastructure platform that enables them to deliver more business value and innovation through AI/ML.

What is the state of AI/ML in the financial services industry

From banking and fintech institutions to insurance and asset management firms, the goal remains the same - how to manage risk, enhance efficiencies to reduce operating costs, and improve experiences for clients and customers more accurately. NLP (Natural Language Processing) and LLM (Large Language Models) are core technologies powering financial services - similar technologies have recently captured public attention via ChatGPT. NVIDIA's "State of AI in Financial Services: 2023 Trends " survey of nearly 500 global financial services professionals reveals these top four takeaways:

1. AI delivers meaningful business impacts to financial services firms

TAKEAWAY: AI is a strategic imperative. Companies that are not investing in AI and cannot migrate AI-enabled applications from research to production will be at a competitive disadvantage.

2. Executive support for AI at new high

TAKEAWAY: Executive support for AI is higher than ever before. Financial services executives value and believe in AI more than ever before.

3. Financial services institutions are investing in trustworthy AI & explainability to support new AI use cases

72% percent of respondents are building a framework for AI governance and risk management to help ensure their AI systems are trustworthy and explainable. With Explainable AI, companies will be able to apply deep learning to additional use cases. Today, internal compliance officers and auditors restrict the application of deep learning, as they require the model outputs to have an explanation, should the outcome be questioned by customers, regulators or other stakeholders.

TAKEAWAY: Expanding the use cases through explainability will increase demand for accelerated computing.

4.Multi-cloud is coming on strong and is critical to AI in financial services

44% of financial services respondents' companies use hybrid infrastructure for their AI workloads and project. In reality, sensitive data cannot always be migrated to the cloud and certain workloads are cheaper to run on premises. Because of this, the debate is not about on premises versus the cloud-rather, it is about how to optimize both and reduce costs in the process.

TAKEAWAY: Many financial services firms state they are cloud first. Yet, the reality is that few companies migrate all their data to the cloud, leading them to turn to a well-managed multi-cloud infrastructure solution.

Challenges faced by financial services customers

AI practitioners (data scientists, quants, analysts, actuaries, etc.) need to experiment quickly, prototype, train, iterate, and adapt. They are under pressure to deliver results to their lines of business, and for a good reason. This can often affect the repeatability and consistency of model results, eroding confidence in the modeling. They want to deliver results to their line of business customers and do not and should not care where the resources are located. Left unmanaged, this has recently been referred to as the 'wild west' of modeling and introduces its own business risk.

On the other hand, IT operations teams are acutely aware of the various constraints they must adhere to, such as data security, operational resilience, data residency, and new and evolving governance frameworks (e.g., DORA, Gaia-X, etc.) in addition to, cost and efficiency concerns. Offering choice to the data scientists, utilizing the most appropriate technology and location for the task at hand while not adversely impeding the data scientists' flow.

The topic of ethical and explainable AI is a concern for both groups and is addressed in different ways. The tension is real, but there are significant opportunities to help both parties to benefit the business.

Can this tension be solved with MLOps?

Yes. Businesses should embrace MLOps principles and platforms to help manage the complexity and risk associated with this rapidly evolving technology.

Data science, machine learning and AI development present challenges that are distinct from the mainstream software engineering lifecycle, let alone the requirements of IT operations teams. MLOps aims to address the differences, applying DevOps principles to the development and deployment of machine learning models. This involves the complete lifecycle from data provisioning and model development through deployment. MLOps incorporates model integration into automated CI/CD pipelines. Deployed models are monitored for precision and performance in production. MLOps seeks to increase the efficiency, scalability, and reliability of machine learning applications. It enables efficient, reliable, and predictable processes to obtain model-driven insights and predictive capabilities from data.

A unified data science and infrastructure solution from VMware, NVIDIA, and Domino Data Lab

Financial services institutions must carefully balance innovation, risk management, and regulatory compliance to adapt to changing market conditions, find new revenue streams, model risk, engage customers, and automate business processes.

VMware has collaborated with Domino Data Lab, NVIDIA to provide a unified analytics, data science, and infrastructure platform optimized, validated, and supported, purpose-built for AI/ML deployments.

This diagram represents the solution we created along with NVIDIA and Domino Data Lab. Diving into each section of the integrated stack in turn, we have:

A - Domino Data Lab

Domino Data Lab provides an enterprise-grade MLOps platform, acting as an enterprise's system of record for all its AI/ML artifacts and enabling the complete data science and AI/ML lifecycle.

Domino helps enterprises address the main challenges of scaling data science. Wasted work is reduced thanks to a rich set of collaboration tools that align business goals and project milestones with model development. Data scientists are empowered with access to data in any format on any infrastructure alongside the best and most modern development tools and the ability to access powerful GPU (Graphics Processing Unit) and computing clusters to accelerate model training and boost productivity. Model deployment delays are avoided with the ability to train and deploy models as APIs or containers on premises and in the cloud. And once your models are in production Domino can monitor model performance to ensure accuracy and avoid financial and reputational risks.

Despite the open, flexible nature of the platform, IT maintains control with the ability to specify hardware and software standards and more securely provide access to data repositories.

Finally, Domino offers hybrid- and multi-cloud workload execution. With enterprise data spread across on-premises and cloud platforms, Domino can execute algorithms with data in place. This capability helps financial services organizations comply with data privacy and sovereignty regulations. Better yet, data is processed in place, avoiding slow and costly transfers.

B - NVIDIA AI Enterprise

NVIDIA AI Enterprise is an end-to-end, secure, cloud-native suite of AI software, enabling organizations to solve new challenges while increasing operational efficiency. It accelerates the data science pipeline and streamlines the development and deployment of predictive AI models to automate essential processes and gain rapid insights from data. It includes an extensive library of full-stack software, including AI solution workflows, frameworks, pretrained models and infrastructure optimization. Available in the cloud, the data center and at the edge, NVIDIA AI Enterprise enables organizations to develop once and run anywhere. Global enterprise support and regular security reviews ensure business continuity and AI projects stay on track.

C - VMware Cloud Foundation

VMware Cloud Foundation is a multi-cloud infrastructure and management platform capable of running both enterprise and cloud native workloads across private, public and edge environments. VMware Cloud Foundation enables infrastructure and operations teams to utilize a consistent infrastructure and cloud operating model.

With VMware Cloud Foundation, NVIDIA AI Enterprise software and accelerated computing, as well as Domino Data Lab software, organizations are able to extend VMware's proven cloud platform for AI. The result is an AI-ready platform that runs all enterprise apps - VMs, containers, Kubernetes and AI - and a single set of operations, tools, and processes for all apps - significantly expanding the reach of AI while simplifying operations.

This solution provides the following key benefits:

• Delivers a fully engineered and integrated solution that reduces the risk of internal projects that require a higher level of maintenance and engineering effort to deliver the same outcome.
• Supports regulatory compliance through intrinsic security, operational resilience and the agility and choice that the platform provides.
• Includes essential capabilities such as disaster recovery, ransomware protection, capacity optimization and planning, and more.
• Increases efficiencies to contribute ESG carbon targets, improve efficiencies for high value systems and direct cost savings.

D - Accelerated hardware

Using accelerated compute for AI delivers better performance which also improves efficiency and reduces costs, including savings from reduced energy consumption. NVIDIA delivers high performing computing systems while optimizing for efficiency^[1].

To be successful with machine learning and AI, enterprises need a modern coherent computing infrastructure that provides functionality, performance, security, and scalability. Organizations also benefit when they can run both development and production workloads with common technology. NVIDIA-Certified Systems go through additional certification, including VMware GPU certification, to help ensure compatibility with NVIDIA AI Enterprise. An NVIDIA-Certified System that is compatible with NVIDIA AI Enterprise conforms to NVIDIA design best practices and has passed certification tests that address a range of use cases on VMware vSphere infrastructure. These use cases include deep learning training, AI inference, data science algorithms, intelligent video analytics, security, and network and storage offload for both single-node and multi-node clusters.

Summary

An optimized, validated, and unified data science and infrastructure platform solves the tensions between disparate groups - IT Operations and AI practitioners. Allowing financial institutions to concentrate on deriving business value from the increased focus and innovation surrounding the use of AI/ML is critical to actively managing risk, efficiency gains and improved customer/client experience. A platform that continually addresses the concerns around security, compliance, operational efficiencies, TCO and more accurate/explainable models benefits all stakeholders. Access to accelerated hardware is democratized and enhanced allowing organizations to be faster and more efficient.

This gives data scientists what they want and need while keeping them safe, secure, and compliant, thus creating better business outcomes with reduced risk.

Learn More

To experience this solution in a live environment, try NVIDIA's hands-on-lab: NVIDIA LaunchPad lab with Domino MLOps platform. With ready-to-use infrastructure running in a VMware virtualized environment, you can take a curated walk through our MLOps solution from infrastructure optimization to application deployment for up to two weeks for free.

Learn more about VMware Cloud Foundation AI-Ready Enterprise Platform and the vSphere AI/ML solution.

As Diamond sponsors of NVIDIA GTC, both VMware and Domino Data Lab have sponsor pages containing links to further resources, including videos, downloadable e-books and more. Of particular interest are the sessions that each company is running at NVIDIA GTC.

^[1] References: NVIDIA MLPerf Benchmarks; NVIDIA A100 Tensor Core GPU

Public link

Please use the above public link if you want to share this noodl on another website.