Banking Challenges with AI and Agents

When BNY deployed agents at enterprise scale, they faced a choice every bank encounters: how do you give AI systems the autonomy to be useful while maintaining the control regulators demand? This article breaks down the architectural decisions banks are making today—from runtime hosting to network isolation—and the Microsoft technologies powering production deployments.

Today: From Models to Agent Harness

Overview

The practical production-level deployments related to Microsoft AI and Agent services that banks are using today largely fall into two categories: 1) Copilot Studio and the new Copilot integrations, and 2) Agent Harness or similar frameworks with Microsoft Foundry model deployments and other components from the Foundry platform. For the second category, which is the focus of this article, there are many examples of ‘wins’ or success stories for Microsoft services used as the primary agent architecture stack, like this article covers for BNY. Available to all customers, these are the key services for agents that Microsoft offers today: GitHub to build, Foundry to run, deploy and scale, IQ to ground, Agent 365 to govern, and M365 or Teams to distribute. The diagram below illustrates Microsoft's enterprise platform for agentic AI.

To dig into the specifics, the way that Microsoft experts from the Global Black Belt team have described the agent platform is very much akin to Microsoft’s offerings as a suite of tools rather than one application: a platform with layers and components that customers can adapt and develop for their own customized solutions. The baseline component is a large-language model, which in this year of 2026, most people already know and implement at scale. The key next component is an agent harness. Below is Microsoft’s definition for a harness, which can also be described when done correctly as the cohesive framework that directs the agent how to interact with prompts, tools, and any other services. From Copilot Studio with a built-in harness to custom Foundry solutions based on the Microsoft Agent Framework (MAF) harness, there are various options within Microsoft to customize and personalize a harness for every agent or multi-agent system. The actual design of the harness will vary by use-case, with some built-in and available, customizable options via Microsoft Agent Framework. 

An agent harness, sometimes called an AI harness, is the runtime scaffolding that turns a language model into an agent that can perform work.

Agent Harness | Microsoft Learn 

Enterprise Layers

For enterprise agents, industry patterns in banking today include these layers in addition to a model: 1) grounding, 2) runtime, 3) orchestration, and 4) security. The agent harness is what connects all these layers and components, defining the exact flows, connections, and actions the agent system can undertake. An agent harness is necessary for any multi-step task or other long-running work, according to industry standards today. For each agent you build, the specifics of these components and how they are organized, reviewed by information security, and implemented into real deployments will vary, but the fundamental components are standard and currently deployable today from easy-to-use Azure samples and accelerators. 

Copilot-generated image of article explanation of layers

Challenges with Agents  

How to define an agent? 

Here’s the first challenge that companies should identify: what is an agent? All companies building AI systems should have a definition of what an agent is and what level of autonomy defines different types of agents. The primary reason for this is that everyone should understand and stop to think if they should even build an AI system, as often the answer may be that you don’t need generative AI but rather improved machine learning systems. It’s critically important to ask if generative AI and agent autonomy are needed and if so, what metrics to use to measure success per use-case. This is the most critical first step, and skipping it often leads to costly rework or failed deployments.' Microsoft recommendations for companies starting in AI and moving to agents include establishing a responsible AI framework, a governance review board, among other organizational structures that may not exist today in most companies. Here’s practical guidance on this and how to get started with Microsoft Foundry and agents: Microsoft Foundry rollout across my organization - Microsoft Foundry | Microsoft Learn. 

AI Agents have evolved from local chatbots to LangChain systems to reasoning loops with tools and actions, and more recently to production level systems with even less supervision like long-running agentic systems with recursive learning. This is exactly the process banks and similar regulated organizations should go through to define the spectrum of agents possible and to scale out securely, adding components incrementally, which you can do with Microsoft Foundry today.

Copilot-generated image of article explanation of agent evolution

Why do agents need a harness? 

Here’s the second challenge: how does a harness establish reliability, monitoring, and security? Across the fast-paced changes and continuous development in the field of LLMs, there is no standardized operating manual and criteria for the equivalent of what most production level applications have for reliability, monitoring, and security. This reality is unavoidable as LLMs and agent systems are actively redefining the cybersecurity principles we've traditionally relied on for production criteria. Back to the practical challenge and what it means for developers and AI engineers today, or anyone building AI systems. The larger question is can we “trust” agents? This is hard question and this article covers some parts of this from the technical perspective, but not the philosophical side which is coming up more in the AI industry and at various policy levels. There are controls and options for how a harness can improve trust, but it’s also important to remember the building blocks of these systems are stochastic processes that can always result in random and unpredictable outputs. 

How to scale safely? 

Here’s the third challenge: How does the agent work in my production environment on a large scale? Often, regulated customers like banks can build a proof-of-concept, or POC, but the real problems arise in trying to scale out for production-level workloads. Instead of dozens of internal employees talking with agent, in production you have hundreds of users, your app is reaching the capacity limit from model providers, your security team asks about rogue agents and prompt injection attacks, and you come across many other risks and new problems with varying size and severity. If you have never worked in cybersecurity or strategic planning, you may not be fully aware of the criticality of ongoing evaluation, monitoring, and updates – and in this age of AI, the incident responses required are not every week or every day, but maybe every hour. This is less of a challenge and more like the start of a forever war – but it’s not a losing one (hopefully). 

While these challenges are not comprehensive nor limited to banks, these are the primary hurdles that enterprises are facing today when it comes to agents. From these challenges, there are tentative solutions, workarounds, and recommended architectures developed by Microsoft and available today, with ongoing updates and releases to come. To understand the options, let’s concretely go through technical decisions that have already been identified by banks with lessons learned that could apply to many other scenarios.

Key Decisions 

Runtime and Hosting

The first key decision that often determines many follow-on decisions is where your agent runs. This is the execution component in the earlier diagram, and many people use the words “host” and “run” interchangeably to denote the level of infrastructure complexity required for an agent system. The definition largely used in the AI industry: host means the computing resource that manages and provides access to the agent’s workload. A hosting option could be what many applications use like a virtual machine from cloud providers in Infrastructure as a Service (IaaS) or a more managed option like Microsoft Foundry which is akin to platform as a service (PaaS).  

With Microsoft, your hosting options for most agents include Azure Virtual Machines, Azure Container Apps, Azure Kubernetes Service, Azure Functions, and Foundry Hosted Agents. The main factors in deciding between these hosting options are what level of control you, as the customer wants to have, and what level of complexity it takes to manage that control. At the highest control and most complex level is hosting directly on a virtual machine, meaning you build everything besides the literal machine on your own, like the image and containerization, and so this requires high operational overhead and self-managed controls. At the opposite end, with the lowest control and least complexity is Foundry agents. The low-code version is Copilot agents or already built-in and deployable agents, but this article does not cover these offerings in detail. More information on those here: Copilot and AI Agents | Microsoft Copilot. How to choose which hosting option? This is the main reference in Azure documentation to proceed in choosing a runtime or hosting option for any application: Choose an Azure Compute Service - Azure Architecture Center. When integrating with Foundry capabilities and for running an agent you have already defined, the recommended option is Foundry Hosted Agents. 

Environment and network isolation

Based on the hosting option the agent uses, there are two key next decisions: environment and network isolation. These two items fall into the environment component of the initial diagram in this article. For banks and similarly regulated customers, environment isolation means that the security controls and risks for AI must be understood, identified, and traceable before any level of autonomy and agentic capability is implemented. Typically, this means network isolation, private networking, and inspection and logging as required by security standards and information security reviews. In terms of controls, this practically looks like guardrails, safety services, and access limitations. For example, an agent should only be accessible by authenticated users, only allow approved prompts for business-related questions, and anything uploaded or communicated with the agent should be traceable for audits or review. These decisions are largely determined by the existing security components and services in banks, like what they are using for firewalls or network inspection or model approval processes.  

AI and agents introduce new risks, and so often there is lots of customization and case-by-case support required to integrate and approve an agent environment into the existing security protocols. This is not easily or quickly done today, as much research has already identified the difficulty in scaling and realizing value of agent systems often due to similar obstacles across various industries. But there are options today, with many features recently released or upcoming in future releases and updates. Here are key options available through Microsoft Foundry for environment isolation and controls, as commonly integrated with other Microsoft security services and products. 

Tomorrow  

Even with all the challenges today for AI and agents in banks, it’s likely that no one is going to slow down. Enterprise use-cases from improving employee productivity to software development automation are already benefitting from agents today, and there is lots of room for improvement and scaling out. Microsoft has various sources for the latest updates in AI and agents, with two key events and times that releases happen being the annual conferences of Microsoft Ignite in the winter and Microsoft Build in the summer. This article does not cover all the upcoming features and services for the challenges surfaced, but here are some items to track that showcase Microsoft’s progress and rapid development to address these challenges if you are building agents in the banking industry. 

• Upcoming Ignite 2026 sessions on agents including a session on choosing the right agent architecture and production failure patterns. 

• Previous presentations on building and running agents at scale, including this one from Build 2026, covering Toolbox features built-in to Foundry.  

• Recent announcements for 1) memory management and new configurations within Foundry, in preview today, and 2) improved observability and optimization including for costs, as shown in this announcement from Build 2026 and this one from September 2026 covering Model Router and Agent Optimizer.