Hybrid intelligence makes the device part of enterprise AI architecture

Historically, endpoint strategy and AI strategy were largely separate conversations. As AI workloads become more distributed and devices become increasingly capable execution environments, those decisions are increasingly connected. IT leaders making device decisions today are also shaping the AI platform their organizations will rely on through the next refresh cycle. 

Enterprise AI architecture has largely centered on platforms, models, and cloud services. Capable AI PCs add an execution environment that application teams need to consider: where should each part of an AI experience run? 

Different workloads can call for different execution environments. Sensitive processing, including workloads with specific data residency or sovereignty requirements, may benefit from execution on the endpoint. Latency-sensitive interactions may work better without a network round trip. A cloud model may be the right resource for tasks requiring greater scale, context, or reasoning capability. 

Hybrid intelligence brings local and cloud resources into one application architecture. The device becomes an active part of the enterprise AI platform, giving developers another place to run useful work close to users and data. That affects application design, testing, security planning, and the economics of inference. 

Agents make these architectural decisions more consequential. Unlike an isolated inference request, an agent may operate over time, use tools, access organizational or local data, and take actions on a user’s behalf. Hybrid intelligence architecture therefore has to address not only where a model runs, but also how agent access, execution, identity, and activity are controlled across device and cloud environments. 

For IT leaders, hybrid intelligence creates a new intersection between application architecture and endpoint strategy. Endpoint teams, security teams, architects, and application owners all have a stake in deciding where workloads run and what the supporting devices must provide. A small set of standard configurations may still serve most employees, but roles expected to run demanding AI workloads locally may require different levels of RAM, compute, or sustained performance. 

With AI-ready devices like the latest Surface portfolio, organizations can preserve the operational benefits of standardization while expanding device profiles to support appropriate workloads for each role. 

Extend existing AI experiences

Organizations have already invested in building AI applications and agents with Microsoft Foundry and related cloud services. Those solutions may connect organizational data, incorporate business workflows, enforce access controls, and operate within existing security and governance requirements. 

Local inference extends that foundation. Developers can examine an existing application task by task, identify where AI can add value, and align each AI-enhanced workload with the execution environment best suited to it. 

Consider a document-review workflow. A Surface device could use a local model to summarize selected material or extract key topics locally, while server-based systems support broader retrieval, reasoning, or workflow integration. Processing can be distributed between device and cloud according to what each task requires. 

That design also needs explicit rules for failure and fallback. Workloads governed by endpoint data requirements need local failure paths that preserve those boundaries. Other workloads may move to a cloud model when local resources are unavailable or insufficient. Applications also need to account for loss of connectivity, unavailable local models, and changes in available device resources. Those decisions belong in the application's security, privacy, and reliability architecture. 

Agentic applications also require controls beyond model placement and fallback. Organizations need ways to constrain the files, tools, applications, and network resources an agent can reach; distinguish agent activity from user activity; and apply policy consistently across a managed fleet. 

Windows is introducing Microsoft Execution Containers as a policy-driven environment for isolating agent activity, with Microsoft Agent 365 providing a broader management and governance layer. Together, these capabilities can help organizations treat agents as managed enterprise actors rather than extensions of unrestricted user access. 

Match each AI workload to the right resources

Workload placement depends on factors such as response time, privacy requirements, data residency and sovereignty requirements, connectivity, context length, RAM, organizational policy, available compute, and required output quality. For organizations operating across jurisdictions, some data or processing may need to remain within defined geographic, organizational, or device boundaries. Local execution provides another option for workloads where data should remain on the device, while cloud resources can be used where policy and workload requirements allow. 

Applications do not necessarily need to treat this placement as a fixed choice. A hybrid runtime can evaluate available device resources, model capabilities, connectivity, policy, and task requirements, then direct work to an appropriate local or cloud resource. Developers still need to define permitted paths, quality thresholds, and fallback behavior, but the platform can support more dynamic routing than separate, hard-coded local and cloud experiences. 

With just a few lines of code, businesses can tell their app to switch between a cloud AI model and a local AI model

Software development offers a useful example. A developer might use a cloud model for a complex architectural problem and a smaller local model for a targeted code transformation, documentation update, or explanation of a function. The right choice can change with the task, repository context, system resources, development tools, and policies governing source code. 

Developers need to understand what each execution environment can support. They need to select models and runtimes that fit the workload, review model output, run appropriate tests, and define what the application should do when the selected resource cannot complete a task. 

An AI application installed on a device may still rely entirely on cloud inference. Local execution requires a supported model and runtime, an application path that invokes local resources, and enough system capacity to deliver the expected experience. Windows ML provides a common foundation for running supported models across available CPU, GPU, and NPU resources, allowing developers to target Windows devices without building a separate integration for every hardware configuration. An NPU adds another compute resource, but it does not automatically move an existing application or agent onto the device.

Make the device part of the architecture 

Once inference moves to the endpoint, hardware becomes part of the application architecture. CPU, GPU, and NPU resources provide different kinds of compute. Memory capacity can determine which models and workloads are practical. Runtime support, power consumption, and competition from other applications can also affect performance. 

This does not mean abandoning a standardized device strategy. It means defining device profiles against the AI workloads associated with each employee persona. Many employees may remain well served by a common configuration, while roles that run larger local models, process richer inputs, or sustain AI activity alongside other business applications may need additional RAM or compute capacity. Representative testing can show whether current device tiers are sufficient and where a differentiated configuration is justified. 

The available range may also extend beyond conventional employee PC tiers. Some workflows may use compact, always-on systems or specialized deskside AI compute alongside general-purpose laptops and desktops. Device planning should therefore consider both the employee’s primary device and any dedicated local infrastructure required by especially demanding workloads.

A field inspection workflow makes these dependencies tangible. Voice dictation could populate structured inspection forms. The camera could capture images for review, while pen annotations on diagrams provide additional input. AI could generate a structured inspection summary locally, allowing portions of the workflow to continue without a continuous cloud connection. Outputs would remain subject to application requirements and human review. 

Copilot provides another illustration of this model. With appropriate permission, an experience can combine cloud intelligence with local context, local models, and actions performed on the device. The relevant architectural question is not whether the experience is local or cloud, but which parts of the experience should use each environment and what controls govern the exchange between them. 

With appropriate permission, an experience can combine cloud intelligence with local context, local models, and actions performed on the device.

Local AI workloads still have to share the device with everything else the employee is running. Speech, images, ink, and model output may be processed while the device is also running a browser, collaboration software, and business applications. Local AI therefore has to be evaluated as part of the complete device workload rather than as an isolated benchmark. 

At Surface, we are working alongside Windows to help shape a portfolio of devices designed for the evolving AI landscape. Surface Pro and Surface Laptop support flexible work styles, providing intuitive interactions and premium experiences spanning on-device and cloud-based AI. Surface Laptop Ultra extends the portfolio with portable performance for developers, AI builders, creators, and others working at the frontier of local AI. Configurations across the portfolio provide additional flexibility to match devices to roles and workloads. 

Surface Laptop Ultra extends the portfolio with portable performance for developers, AI builders, creators, and others working at the frontier of local AI

As we design these devices, we consider how compute, RAM, power, security, and manageability affect which workloads can run effectively near the user and how local processing fits into an organization’s broader AI environment and how customers can choose the configuration that best meets their needs. 

Testing the intended application on representative employee hardware can provide workload-specific evidence that complements component specifications. An accelerator specification alone cannot show how the model, runtime, operating system, business applications, and device resources will behave together throughout a working day. Teams may need to measure response time, RAM pressure, sustained performance, power use, and application behavior under realistic workloads before deciding where inference should run.

Account for the economics of local inference 

Workload placement is also a factor in evaluating operating costs. When an appropriate workload runs through an on-device model, it uses local compute rather than making a cloud inference call. For suitable, frequently repeated tasks, local inference may reduce reliance on recurring cloud inference, subject to the full costs of development, deployment, management, and support. 

Local processing may also allow organizations to reserve cloud tokens and higher-cost reasoning capacity for the tasks that need them most. The value is not simply avoiding an individual cloud call; it is allocating cloud resources to work where their additional capability materially improves the result. 

Local inference still carries costs for hardware, power, software licensing, model deployment, management, servicing, and support. Model quality matters as well. If a local model requires multiple attempts before a cloud model ultimately completes the task, the organization needs to account for the entire path. 

Organizations therefore need to evaluate the overall result, including output quality, responsiveness, reliability, and operating cost. A task performed occasionally may offer little economic reason to run locally. A focused operation repeated thousands of times across a large fleet may present a stronger case when the cost of developing, deploying, and maintaining a local model is weighed against the cloud inference costs it could avoid. 

Start with the workload and measure the result 

IT and application teams can begin with a familiar workflow and identify a specific task that may benefit from local, cloud, or hybrid execution. Requirements for output quality, response time, privacy, connectivity, failure handling, and permitted cloud fallback provide criteria for determining the best placement and whether it improves the experience. 

The Microsoft platform gives organizations flexibility to run workloads locally, run them in the cloud, or distribute them across both environments within the same application. Teams can also select the Microsoft or third-party models and services that best meet each workload’s requirements, allowing architecture decisions to follow business, security, performance, and operational needs rather than a single prescribed execution model.

The selected workload and application path should then be tested on the hardware employees actually use under normal working conditions. Application owners, endpoint teams, and security teams should evaluate access, stored outputs, logs, model updates, endpoint protection, and connections to cloud services. When agents can use tools or take actions, teams should also validate containment boundaries, agent identity, permitted resources, approval requirements, and the ability to audit or interrupt agent activity. 

A practical starting point is to identify priority AI workloads, map them to the employee personas expected to use them, and test the appropriate local, cloud, or hybrid execution path on representative configurations. Organizations can then retain standard device tiers where they meet the requirement and adjust profiles only where measured results support the change. 

Windows is evolving from a device operating system into an execution and management layer for hybrid intelligence, bringing local models, cloud services, agents, tools, and policy-controlled resources into a common environment. 

As enterprise AI architecture becomes more distributed, endpoint strategy becomes part of AI strategy. Models and workloads can operate across devices, organizational infrastructure, and cloud platforms within the same application. Managing that environment requires teams to understand each workload’s requirements, place it deliberately, and measure the complete result. 

Surface is designed to represent the best of Windows

Bringing hybrid intelligence to the endpoint

Surface is designed to represent the best of Windows and demonstrate the Microsoft platform on first-party hardware. Innovative devices from trusted Windows partners expand the range of devices and configurations available to customers. Together, these first- and third-party options give organizations greater choice in how they adopt AI across their workforce. 

As the PC takes on a larger role in that architecture, endpoint strategy becomes part of AI strategy. Ultimately, organizations that evaluate device compute alongside cloud resources can make more deliberate decisions about where inference runs, build applications around the strengths of each environment, and make effective use of the infrastructure already in employees’ hands. 

This article is part of a series designed to help IT leaders understand how hybrid intelligence is changing application architecture, endpoint strategy, and device planning. Stay tuned to the IT Pro Blog for more practical guidance in upcoming articles.