AI agents on Microsoft Fabric for faster retail merchandising decisions
For our latest in the Partner Spotlight series, we’re highlighting a partner building business-ready AI agents on Microsoft Fabric, so organizations can turn governed enterprise data into faster decisions and automated workflows. I connected with the team at Lucid Data Hub to learn how Lucid Agents Hub brings agentic experiences directly to customers’ data in OneLake, helping retail teams move beyond manual reporting and into repeatable, insight-driven action. About Venu Amancha, Founder & CEO, Lucid Data Hub builds business-ready AI agents that run directly on enterprise data within Microsoft Fabric. Our platform, Lucid Agents Hub, enables organizations to move beyond reporting and into automated, insight-driven workflows without moving data outside their existing security boundaries. _______________________________________________________________________________________________________________________________________________________________ [JR] Who is your solution designed for, and what does it help them do? [VA] Lucid Agents Hub is designed for teams who need to make frequent, high-impact decisions from large volumes of operational data especially merchandising teams, buyers, and store operations leaders in retail. Instead of spending hours assembling recaps and interpreting dashboards, they can receive agent-generated insights and clear, actionable recommendations on a predictable cadence. The AI agent eliminated that manual cycle entirely. It now surfaces those insights automatically, every week, in minutes. One example is our Retail Sales Performance AI Agent, which automates the weekly sales insights cycle for merchandising teams and buyers by analyzing millions of rows of weekly sales, item, and store data across banners and store clusters. [JR] Can you give an example of how the Retail Sales Performance AI Agent solved a customer’s problem? [VA] At Heritage Grocers Group, merchandising teams spent 5+ hours every week manually building sales recaps. They could see what happened—but not why. Buyers lacked a clear view of category trends, item-level performance, quantity shifts, and store-cluster patterns. The Retail Sales Performance AI Agent eliminated that manual cycle. It now surfaces those insights automatically every week in minutes detecting item-level declines, identifying fast-moving margin-positive SKUs, flagging underperforming items by store cluster, and delivering recommendations directly to buyers and store managers. [JR] Which Microsoft technologies or services are foundational to what you’re building? [VA] The solution runs natively on Microsoft Fabric, using OneLake as the unified data layer. Our agents operate directly on enterprise data and inherit existing governance and access controls without additional configuration. Outputs flow into the dashboards, collaboration platforms, and reporting workflows customers already use, so insights show up where decisions get made. Microsoft Fabric was a deliberate choice, not just a default. Our enterprise customers especially in retail already have their critical data living in the Microsoft ecosystem. OneLake means there’s a single, governed copy of that data. No duplication, no movement, no additional risk surface. Our agents read directly from that layer, which means the security and compliance boundaries that customers have already invested in carry over automatically. The value of building on Fabric goes beyond the technical architecture. It fundamentally changes how enterprise buyers evaluate and procure a solution like ours. When IT and security teams see that agents operate entirely within their existing Fabric environment with role-based access controls, workspace permissions, and audit logs they already control. It removes the largest barrier to enterprise adoption: trust. Procurement conversations that used to require months of security review cycles are now dramatically faster. What we didn’t fully anticipate was how much Fabric’s native integration capabilities would simplify end-to-end delivery. Going in, we expected to spend significant engineering time on data pipeline infrastructure. What we found instead was that Fabric’s data ingestion, lakehouse, and compute layers fit together in a way that let our team focus almost entirely on agent logic and business outcomes, not infrastructure plumbing. That shift in where we spend our effort has meaningfully accelerated how quickly we can deploy for new customers and extend to new use cases. [JR] How are you using AI today in Lucid Agents Hub, and what business outcomes have customers seen? [VA] We use Microsoft Azure AI Foundry for core AI and language model capabilities, and Microsoft Fabric Copilot (Fabric IQ) as the data and compute backbone. Together, they power agents that analyze weekly sales data across banners and store clusters, generate narrative-quality insights at the category and SKU level, and deliver clear recommendations without human intervention in the analysis cycle. 5+ hours of weekly manual effort eliminated Item-level sales declines and fast-moving margin-positive SKUs surfaced automatically Top-growth categories and underperforming items identified by store cluster Recommendations delivered directly to buyers and store managers weekly This all leads to faster decisions, stronger merchandising actions, and measurable improvements in product mix, availability, and overall sales performance. [JR] Any architectural decisions or best practices you’d recommend to other partners building agents? How did you approach building securely? [VA] We designed the solution as a coordinated set of specialized agents one for data ingestion, one for validation, and one for insight generation and delivery. Each agent owns a focused task, and together they run as a connected, end-to-end workflow. This makes the system easier to maintain, consistent in its logic, and straightforward to extend to new banners, categories, or use cases. Agents run entirely within the customer’s Microsoft Fabric environment data never leaves the customer’s security perimeter. All access controls, role-based permissions, and governance policies are inherited directly from Fabric. [JR] What motivated you to publish on Microsoft Marketplace? And did you use any Microsoft tools or benefits to support your publishing process? [VA] Publishing on Microsoft Marketplace was a straightforward decision. It gives enterprise customers immediate confidence that they’re procuring from a trusted, Microsoft-validated source instead of navigating a separate vendor relationship. It also simplifies procurement transactions run through an established Microsoft channel; so, customers can move faster than in traditional sales cycles. And it expands our reach to buyers already operating in the Microsoft ecosystem who actively look to Marketplace for solutions. We actively use Marketplace Rewards, which has been valuable for amplifying go-to-market efforts and accessing Microsoft co-marketing resources. We also leverage AI-enabled Marketplace Listing Optimization and related Marketplace content guidance provided through Marketplace Rewards. We used this support primarily to improve our marketplace messaging, positioning, and listing content so it would better resonate with enterprise buyers evaluating solutions within the Microsoft ecosystem. [JR] What key takeaways would you share with other partners building and publishing agents? Any unexpected wins or challenges along the way? [VA] Building and publishing agents can be a complicated endeavor. To other partners, we’d say, start with workflows that are repetitive and directly tied to decisions weekly merchandising recaps are a perfect example. Think end-to-end, not task by task. And build on governed enterprise data from the start, because that’s what drives trust and adoption. An unexpected win was how quickly merchandising teams adapted. Receiving plain-language summaries broken down by banner, store cluster, category, and SKU was more accessible than navigating dashboards. Teams made faster, more confident decisions without needing to interpret raw data themselves. _______________________________________________________________________________________________________________________________________________________________ Closing reflection Lucid Data Hub shows how agents built on Microsoft Fabric can turn governed enterprise data into repeatable, decision-ready insight helping teams act faster while keeping security boundaries and access controls intact.How governed agentic AI is transforming professional services workflows
Discover how partners are building industry-specific AI solutions in Microsoft Marketplace using governed, agentic workflows tailored for professional services. In the latest Partner Spotlight, Richard Baskerville, Senior Director of Strategic Global Alliances at Intapp, shares how “Firm AI” delivers purpose-built capabilities aligned to the unique needs of professional and financial services firms—combining deep workflow expertise with the scale, security, and compliance of Microsoft Azure. This article explores how agentic AI enables end-to-end workflow automation while maintaining human accountability, helping firms modernize operations across client engagement, risk management, and business processes. See how Microsoft Marketplace empowers partners to bring trusted, enterprise-ready AI solutions to customers, accelerating adoption and unlocking scalable growth opportunities. Read the full article: Firm AI for professional services: governed, agentic workflows built on Microsoft Azure | Microsoft Community HubCustomer office hour: Dragon Copilot AI apps and agents in Microsoft Marketplace
May 28 | 8:30 AM PT Join the next Microsoft Marketplace customer office hours session to explore how AI apps and agents and Dragon Copilot can help healthcare organizations reduce administrative burden and streamline clinical workflows. In this session, you’ll learn how to move from AI exploration to real world deployment. Discover how Dragon Copilot can help accelerate time to value by connecting with trusted partner solutions that streamline workflows and embed AI directly into daily operations. Learn more and attend Dragon Copilot AI apps and agents in Microsoft Marketplace - Microsoft Marketplace CommunityWhen cloud apps become a weak link: How FortiAppSec Cloud in Microsoft Marketplace bridges the gap
In this guest blog post, Srija Reddy Allam, Cloud Security/DevOps Architect, Fortinet, discusses the increase of attacks targeted at web applications and APIs and how FortiAppSec Cloud in Microsoft Marketplace provides a layer of adaptive security to address the challenge.Design CI/CD for AI apps and agents selling through Microsoft Marketplace
In the previous post, Design observability for AI apps and agents selling through Microsoft Marketplace, we focused on observability—making AI app and agent behavior visible and explainable. Execution paths, retries, degradation patterns, and agent decisions can now be observed across environments and tenants. With that visibility in place, a new challenge emerges: how do you safely modify an AI system whose behavior you can now observe? You can always get curated step-by-step guidance through building, publishing and selling apps for Marketplace through App Advisor. This post is part of a series on building and publishing well-architected AI apps and agents in Microsoft Marketplace. The series focuses on AI apps and agents that are architected, hosted, and operated on Azure, with guidance aligned to building and selling solutions through Microsoft Marketplace. Using continuous integration/continuous delivery (CI/CD) to control AI system evolution AI apps and agents introduce numerous novel ways that production behavior can change. In addition to application code, updates to configuration, prompts, models, and guardrails, agent logic can alter execution, cost, and outcomes—often immediately and across tenants. CI/CD defines how these changes reach production. Without a structured delivery path, behavior‑shaping updates risk entering runtime without validation or recovery paths, making system behavior difficult to explain or reverse once customers encounter it. AI solutions are typically built and operated as cloud applications. Software delivery of cloud services, and the supporting components that enable it, remains part of the CI/CD pipeline, and any instability in these foundational components directly propagates into AI behavior. AI systems add two additional sources of change that require explicit control. MLOps governs model evolution. Agents introduce further variability, as agent logic and configuration evolve. CI/CD is what prevents these change vectors from interacting unpredictably across both publisher and customer environments. Core CI/CD requirements for AI apps and agents For AI apps and agents, CI/CD determines whether deployment strategies can be applied safely. Progressive rollouts, ring deployments, feature flags, and kill switches all rely on pipelines that isolate change, validate behavior, and support rollback. Observability provides insight into behavior; CI/CD controls when and how that behavior is allowed to change. CI/CD must reliably provision, configure, and promote cloud native infrastructure, including but not limited to front-end services, APIs, storage, identity, and networking across environments. Agent behavior depends directly on the stability of the platform it runs on. AI systems introduce additional CI/CD requirements through MLOps and agents. Model versions, routing logic, and evaluation configurations must move through pipelines as deployable artifacts, with isolation, validation, and rollback built in. Changes to models affect latency, cost, and outcomes even when application code remains unchanged, making promotion controls necessary at the model layer. A well-run CI/CD pipeline should positively impact AI models and agents in the following ways: Change isolation ensures code, prompts, models, and configuration evolve independently. Artifact versioning beyond code treats prompts, policies, tools, and models as release assets. Behavioral validation evaluates outcomes, constraints, and patterns rather than single responses. Safe promotion controls gate model and agent releases based on observed behavior. Rollback readiness allows fast reversion when model or agent behavior degrades. Building behavioral baselines for AI solutions using CI/CD Before an AI system is built by a pipeline, it is built by a team. CI/CD build pipelines are where these contributions are stitched together. Product managers define scope and constraints. UX designers shape how behavior is experienced. Full‑stack engineers assemble application logic. AI engineers wire reasoning and tools. Data engineers and data scientists curate data and models. In AI systems, a build does more than compile code. It captures a shared agreement across roles about what the system is expected to do. Application code, orchestration logic, prompts, configuration, guardrails, routing rules, and trained or fine‑tuned models are assembled into a single versioned artifact. That artifact represents a coordinated snapshot of intent, behavior, and constraints. This coordination must declare which models, prompts, policies, and tool definitions are included. Implicit dependencies—such as dynamically changing prompts or unpinned models—break shared understanding across teams and introduce behavior changes without acknowledgement. A successful build confirms that contributions from multiple roles are compatible and executable together. It does not decide when customers see the change. That decision belongs later, where behavior can be evaluated deliberately, enforced by build pipelines that are separating assembly from release. Testing AI solutions with CI/CD pipelines When an agent is updated, the first task is straightforward: the agent’s code changes. Logic is refined, tools are added, limits are adjusted. That change moves through the CI/CD pipeline, where it is built, packaged, and validated in isolation. At this point, the focus is narrow—does this agent compile, configure, and execute as expected? The second step widens the lens. The update now moves through testing aligned to the layers beneath it. For cloud solutions, tests confirm the platform still behaves as assumed: infrastructure provisions correctly, APIs and identity boundaries remain intact, and dependencies remain reachable. These tests ensure the environment can support execution before behavior is evaluated. Next, MLOps tests assess whether model behavior still aligns with system expectations. New model versions, routing logic, or provider changes are evaluated for cost, latency, and outcome consistency. The goal is not identical responses, but bounded behavior within known limits. Finally, testing shifts to the agentic system as a whole. Other agents need to be made aware of the new capabilities. When you go to update the agent the first job you have to do is update the agent code. The second job you have to do is to use your CI/CD pipeline to build, test and release that code. The third job is to test that the entire agentic system is running smoothly together. At this stage, testing answers a different question: not does the agent work, but does the system still work together. CI/CD release management as team coordination Once testing confirms that behavior remains within expected bounds, release management determines how changes are introduced and observed under real conditions. In AI systems, release management must reflect where change originates and how risk propagates across layers. Within the cloud services that support the AI solution, release management focuses on scope and blast‑radius control. Examples include staged rollout of infrastructure updates, controlled exposure of new API versions, and limiting configuration changes to specific environments or tenants before going global. These steps allow both publisher and customer teams to observe stability and dependency behavior under load. For MLOps, release management governs behavioral shifts introduced by model changes. Common patterns include routing a small percentage of requests to a new model version, limiting exposure to specific customer segments, or restricting usage to defined request types. This allows teams to compare cost, latency, and outcome patterns before expanding exposure. For agents, release management controls how new behaviors surface. Prompt updates, tool access changes, or guardrail adjustments may be released to specific workflows, tenants, or traffic slices. This makes it possible to observe planning depth, retry behavior, and termination patterns without affecting all users simultaneously. Rollback readiness remains essential. Release paths must allow fast reversion using version pinning or traffic shifting rather than full redeployment. Release management creates space to observe, adjust, and respond before changes reach full Marketplace scale. Deployment as a shared boundary Effective deployment pipelines ensure that software, models, and agent behavior enter production together, with changes explicitly acknowledged and observable. Versioning and rollback remain available, but deployment defines the moment when coordinated decisions become customer‑visible. Cloud service—For the software, deployment governs application code and supporting platform changes. These remain necessary foundations. Application binaries, infrastructure templates, runtime configuration, and orchestration must enter production in a known, versioned state so operational behavior can be correlated with specific changes. MLOps—Model version updates, routing rules, provider switches, and evaluation configurations can change system behavior without modifying application code. Deployment pipelines must therefore treat these artifacts as deployable units, subject to the same versioning, promotion, and rollback mechanics as software releases. Agent—Deployment includes behavior‑defining inputs such as prompts and system messages, tool definitions and permissions, guardrails, and execution limits. Changes directly affect how agents plan, execute, and terminate work. Allowing these inputs to change outside deployment pipelines breaks traceability and weakens accountability across teams. How CI/CD best practices positively impact marketplace readiness Customers expect updates to arrive in predictable ways. They expect that behavior changes can be explained, that issues can be reversed without prolonged disruption, and that outcomes remain consistent across trials and production use. CI/CD pipelines make these expectations achievable by ensuring changes are versioned, staged, and observable as they move through environments. Reliability depends on limiting how far unstable behavior propagates. Billing accuracy depends on knowing when changes alter execution paths, token usage, or metering logic. Compliance depends on being able to identify which versions of software, models, and agent configurations were active at a given time. Offer type shapes how CI/CD is applied. For transactable SaaS offers, CI/CD operates entirely within the publisher’s environment. For container offers and Azure Managed Applications, deployment boundaries extend to customer environments requiring a CI/CD hand-off between publisher and customer pipelines. Publisher CI/CD responsibilities for AI solutions Publishers must define what constitutes a deployable change. Updates to software, models, prompts, agent configuration, guardrails, or limits should not enter customer environments or generally available code implicitly. Each change that can influence behavior must flow through the publisher’s CI/CD pipelines so it can be versioned, observed, and reversed if necessary. Additionally, CI/CD pipelines require validation and approval before promotion, ensuring that behavior‑altering updates do not reach customers without visibility or control. Publishers are also responsible for communicating behavior changes. Customers should be able to understand when updates affect outcomes, performance, or cost profiles. Customers should never experience silent behavior shifts, undocumented updates, or releases that cannot be recovered cleanly. When those occur, trust erodes quickly. In this context, CI/CD is part of how publishers establish reliability, accountability, and trust with Marketplace customers. Customer’s responsibility: CI/CD across environments (Dev / Stage / Prod) While publishers own CI/CD pipelines, customers play an important role in how AI systems are evaluated and adopted across environments. AI behavior often manifests differently across Dev, Stage, and Prod because operating conditions change as systems move toward real usage. As environments scale, dependency interactions increase, traffic patterns diversify, and tenant behavior becomes less predictable—revealing execution paths and constraints that are not exercised earlier. These differences affect how behavior appears during evaluation and rollout. To keep behavior interpretable across environments, pipeline structure matters. CI/CD pipelines, validation steps, and promotion criteria should operate consistently so signals observed earlier can be understood later. When these mechanics diverge between environments, it becomes difficult to attribute changes in behavior to specific updates or conditions. Staging environments serve as a behavioral proving ground. They allow customers to observe retries, limits, degradation paths, and cost behavior under conditions that more closely resemble production. Trials often run against production‑like configurations, which means CI/CD gaps surface early. When behavior differs from expectations, the consistency of pipelines determines how quickly teams can diagnose and respond. What’s next in the journey With CI/CD establishing control over how AI systems change, the next focus is how those changes are introduced safely at runtime. The following posts cover deployment strategies, progressive rollouts, and operational patterns that allow AI apps and agents to evolve while remaining stable, observable, and ready for Marketplace scale. Key resources See curated, step-by-step guidance to help you build, publish, or sell your app or agent (no matter where you start) in App Advisor Quick-Start Development Toolkit can connect you with code templates for AI solution patterns Microsoft AI Envisioning Day Events How to build and publish AI apps and agents for Microsoft Marketplace Get over $126K USD in benefits and technical consultations to help you replicate and publish your app with ISV Success
Build observability for scalable AI apps and agents selling through Microsoft Marketplace
Discover how to design observability for AI apps and agents selling through Microsoft Marketplace. This Marketplace Community article explains why visibility into execution behavior is essential for operating AI systems confidently at scale—not just keeping them running. As AI apps and agents reason, branch, retry, and exit dynamically at runtime, traditional infrastructure metrics fall short. Behavioral signals such as execution flow, token usage, latency, and failure patterns help explain what systems are doing, why outcomes occur, and how limits and safeguards shape behavior across tenants and environments. Learn how observability turns runtime telemetry into clarity that supports customer trust, usage‑based billing, and scalable operations. Read more: Design observability for AI apps and agents selling through Microsoft MarketplaceFirm AI for professional services: governed, agentic workflows built on Microsoft Azure
In this installment of our Partner Spotlight series, we’re highlighting partners building industry-focused AI solutions and bringing them to customers through Microsoft Marketplace. I connected with Richard Baskerville from Intapp to learn how the company is delivering Firm AI—governed, agentic capabilities designed specifically for professional and financial services firms—while aligning with Microsoft Azure for security, scale, and enterprise procurement. Intapp’s approach shows what it looks like to pair deep vertical workflow expertise with a trusted cloud platform so customers can adopt AI in a way that is both practical and accountable. About Richard Baskerville Richard Baskerville is a Senior Director at Intapp, where he helps shape strategic AI alliances across the Microsoft ecosystem and beyond. _______________________________________________________________________________________________________________________________________________________________ [JR] Tell us about Intapp. What inspired the founding of the company, and what problems do your solutions help customers solve? [RB] Intapp was founded on a durable observation: professional firms—law firms, accounting practices, private capital firms, and investment banks—operate differently than general enterprises. Their workflows are shaped by client relationships, professional obligations, and regulatory requirements that generic software was never designed to handle. Our founders saw that these firms were either building bespoke systems at enormous cost or forcing themselves into enterprise tools that didn’t fit. Today, Intapp delivers Firm AI—governed AI purpose-built for professional services. Our solutions span the full firm lifecycle: business development through Intapp DealCloud, time capture and billing through Intapp Time, and risk and compliance management across Intapp Conflicts, Intake, Terms, Walls, and Employee Compliance. Underpinning it all is Intapp Celeste, our agentic AI platform, which puts AI to work on the specific workflows that drive firm performance—while keeping humans accountable and in control. For firms where every engagement carries professional liability, that governance layer isn’t a feature; it’s the foundation. [JR] What industries or types of organizations do you primarily serve today? [RB] Intapp serves professional and financial services firms exclusively. That focus is intentional—and it’s what makes us different. Our customers are law firms, accounting and consulting practices, investment banks and advisory firms, private capital managers, and real assets firms. Many are among the largest in their categories globally. These firms share characteristics that set them apart from general enterprises: partnership structures, billable-hour economics, client conflict management, regulatory oversight, and relationship networks spanning decades. Generic CRM or ERP tools aren’t built for these dynamics. Intapp is. That vertical depth—built over more than 20 years—is what Firm AI means in practice: AI that understands the context of a law firm partner’s client obligations or a dealmaker’s fund-level requirements, not just the general shape of business software. [JR] What were your initial expectations for Microsoft Marketplace when you first started your journey? [RB] Our initial expectation was straightforward: Marketplace would give us a cleaner path to transact with customers already operating inside the Microsoft ecosystem. Many of our customers—large law firms and financial services firms—had already committed significant Azure spend through enterprise agreements. Marketplace offered a way to meet them commercially where they already were. What we underestimated was how much Marketplace would shape the broader partnership. We went in expecting a distribution channel. What we found was a framework that connected co-sell, Azure consumption alignment, and joint go-to-market in ways that changed how our teams engaged. The commercial mechanics—particularly MACC drawdown eligibility—became a real conversation-opener with procurement and finance stakeholders, not just a contract path. [JR] What applications do you have available in Microsoft Marketplace, and how do they help customers? [RB] Intapp has 12 SaaS solutions available in Microsoft Marketplace today, transacted via private offers. They span the full professional services firm lifecycle—from relationship intelligence and deal management with Intapp DealCloud, to time capture with Intapp Time, to risk and compliance management across Intapp Conflicts, Intake, Terms, Walls, and Employee Compliance. Because we focus exclusively on professional and financial services, customers aren’t buying horizontal software adapted for their industry; they’re buying solutions designed for their workflows, compliance obligations, and client structures. Looking ahead, Intapp Celeste—our agentic AI platform—will be available in Marketplace via a consumption-based, metered model. That structure matches how agentic AI gets used: variably, tied to real firm activity, and governed end-to-end. [JR] What were the biggest lessons you learned early on when selling through Marketplace? [RB] Three lessons stood out early: Listing is the beginning, not the end. Initial traction required deliberate investment in co-sell enablement—ensuring Microsoft field teams could position Intapp clearly, not just point to a catalog entry. Specificity wins. Our customers are sophisticated buyers. What worked was leading with vertical relevance—speaking directly to the compliance requirements of a law firm or the relationship-data challenges of a private capital manager. Private offers require commercial fluency. Helping customers understand how Intapp maps to Azure commitments—and helping Microsoft sellers tell that story—made a material difference in deal velocity. [JR] How has your business changed with a transactable offer on the Marketplace? [RB] Transactable offers have changed how deals close. Customers with existing Azure commitments can apply Intapp spend against their MACC, removing a procurement obstacle that previously added months to cycles. Finance and procurement can work within familiar Microsoft frameworks rather than running separate vendor onboarding. Marketplace has also expanded our reach. Microsoft’s field organization has relationships we can’t replicate at scale, and co-sell has helped translate that reach into qualified pipeline—especially in segments where we previously had limited coverage. And the signal matters: being transactable in Marketplace reinforces that Intapp is an enterprise-grade partner, not a niche point solution. [JR] How has collaborating with Microsoft sellers impacted your Marketplace growth? [RB] Microsoft sellers are the activation mechanism for our Marketplace offers. Without field alignment, a listing is a catalog entry. With it, it becomes a joint pipeline motion. We’ve invested in enablement—giving sellers the vertical context to position Intapp credibly in front of legal and financial services CIOs, and making it easy to bring us into deals where Azure capabilities are already in the conversation. That alignment shows up in specific segments. In private capital and investment banking, Microsoft enterprise relationships often predate ours—co-sell provides warm introductions backed by a trusted infrastructure partner. In legal, where Microsoft 365 is near-universal, that adjacency and deep interoperability creates natural entry points. Co-sell turns those adjacencies into active pipeline rather than theoretical opportunity. [JR] What has made the co-sell relationship with Microsoft particularly valuable for Intapp? [RB] Co-sell works because it’s structurally aligned, not just commercially convenient. Microsoft’s investment in these verticals—through industry clouds, compliance frameworks, and dedicated field teams—maps directly onto Intapp’s customer base. We’re selling into the same firms, with the same platform expectations underpinning both offerings. What makes it particularly valuable is the mutual trust transfer. Firms hold Microsoft to a high standard for security, data governance, and regulatory compliance. When Microsoft sellers bring Intapp into the conversation, that credibility extends to us and compresses the trust-building phase of an enterprise cycle—especially in regulated industries. [JR] How does Microsoft Marketplace fit into Intapp’s long-term growth strategy? [RB] Marketplace is central to how we scale Firm AI globally. Our ambition is to be the governed AI platform of record for professional firms across legal, private capital, accounting, and advisory. Helping customers decrement MACC through Marketplace purchases is a clear win-win because it aligns platform investment with workflow outcomes. The upcoming Marketplace availability of Intapp Celeste which will offer different commercial models for customers (e.g. consumption based) marks the next phase. As Celeste deepens integration with Microsoft 365, Teams, and Azure AI services, the commercial and technical stories converge—customers can both buy and operate within an architecture where Firm AI and Microsoft’s platform reinforce each other. [JR] What advice would you give other SDCs who are just starting their Microsoft Marketplace journey? [RB] Three things matter most early on: Earn the co-sell relationship before you need it. Invest in enablement early so Microsoft sellers have enough vertical context to represent your value clearly. Get your commercial model right for the channel. Understand how private offers interact with Azure commitments, and plan for consumption-based pricing where it fits AI usage. Lead with a sharp point of view. The partners who gain traction fastest are the obvious choice for a specific industry workflow or problem—know what that is and communicate it consistently. _______________________________________________________________________________________________________________________________________________________________ Closing reflection Intapp’s Marketplace journey shows that industry-specific, governed AI wins when it’s paired with an enterprise platform customers already trust. By making solutions transactable—especially through private offers that align to customer’s existing Azure commitments—Intapp reduces procurement friction and accelerates adoption. And like many successful partners, their growth ultimately comes down to enablement: clear vertical messaging and tight co-sell alignment that turns Marketplace presence into a real, qualified pipeline.Design observability for AI apps and agents selling through Microsoft Marketplace
In the last post, API resilience and reliability patterns for AI apps and agents, we focused on what happens when AI systems encounter failure—and how resilient execution paths keep that failure contained. Timeouts fire with intent. Retries stay bounded. Circuit breakers provide overload protection. When resilience is designed well, your system continues to function even as conditions change. You can always get curated step-by-step guidance through building, publishing and selling apps for Marketplace through App Advisor. This post is part of a series on building and publishing well-architected AI apps and agents in Microsoft Marketplace. The series focuses on AI apps and agents that are architected, hosted, and operated on Azure, with guidance aligned to building and selling solutions through Microsoft Marketplace. Observability for AI systems AI apps and agents are shifting traditional observability, which was designed for systems based on simple assumptions, where requests followed linear paths and workloads behaved predictably. Execution in AI systems consumes tokens at a highly variable rate rather than fixed compute units. Requests unfold across multiple reasoning steps. Agents perform work that spans APIs, models, retrieval layers, and applications. A single interaction may pause, branch, retry, or exit early depending on inferred intent, context, and constraints. Instead of asking whether services are running, observability for AI systems asks: what is the system doing right now—and why? Is an agent spending its time reasoning, waiting on dependencies, retrying tool calls, or exiting early due to enforced limits? Is cost increasing because value is increasing, or because execution paths are expanding without progress? AI observability requirements shift the focus in the following subtle, but critical ways: From resource availability to workflow state From performance metrics to signals From incidents to patterns Core observability dimensions for AI apps and agents Once observability shifts toward understanding behavior, clarity comes from tracking state across the agents in the workflow. For AI apps and agents, observable indicators, such as those detailed below, show how work unfolds and changes during real usage—especially in trials and early adoption: Execution flow shows how a request moves through agents, tools, and workflows. This highlights where execution progresses smoothly, where it slows, and where it concludes early. This makes agent outcomes explainable and keeps behavior consistent across tenants. Cost and token behavior reveals how execution translates into consumption. Token usage per request, per agent step, and per retry shows where value is being delivered and where execution paths expand without proportional benefit. This insight connects runtime behavior directly to Marketplace billing expectations and evaluations. Latency and wait states distinguish active processing from time spent waiting on dependencies. Seeing where time is consumed helps explain slow experiences and guides decisions about optimization, caching, or resilience improvements. Failure classification provides structure when systems degrade. Separating tool failures from planning failures, and transient issues from terminal exits, keeps investigations focused and prevents protective behavior from being misread as instability. Tenant‑level patterns surface how behavior repeats at scale. Uneven load, and recurring degradation often appear first during trials and shape the customer's perception. Together, these dimensions turn telemetry into understanding—supporting clearer conversations, faster triage, and predictable execution as usage grows. Why observability matters By this point in the journey, your AI app or agent has implemented bounded execution paths, cost controls, and quality of service safeguards. As a result, failure degrades gracefully instead of spreading. These resilience techniques determine how your solution behaves under pressure. The data gathered from observability platforms like Application Insights and Azure Monitor explains why it behaves that way. For AI and agentic systems, infrastructure health alone rarely answers the questions that matter. Services can be up, CPUs can be idle, and queues can look healthy while agents loop inefficiently, retries quietly expand cost, or workflows exit early without delivering value. From the customer’s perspective, the experience feels inconsistent even though the platform appears stable. Observability closes this gap by revealing system behavior rather than system status. It shows how requests move, where work concentrates, and how constraints shape outcomes. At Marketplace scale, these patterns repeat across tenants and trials. What appears once during an evaluation often appears again as adoption grows. Observability connects runtime behavior back to the design choices introduced in earlier posts: Usage‑based billing introduced variability in consumption Performance optimization introduced tradeoffs among latency, quality, and cost Resilience patterns introduced controlled failure and bounded execution Observability allows you to explain outcomes during trials, validate assumptions as usage grows, and operate with confidence across customers and environments. Without this visibility, teams react to symptoms. With it, they recognize patterns. From execution paths to behavioral signals Observability begins at the same place resilience begins—API boundaries. These boundaries define where responsibility shifts and where behavior becomes visible. Observability focuses on signals that explain decisions made by the system as it executes instead of relying on raw logs that describe isolated events. Every resilience mechanism emits behavioral signals. Viewed together, these signals provide far more value than logs alone. Logs answer whether something happened. Behavioral signals explain why it happened and how the system responded. Circuit breakers change state as load builds and recedes. Retry loops show whether failures resolve quickly or exhaust their limits. Timeout enforcement reveals where dependencies slow execution. Fallback paths and early terminations show how the system protects itself while preserving outcomes for customers. This perspective matters most for agents. Agent execution unfolds as a series of choices—plan, call a tool, retry, exit early—rather than a single request‑response cycle. Observability that tracks these decisions makes agent behavior understandable, consistent, and defensible as usage grows across customer tenants. Observability at the agent layer As AI systems become more agent‑driven, observability needs to move closer to where decisions are made. Agents introduce variability by design. They plan, adapt, and choose workflow paths dynamically. Without first‑class visibility into that behavior, execution can appear unpredictable even when the underlying system is healthy. Observability at the agent layer acts as the feedback loop that keeps execution safely bounded. It shows how agents use the freedom you give them—and where that freedom begins to stretch into inefficiency. Observability follows how the agent did its job instead of treating the agent’s interaction as a single outcome. Several indicators help make agent behavior understandable. Step count per request reveals how much reasoning effort a prompt requires. Planning iterations show whether an agent converges quickly or cycles through alternatives. Tool invocation frequency highlights when agents rely heavily on external systems. Early exits compared to full completion explain whether limits and fallbacks activate as designed. Taken together, these indicators help distinguish healthy exploration from inefficient reasoning and degraded execution. An agent exploring briefly before converging adds value. An agent looping through tools without progress signals pressure, uncertainty, or dependency issues. This distinction reinforces a core principle of agentic systems: models reason probabilistically, adapting to context as it changes. Your system observes deterministically—measuring execution, enforcing boundaries, and clarifying outcomes. When those roles stay separate and well‑instrumented, agent behavior becomes transparent, predictable, and ready for Marketplace scale. Observability across environments The type of Marketplace offer you choose shapes what observability customers expect and how responsibility is shared. For SaaS offers, publishers typically own end‑to‑end execution. Observability centers on agent behavior, workflow completion, token usage, latency, and dependency impact across tenants. Publishers rely on consistent signals—often surfaced through tools like Azure Monitor, Application Insights, and Microsoft AI Foundry—to explain how requests behave as scale and load increase. For container‑based offers and Azure Managed Applications, observability expectations are more distributed. Publishers expose clear execution outcomes, limits, and failure signals at application boundaries. Customers, in turn, observe infrastructure health, scaling behavior, and downstream systems within their own environments. This separation ensures each party has visibility into what they control without creating ambiguity. Learn more about Choosing your marketplace offer type for AI Apps and agents. Execution behavior differs across environments for predictable reasons. Scale increases, tenant mix broadens, and external dependencies behave differently under real load. What must stay consistent is how behavior is interpreted. Signal definitions, thresholds, and failure classification should mean the same thing in Dev, Stage, and Prod. Learn more about designing a reliable environment strategy for Microsoft Marketplace AI apps and agents. Staging environments are where this consistency is validated. Observing retries, timeouts, and graceful degradation before production prepares you for Marketplace evaluations, which often resemble production conditions. Observability gaps tend to appear first during customer evaluation—when clarity matters most. Publisher and customer visibility boundaries Purpose: Parallel Post #13 responsibility clarity, now for observability As observability matures across environments, clarity around responsibility becomes essential. For Marketplace solutions, trust grows when publishers and customers each see what they own—and understand where that visibility ends. Publishers are responsible for instrumenting execution paths end to end. That means making workflows traceable, limits visible, and failure modes explainable. Observability should surface behavior—how requests progressed, where execution concluded, and why—rather than exposing raw internal errors that require insider knowledge to interpret. Customers focus their observability on what they control. This includes monitoring downstream systems, infrastructure behavior, and environment‑level alerts within their own estate. When visibility aligns with ownership, teams can act quickly and decisively. Exposing too much internal detail can overwhelm customers and blur accountability. Observing too little behavior creates friction, especially when issues cross boundaries and lack context. Clear visibility enables faster triage, sharper ownership boundaries, and fewer escalations rooted in ambiguity. Observability as an enabler for scale, billing, and trust From a customer’s perspective, observability answers two fundamental questions: Can I understand what happened? and Can I trust this at scale? When the answer to both is clear, observability becomes part of the value your Marketplace offering delivers. When system behavior is visible and explainable, customers gain confidence that adoption and growth will remain predictable. Observability directly supports usage‑based billing by tying execution behavior to measured consumption. Clear visibility into token usage, retries, and execution paths helps validate how usage is calculated and supports transparent billing conversations. It also enables ongoing performance tuning and caching strategies by showing where latency accumulates, where work repeats, and where optimization delivers measurable impact. Observability reinforces confidence in resilience mechanisms, confirming that limits, fallbacks, and degradation paths activate as designed under real‑world conditions. Beyond validation, observability creates a continuous feedback loop. Execution data informs pricing adjustments, guides changes to limits, and helps refine default configurations as customer behavior evolves. What’s next in the journey With execution behavior observable and explainable, the focus shifts to how AI systems are operated safely as change accelerates. The upcoming posts will discuss deployment strategies, CI/CD pipelines for agents, and progressive rollouts build on this foundation—ensuring AI apps evolve confidently as usage and expectations grow. Key Resources See curated, step-by-step guidance to help you build, publish, or sell your app or agent (no matter where you start) in App Advisor Quick-Start Development Toolkit can connect you with code templates for AI solution patterns Microsoft AI Envisioning Day Events How to build and publish AI apps and agents for Microsoft Marketplace Get over $126K USD in benefits and technical consultations to help you replicate and publish your app with ISV SuccessAPI resilience and reliability patterns for AI apps and agents selling through Microsoft Marketplace
Why API resilience is a Marketplace readiness requirement The previous post Design Predictable AI Performance for Apps Selling Through Microsoft Marketplace showed how to design systems that behave predictably when things go right. This post focuses on what happens when they do not. Imagine an enterprise customer launching a trial of your AI agent from Microsoft Marketplace. The first few interactions work beautifully. Then a more complex request triggers a multi‑step agent workflow: retrieval, enrichment, validation, approval. One downstream API stalls for just long enough to push the workflow beyond its timeout. The agent retries. The retry fans out into additional calls. Tokens burn. Costs rise. Eventually the entire interaction fails ambiguously. From the customer’s perspective, the trial just “didn’t work” with no explanation or architecture diagram. Just a stalled agent and decreased confidence. AI apps and agents treat APIs as their execution backbone. Every model invocation, tool call, retrieval query, and workflow step depends on APIs behaving within expected bounds. Solutions with a single unstable dependency can affect many tenants simultaneously. You can always get curated step-by-step guidance through building, publishing and selling apps for Marketplace through App Advisor. This post is part of a series on building and publishing well-architected AI apps and agents in Microsoft Marketplace. The series focuses on AI apps and agents that are architected, hosted, and operated on Azure, with guidance aligned to building and selling solutions through Microsoft Marketplace. How AI and agentic workloads stress APIs differently Traditional API platforms often assume linear, predictable request patterns. One request in, one response out. AI apps produce bursty, non‑linear traffic shaped by user behavior, token budgets, and inference variability. Agents amplify this further. A single user request may trigger planning, branching logic, parallel tool calls, and dynamic retries—all before returning a result. Single‑turn inference calls tend to be synchronous and bounded. Agent workflows may run for minutes, traverse multiple services, and consume tokens unpredictably depending on intermediate outcomes. Happy‑path assumptions break down quickly. Reliability also compounds mathematically. If you chain five APIs, each with 99.9% availability, the composite reliability drops to roughly 99.5%. Add retries without bounds, and the system can degrade traffic rather than absorb failure. For AI systems, reliability must be defined across multiple dimensions: Availability: Are dependencies reachable? Timeout behavior: How long will the system wait? Error propagation: What information crosses boundaries? Recovery safety: Can operations be retried without harm? Data access and integrity: Is contextual data available, relevant, and trustworthy? Defining reliability for AI systems Reliability becomes the mechanism that preserves trust when uncertainty appears. Reliability in AI systems is more than “the model didn’t fail.” That framing is incomplete. True reliability means providing predictable behavior under partial failure, bounding execution when dependencies degrade, and failing clearly, safely, and consistently instead of unpredictably. For publishers providing AI solutions on Marketplace, this includes protecting customers from ambiguous states—workflows that half‑complete, retries that silently multiply costs, or agents that continue planning after their assumptions are no longer valid. Designing resilient API boundaries The shift toward reliable AI systems starts with how you think about API boundaries. In this context, an API boundary is the line where responsibility changes—between your app and a dependency, between orchestration and execution, or between your system and a customer‑ or partner‑owned service. These boundaries are deliberate points of control. You must decide: how long is a call allowed to run? What happens if it fails? Is a retry safe, and if so, how many times? When agents assume that APIs will be reliable, fast, or always available, failure starts becoming systemic. Well‑designed API boundaries stop execution early when reliability assumptions break. Explicit timeouts keep your system from waiting indefinitely when a dependency slows or an API call hangs. Bounded retries allow brief recovery without inflating cost, load, or complexity. Together, these constraints help your system behave predictably, even under stress. This is where your enforcement layers come into focus. For many Marketplace solutions, Azure API Management is where you turn design intent into predictable behavior. At this boundary, you define how your system responds under pressure—how much traffic is allowed, how tokens are budgeted, and how long requests are permitted to run. These policies give you a steady way to shape execution across tenants, even when the systems behind the boundary behave unpredictably. As workflows grow more complex, orchestration layers such as Azure Durable Functions or Logic Apps carry that intent forward. They give you a way to manage long‑running or multi‑step operations explicitly, with clear execution limits, defined retry behavior, and compensating actions when steps fail so you can keep control over how work progresses and how it concludes. Core API resilience patterns for AI apps and agents Several foundational patterns appear repeatedly in resilient AI solutions published on Marketplace. Timeouts and deadline propagation ensure no call waits indefinitely. For AI workloads, these limits should be token‑aware—longer prompts or higher‑cost models require proportional constraints. Deadlines should propagate across calls so upstream services remain informed. Bounded retries protect against transient failures but with pre-defined limits and quotas. In agent workflows, retries should be explicit, counted, and observable. Retrying API calls that execute actions, attempt and fail authentications, or create updates that exceed quotas can lead to runaway failures. Circuit breakers prevent cascading failure by opening when error rates exceed thresholds. Unlike guardrails—which enforce policy by intent—circuit breakers react to system state by pausing execution paths that are no longer reliable. Azure API Management and resilience libraries such as Polly in .NET provide practical implementations. Bulkheads isolate high‑risk or high‑cost operations. Separate concurrency pools, queues, or compute tiers prevent one tenant or workflow from consuming disproportionate resources. This is especially critical for expensive reasoning paths or third‑party dependencies. Idempotency keeps retries safe by ensuring that repeating the same request produces the same result. Agents that take real‑world actions—creating records, approving workflows, triggering payments—must attach idempotency keys so repeat attempts do not multiply side effects. Together, these patterns do not eliminate failure. They contain it. Agent‑specific reliability risks and mitigations Agent autonomy shifts how reliability behaves in practice. Agents change the shape of failure. Because they plan, reason, and act across multiple steps, a single issue rarely stays isolated. When autonomy increases, failures affect more of the workflow and do so faster. Most agent failures fall into two categories and treating them the same way creates instability. Tool failures occur when an external dependency slows, times out, or becomes unavailable. An API may reject a request, enforce a quota, or fail temporarily. These failures require containment. Your system should pause execution, apply fallback behavior, or exit cleanly once limits are reached. Allowing the agent to keep calling tools under these conditions increases cost and load without improving results. Planning failures occur when the agent’s reasoning breaks down. The plan itself is flawed, incomplete, or loops without converging on an outcome. These failures require correction. Step limits, loop detection, and execution caps keep planning from expanding indefinitely and signal when the system should stop and reassess. Making this distinction explicit is what keeps agent behavior predictable. You define how far execution can go—how many steps are allowed, how long a request may run end‑to‑end, and when the system should pause or conclude. By enforcing these limits outside the model, you give agents room to reason while your system provides the structure that contains failure and keeps execution steady as conditions change. As explored in Designing AI Guardrails for Apps and Agents in Microsoft Marketplace, guardrails define what an agent is allowed to do. Resilience patterns determine how your system holds up when dependencies degrade. Together, they enable agents that feel capable and autonomous while remaining stable, bounded, and ready for Marketplace scale. Reliability across external and third‑party APIs Marketplace AI apps rarely operate in isolation. They depend on customer‑owned systems, partner services, SaaS platforms, and external LLM APIs—each with different SLAs and failure modes. Publishers must absorb this variability rather than pass it directly to customers. That means handling throttling gracefully, surfacing authentication failures clearly, and isolating quota exhaustion. Token‑based rate limiting via Azure API Management is especially important for downstream LLM calls, where cost and availability intersect. Remember the SLA math: your effective reliability is the product of every dependency. Designing for the weakest link protects customer perception—and your own margins. Environment‑aware reliability validation As outlined in Designing a reliable environment strategy for Microsoft Marketplace, environment strategy underpins reliable promotion and confident scaling. Reliability cannot be tested only in production. Before Marketplace submission, failure behavior should be validated in staging. Timeouts should trigger as expected. Retries should stop when designed to stop. Circuit breakers should open—and close—predictably. Equally important is environment consistency. Dev, Stage, and Prod environments should enforce the same resilience policies, even if scale differs. Otherwise, failures will appear only when customers are watching. Azure Chaos Studio provides controlled fault injection to test these scenarios intentionally. The goal is to confirm that systems behave consistently under stress. Reliability, ownership, and Marketplace readiness As a publisher, you are responsible for resilient defaults, protection against cascading failures, predictable failure modes, and documented service expectations. Customers, in turn, remain responsible for the reliability of their downstream systems, environment‑level scaling, and internal monitoring. When this boundary is explicit, teams know where responsibility sits and how to respond when conditions change. When ownership is unclear, support escalations increase, accountability blurs, and confidence drops on both sides. Marketplace customers expect clarity about what your solution controls, what it depends on, and how issues are handled when they arise. That clarity directly shapes Marketplace readiness. Reliable execution paths influence certification reviews, determine whether enterprise pilots progress, and establish long‑term operational confidence. During trials, predictable behavior feels professional. It reduces surprise costs, shortens evaluation cycles, and makes adoption decisions easier. In this way, reliability acts as a trust signal and a sales enabler. When customers see that ownership is well-defined and failure is handled intentionally, AI adoption through Marketplace feels safe, bounded, and ready to scale. What’s next in the journey Once execution paths are resilient, your solution’s behavior becomes visible. Circuit breaker transitions, retry frequency, timeout events, and error propagation turn into operational signals that show how your AI app or agent behaves under real load and across customer tenants. This foundation enables the next layer of operational maturity—observability, safe deployment practices, CI/CD for agents, and ongoing evaluation—so you can understand behavior end‑to‑end and operate confidently as usage grows. Reliability makes AI adoption safe; observability makes it sustainable. Key Resources See curated, step-by-step guidance to help you build, publish, or sell your app or agent (no matter where you start) in App Advisor Quick-Start Development Toolkit can connect you with code templates for AI solution patterns Microsoft AI Envisioning Day Events How to build and publish AI apps and agents for Microsoft Marketplace Get over $126K USD in benefits and technical consultations to help you replicate and publish your app with ISV SuccessDesign predictable AI performance to scale selling through Microsoft Marketplace
Trade-offs in AI performance: latency, quality and cost Imagine a software company launches a customer trial for its new AI assistant through Microsoft Marketplace. The trial begins smoothly — until more complex queries take longer than a few seconds to return a response. The cause isn’t model failure. It’s an unbounded Retrieval‑Augmented Generation (RAG) pipeline retrieving 50 documents per query before synthesizing an answer. Latency increases. Runtime token usage expands. Trial‑stage infrastructure cost rises immediately. This exposes the core runtime tradeoff in enterprise AI systems: Latency ↔ Quality ↔ Cost Improving response quality often increases retrieval depth. Increasing retrieval depth expands token usage. Expanded token usage drives both cost and latency upward. This post is part of a series on building and publishing well-architected AI apps and agents in Microsoft Marketplace. The series focuses on AI apps and agents that are architected, hosted, and operated on Azure, with guidance aligned to building and selling solutions through Microsoft Marketplace. You can always get curated step-by-step guidance through building, publishing and selling apps for Marketplace through App Advisor. How traditional cost model assumptions break down for AI In classic software models, you expect predictable runtime costs such as license allocations, storage, compute time, bandwidth consumption, etc. But in AI-powered systems, that stability gives way to new complexities driven by token-based cost structures. These costs scale in unexpected ways, depending on the length of generated outputs, the depth of information retrieval, the number of reasoning steps an agent performs, and how often external tools are invoked. Consider the RAG pipeline scenario: retrieving five documents for a single query might create a 3,000-token prompt. If the pipeline instead pulls 50 documents, that prompt balloons to 15,000 tokens—before the AI even begins to infer an answer. And the unpredictability doesn’t stop there. Agent orchestration can introduce even more variability. Planning steps may stretch or shrink depending on the query, tool-calling systems might retry failed executions multiple times, and multi-branch workflows can run in parallel, all amplifying token consumption and cost. Keep costs bounded without sacrificing quality While unpredictable token usage and orchestration steps can quickly escalate infrastructure costs in AI-powered systems, design choices can prevent runaway expenses without compromising the quality of responses. To achieve this, engineers must balance procurement expectations set by pricing with real-time operational controls. For instance, use a multi-model tiered routing strategy to allow less complex queries to be handled by lightweight models, reserving advanced reasoning models for more demanding tasks. Combining this with token budgeting strategies—such as per-session caps and API Management token-limit policies—ensures that each interaction remains within defined boundaries. Cost-aware orchestration paths become essential when running AI workloads across multiple tenants, especially when retries and multi-branch workflows threaten to multiply inference consumption. By calibrating runtime guardrails to performance and cost signals, AI systems can be designed to fail gracefully and predictably, preventing ambiguous and expensive failures. Ultimately, the goal is to deliver high-quality results at scale, maintaining control over both costs and performance as usage grows. Achieving predictable latency: Business best practices across each layer For enterprise AI systems, ensuring fast and consistent response times—while balancing quality and cost—is a top priority. Predictable latency requires intentional design at every layer of your architecture. Interaction Layer: Set clear boundaries for incoming requests using Azure API Management rate‑limit and quota policies, such as rate-limit-by-key, scoped per subscription or tenant. These controls cap request throughput and request volume over time, preventing traffic spikes from overwhelming downstream AI services and ensuring consistent, predictable response behavior across tenants. Orchestration Layer: Define and restrict system execution paths. Limit reasoning depth in workflows so complex operations don’t unexpectedly slow things down. This keeps your business processes running smoothly and predictably. At the API boundary, Azure API Management can enforce deterministic routing, retry limits, and timeout policies, while backend orchestration services such as Azure Durable Functions or Logic Apps manage multi‑step workflows with explicit bounds on execution depth and retries. Model Layer: Choose models based on expected concurrency needs. Use fallback routing to redirect traffic during busy periods—so users don’t experience delays. Rely on Azure OpenAI Provisioned Throughput Units (PTUs) for steady baseline performance and enable PAYG overflow to handle temporary surges without sacrificing speed. Microsoft AI Foundry can be used to centrally manage model selection and routing policies, enabling consistent fallback strategies and governed use of multiple models across agents and workloads. Retrieval Layer: Optimize your document indexing and narrow the scope of data being searched. This means users get relevant information faster, and your system avoids unnecessary slowdowns. Services such as Azure AI Search enable scoped, indexed retrieval over structured and unstructured content, while integrating with Azure Blob Storage or Azure Cosmos DB as source data stores to support predictable, low‑latency access for RAG‑based AI workflows. Data Layer: Keep your compute and storage resources close together and aligned regionally. By minimizing cross-region data transfers, you reduce latency and boost reliability—critical for enterprise-grade AI. Across every layer, publishers are responsible for designing bounded, predictable defaults, while customers govern configuration, scale, and operational posture—a clear separation that reduces friction, improves trial outcomes, and accelerates Marketplace adoption. By applying these best practices decisively at every layer, software development companies can move beyond isolated optimizations and design AI solutions that behave predictably under real customer load. This approach enables customers to run meaningful trials, validate performance and cost assumptions early, and scale with confidence as demand grows. More importantly, it establishes a repeatable engineering foundation—one that supports faster iteration, clearer operational ownership, and successful commercialization through Microsoft Marketplace. Design caching into your architecture from the start Predictable AI performance relies on caching that’s intentionally designed into the architecture—not added after systems are already under load. In agent‑driven and retrieval‑augmented workflows, caching is foundational to controlling latency, stabilizing runtime costs, and keeping execution behavior consistent as usage scales. Effective designs cache work wherever outcomes are deterministic. Request‑level and semantic caching reduce redundant inference when users submit identical or meaning‑equivalent queries, while Azure API Management paired with Azure Managed Redis enables governed reuse at the intent level. Retrieval pipelines benefit from embedding and retrieval caching, which avoids repeated vectorization and unnecessary search overhead. Within orchestration flows, tool‑level caching ensures stable responses for deterministic calls such as policy checks or configuration lookups, and agent plan caching allows reasoning paths to be reused without re‑incurring planning cost. Caching must be paired with clear invalidation strategies—time‑based expiration, context‑aware refresh, and event‑driven updates—to preserve correctness and trust. In Marketplace deployments, multi‑tenant cache isolation and observability are essential. When caching is visible, governed, and intentional, it becomes a powerful enabler of predictable scale. What’s next in the journey With performance and cost under control, the next question is how your system behaves when something goes wrong. The next post explores API resilience and reliability patterns—because predictable performance only matters if your AI system continues to function through the inevitable failures that occur at Marketplace scale. Key Resources See curated, step-by-step guidance to help you build, publish, or sell your app or agent (no matter where you start) in App Advisor Quick-Start Development Toolkit can connect you with code templates for AI solution patterns Microsoft AI Envisioning Day Events How to build and publish AI apps and agents for Microsoft Marketplace Get over $126K USD in benefits and technical consultations to help you replicate and publish your app with ISV Success
