CI/CD for AI Agents on Microsoft Foundry
Introduction Building an AI agent is the straightforward part. Shipping it reliably to production with version control, evaluation-driven quality gates, multi-environment promotion, and enterprise governance is where most teams run into friction. Microsoft Foundry changes this. It is Microsoft's AI app and agent factory: a fully managed platform for building, deploying, and governing AI agents at scale. It provides a first-class agent runtime with built-in lifecycle management, making it possible to apply the same CI/CD rigour you already use for application software to AI agents — regardless of whether you are building containerised hosted agents or declarative prompt-based agents. This post walks through a complete, production-ready reference architecture for doing exactly that. You will find the GitHub Actions workflow, the Azure DevOps pipeline YAML, and the architecture diagram linked throughout. Reference implementation repository: foundry-agents-lifecycle and CI/CD for AI Agents on Microsoft Foundry Why Agent CI/CD Is Different Traditional software pipelines gate releases on test pass/fail. Agent pipelines require an additional, critical layer: evaluation-driven quality gates. Before any agent version can be promoted to the next environment, it must pass three categories of evaluation: Quality — answer correctness, task completion rate, hallucination rate Safety — grounded responses, policy compliance, tool usage validation Performance — token usage per query, p95 response latency A second key difference is the deployment unit. You are not deploying a binary or a container tag in isolation. You are deploying an agent version — an immutable artefact that bundles the model selection, system instructions, tool definitions, and configuration together. This is what enables deterministic promotion and full auditability across environments. "Agents follow a standard CI/CD pattern, but with a critical shift: promotion happens at the agent version level, and release gates are driven by evaluation outcomes, not just test results." Reference Architecture Figure 1: End-to-end CI/CD reference architecture for hosted and prompt-based agents on Microsoft Foundry. The architecture has five logical layers, flowing from developer commit to production monitoring: Layer 1 — Developer Layer The developer layer is a standard source-controlled repository in GitHub or Azure DevOps. It contains: Agent code written in Python or .NET agent.yaml or prompt definition files for prompt-based agents Tool configurations: MCP servers, REST API connectors, or other integrations Infrastructure as Code: Bicep or ARM templates for provisioning the Foundry project and dependencies Layer 2 — CI Pipeline (Build · Validate · Evaluate) Every push or pull request triggers the CI pipeline. It performs five steps: Docker build — for hosted agents, build and tag the container image Static checks — lint with ruff , security scan with bandit , agent YAML schema validation Unit and tool tests — pytest suites covering agent logic and tool integrations Evaluation gate — run evaluation datasets; fail the pipeline if thresholds are breached Image push — push the validated container to Azure Container Registry (ACR) Prompt-based agents skip the Docker build step. Instead, the YAML definition and prompt bundle are validated against schema and evaluated against golden datasets. Layer 3 — CD Pipeline (Multi-stage Promotion) A single agent version is promoted through three Foundry project environments: Stage Environment Activities Gate Stage 1 Dev Foundry Project Deploy vNext version, smoke tests, developer evals Eval quality thresholds Stage 2 Test / QA Foundry Project Scenario tests, HITL validation, safety evaluation Eval gates + human approval Stage 3 Production Foundry Project Promote version, enable endpoint, post-deploy smoke test Required reviewer approval Rollback is straightforward: switch the active version pointer back to the previous agent version. No re-deployment is needed. Layer 4 — Microsoft Foundry Agent Service The Foundry Agent Service runtime provides: Hosted agent runtime — managed container execution supporting Agent Framework, LangGraph, Semantic Kernel, or custom code Prompt-based agent runtime — declarative agent definitions, no container required Built-in lifecycle operations — version, start, stop, rollback Entra Agent Identity — each deployed version receives a dedicated Microsoft Entra managed identity RBAC and policy enforcement — Azure role-based access controls per project Observability — distributed traces, structured logs, and evaluation signals Layer 5 — Monitoring, Governance, and Control Plane Foundry control plane: agent registry, environment configuration, version history OpenTelemetry forwarded to Azure Monitor and Application Insights Continuous evaluation pipelines for ongoing quality, grounding, and safety monitoring Azure Policy and RBAC enforcement at the platform level Environment Topology There are two topology options. We recommend Option A for all production workloads: Option Structure Best for Trade-off A — Recommended Dev Project → Test Project → Prod Project (separate Foundry projects) Enterprise workloads Full isolation, clean RBAC boundaries, easier governance B — Lightweight Single Foundry project with agent version tags (dev/test/prod) Small teams, prototyping Simpler setup, but weaker environment separation Separate projects mean separate RBAC policies, separate connection strings, and separate evaluation signals. A developer service principal has access only to the Dev project; the CI/CD identity has restricted access to promote to Test and Production. Evaluation Gates — The Core Difference Evaluation gates transform a standard software pipeline into an AI-safe deployment pipeline. They run at two points: pre-merge (CI) and pre-promotion (CD). Defining the Gates Category Metric CI threshold Prod threshold Quality Hallucination rate < 5% < 3% Quality Task completion rate > 90% > 95% Safety Grounded response rate > 95% > 98% Safety Policy violations 0 0 Performance p95 latency < 4 000 ms < 3 000 ms Cost Token usage per query Track only Alert on > 20% regression Gate Enforcement (Python) import json import sys def check_gates(results_path: str) -> None: with open(results_path) as f: results = json.load(f) failures = [] if results["hallucination_rate"] > 0.05: failures.append(f"Hallucination rate {results['hallucination_rate']:.1%} exceeds 5% threshold") if results["task_completion_rate"] < 0.90: failures.append(f"Task completion {results['task_completion_rate']:.1%} below 90% threshold") if results["latency_p95_ms"] > 4000: failures.append(f"p95 latency {results['latency_p95_ms']}ms exceeds 4000ms threshold") if results.get("policy_violations", 0) > 0: failures.append(f"Policy violations detected: {results['policy_violations']}") if failures: for f in failures: print(f"GATE FAILED: {f}", file=sys.stderr) sys.exit(1) print("All evaluation gates passed — proceeding to deployment") if __name__ == "__main__": check_gates(sys.argv[1]) Hosted vs Prompt-Based Agents — Pipeline Differences Capability Hosted Agents Prompt-Based Agents Deployment unit Container image + agent definition YAML / prompt configuration bundle Build step required Yes — Docker build + ACR push No — YAML validation only Supported frameworks Agent Framework, LangGraph, Semantic Kernel, custom Foundry declarative runtime Promotion artefact Versioned agent with container image reference Versioned prompt/config bundle CI focus Code quality, tool tests, evaluation Prompt schema validation, evaluation Rollback mechanism Switch active agent version Switch active agent version Runtime management Foundry manages container lifecycle Foundry manages declarative runtime CI Pipeline Walkthrough The following steps are representative of the full GitHub Actions workflow available in github-actions-pipeline.yml alongside this post. Hosted Agent CI # 1. Static checks ruff check . bandit -r src/ -ll python scripts/validate_agent_config.py --config agent.yaml # 2. Tests pytest tests/unit/ -v --tb=short pytest tests/tools/ -v --tb=short # 3. Evaluation gate python scripts/run_evaluations.py \ --dataset eval/datasets/golden_set.jsonl \ --output eval/results/results.json python scripts/check_eval_gates.py \ --results eval/results/results.json \ --max-hallucination 0.05 \ --min-task-completion 0.90 \ --max-latency-p95 4000 # 4. Push container image az acr build \ --registry myregistry.azurecr.io \ --image "myagent:$SHA" \ --file Dockerfile . Prompt-Based Agent CI # Validate YAML / prompt definitions python scripts/validate_agent_config.py --config agent.yaml # Evaluation against golden dataset python scripts/run_evaluations.py \ --dataset eval/datasets/golden_set.jsonl \ --output eval/results/results.json python scripts/check_eval_gates.py \ --results eval/results/results.json CD Pipeline Walkthrough Stage 1 — Dev Deployment python scripts/deploy_agent.py \ --env dev \ --image "myregistry.azurecr.io/myagent:$SHA" \ --foundry-endpoint $FOUNDRY_ENDPOINT_DEV \ --agent-config agent.yaml # Returns the new agent version ID, stored for promotion AGENT_VERSION=$(python scripts/get_active_version.py --env dev) Stage 2 — Promote to Test (after approval gate) python scripts/promote_agent.py \ --from-env dev \ --to-env test \ --agent-version $AGENT_VERSION \ --foundry-endpoint $FOUNDRY_ENDPOINT_TEST # Run scenario tests and safety evaluation python scripts/run_evaluations.py \ --dataset eval/datasets/scenario_set.jsonl \ --output eval/results/test-results.json python scripts/check_eval_gates.py \ --results eval/results/test-results.json \ --max-hallucination 0.03 \ --min-task-completion 0.95 Stage 3 — Promote to Production (after required reviewer approval) python scripts/promote_agent.py \ --from-env test \ --to-env prod \ --agent-version $AGENT_VERSION \ --foundry-endpoint $FOUNDRY_ENDPOINT_PROD # Enable the production endpoint python scripts/enable_agent_endpoint.py \ --agent-version $AGENT_VERSION \ --foundry-endpoint $FOUNDRY_ENDPOINT_PROD Rollback # Switch the active version to the previous known-good version python scripts/promote_agent.py \ --from-env prod \ --to-env prod \ --agent-version $PREVIOUS_AGENT_VERSION \ --foundry-endpoint $FOUNDRY_ENDPOINT_PROD # OR delete the failing version python scripts/delete_agent_version.py \ --agent-version $AGENT_VERSION \ --foundry-endpoint $FOUNDRY_ENDPOINT_PROD Deployment Using the Azure AI Projects SDK The azure-ai-projects SDK provides programmatic control over the full agent lifecycle. This is the recommended approach for CI/CD scripts where you need deterministic, scriptable deployment. from azure.identity import DefaultAzureCredential from azure.ai.projects import AIProjectClient # Connect to the Foundry project client = AIProjectClient( endpoint=FOUNDRY_PROJECT_ENDPOINT, credential=DefaultAzureCredential() ) # List existing agents (useful for idempotent deploy scripts) for agent in client.agents.list(): print(f"Agent: {agent.name} version: {agent.id}") # Create a new agent version (hosted agent) agent = client.agents.create_agent( model="gpt-4o", name="my-enterprise-agent", instructions="You are a helpful assistant ...", tools=[...], # tool definitions metadata={"version": GIT_SHA, "environment": "dev"} ) print(f"Created agent version: {agent.id}") For hosted agents, the SDK call also references the container image pushed to ACR. Refer to the Deploy a hosted agent — Microsoft Foundry documentation for the full SDK flow including container image registration and version polling. Reference Implementation Stack Concern Technology Source control and pipelines GitHub Actions or Azure DevOps Pipelines Infrastructure and agent deployment Azure Developer CLI ( azd up ) Programmatic agent lifecycle azure-ai-projects Python SDK Agent evaluation azure-ai-evaluation Python SDK Agent runtime Microsoft Foundry Agent Service Container registry Azure Container Registry (hosted agents only) Observability OpenTelemetry, Azure Monitor, Application Insights Identity and access Microsoft Entra (Agent ID, OIDC workload identity federation) Governance Azure Policy, RBAC, Foundry control plane Governance and Responsible AI Shipping AI agents at enterprise scale requires governance beyond what a traditional CI/CD pipeline provides. Microsoft Foundry addresses this at the platform level: RBAC per environment — each Foundry project has independent access controls. Developers deploy to Dev; only CI/CD service principals (with audited OIDC tokens) can promote to Test and Production. Agent registry and audit trail — the Foundry control plane records which agent version is active in each environment, who deployed it, and when. This satisfies enterprise audit requirements without additional tooling. Content safety and policy enforcement — Azure Policy governs model access, data handling, and content safety rules at the infrastructure level, not just at the application code level. Policy violations block deployment automatically. Entra Agent Identity — each deployed agent version receives a dedicated, short-lived managed identity. Agents authenticate to downstream services using least-privilege credentials scoped to that specific deployment. Continuous evaluation in production — evaluation pipelines run on sampled production traffic, alerting when quality, safety, or cost metrics drift from their baseline. A key trade-off to be transparent about: evaluation datasets must be maintained and updated as the agent's tasks evolve. Stale datasets produce misleading pass/fail signals. Treat your golden evaluation set as a first-class engineering artefact alongside the agent code itself. Pipeline Files Two pipeline files accompany this reference architecture. Both implement the same four-stage pipeline (CI Build, CI Evaluate, CD Dev, CD Test, CD Production) with environment-appropriate approval gates. github-actions-pipeline.yml — GitHub Actions workflow. Uses GitHub Environments for approval gates and OIDC Workload Identity Federation for passwordless Azure authentication. No stored Azure credentials required. azure-devops-pipeline.yml — Azure DevOps multi-stage YAML pipeline. Uses ADO Environments with required approvers and variable groups per environment. Both pipelines share these security practices: OIDC / Workload Identity Federation — no long-lived Azure credentials stored in pipeline secrets Per-environment variable groups, each with scoped connection strings and endpoints Evaluation quality gates enforced before every promotion step Mandatory human approval before production deployment Summary The full pipeline in one view: Developer commit | CI Pipeline ├── Docker build (hosted agents) / YAML validation (prompt agents) ├── Static checks + unit tests + tool tests └── Evaluation gate ← quality · safety · performance | Agent Version created ← immutable, versioned artefact | CD Pipeline ├── Deploy to Dev → smoke tests + eval gate ├── Promote to Test → scenario tests + HITL + approval gate └── Promote to Prod → enable endpoint + monitoring | Microsoft Foundry Agent Service └── Versioned runtime · Entra identity · RBAC · Observability | Control Plane └── Agent registry · Governance · Continuous evaluation Microsoft Foundry provides the platform primitives — versioned agent deployments, multi-environment Foundry projects, built-in lifecycle management, and an enterprise observability stack — needed to operate AI agents with the same confidence as any production software system. The key takeaway: treat the agent version as your deployment artefact, and evaluation outcomes as your release gate. The rest follows familiar CI/CD patterns you already know and trust. Next Steps Clone the CI/CD Repo at leestott/foundry-cicd Clone the reference demo: foundry-agents-lifecycle on GitHub Set up your environment: Set up your environment for Foundry Agent Service Deploy your first hosted agent: Quickstart: Deploy your first hosted agent Understand hosted agent concepts: Foundry Hosted Agents concepts Automate deployments in CI/CD: Automate deployment of Microsoft Foundry agents Manage agent versions: Manage hosted agents — Microsoft Foundry Deploy via SDK: Deploy a hosted agent — Microsoft Foundry SDK and endpoint reference: Microsoft Foundry SDK and Endpoints reference Azure AI Projects SDK: azure-ai-projects Python SDK Azure Developer CLI: Azure Developer CLI (azd) overview Microsoft Foundry documentation hub: Microsoft Foundry on Microsoft Learn
Integrating Azure DevOps with Jira Service Management: Real-World Use Cases
If your development team works in Azure DevOps while support operates in Jira Service Management (JSM), you're probably dealing with manual ticket updates, information silos, and delayed responses. This friction slows down ticket resolution and creates unnecessary back-and-forth between teams. You can integrate both systems to automate data exchange and keep everyone on the same page. In this post, we'll explore why this integration matters, common use cases I've seen from teams using both platforms, and the key features you should consider when setting up your integration. Why Integrate Azure DevOps with Jira Service Management? When you integrate Azure DevOps with JSM, ticket escalation becomes automatic. A critical bug reported in JSM creates a work item in Azure DevOps with complete context—error logs, customer details, priority level, and all relevant information. Status updates sync bidirectionally. Your support team sees development progress without switching tools. Developers get full customer context without leaving Azure DevOps. The real benefits: Eliminate copy-paste errors Real-time visibility into work status Faster incident resolution Clear audit trails for SLAs and post-mortems Each team stays productive in their preferred environment Without integration, support agents need to check Azure DevOps regularly for updates to relay to customers. This creates delays, introduces errors, and wastes time on both sides. Common Use Cases for Azure DevOps + JSM Integration I have worked with hundreds of teams integrating these platforms. Here are the most common scenarios: 1. Incident and Bug Escalation This is probably the most common use case. Critical bugs reported in JSM automatically create high-priority work items in Azure DevOps with error logs, affected user details, and complete customer context. As developers update the work item, adding comments, changing status, or resolving the bug, those changes flow back to JSM automatically. Support agents can keep customers informed without constantly asking the dev team for updates. Use Case: Current Setup: Support uses JSM for customer tickets. Development uses Azure DevOps for bug tracking. Problem: Manually updating both systems is time-consuming and error-prone. Solution: Two-way sync ensures bugs and updates flow automatically between both systems. 2. Feature Request Management When customers submit feature requests through JSM and they get approved, they automatically flow to Azure DevOps as backlog items with inline images, custom fields, attachments, and more. When development completes the feature, the original JSM request closes automatically and notifies the customer. Use Case: Current Setup: Product managers collect feature requests in JSM. Developers track work in Azure DevOps. Problem: Manually creating work items for approved requests takes time, and context gets lost. Solution: Approved JSM requests automatically create Azure DevOps work items with full context. 3. Multi-Platform Sync for MSPs A central JSM instance can route tickets to different Azure DevOps projects based on work item type. This works especially well for MSPs managing multiple clients. You can connect your JSM instance with multiple client Azure DevOps environments while keeping data completely isolated per client. Use Case: Current Setup: An MSP uses one JSM instance. Multiple clients use separate Azure DevOps environments. Problem: Routing tickets manually to the right client's Azure DevOps is inefficient. Solution: Conditional routing based on customer tags or custom fields automatically sends tickets to the correct Azure DevOps project. 4. Post-Merger System Integration When two companies merge, one might use JSM for service management while the other uses Azure DevOps for development and QA. Rather than forcing everyone onto a single platform immediately, you can connect both systems to let teams continue using their existing tools during the transition. Use Case: Current Setup: Merged company with different tool stacks. Problem: Forcing immediate migration disrupts workflows. Solution: Integration bridges the gap while you plan a longer-term consolidation strategy. Key Features to Consider When Choosing Your Integration Approach Bidirectional vs. Unidirectional Sync Bidirectional sync is essential when both teams need to update shared information like status, priority, and comments. Updates flow both ways automatically without sync conflicts. For some use cases, you might only need one-way sync. For example, JSM → Azure DevOps for escalations where only support creates tickets, but developers provide all updates. Selective Filtering You don't want to sync everything. Look for solutions that let you sync only tickets meeting specific criteria: priority levels, labels, custom fields, or status values. Example filters: Only sync JSM tickets with "escalate-to-dev" label Only sync Azure DevOps bugs tagged "customer-reported" Only sync high and highest priority items This keeps Azure DevOps boards focused on actionable work rather than cluttered with routine requests. Field Mapping Flexibility JSM and Azure DevOps use different field structures. Your integration needs to handle transformations between JSM's field structure and Azure DevOps work item fields without losing data. Common mappings: JSM Status → Azure DevOps State JSM Priority → Azure DevOps Priority Custom fields require explicit mapping rules Scalability The solution should handle your current ticket volume and grow with your organization. Look for reliable performance, error handling, retry mechanisms, and the ability to add more integrations as your needs expand. Security and Compliance Essential security features: Encryption in transit and at rest OAuth or Basic authentication ISO certification Role-based access controls For MSPs: Complete data isolation between client environments Audit logging for compliance requirements Conflict Resolution You need clear rules for what happens when both sides update the same field simultaneously. Common approaches include last-write-wins logic or timestamp-based priority. Technical Implementation Approaches Webhooks + REST APIs Azure DevOps Service Hooks, combined with JSM REST API, provide real-time bidirectional sync. This is the recommended approach for most teams. The flow works like this: Change happens in Azure DevOps Service Hook triggers webhook Integration middleware receives a webhook Middleware calls the JSM REST API to update the ticket The same flow works in reverse for JSM → Azure DevOps updates. Custom Middleware For complex requirements, custom middleware gives you maximum flexibility: Custom field transformation logic Complex routing rules Conditional synchronization Workflow orchestration Error handling and retry logic Common technology stacks include Azure Functions, Logic Apps, or custom Node.js/Python microservices. Third-Party Integration Platforms Many teams opt for dedicated integration platforms rather than building from scratch. These platforms offer pre-built connectors for both JSM and Azure DevOps, significantly reducing implementation time. What third-party platforms typically provide: Pre-configured connectors that understand both JSM and Azure DevOps data structures out of the box Visual or scripting interfaces for setting up field mappings, filters, and sync rules with or without writing code Managed infrastructure so you don't need to host and maintain your own integration servers Built-in error handling and retry logic that handles API failures automatically Audit logging and monitoring dashboards for tracking sync activity and troubleshooting issues Support for complex scenarios like multi-project routing, conditional logic, and custom field transformations Regular updates to keep pace with API changes in both platforms When to consider third-party platforms: You need to get integration running quickly without significant development effort Your team lacks in-house expertise in API integration You want managed infrastructure rather than maintaining your own servers You need support and documentation for troubleshooting You plan to integrate multiple tools beyond just JSM and Azure DevOps You require complex field mappings and conditional routing that would be time-consuming to build Trade-offs to consider: Recurring subscription costs vs. one-time development investment Less control over the exact implementation compared to custom solutions Dependency on the platform's feature set and release cycle Data flows through a third-party service (though reputable platforms offer strong security and compliance) Most platforms available in the Azure DevOps marketplace or Atlassian marketplace offer free trials, allowing you to test their capabilities before committing. Choose the right approach considering the above trade-offs and advantages I have discussed. Good luck! Let's discuss if you have anything specific in mind related to this post.Optimizing Azure DevOps Jira Integration: 5 Practical Use Cases for DevOps Teams
Many teams rely on Azure DevOps (ADO) for development and Jira for project or product management. While each tool is powerful on its own, things often get messy when work items, statuses, and updates live in separate systems. Integrating the two platforms can remove a lot of friction. Below are six common use cases I have seen from real teams, with concrete problems and solutions to make the connection between Jira and Azure DevOps work smoothly. 1. Keeping User Stories and Bugs in Sync Challenge: Teams use Jira for user requests and Azure DevOps for development tasks. Manually updating both systems is tedious and error-prone. Solution: Enable two-way synchronization so that changes in Jira automatically reflect in Azure DevOps and vice versa (including comments and status updates). This keeps bugs and stories aligned without duplicate work. “Before we integrated Jira with Aure DevOps, I spent too much time manually updating task statuses in both systems. Now, with the automatic sync, my team is focused on actual coding work instead of managing project statuses across platforms.” — DevOps Engineer 2. One-Way Sync for Project Management–First Teams Challenge: Some organizations plan and track everything in Jira but manage code exclusively in Azure DevOps. Developers only need the essentials pushed across. Solution: Use a one-way sync from Jira → Azure DevOps to bring over metadata like titles, statuses, sprints, and due dates. Developers see the context they need without cluttering both systems with manual updates. “We rely on Jira for all project planning and management, but the developers need a clean workspace in Azure DevOps. A one-way sync from Jira to ADO helps us keep things efficient and ensures developers always have the latest information without double entry.” — Product Owner 3. Creating Jira Tickets from Azure DevOps Tasks or Bugs Challenge: External partners or stakeholders may only work in Jira Service Management to manage tickets. Developers in Azure DevOps often need their work mirrored for transparency. Solution: Configure automated ticket creation in Jira when certain ADO tasks are tagged. Both teams can track progress in their preferred tool without duplicating effort. “We use Azure DevOps internally, but our external stakeholders only work in Jira. Automating the creation of Jira tickets based on Azure DevOps tasks or bugs has made collaboration seamless and ensured no work is lost in translation.” — DevOps Lead 4. Syncing Epics, Features, and Work Items Challenge: High-level epics might live in Jira, while features and tasks are managed in Azure DevOps. Without integration, visibility across systems is fragmented. Solution: Sync epics and features so Jira provides portfolio-level visibility, while Azure DevOps remains the system of record for detailed development work. This keeps roadmaps and execution aligned. “Tracking epics in Jira while managing the technical work in Azure DevOps used to cause us to lose visibility. Now, everything from high-level epics to individual tasks is in sync, so we always know where we stand.” — Azure DevOps Product Manager 5. Managing Multiple Jira Projects with One Azure DevOps Project Challenge: Large organizations often run multiple Jira projects (by teams or business units) but only one Azure DevOPs project for development. Syncing everything consistently is tough. Solution: Map multiple Jira projects to a single Azure DevOps project, syncing only the key data (titles, statuses, sprints, custom fields). This creates a unified development view without losing project-specific details. “We have multiple teams using different Jira projects, but we consolidate all development work into a single Azure DevOps project. Syncing across these platforms used to be a nightmare, but now everything stays aligned, and we’re able to track all initiatives in one place.” — Azure DevOps Engineer 💬 Have you integrated Jira with Azure DevOps in your team? What worked well, and what challenges did you run into?
Exciting Announcements: New Data Connectors Released Using the Codeless Connector Framework
Microsoft Sentinel’s Codeless Connector Framework or ‘CCF’ (formerly called Codeless Connector Platform [CCP]) represents a paradigm shift in data ingestion, making it easier than ever for organisations to do more with Microsoft Sentinel by integrating diverse data sources seamlessly. Designed to simplify and expedite the onboarding of data sources, CCF eliminates the need for extensive coding expertise and maintaining additional services to facilitate ingestion, allowing security teams to focus on what truly matters – safeguarding their environment. Advantages of the Codeless Connector Framework The Codeless Connector Framework offers several compelling benefits: Ease of Use: CCF configuration-based templates allows advanced users to create data connectors without writing exhausting code, making the onboarding process quicker and more accessible to a broader audience. Flexibility: Users can customise data streams to meet their specific needs; optimizing efficacy while ensuring more control on the data being ingested. Scalability: The connectors built using CCF follows a true SaaS auto-expansion model making them highly scalable and natively reliable for large data volumes. Efficiency: By reducing the time and effort required to develop and deploy data connectors, CCF accelerates the availability of critical insights for security monitoring and more rapidly expands the value Microsoft Sentinel provides. What are we up to? We recognize that Codeless Connectors offer substantial advantages over Azure Function App based ingestion in Microsoft Sentinel in most cases. That motivates us to continue investing in modernizing our ingestion patterns for out-of-box connectors; one connector at a time. Another goal of modernizing these connectors is to replace the deprecated HTTP Data Collector API with the Log Ingestion API to send data to Microsoft Sentinel. Announcing the General Availability of New Data Connectors We are continually improving the Data Collection experience for our customers and are thrilled to announce that the following data connectors are now Generally Available (GA) on the Codeless Connector Framework. Atlassian Confluence Ingesting Confluence audit logs allows organizations to monitor collaboration activity, detect security risks, and troubleshoot configuration issues using Confluence audit records. Auth0 With the Auth0 Connector, organizations can effortlessly integrate authentication and authorization data from Auth0 into Microsoft Sentinel. This connector provides valuable insights into user activities and access patterns, bolstering identity security and compliance efforts. Azure DevOps Audit logs from Azure DevOps, allows security teams to monitor user activities, detect anomalous behavior, and investigate potential threats across DevOps environments. Box The Box Connector facilitates the ingestion of file storage and sharing data from Box into Microsoft Sentinel. By leveraging this connector, security teams can monitor file access and sharing activities, ensuring data integrity, and preventing unauthorized access. Google Cloud Platform Load Balancer With GCP Load Balancer and Web Application Firewall (Cloud Armor) logs, security teams can monitor inbound network activity, enforce security policies, and detect threats across GCP environments. Proofpoint POD The ingestion of email security logs allows organizations to monitor message traceability, detect threats, and investigate data exfiltration attempts by attackers and malicious insiders. Proofpoint TAP Email threat intelligence logs, including message and click events, provides visibility into malware and phishing activity to support custom alerts, dashboards, and threat investigation. SentinelOne The SentinelOne Connector enables seamless ingestion of threat intelligence and endpoint security data from SentinelOne into Microsoft Sentinel. This integration empowers security teams to enhance their threat detection capabilities and respond swiftly to potential threats. New Connectors in Public Preview CrowdStrike Falcon Data Replicator (S3 based Polling) Google Cloud Platform VPC Flow Google Cloud Platform DNS Google IAM These new additions are not new out-of-box sources in Microsoft Sentinel, but they do improve how data is collected. The previously Azure Function App based polling has now been upgraded to the Codeless Connector Framework for these products to ensure data collection adheres to the more scalable; advantageous pattern with CCF. As noted previously, the newer version of these connectors replaces the deprecated HTTP Data Collector API with the Log Ingestion API to send data to Microsoft Sentinel. Call to Action! Microsoft Sentinel customers collecting data from any of the mentioned sources using Azure Function Apps are advised to migrate their ingestion streams to newer versions to utilize the Codeless Connector Framework. While we continue to improve the data collection experience across all connectors, we encourage our customers and partners to join the Microsoft Security Communities to benefit from early insights about the latest and greatest with Microsoft Security. Call to Action for ISV Partners We invite our ISV partners to migrate their Azure Function App-based data connectors to the Codeless Connector Framework. By leveraging CCF for data ingestion, we can ensure that our mutual customers benefit from streamlined data integration and enhanced security monitoring in Microsoft Sentinel. We are committed to ensuring partners have all the support needed in this transformation. For any support, please reach out to us at Microsoft Sentinel Partners. Join us in this transformative journey to empower our customers by unlocking the full potential of their security investments with Microsoft Sentinel’s Codeless Connector Framework. References Create a codeless connector for Microsoft Sentinel Migrate from the HTTP Data Collector API to the Log Ingestion API to send data to Azure Monitor Logs
azure-pipelines-agent on NixOS
I am trying to install azure-pipelines-agent on NixOS, because I want to use nix in my pipeline. I am finding this to be very difficult; the scripts like installdependencies.sh do not support NixOS as a distribution. Is there a known solution or workaround for this, or is it something that may be supported in the future?How to sync sprints between Atlassian Jira and Microsoft Azure DevOps?
Teams involved in a sprint need to be able to exchange information in real time. This will keep all stakeholders and team members in perfect sync. Let’s say the connection is between a team of developers handling work items in Azure DevOps and the IT team using Jira Cloud for service management. To make all updates on one system reflect on the other one, both Jira Cloud and Azure DevOps need to be connected. In the absence of native integration solutions, tools like Exalate can bridge the gap between both platforms. Let me show you how teams can sync sprints between Jira and Azure DevOps. What to consider when syncing sprints between Jira and Azure DevOps? The first thing to bear in mind when syncing Azure DevOps and Jira is security. Considering that both sensitive data will be flowing between the systems, there should be measures to keep the information safe at rest and in transit. Protocols like tokenization, pseudonymization, and firewalls will keep data secure from unauthorized users. Other features like authentication, endpoint detection, and role-based access controls can bolster your security. With security nailed down, pay attention to the flexibility of the integration solution. As the number of tickets increases, the integration should be able to handle the network demands. Due to the dynamic nature of sprints, the tool connecting Jira Cloud and Azure DevOps should also have the scalability to handle an increase in ticket volumes. Another thing to check for is customization. The more sync options and custom connectors available, the broader the application of use cases. How to sync sprints between Jira and Azure DevOps? Assuming you want to sync a Jira sprint with Azure DevOps. Let’s break down what this entails from a technical standpoint. The connection should support data synchronization between standard fields (comments, attachments, etc.) and custom fields. If a sprint gets created in Azure DevOps, it should be automatically replicated on the Jira side with the same information. Issues created in Jira should be channeled to the correct sprint with the iteration path value from Azure DevOps. A custom field named “Team” (Azure DevOps side) should sync to a custom select field called “ADO Team” (Jira side). If any new values are added to the custom field on the Azure DevOps side, these should also be dynamically created on the Jira side. Exalate also supports AI-powered Groovy scripting for setting up syncs between Jira and Azure DevOps. You can use it to set up a two-way integration as well as event triggers for real-time sync and bulk operations. First, install it on both Jira and Azure DevOps. Follow this step-by-step Getting Started guide to connect both of them. This use case can only work with the Exalate Script Mode, which allows you to configure the sync however you want. Open Exalate in your Azure DevOps dashboard, go to the connection you want to edit, and click on the “Edit connection” icon. You have two options: Outgoing sync (on the Azure DevOps side) refers to the data being sent over to Jira. Incoming sync (on the Jira side) refers to the data to be received from Azure DevOps. Here is the code to control what fields and entities go out from Azure DevOps as part of the sprint [Azure DevOps Outgoing sync]: def res = httpClient.get("/<<project_name>>/<<team_name_in_ADO>>/_apis/work/teamsettings/iterations",true) def flag = 0 int i = 0 for (;i<res.value.size(); i++){ if (res.value[i].path == replica.iterationPath){ flag =1 break } } if (flag == 1){ replica.customKeys."sprint_name" = res.value[i].name replica.customKeys."sprint_start" = res.value[i].attributes.startDate replica.customKeys."sprint_end" = res.value[i].attributes.finishDate } The code snippet uses httpClient to fetch the iteration path and uses the flag to assign the name of the sprint (sprint_name), the start date (sprint_start), and the end date (sprint_end). Let’s check out the code to control what’s coming into the Jira Cloud project in the sprint [Jira Cloud Incoming Sync]. def list = httpClient.get("/rest/agile/1.0/board/3/sprint") int flag = 0 for (int i=0; i<list.values.size(); i++){ if (list.values[i].name == replica.customKeys.'sprint_name') flag = 1 } String startDate, endDate; if (flag == 0){ if (replica.customKeys."sprint_start"){ startDate = replica.customKeys."sprint_start".trim() startDate = startDate.replaceAll("Z",".000+05:00").trim(); } if (replica.customKeys."sprint_end"){ endDate = replica.customKeys."sprint_end".trim() endDate = endDate.replaceAll("Z",".000+05:00").trim(); } def res = httpClient.post("/rest/agile/1.0/sprint", "{\"name\": \"${replica.customKeys.'sprint_name'}\", \"startDate\": \"${startDate}\", \"endDate\": \"${endDate}\", \"originBoardId\": boardId}") } def res = httpClient.get("/rest/agile/1.0/board/"Board id"/sprint") for (int i=0; i<res.values.size(); i++){ if (res.values[i].name == replica.customKeys.'sprint_name') issue.customFields.Sprint.value = res.values[i].id } Similar to the Azure DevOps incoming sync snippet, this code fragment fetches the sprint startDate and endDate as customKeys. The httpClient uses POST and GET methods to point to the path or the board containing the sprint name. Congratulations! You have successfully synced your Jira sprint with Azure DevOps. Triggers for the Sync Azure DevOps users can use the Work Item Query Language to configure triggers in Exalate. [Work Item Type] = 'Task' AND System.TeamProject = 'SprintMarch' This triggers the synchronization of any work item (task) in the project named “SprintMarch”. On Jira Cloud, triggers are available via Jira Query Language. project = SprintMarch AND labels = sprint This trigger starts the synchronization if any issue in the project “SprintMarch” has been updated with the label “sprint”. If you still have questions or want to see how Exalate is tailored to your specific use case, discuss your use case with us.Jira Service Management and Azure DevOps Integration: Optimize ITSM and Development Workflows
This article dives into how integrating Jira Service Management (JSM) with Azure DevOps can improve ITSM and development workflows. Let’s face it, businesses can no longer ignore the friction that comes from siloed support and dev teams. Integration, when done right, brings real-time updates, better visibility, and a smoother customer experience. For such integrations, you need tools that help you connect these multiple platforms together. Integration isn’t about one side changing its behavior to meet the needs of the other. It’s about combining strengths, working together, and reducing waste of time and resources on both sides. Exalate connects teams within and across companies by providing a scalable, reliable, and AI-assisted integration solution, eliminating the need to switch between multiple ITSM systems. How does Exalate work? Exalate works as a dedicated app on each system you want to integrate. Each tool admin stays in control. You decide what goes out and what comes in. Exalate is a script-based integration solution. It’s Groovy-based scripting engine allows the flexibility to implement deep integration between Jira and Azure DevOps. Got an edge case that doesn’t quite fit in the standard mold? Bring it on. It’s also available for other systems like Salesforce, ServiceNow, Freshdesk, Zendesk, and more. To make scripting faster (and more approachable), it also provides AI Assist. You describe your sync logic in plain language, and it turns it into dynamic sync rules, right inside the Exalate admin console. Replica and Triggers Exalate allows you to define sync rules that hold what data gets shared and how it maps across systems. Sync rules have a replica. A replica is a copy of an issue/work item that holds the data you want to share. Each integrating side has incoming and outgoing sync rules. In Jira, the outgoing sync will define what information should be transferred to Azure DevOps, and the incoming sync will decide how you map the information coming from Azure DevOps. Triggers kick off syncs automatically, based on conditions written in native query languages like JQL (Jira Query Language) or WIQL (Work Item Query Language). Some common use cases that you can implement. First Use Case: Support Escalation to Dev When a customer raises a ticket in JSM, some of those need to be escalated to the dev team in Azure DevOps, either as Bugs or Features. Map request types from JSM to work item types in Azure DevOps e.g., ‘Report a bug’ → Bug | ‘Suggest a feature’ → Feature Sync status and priority between both platforms. This ensures both teams stay aligned as tickets progress Triggers Used When the project name is SUPP and the request type is a bug or feature, send the ticket over to Azure DevOps. Second Use Case: Product Support Flow The product team creates epics and user stories in Azure DevOps. These entities on the project board are unidirectionally synced to Jira Cloud as epics and stories. The relation hierarchy between Azure DevOps and Jira is maintained. For instance, ‘Relations’ in Azure DevOps are mapped as ‘Issue links’ in Jira. Statuses are synced between Jira Cloud and Azure DevOps to reflect accurate progress. Integrate Azure DevOps and Jira: Get Started Integrating Jira and Azure DevOps is not only a tech decision, it’s a business strategy. With Exalate, you can tailor the integration to your workflow, your logic, and your comfort level. Got a unique use case? Think Exalate might be the answer to your scattered support processes and manual ticket escalations? Drop a comment below, or if you’d rather chat one-on-one, book a call with us. Let’s make your integration work for you, not the other way around.Azure Event Grid Domain Creation: Overcoming AZ CLI's TLS Parameter Limitations with Workaround
Introduction: The Intersection of Security Policies and DevOps Automation In the modern cloud landscape, organizations increasingly enforce strict security requirements through platform policies. One common requirement is mandating latest TLS versions for example TLS 1.2 across all deployed resources to protect data in transit. While this is an excellent security practice, it can sometimes conflict with the available configuration options in deployment tools, particularly in the Azure CLI. This blog explores a specific scenario that many Azure DevOps teams encounter: how to deploy an Azure Event Grid domain when your organization has a custom policy requiring latest version considering TLS 1.2, but the Azure CLI command doesn't provide a parameter to configure this setting. The Problem: Understanding the Gap Between Policy and Tooling What Is Azure Event Grid? Azure Event Grid is a serverless event routing service that enables event-driven architectures. It manages the routing of events from various sources (like Azure services, custom applications, or SaaS products) to different handlers such as Azure Functions, Logic Apps, or custom webhooks. An Event Grid domain provides a custom topic endpoint that can receive events from multiple sources, offering a way to organize and manage events at scale. The Policy Requirement: Many organizations implement Azure Policy to enforce security standards across their cloud infrastructure. A common policy might look like this: { "policyRule": { "if": { "allOf": [ { "field": "type", "equals": "Microsoft.EventGrid/domains" }, { "anyOf": [ { "field": "Microsoft.EventGrid/domains/minimumTlsVersion", "exists": false }, { "field": "Microsoft.EventGrid/domains/minimumTlsVersion", "notEquals": "1.2" } ] } ] }, "then": { "effect": "deny" } } } This policy blocks the creation of any Event Grid domain that doesn't explicitly set TLS 1.2 as the minimum TLS version. The CLI Limitation: Now, let's examine the Azure CLI command to create an Event Grid domain: az eventgrid domain | Microsoft Learn TLS property is unrecognized with the latest version of AZ CLI version. Current Status of This Limitation: It's worth noting that this limitation has been recognized by the Azure team. There is an official GitHub feature request tracking this issue, which you can find at => Please add TLS support while creation of Azure Event Grid domain through CLI · Issue #31278 · Azure/azure-cli Before implementing this workaround described in this article, I recommend checking the current status of this feature request. The Azure CLI is continuously evolving, and by the time you're reading this, the limitation might have been addressed. However, as of April 2025, this remains a known limitation in the Azure CLI, necessitating the alternative approach outlined below. Why This Matters: This limitation becomes particularly problematic in CI/CD pipelines or Infrastructure as Code (IaC) scenarios where you want to automate the deployment of Event Grid domain resources. Workaround: You can utilize below ARM template and deploy it through AZ CLI in your deployment pipeline as below: Working ARM template: { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "domainName": { "type": "string", "metadata": { "description": "Name of the Event Grid Domain" } }, "location": { "type": "string", "defaultValue": "[resourceGroup().location]", "metadata": { "description": "Azure region for the domain" } } }, "resources": [ { "type": "Microsoft.EventGrid/domains", "apiVersion": "2025-02-15", "name": "[parameters('domainName')]", "location": "[parameters('location')]", "properties": { "minimumTlsVersionAllowed": "1.2" } } ] } Please note I've used latest API version from below official Microsoft documentation : Microsoft.EventGrid/domains - Bicep, ARM template & Terraform AzAPI reference | Microsoft Learn Working AZ CLI command: az deployment group create --resource-group <rg> --template-file <armtemplate.json> --parameters domainName=<event grid domain name> You can store this ARM template in your configuration directory with replacement for Azure CLI command. It explicitly sets TLS 1.2 for Event Grid domains, ensuring security compliance where the CLI lacks this parameter. For example: az deployment group create --resource-group <rg> --template-file ./config/<armtemplate.json> --parameters domainName=<event grid domain name> Disclaimer: The sample scripts provided in this article are provided AS IS without warranty of any kind. The author is not responsible for any issues, damages, or problems that may arise from using these scripts. Users should thoroughly test any implementation in their environment before deploying to production. Azure services and APIs may change over time, which could affect the functionality of the provided scripts. Always refer to the latest Azure documentation for the most up-to-date information. Thanks for reading this blog! I hope you've found this workaround valuable for addressing the Event Grid domain TLS parameter limitation in Azure CLI. 😊Azure Event Grid CLI Identity Gaps & Workarounds with Python REST and ARM Templates
Azure Event Grid has become a cornerstone service for building event-driven architectures in the cloud. It provides a scalable event routing service that enables reactive programming patterns, connecting event sources to event handlers seamlessly. However, when working with Event Grid through the Azure CLI, developers often encounter a significant limitation: the inability to configure system-assigned managed identities using CLI commands. In this blog post, I'll explore this limitation and provide practical workarounds using Python REST API calls and ARM templates with CLI to ensure your Event Grid deployments can leverage the security benefits of managed identities without being blocked by tooling constraints. Problem Statement: Unlike many other Azure resources that support the --identity or ---assign-identity parameter for enabling system-assigned managed identities, Event Grid's CLI commands lack this capability while creating event subscription for system topic at the moment. This means that while the Azure Portal and other methods support managed identities for Event Grid, you can't configure them directly through the CLI in case of system topic event subscriptions For example you can add managed identity for delivery through portal but not through AZ CLI: If you try to use the following CLI command: az eventgrid system-topic event-subscription create \ --name my-sub \ --system-topic-name my-topic \ --resource-group my-rg \ --endpoint <EH resource id> --endpoint-type eventhub \ --identity systemassigned You'll run into a limitation: The --identity flag is not supported or unrecognized for system topic subscriptions in Azure CLI. Also, --delivery-identity is in preview and under development Current Status of This Limitation: It's worth noting that this limitation has been recognized by the Azure team. There is an official GitHub feature request tracking this issue, which you can find at Use managed identity to command creates an event subscription for an event grid system topic · Issue #26910 · Azure/azure-cli. Before implementing any of the workarounds described in this article, I recommend checking the current status of this feature request. The Azure CLI is continuously evolving, and by the time you're reading this, the limitation might have been addressed. However, as of April 2025, this remains a known limitation in the Azure CLI, necessitating the alternative approaches outlined below. Why This Matters: This limitation becomes particularly problematic in CI/CD pipelines or Infrastructure as Code (IaC) scenarios where you want to automate the deployment of Event Grid resources with managed identities. Solution 1: Using Azure REST API with Python request library: The first approach to overcome this limitation is to use the Azure REST API with Python. This provides the most granular control over your Event Grid resources and allows you to enable system-assigned managed identities programmatically. System Topic Event Subscriptions - Create Or Update - REST API (Azure Event Grid) | Microsoft Learn You can retrieve Azure Entra token using below CLI command: az account get-access-token Sample working code & payload: import requests import json subscription_id = <> resource_group = <> system_topic_name = <> event_subscription_name = <> event_hub_resource_id = <> access_token = <> url = f"https://management.azure.com/subscriptions/{subscription_id}/resourceGroups/{resource_group}/providers/Microsoft.EventGrid/systemTopics/{system_topic_name}/eventSubscriptions/{event_subscription_name}?api-version=2024-12-15-preview" payload = { "identity": { "type": "SystemAssigned" }, "properties": { "topic": "/subscriptions/<>/resourceGroups/<>/providers/Microsoft.EventGrid/systemTopics/<>", "filter": { "includedEventTypes": [ "Microsoft.Storage.BlobCreated", "Microsoft.Storage.BlobDeleted" ], "advancedFilters": [], "enableAdvancedFilteringOnArrays": True }, "labels": [], "eventDeliverySchema": "EventGridSchema", "deliveryWithResourceIdentity": { "identity": { "type": "SystemAssigned" }, "destination": { "endpointType": "EventHub", "properties": { "resourceId": "/subscriptions/<>/resourceGroups/rg-sch/providers/Microsoft.EventHub/namespaces/<>/eventhubs/<>", "deliveryAttributeMappings": [ { "name": "test", "type": "Static", "properties": { "value": "test", "isSecret": False, "sourceField": "" } }, { "name": "id", "type": "Dynamic", "properties": { "value": "abc", "isSecret": False, "sourceField": "data.key" } } ] } } } } } headers = { "Authorization": f"Bearer {access_token}", "Content-Type": "application/json" } response = requests.put(url, headers=headers, data=json.dumps(payload)) if response.status_code in [200, 201]: print("Event subscription created successfully!") Remember that these tokens are sensitive security credentials, so handle them with appropriate care. They should never be exposed in logs, shared repositories, or other insecure locations. Solution 2: Using ARM Templates & deploying it through CLI Another solution is to use Azure Resource Manager (ARM) templates, which fully support system-assigned managed identities for Event Grid. This approach works well in existing IaC workflows. Microsoft.EventGrid/systemTopics/eventSubscriptions - Bicep, ARM template & Terraform AzAPI reference | Microsoft Learn Here's a sample ARM template that creates an Event Grid topic with a system-assigned managed identity: { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "systemTopicName": { "type": "string", "metadata": { "description": "Name of the existing system topic" } }, "eventSubscriptionName": { "type": "string", "metadata": { "description": "Name of the event subscription to create" } }, "eventHubResourceId": { "type": "string", "metadata": { "description": "Resource ID of the Event Hub to send events to" } }, "includedEventType": { "type": "string", "defaultValue": "Microsoft.Storage.BlobCreated", "metadata": { "description": "Event type to filter on" } } }, "resources": [ { "type": "Microsoft.EventGrid/systemTopics/eventSubscriptions", "apiVersion": "2024-06-01-preview", "name": "[format('{0}/{1}', parameters('systemTopicName'), parameters('eventSubscriptionName'))]", "identity": { "type": "SystemAssigned" }, "properties": { "deliveryWithResourceIdentity": { "destination": { "endpointType": "EventHub", "properties": { "resourceId": "[parameters('eventHubResourceId')]" } }, "identity": { "type": "SystemAssigned" } }, "eventDeliverySchema": "EventGridSchema", "filter": { "includedEventTypes": [ "[parameters('includedEventType')]" ] } } } ] } How to deploy via Azure CLI: az deployment group create \ --resource-group <your-resource-group> \ --template-file eventgridarmtemplate.json \ --parameters \ systemTopicName=<system-topic-name> \ eventSubscriptionName=<event-subscription-name> \ eventHubResourceId="/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.EventHub/namespaces/<namespace>/eventhubs/<hub>" Disclaimer The sample scripts provided in this article are provided AS IS without warranty of any kind. The author is not responsible for any issues, damages, or problems that may arise from using these scripts. Users should thoroughly test any implementation in their environment before deploying to production. Azure services and APIs may change over time, which could affect the functionality of the provided scripts. Always refer to the latest Azure documentation for the most up-to-date information. Thanks for reading this blog! I hope you've found these workarounds valuable for addressing the Event Grid identity parameter limitation in Azure CLI. 😊
