This post walks through building a Microsoft Sentinel Codeless Connector Framework (CCF) RestApiPoller connector that uses the nested API polling pattern. The nested pattern exists for a specific reason: many enterprise APIs do not return enriched records from a single call. Instead, they use a two-step model: a list endpoint that returns identifiers, followed by a detail endpoint that accepts one identifier and returns the full record. Without native support for this pattern, building a Sentinel connector for such an API means writing custom orchestration in code. The CCF nested pattern re places that with a JSON configuration that the engine handles directly.
The lab is built around a mock Contoso Incident API hosted as an Azure Function App. The purpose is to give you a live API you can deploy, inspect, and connect to Sentinel without needing a real product. Once deployed, you will walk through each of the four connector files that make up the integration (the poller config, the DCR, the table schema, and the connector definition) and see exactly how the two-call chain is wired together. By the end of the walkthrough, you will have a working connector ingesting enriched incident records into ContosoIncidents_CL in your Sentinel workspace.
Lab repository: CCF Pull Connector - Nested API Accelerator: clone the repository, then deploy the mock API and connector with a single Copilot prompt. See the README for full instructions. (link to be confirmed on merge)
Related documentation: CCF Nested API Polling Reference
Quick Start
Already familiar with CCF nested polling?
Complete these sections:
The remaining sections explain how the nested polling configuration, KQL extraction, DCR transform, and connector artifacts work behind the scenes.
Prerequisites
Before starting, make sure you have the following:
- Azure subscription with Contributor access on a resource group for deploying the Function App, and Microsoft Sentinel Contributor on a Sentinel-enabled workspace for deploying the connector
- Microsoft Sentinel workspace: an existing Log Analytics workspace with Sentinel enabled (Quickstart: Onboard Microsoft Sentinel)
- Azure CLI (Install the Azure CLI) installed and authenticated
- VS Code with GitHub Copilot installed if you want to use the one-prompt agent deployment option
- Lab repository: clone the CCF Pull Connector - Nested API Accelerator repository locally
The Lab Environment
The mock Contoso Incident API exposes exactly the two-endpoint pattern the nested connector is designed for. The list endpoint returns a set of incident identifiers scoped to a time window. The detail endpoint accepts a single identifier and returns the full enriched record for that incident. There is no way to get the full records in a single call; you have to ask for them one at a time.
A call to the list endpoint looks like this:
{
"incidents": [
{ "incidentId": "INC-001" },
{ "incidentId": "INC-002" },
{ "incidentId": "INC-003" },
{ "incidentId": "INC-004" },
{ "incidentId": "INC-005" }
]
}
And a call to the detail endpoint for INC-00 returns:
{
"incidentId": "INC-001",
"title": "Suspicious login attempt",
"severity": "High",
"status": "Active",
"affectedUser": "alice@contoso.com",
"sourceIp": "198.51.100.42",
"createdAt": "2026-05-30T14:22:00Z"
}
So for every polling cycle the connector needs to call the list endpoint once, extract the five incidentId values, and then call the detail endpoint five times. That fan-out logic (taking a value from the first response and injecting it into the URL for each subsequent call) is what the nested step configuration handles. The repo contains everything needed for the walkthrough: an ARM template to deploy the Function App, the four connector artifact files, and a solution package that deploys the connector into Sentinel.
Deploying the Mock API
The quickest way to deploy is to open Copilot Chat in Agent mode in VS Code and paste:
Load and follow the deployment instructions at Tools/CCF-Pull-Connector-Nested-Accelerator/agent-instructions.md. Let's deploy a CCF nested API connector.
The agent reads the deployment instructions, collects the values it needs, generates names for anything you do not provide, and deploys end-to-end. The only manual action in the entire flow is clicking Connect in the Sentinel portal once the ARM template has been deployed.
If you prefer to deploy manually, the steps are straightforward. Create a resource group, deploy the Function App using the included ARM template, zip the MockApi/ folder and push the code, then retrieve the Function App API key:
az group create --name contoso-mock-api-rg --location eastus
az deployment group create `
--resource-group contoso-mock-api-rg `
--template-file "Tools/CCF-Pull-Connector-Nested-Accelerator/MockApi/azuredeploy_MockApi.json" `
--parameters FunctionAppName=ContosoMockApi Location=eastus
$outputs = az deployment group show `
--resource-group contoso-mock-api-rg --name azuredeploy_MockApi `
--query properties.outputs -o json | ConvertFrom-Json
$functionAppName = $outputs.functionAppName.value
$mockApiBaseUrl = $outputs.mockApiBaseUrl.value
Compress-Archive -Path "Tools/CCF-Pull-Connector-Nested-Accelerator/MockApi/*" -DestinationPath contosoapi.zip -Force
az functionapp deployment source config-zip --name $functionAppName --resource-group contoso-mock-api-rg --src contosoapi.zip
$apiKey = (az functionapp keys list --name $functionAppName --resource-group contoso-mock-api-rg --query functionKeys.default -o tsv).Trim()
Once deployed, verify both endpoints are responding before moving on:
Invoke-RestMethod "$mockApiBaseUrl/incidents" -Headers @{"x-functions-key" = $apiKey}
Invoke-RestMethod "$mockApiBaseUrl/incidents/INC-001/details" -Headers @{"x-functions-key" = $apiKey}
How the Nested Step Pattern Works
The entire two-call chain lives inside ContosoIncidents_PollerConfig.json. Here is the full config:
{
"type": "Microsoft.SecurityInsights/dataConnectors",
"kind": "RestApiPoller",
"properties": {
"connectorDefinitionName": "ContosoIncidentsConnector",
"dataType": "ContosoIncidents_CL",
"dcrConfig": {
"streamName": "Custom-ContosoIncidents_CL",
"dataCollectionEndpoint": "{{dataCollectionEndpoint}}",
"dataCollectionRuleImmutableId": "{{dataCollectionRuleImmutableId}}"
},
"auth": {
"type": "APIKey",
"ApiKey": "{{ApiKey}}",
"ApiKeyName": "x-functions-key"
},
"request": {
"apiEndpoint": "{{mockApiBaseUrl}}/incidents",
"httpMethod": "GET",
"queryWindowInMin": 5,
"queryTimeFormat": "yyyy-MM-ddTHH:mm:ssZ",
"startTimeAttributeName": "startTime",
"endTimeAttributeName": "endTime",
"headers": { "Accept": "application/json" }
},
"response": {
"eventsJsonPaths": [ "$.incidents" ],
"format": "json"
},
"stepInfo": {
"stepType": "Nested",
"nextSteps": [
{
"stepId": "fetchIncidentDetails",
"stepPlaceholdersParsingKql": "source | project res = parse_json(data) | project incidentId = res.incidentId"
}
]
},
"stepCollectorConfigs": {
"fetchIncidentDetails": {
"shouldJoinNestedData": false,
"request": {
"httpMethod": "GET",
"apiEndpoint": "{{mockApiBaseUrl}}/incidents/$incidentId$/details",
"headers": { "Accept": "application/json" }
},
"response": {
"eventsJsonPaths": [ "$" ],
"format": "json"
}
}
}
}
}
The request block at the top defines the parent call, the list endpoint. On every polling cycle the CCF engine calls GET /incidents with startTime and endTime query parameters derived from queryWindowInMin: 5. The response.eventsJsonPaths: ["$.incidents"] tells the engine where to find the data in the response; it walks to the incidents array and treats each element as a separate row.
Those rows do not go to the DCR. Instead, because stepInfo.stepType is set to "Nested", the engine passes each row to the stepPlaceholdersParsingKql expression before doing anything else.
The KQL That Links the Two Calls
This is the most important line in the entire connector:
source | project res = parse_json(data) | project incidentId = res.incidentId
Each row from the list response arrives as raw JSON in a column called data. The parse_json(data) call converts that string into a dynamic object so individual fields can be addressed. The final project extracts incidentId and gives it a column name. That column name, incidentId, is not arbitrary. It must exactly match the $incidentId$ token in the child endpoint URL. The engine reads the output of this expression, takes every value in the incidentId column, and substitutes it into the URL for the next step.
For the Contoso API this produces five values (INC-001 through INC-005), and the engine issues five parallel GET requests:
GET /api/incidents/INC-001/details
GET /api/incidents/INC-002/details
GET /api/incidents/INC-003/details
GET /api/incidents/INC-004/details
GET /api/incidents/INC-005/details
Each of those calls returns a full incident record. The eventsJsonPaths: ["$"] in the child response block captures the entire response body as a single event. Those five events are what get sent to the DCR stream, not the five lightweight rows from the list call.
shouldJoinNestedData
The shouldJoinNestedData: false setting on the child step tells the engine not to merge the parent row's fields into the child row before sending to the DCR. In this case the detail response already contains incidentId and all other required fields, so there is nothing to carry forward from the parent. If your list endpoint returns fields that are not present in the detail response (a category, a tenant identifier, a product namespace), setting this to true will merge them in automatically.
The DCR and the KQL Transform
The Data Collection Rule handles what happens to each detail response once it arrives at the Data Collection Endpoint. The streamDeclarations block defines the inbound schema (the shape of the data the CCF engine sends), and the transformKql maps it to the destination table columns.
The transform for this connector is:
source
| extend TimeGenerated = now()
| project
TimeGenerated,
IncidentId = ['incidentId'],
Title = ['title'],
Severity = ['severity'],
Status = ['status'],
AffectedUser = ['affectedUser'],
SourceIp = ['sourceIp']
The extend TimeGenerated = now() sets the ingestion timestamp. The mock API returns a createdAt field on every detail record, but it is not used here; TimeGenerated reflects when the record was received by the pipeline rather than when the incident was created on the source system. The project operators rename each snake_case field from the API response to the PascalCase column names defined in the table schema.
The column names produced by transformKql must exactly match the column names in ContosoIncidents_Table.json. A mismatch causes rows to be silently dropped at ingestion with no error surfaced to the connector status page, which makes it one of the more frustrating issues to debug after the fact.
The Table Schema and Connector Definition
Before looking at the connector definition, it helps to understand what the table object itself is. ContosoIncidents_Table.json deploys a Microsoft.OperationalInsights/workspaces/tables resource, the same resource type that creates any custom log table in your Log Analytics workspace. Its schema block is an array of column definitions, each with a name, a type, and a description:
"schema": {
"name": "ContosoIncidents_CL",
"columns": [
{ "name": "TimeGenerated", "type": "datetime", "description": "Ingestion timestamp." },
{ "name": "IncidentId", "type": "string", "description": "Unique identifier of the incident (e.g. INC-001)." },
{ "name": "Severity", "type": "string", "description": "Incident severity: Critical, High, Medium, or Low." }
]
}
This is the definition that shows up when you query the table. Each column name becomes a field you can reference directly in KQL, and each column type determines how that field behaves when you filter or aggregate on it. TimeGenerated being a datetime is what lets you sort or filter with operators like ago() and between.
The table schema in ContosoIncidents_Table.json defines ContosoIncidents_CL with seven columns: TimeGenerated, IncidentId, Title, Severity, Status, AffectedUser, and SourceIp.
The connector definition in ContosoIncidents_ConnectorDefinition.json drives the portal UI. It defines the connector page title and description, the ingestion activity graph, the sample KQL queries that are surfaced to analysts directly from the connector page, and the connectivity check query that determines whether the connector shows as Connected or Disconnected in the Data Connectors blade. The two Textbox inputs in the instructionSteps block (mockApiBaseUrl and ApiKey) are what feed the {{template}} variables in the poller config when a user clicks Connect. Getting those input names right is what makes the credential handoff from the portal to the connector work without manual intervention.
Deploying the Sentinel Connector
With the mock API running, deploy the Sentinel connector solution using the included mainTemplate.json. The template creates the Data Collection Endpoint, the ContosoIncidents_CL table, the DCR, the poller config, and the connector definition in the correct dependency order.
If you would rather not use the CLI, you can deploy the same template through the Azure portal. Open the Custom deployment blade, choose "Build your own template in the editor," and paste in the contents of mainTemplate.json, then fill in the parameters (workspace, workspace-location, and so on) through the generated form instead of a parameters file. See Deploy template - Azure portal - Azure Resource Manager for the full walkthrough. The CLI steps below are the faster path if you already have the CLI authenticated, but the portal works just as well.
The workspace-location parameter requires special handling because the hyphen in the parameter name causes issues when passed inline to the Azure CLI. Write a parameters file first, then reference it with @:
@{
'$schema' = 'https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#'
contentVersion = '1.0.0.0'
parameters = @{
workspace = @{ value = '<your-workspace-name>' }
'workspace-location' = @{ value = '<region>' }
}
} | ConvertTo-Json -Depth 5 | Out-File deploy-params.json -Encoding utf8
az deployment group create `
--resource-group <sentinel-workspace-rg> `
--name mainTemplate `
--template-file "Tools/CCF-Pull-Connector-Nested-Accelerator/ContosoIncidents/Package/mainTemplate.json" `
--parameters "@deploy-params.json" `
--output table</sentinel-workspace-rg></region></your-workspace-name>
The workspace-location value must exactly match the region string returned by az monitor log-analytics workspace show --query location for your workspace.
Enabling and Verifying Data
The ARM deployment registers the connector definition but does not start data collection. To start polling you need to click Connect once in the portal. Navigate to your Sentinel workspace via security.microsoft.com, go to Data Connectors, and find Contoso Incidents (CCF Nested API Accelerator). If it is not immediately visible, click Refresh and wait two to three minutes for the deployment to propagate.
Open the connector page and under STEP 2 - Connect to the Contoso Mock API, enter the mockApiBaseUrl from the deployment output and the ApiKey retrieved earlier, then click Connect. The CCF engine begins polling immediately.
Allow five to ten minutes for the first poll cycle to complete, then confirm data is arriving:
ContosoIncidents_CL
| sort by TimeGenerated desc
| take 10
You should see five rows, one per mock incident, with all columns populated. From there you can use the sample queries built into the connector page to start exploring the data:
// High and Critical incidents
ContosoIncidents_CL
| where Severity in ('Critical', 'High')
| sort by TimeGenerated desc
// Active incidents with source IP
ContosoIncidents_CL
| where Status == 'Active'
| project TimeGenerated, IncidentId, Title, Severity, AffectedUser, SourceIp
Adapting This Pattern to Your Own API
The nested pattern is reusable for any two-tier REST API. The changes needed to adapt the accelerator are mechanical: replace the parent endpoint and update eventsJsonPaths to point to your identifier array, rewrite the stepPlaceholdersParsingKql to extract your identifier field (the column name in the final project must match the $token$ in your child URL), replace the child endpoint URL, and update the DCR stream declaration, transform, and table schema to match your detail response fields.
The shouldJoinNestedData flag is worth thinking through for your specific case. If your list endpoint returns context that is not replicated in the detail response (a tenant ID, a data source label, a parent category), set it to true and those fields will be merged into every child row automatically before it reaches the DCR.
Next Steps
Review the CCF Nested API Polling Reference for the full specification of stepInfo, stepCollectorConfigs, shouldJoinNestedData, and multi-level nesting: CCF Nested API Polling Reference
Review the codeless connector documentation for the full range of supported auth types, pagination patterns, and incremental pull strategies: Create a codeless connector
Explore the Microsoft Sentinel content hub to see how published connectors are structured and what the certification requirements look like for production submissions.
Conclusion
Following these steps, you saw how a CCF nested API connector works at the configuration level: how the stepPlaceholdersParsingKql expression extracts identifier values from the list response, how those values are substituted into the child endpoint URL to produce per-record detail calls, and how the DCR transform maps the raw API fields into a clean Sentinel table. The pattern is deliberately simple to adapt: the KQL expression and the $token$ in the URL are the only two moving parts that link the two calls together.
If you are an ISV building a Sentinel integration and want hands-on support, Microsoft's App Assure program is available to help with connector development, validation, and readiness for production. You can get started by reaching out through our intake form.
Microsoft