Forum Discussion
Codeless Connect Framework (CCF) Template Help
As the title suggests, I'm trying to finalize the template for a Sentinel Data Connector that utilizes the CCF. Unfortunately, I'm getting hung up on some parameter related issues with the polling config.
The API endpoint I need to call utilizes a date range to determine the events to return and then pages within that result set. The issue is around the requirements for that date range and how CCF is processing my config. The API expects an HTTP GET verb and the query string should contain two instances of a parameter called EventDates among other params. For example, a valid query string may look something like:
../path/to/api/myEndpoint?EventDates=2025-08-25T15%3A46%3A36.091Z&EventDates=2025-08-25T16%3A46%3A36.091Z&PageSize=200&PageNumber=1
I've tried a few approaches in the polling config to accomplish this, but none have worked. The current config is as follows and has a bunch of extra stuff and names that aren't recognized by my API endpoint but are there simply to demonstrate different things:
"queryParameters": {
"EventDates.Array": [ "{_QueryWindowStartTime}", "{_QueryWindowEndTime}" ],
"EventDates.Start": "{_QueryWindowStartTime}",
"EventDates.End": "{_QueryWindowEndTime}",
"EventDates.Same": "{_QueryWindowStartTime}",
"EventDates.Same": "{_QueryWindowEndTime}",
"Pagination.PageSize": 200
}
This yields the following URL / query string:
../path/to/api/myEndpoint?EventDates.Array=%7B_QueryWindowStartTime%7D&EventDates.Array=%7B_QueryWindowEndTime%7D&EventDates.Start=2025-08-25T15%3A46%3A36.091Z&EventDates.End=2025-08-25T16%3A46%3A36.091Z&EventDates.Same=2025-08-25T16%3A46%3A36.091Z&Pagination.PageSize=200
There are few things to note here:
- The query param that is configured as an array (EventDates.Array) does indeed show up twice in the query string and with distinct values. The issue is, of course, that CCF doesn't seem to do the variable substitution for values nested in an array the way it does for standard string attributes / values.
- The query params that have distinct names (EventDates.Start and .End) both show up AND both have the actual timestamps substituted properly. Unfortunately, this doesn't match the API expectations since the names differ.
- The query params that are repeated with the same name (EventDates.Same) only show once and it seems to use the value from which comes last in the config (so last one overwrites the rest). Again, this doesn't meet the requirements of the API since we need both.
I also tried a few other things ...
- Just sticking the query params and placeholders directly in the request.apiEndpoint polling config attribute. No surprise, it doesn't do the variable substitution there.
- Utilizing queryParametersTemplate instead of queryParameters. https://learn.microsoft.com/en-us/azure/sentinel/data-connector-connection-rules-referenceindicates this is a string parameter that expects a JSON string. I tried this with various approaches to the structure of the JSON. In ALL instances, the values here seemed to be completely ignored. All other examples from Azure-Sentinel repository utilize the POST verb. Perhaps that attribute isn't even interpreted on a GET request???
- And because some AI agents suggested it and ... sure, why not??? ... I tried queryParametersTemplate as an actual query string template, so "EventDates={_QueryWindowStartTime}&EventDates={_QueryWindowEndTime}". Just as with previous attempts to use this attribute, it was completely ignored.
I'm willing to try anything at this point, so if you have suggestions, I'll give it a shot! Thanks for any input you may have!
4 Replies
Thanks for the suggestion pradejain! I did attempt placing the parameters directly in the apiEndpoint value, but since you suggested this along with removing the queryParameters section, I gave it another shot. Unfortunately, CCF does not seem to check the apiEndpoint value when it's doing variable substitution, so you just end up with the literal values in the resulting URL ...
The configuration:
"apiEndpoint": "https://my.hostname.com//path/to/api/myEndpoint?EventDates={_QueryWindowStartTime}&EventDates={_QueryWindowEndTime}&EventDates.SelectionMode=Range&PageSize=200",
And here's the URL it generated:
"GET /../path/to/api/myEndpoint?EventDates=%7B_QueryWindowStartTime%7D&Event.Dates=%7B_QueryWindowEndTime%7D&EventDates.SelectionMode=Range&PageSize=200 HTTP/1.1"jamos Sorry to hear that. The second trick that comes to my mind is to leverage the pagination attribute in your polling configuration. While not its intended purpose, you can use it to build the required date parameters with proper variable substitution.
"pollingConfig": {
"request": {
"apiEndpoint": "https://my.hostname.com/path/to/api/myEndpoint",
"httpMethod": "GET"
},
"queryParameters": {
"EventDates": "{_QueryWindowStartTime}"
},
"pagination": {
"type": "custom",
"paginationType": "linkHeader",
"link": "{_nextPageLink}",
"additionalQueryParameters": {
"EventDates": "{_QueryWindowEndTime}"
}
},
...
}- The initial request will send the EventDates={_QueryWindowStartTime} parameter, which is correctly substituted from the queryParameters section.
- The pagination attribute is configured to add additionalQueryParameters to the next page request.
- By setting the pagination type to custom and the link to a placeholder ({_nextPageLink}), you effectively bypass the standard pagination logic.
- The additionalQueryParameters section will then append EventDates={_QueryWindowEndTime} to the URL, resulting in a single URL with both repeated parameters properly substituted.
Unfortunately, another dead end. I didn't see anything in the documentation to indicate pagination.additionalQueryParameters was an expected / accepted path, but I did try it. The section was completely ignored resulting in NO parameters other than the page size (which is also defined under the pagination path) being present in the constructed URL.
With that said, I also found that the pagination capabilities aren't going to be sufficient for some of the APIs I need to call since they only pass the current page and no next page number or token. This means CCF simply isn't going to be a viable solution for this project.
I do appreciate your time and suggestions, though!
Hi jamos The most reliable way to fix this issue is to remove the queryParameters section and manually build the query string directly within the request.apiEndpoint attribute. This method allows for both repeated parameter names and proper variable substitution.
"pollingConfig": {
"request": {
"apiEndpoint": "../path/to/api/myEndpoint?EventDates={_QueryWindowStartTime}&EventDates={_QueryWindowEndTime}&PageSize=200",
"httpMethod": "GET"
},
...
}This configuration ensures the CCF correctly creates a URL with two EventDates parameters, each containing the appropriate time value from the polling window.
