Azure AD SCIM Validator is in General Availability (GA) Status

Microsoft

You can now validate the compatibility of your SCIM provisioning endpoint and Azure AD code base using our Azure AD SCIM Validator. This tool can be used by ISVs who want to build SCIM compatible servers either for gallery app or generic app and developers building their line of business SCIM apps. https://learn.microsoft.com/azure/active-directory/app-provisioning/scim-validator-tutorial

45 Replies

@owinoakelo 

On another note, I am not convinced that the Validator treats complex multiValued 'members' attribute for Groups correctly, if they are exposed in Schema.
Specifically, if I have Group.members[type ="User"] collection defined in Schema like shown below, then the Validator would attempt to generate a Patch request as shown on the screenshot with some bogus(?) value in the Replace operation for users. That bogus value such as "QQ7M8HRCRACN" looks like a display name for a group, rather than User ID that I would expect it to be.
Or am I missing SCIM spec completely here, and this Replace request has totally different semantics?

See image attached.

norfas_0-1679336581527.png

 

To compare, Add Member operation on Group looks fine, with proper User ID:

norfas_1-1679337087433.png

 

My schema for the group is as follows:

{
            "name": "Group",
            "description": "Group",
            "meta": {
                "resourceType": "Schema",
                "created": "0001-01-01T00:00:00",
                "lastModified": "0001-01-01T00:00:00"
            },
            "id": "urn:ietf:params:scim:schemas:core:2.0:Group",
            "schemas": [
                "urn:ietf:params:scim:schemas:core:2.0:Schema"
            ],
            "attributes": [
                {
                    "caseExact": false,
                    "type": "string",
                    "description": "A human-readable name for the Group. REQUIRED.",
                    "mutability": "readWrite",
                    "name": "displayName",
                    "multiValued": false,
                    "required": true,
                    "returned": "default",
                    "uniqueness": "server"
                },
                {
                    "caseExact": false,
                    "type": "complex",
                    "description": "A list of members of the Group.",
                    "mutability": "readWrite",
                    "name": "members",
                    "multiValued": true,
                    "required": false,
                    "returned": "request",
                    "uniqueness": "none",
                    "subAttributes": [
                        {
                            "caseExact": false,
                            "type": "string",
                            "description": "The significant value for the attribute",
                            "mutability": "readWrite",
                            "name": "value",
                            "multiValued": false,
                            "required": false,
                            "returned": "default",
                            "uniqueness": "none"
                        },
                        {
                            "caseExact": false,
                            "type": "string",
                            "description": "A label indicating the attribute's function",
                            "mutability": "immutable",
                            "name": "type",
                            "multiValued": false,
                            "required": false,
                            "returned": "default",
                            "uniqueness": "none",
                            "canonicalValues": [
                                "Group",
                                "User"
                            ]
                        }
                    ]
                }
            ]
        }

 

@norfas Thank you. We are working on improving the UI to surface this additional call.

The Patch Group: Replace Attributes has a risk of replacing all group members. Due to this risk, this test will be removed from the test suite. Please also note that our provisioning service does not support any Patch Group: Replace attributes operation. You can therefore ignore this test for now until we remove it from the suite.

`urn:ietf:params:scim:schemas:extension:companyName:2.0:User:authCodes[type eq "Card"].value`

When using customExtensions to support multi-valued complex attributes, this removes the URI prefix in the request body, I believe this may be a bug, is this problem isolated to the scim validator, or would the same problem occur with the producton provisioning service?

Screen Shot 2023-04-03 at 3.29.53 pm.png

@owinoakelo 

 

I mark some attributes as immutable but SCIM validator still send PATCH request with Patch Operation to `replace` those fields. 

How can I get rid of it?

And the Validator reports some values missing in the Fetched Resource, but they do exist there.

chanhle_0-1684938368461.png

 
 

 

 

@chanhle The experience with immutable attributes is an issue with the validator and we are working to fix it. You can ignore such calls

 

On the second issue with the patch response: The actual flow is we send a PATCH request>You acknowledge and send a response> we send a GET to confirm that you committed the patch> we compare what we sent in the PATCH with what you return in the GET. This error means that some of the changes have not been done on the identities that GET calls.

 

Hope this clarifies the issue.

hi@owinoakelo 

 

I am sorry. but when I monitor the requests from the validator , I don't see the second `GET request` you mentioned as below


we send a GET to confirm that you committed the patch

And even when I manually send a GET request which has the same path (/scim/Users/<UserID>) with the PATCH request, the response is correct and the same as the response of the PATCH request (sent by the validator).

 

Could you help me verify again this case?

 

PATCH request body 

{
    "schemas": [
        "urn:ietf:params:scim:api:messages:2.0:PatchOp"
    ],
    "Operations": [
        {
            "op": "replace",
            "path": "emails[type eq \"work\"].value",
            "value": "email address removed for privacy reasons"
        },
        {
            "op": "replace",
            "path": "roles[primary eq \"True\"].display",
            "value": "3WF31QEVRLRR"
        },
        {
            "op": "replace",
            "path": "roles[primary eq \"True\"].value",
            "value": "HISMH3ZHYVXI"
        },
        {
            "op": "replace",
            "path": "roles[primary eq \"True\"].type",
            "value": "M4PBMUTFLU7N"
        },
        {
            "op": "replace",
            "value": {
                "userName": "email address removed for privacy reasons",
                "active": false,
                "externalId": "b76619a5-6635-450c-9f61-bb3cb7882460",
                "displayName": "AIPZKDL1CCXH",
                "preferredLanguage": "Q17Q2L9WRNLS"
            }
        }
    ]
}

The PATCH response body

{
    "id": "601E76C3-0BF5-40DD-B0BB-74C766B7E1C9",
    "externalId": "b76619a5-6635-450c-9f61-bb3cb7882460",
    "userName": "email address removed for privacy reasons",
    "active": false,
    "preferredLanguage": "Q17Q2L9WRNLS",
    "schemas": [
        "urn:ietf:params:scim:schemas:core:2.0:User",
        "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User"
    ],
    "meta": {
        "resourceType": "User",
        "location": "/scim/Users/601E76C3-0BF5-40DD-B0BB-74C766B7E1C9"
    },
    "emails": [
        {
            "value": "email address removed for privacy reasons",
            "type": "work"
        }
    ],
    "roles": [
        {
            "value": "HISMH3ZHYVXI",
            "display": "3WF31QEVRLRR",
            "type": "M4PBMUTFLU7N",
            "primary": "True"
        }
    ],
    "displayName": "AIPZKDL1CCXH"
}

 

The GET response body 

 

{
    "id": "601E76C3-0BF5-40DD-B0BB-74C766B7E1C9",
    "externalId": "b76619a5-6635-450c-9f61-bb3cb7882460",
    "userName": "email address removed for privacy reasons",
    "active": false,
    "preferredLanguage": "Q17Q2L9WRNLS",
    "schemas": [
        "urn:ietf:params:scim:schemas:core:2.0:User",
        "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User"
    ],
    "meta": {
        "resourceType": "User",
        "location": "/scim/Users/601E76C3-0BF5-40DD-B0BB-74C766B7E1C9"
    },
    "emails": [
        {
            "value": "email address removed for privacy reasons",
            "type": "work"
        }
    ],
    "roles": [
        {
            "value": "HISMH3ZHYVXI",
            "display": "3WF31QEVRLRR",
            "type": "M4PBMUTFLU7N",
            "primary": "True"
        }
    ],
    "displayName": "AIPZKDL1CCXH"
}

 

I've run into the same issue with phonenumber[0].primary being sent as a string phone number, rather than a boolean.
Anyone found a workaround?

I think we put in a hardcoded True/False value as we have a built-in workaround for Microsoft sending text values for booleans. But that means we can't use schema discovery.
Alternatively, we removed that value from the schema; again, can't use discovery.

Hoping this gets fixed soon, it's a little frustrating.

I've not checked the validator for a while but trying it today I get a couple of failures I didn't get in the past. "Filter for an existing user" and "Filter for an existing user with a different case" aren't returning any users but when I use the same request I do get the user returned. Is this a problem with the validator?

@Doogal Sorry, I don't follow. Is it possible to include some screenshots to help understand the issue?

@owinoakelo, sorry it was my mistake. I've now realised we are doing some caching in our back end so the user the validator creates is not immediately returned for the following /Users request

Hi there,

 

I've been running the SCIM validator against my endpoint and have spotted something that I believe may be a bug, but would like to confirm on here.

 

I'm using the discover schema functionality, my schema gets discovered and in the mappings i'm shown the following:

alexjones_1-1689842450844.png

 

With the JSON from my '/Schemas' endpoint returning the following for the 'primary' sub attribute on 'phoneNumbers':

{
...
    "name""primary",
    "type""boolean",
...
}

 

When I run the SCIM validator, I can see that the validator is sending the following value for a 'phoneNumber' when adding a User through POST

 

{
    "type": "work",
    "value": "1-227-170-3259 x696",
    "primary": "(882)058-8632 x34575"
}

 

As you can see, the primary property is a string. This doesn't appear to be the case for the 'primary' property in the 'emails' array in the same request

 

{
    "type": "work",
    "value": "email address removed for privacy reasons",
    "primary": true
}

 

For a bit more detail, emails in the schema discovery mapping screen shows the following:

 

alexjones_2-1689842882914.png

 

And my '/Schemas' endpoint returns the following for the 'primary' sub attribute for the emails array:

{
...
    "name""primary",
    "type""boolean",
...
}

 

 

Any help would be greatly appreciated

 

@alexjones You are right. We have an issue with the 'phoneNumber' 'primary' attribute which should be sent as a Boolean, but SCIM Validator sends it as string, ignoring the schema. We have this captured in our backlog of issues to be addressed this quarter (ending in September)

@owinoakelo 

 

Brilliant, thanks for your response.

 

Is the issue backlog publicly available? Be it read-only or otherwise?

 

I had another couple of questions but don't want to raise them if they've already been raised before

@owinoakelo 

 

Hi Owino,

 

I just have a couple of questions on some of the PATCH operations in the SCIM validator.

 

Should the following PATCH operation be allowed?

{
    "op": "replace",
    "path": "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:manager",
    "value": ""
}


This looks like it should be a 'remove' operation to me. We're currently running into an issue around the value being empty due to validation in our implementation for 'replace' operations. 

 

Should the following mapping exist in the defaults?

roles[primary eq "True"].display

 

Primary is a boolean attribute and is expected to be a string in the filter here. In addition to this, the initial create request doesn't contain any roles. This means when that even when the filter does include a boolean and the filter is ran, there is no role for primary and so the value for the 'type' attribute cannot be replaced.

{
    "op": "replace",
    "path": "roles[primary eq true].type",
    "value": "I6T3MZAC1HXV"
}


What's the intention of the following PATCH operation?

 

{
    "op": "add",
    "path": "emails[type eq \"work\"].value",
    "value": "email address removed for privacy reasons"
}

 

In the SCIM validator this example has the same issue as the previous, where there are no emails in the initial create request and so the filter fails to find an email to add a 'value' to. I wanted to double check that the intention of the request wasn't to add an email like the following

 

{
    "type": "work",
    "value": "email address removed for privacy reasons"
}

 

I understand there are a few issues here, any help will be greatly appreciated! Thanks!

I keep getting this failure in the validator if my Users schema exposes a groups attribute:

J_Stokes_0-1692286122480.png

What it says just seems incorrect. Direct and indirect groups are part of the core spec in RFC7643 section 4.1.2 (page 24), described exactly as I've used them.

J_Stokes_1-1692286366922.png

 

@owinoakelo 

I was trying to use SCIM sdk library (https://github.com/Captain-P-Goldfish/SCIM-SDK) and they are adding common attributes (like "id") to the schema. According to specification, some providers MAY include the common attributes. https://datatracker.ietf.org/doc/html/rfc7643#section-3.1

For backward compatibility, some existing schema definitions MAY list
   common attributes as part of the schema.  The attribute
   characteristics (see [Section 2.2](https://datatracker.ietf.org/doc/html/rfc7643#section-2.2)) listed here SHALL take precedence
   over older definitions that may be included in existing schemas.

Unfortunatelly, SCIM validator doesn't handle this case correctly. 

ekos2001_0-1692379113567.png

 

ekos2001_1-1692379149722.png

 

Is there any chance that it could be fixed?

 

Thank you

 

@owinoakelo 

 

We've got a very simple schema that only includes the following user attributes (All required) with no group support:

  • • userName
  • • active
  • • emails[type eq "work"].value
  • • name.givenName
  • • name.familyName

 

When I run the validator I'm getting the following error: Unknown test failed. Object ref not set to an instance of an object

        "FailedTests": [

            {

                "$id": "31",

                "Description": "One or more errors occurred. (Object reference not set to an instance of an object.)",

                "Name": "Unknown test failed",

                "Results": [

                    {

                        "$id": "32",

                        "Message": "Internal server error: One or more errors occurred. (Object reference not set to an instance of an object.)",

                        "Outcome": 1,

                        "MoreInformation": ""

                    }

                ],

                "ResponseHeaders": "",

                "ResponseStatus": "One or more errors occurred. (Object reference not set to an instance of an object.)",

                "ResponseBody": "",

                "InitialRequestBody": "",

                "RequestBody": "One or more errors occurred. (Object reference not set to an instance of an object.)",

                "SFComplianceOptional": false,

                "HasTestPassed": false,

                "TestexecutionTime": 0

            }

        ],

I've run through the postman tests directly and they're all passing, or failing for valid reasons i.e.

  • Get User Filters fails due to DisplayName not being supported in our schema
  • Get User1 Check Patch due to Formatted not being supported in our schema
  • Create user1 fails as emailName357 is not a valid email address format

I can change the above 3 tests to use valid data and then they pass as expected

 

Based on the above, I'm not seeing anything wrong with our implementation. Is the error an issue with the scim validator itself or am I missing something? How do we proceed with getting our implementation approved?

For the validator, one of the tests is to attempt to create a duplicate user. The validator expects this to fail when it succeeds and then expects it to succeed when it fails.

 

Does anyone know what the validator is checking for specifically with this test?