Azure AD SCIM Validator is in General Availability (GA) Status

owinoakelo 

Can the SCIM Validator be made open source or at least have a location where issues can be filed against it? We've encountered a couple problems with the SCIM Validator and would like some way of reporting issues or discussing misunderstandings with the developers.

The first is that it does not read the schema correctly. For instance, our endpoint describes the 'email' 'primary' attribute as boolean, but SCIM Validator sets it to string, apparently ignoring our schema.

Another is a failure during group testing where it re-creates a user that already exists. It seems not to expect a 409 reply.

It is very hard to tell *what* the tester expects without a diagnostic error or insight into its source code.

Where's the best place to ask questions about the results I'm getting from the validator? My POST /Users request is failing validation but I'm not getting a reason for the failure

Looks like there was recently a big update on the Validator. The "Discover Schema" now sets the correct type (string/boolean). Thank you!

However, now we're seeing a new complication. It appears the Validator is trying to add new entries using replace with a value selection filter (valuePath). This was discussed in 708183: SCIM PATCH of Complex Multi-Valued Attribute Includes Filter and Sub Attribute in Path in the context of patching with add. And that discussion makes sense as the spec is a little vague.

However, with replace, RFC 7644 is quite specific in 3.5.2.3:

   o  If the target location is a multi-valued attribute for which a
      value selection filter ("valuePath") has been supplied and no
      record match was made, the service provider SHALL indicate failure
      by returning HTTP status code 400 and a "scimType" error code of
      "noTarget".

If the valuePath does not match anything, then you need to return 400 noTarget. Microsoft Azure AD SCIM Validator does not appear to expect that reply and expects us to add new entries on replace.
I disagree that:

 o  If the target location path specifies an attribute that does not
      exist, the service provider SHALL treat the operation as an "add".

would change this interpretation as that does not consider the filter; plus the way these are written they get more and more specific as you go down the list.

A specific example:
The current value of emails:

 "emails": [                     
   {                             
     "type": "work",             
     "value": "some_primary_email",  
     "primary": true             
   }                             

and the replace operation:

"Operations": [{
  "op": "replace",
  "path": "emails[type eq \"home\"].value",
  "value": "some_email"
}]

NOTE: values do not show email addresses as those get removed (privacy checker) - so they are replaced with "some_primary_email" and "some_email"

This should fail as there is no email of type = home to replace.

owinoakelo 

I suspect I found a bug in SCIM Validator. What should I do?

Please refer to the image below.

Just running through the validator, and noticed that for CREATE operations, when the status code returns 200 it fails the test, however the test result doesn't inform us that it's looking for a 201 status (we had to try a lot of things before we figured it out)

owinoakelo 

On another note, I am not convinced that the Validator treats complex multiValued 'members' attribute for Groups correctly, if they are exposed in Schema.
Specifically, if I have Group.members[type ="User"] collection defined in Schema like shown below, then the Validator would attempt to generate a Patch request as shown on the screenshot with some bogus(?) value in the Replace operation for users. That bogus value such as "QQ7M8HRCRACN" looks like a display name for a group, rather than User ID that I would expect it to be.
Or am I missing SCIM spec completely here, and this Replace request has totally different semantics?

See image attached.

To compare, Add Member operation on Group looks fine, with proper User ID:

My schema for the group is as follows:

{
            "name": "Group",
            "description": "Group",
            "meta": {
                "resourceType": "Schema",
                "created": "0001-01-01T00:00:00",
                "lastModified": "0001-01-01T00:00:00"
            },
            "id": "urn:ietf:params:scim:schemas:core:2.0:Group",
            "schemas": [
                "urn:ietf:params:scim:schemas:core:2.0:Schema"
            ],
            "attributes": [
                {
                    "caseExact": false,
                    "type": "string",
                    "description": "A human-readable name for the Group. REQUIRED.",
                    "mutability": "readWrite",
                    "name": "displayName",
                    "multiValued": false,
                    "required": true,
                    "returned": "default",
                    "uniqueness": "server"
                },
                {
                    "caseExact": false,
                    "type": "complex",
                    "description": "A list of members of the Group.",
                    "mutability": "readWrite",
                    "name": "members",
                    "multiValued": true,
                    "required": false,
                    "returned": "request",
                    "uniqueness": "none",
                    "subAttributes": [
                        {
                            "caseExact": false,
                            "type": "string",
                            "description": "The significant value for the attribute",
                            "mutability": "readWrite",
                            "name": "value",
                            "multiValued": false,
                            "required": false,
                            "returned": "default",
                            "uniqueness": "none"
                        },
                        {
                            "caseExact": false,
                            "type": "string",
                            "description": "A label indicating the attribute's function",
                            "mutability": "immutable",
                            "name": "type",
                            "multiValued": false,
                            "required": false,
                            "returned": "default",
                            "uniqueness": "none",
                            "canonicalValues": [
                                "Group",
                                "User"
                            ]
                        }
                    ]
                }
            ]
        }

`urn:ietf:params:scim:schemas:extension:companyName:2.0:User:authCodes[type eq "Card"].value`

When using customExtensions to support multi-valued complex attributes, this removes the URI prefix in the request body, I believe this may be a bug, is this problem isolated to the scim validator, or would the same problem occur with the producton provisioning service?

owinoakelo 

I mark some attributes as immutable but SCIM validator still send PATCH request with Patch Operation to `replace` those fields. 

How can I get rid of it?

And the Validator reports some values missing in the Fetched Resource, but they do exist there.

I've not checked the validator for a while but trying it today I get a couple of failures I didn't get in the past. "Filter for an existing user" and "Filter for an existing user with a different case" aren't returning any users but when I use the same request I do get the user returned. Is this a problem with the validator?

Hi there,

I've been running the SCIM validator against my endpoint and have spotted something that I believe may be a bug, but would like to confirm on here.

I'm using the discover schema functionality, my schema gets discovered and in the mappings i'm shown the following:

With the JSON from my '/Schemas' endpoint returning the following for the 'primary' sub attribute on 'phoneNumbers':

When I run the SCIM validator, I can see that the validator is sending the following value for a 'phoneNumber' when adding a User through POST

{
    "type": "work",
    "value": "1-227-170-3259 x696",
    "primary": "(882)058-8632 x34575"
}

As you can see, the primary property is a string. This doesn't appear to be the case for the 'primary' property in the 'emails' array in the same request

{
    "type": "work",
    "value": "email address removed for privacy reasons",
    "primary": true
}

For a bit more detail, emails in the schema discovery mapping screen shows the following:

And my '/Schemas' endpoint returns the following for the 'primary' sub attribute for the emails array:

Any help would be greatly appreciated