Cosmos DB Data Connection - New role required
Published May 29 2023 04:01 PM 2,571 Views
Microsoft

You might have read it in our recent breaking change bundle last week.

 

Before going Generally Available (GA) in a few weeks, we have indeed enacted a breaking change on the Cosmos DB data connection:  it now requires the managed identity (either system or user) assigned to the data connection to have Cosmos DB Account Reader Role (Control Plane role) on the Cosmos DB account.

 

Now before you read any further, this doesn't actually break existing data connection as we explain in more details below.

 

Since we released the public preview in January, we required Cosmos DB Built-in Data Reader (Data Plane role).  That role allowed Kusto to read the Cosmos DB change feed.

 

The new Control plane role allows Kusto to do a control plane call to Cosmos DB extract the account's URI.  We give details on that below.

 

Although the data connection feature is still in public preview, we wanted to limit the impact on consumer as much as possible.  We deliberately took a step-by-step approach to the change.

Does this break my existing connection?

No.  If your connection was working yesterday, it's going to continue working all the way to General Availability.

 

The impact you will have is if you change / update your data connection.

How can I see if my data connection is affected?

In the Azure Portal, go to your cluster.  From the cluster, pick the Databases pane.  Select your database.  Pick the Data connections pane.

 

If one of your connection is affected, you should see a warning cue next to the Managed Identity:

 

vplauzon_0-1685398791784.png

 

If you click on the down arrow next to the Managed Identity, you should see some ingestion metric graphs.  As stated above, that change shouldn't affect your data connection hence you shouldn't see any issue.

 

At the bottom of that screen you should see this:

 

vplauzon_2-1685399232450.png

 

The green sign tells you the managed identity does have the data plane role (allowing Kusto to read the change feed).  This is why your data connection should still work.

 

The red sign tells you the managed identity does not have the control plane (or RBAC) role and suggest you assign it.

How can I fix it (Portal)?

If you, and by you I mean the logged in user in the Azure Portal, are contributor on the Cosmos DB account, this should be very easy.

 

Simply select the data connection and save it.

 

vplauzon_3-1685399530517.png

 

What happens here is the Portal, on your behalf (i.e. using the logged in user identity), is assigning the role to the Cosmos DB account.

 

If you are not Contributor on the Cosmos DB account, the Portal will display an error and will block the data connection update.  If that is the case, you can ask a colleague who is Contributor to perform the action or one described in the next section.

How can I fix it (Automation)?

As always, you can replace an Azure Portal action by some automation.

 

In the online documentation, we describe two:  using the Azure CLI and using an ARM template.  But of course, you can also accomplish it using PowerShell or one of the Azure Management SDK (e.g. C#).

Step-by-step approach to the change

As stated, we are taking a step-by-step approach into bringing the change:

 

  • Since last week, the Azure Portal data connection pane is trying to assign the RBAC role upon update.  If it wasn't able to (your user account wasn't Contributor), it didn't report any error and continued
  • This week, the Azure Portal will start reporting an error and will stop the update
  • In a few days, the same thing will happen in non-Azure Portal flow, e.g. if you provision a data connection through an ARM portal and the managed identity doesn't have the role, the provisioning will fail
  • Once all those steps are done and we do not detect issues, we will be able to go through General Availability

Summary

Although this is a breaking change, your data connection will not stop working.  Only updates will be blocked until you can give access to your cluster's managed identity.

 

We hope this change produce as little friction as possible!

 

Why do we need that new role?  (Technical discussion)

You might ask yourself why do we need that new role since your data connection has been working fine (and will keep working fine) since January?

 

The reason is a bit technical.  If you are interested in those details, read on, if not, you do not need to understand that part to go forward.

 

When you provision a data connection, under the hood you provide the Cosmos DB account's resource ID.  That is an Azure resource ID (starting with /subscriptions/<your subscription's ID>/...).

 

For Kusto to read from the change feed, it needs the Cosmos DB account Uri.  That is the Cosmos DB REST API Uri.  So far Kusto was inferring the Uri from the resource ID.  That approach isn't future proof.  We wanted the provisioning to be future proof, hence Kusto is now "asking" Cosmos DB for the Uri.  In order to do that, it needs the RBAC role to do so.

 

Why doesn't Kusto take a Cosmos DB Uri instead of resource ID?  For security reason:  we do not want an administrator of your cluster  to provision inadvertently (or maliciously) a Cosmos DB data connection pointing to anything else than a Cosmos DB account.  By taking an Azure resource ID, we ensure the data connection is legitimate.

 

That's it!  I hope this last section satisfied the curious among you!

Co-Authors
Version history
Last update:
‎May 29 2023 04:01 PM
Updated by: