Forum Widgets
Latest Discussions
CosmosDb multi-region writes and creating globally unique value
Hi! I am trying to understand how to deal with conflicts when using multi-region writes. Imagine I am trying to create a Twitter clone and I have to ensure that when a user creates an account, it also select an unique user handle (a unique key like username ). In a single region I would just have a container with no indexing and then create that value as a partition key, if I succeed it means that there was not another handle with that value and from this point nobody else will be able to add it. But when thinking in multi-region writes, two persons in different regions could indeed add the same handle. Then the conflict resolution strategy would need to deal with it. But the only conflict resolution possible here is to delete one of them. But this is happening asynchronously after both persons successfully created their accounts, so one of them would get a bad surprise the next time they log in. As far as I understood, there is no way to have Strong consistency across multiple write regions. After thinking for a while about this problem I think there is no solution possible using multiple write regions. The only solution would be to have this container in an account with a single write region, and although the client could do a "tentative query" to another read-only region to see if a given handle is already taken, in the final step to actually take it I must force the client to do the final write operation in that particular region. Consistency levels here only help to define how close to reality is the "tentative query", but that is all. Does this reasoning make sense? Many thanks.Azure AI Search for offloading cross partition queries in Cosmos Db?
HI Azure Cosmos Db team, We were testing on a design to use Azure AI Search indexing on Cosmos Db and use AI Search for cross partition queries that was coming to cosmos db.AI Search will return the unique Id that can be used for point read in Cosmos Db. Apart from having eventual consistency always when incorporating this design which is a disadvantage can we guarantee accuracy with Azure AI Search equality and greater than filters for transactional workloads? We can ensure cosmos db will give the correct response ( accurate) when requested with a query for transactional workloads? We are not utilizing synapse link because of concurrency and API Centric application architecture. With Regards, Nitin RahimCosmos Db JAVA SDK Retry Policy
Hi Azure Cosmos Db Team, We haven't explicitly set retry policy in the event of throttling. Uses the default throttling retry policy. Below as seen from diagnostics. throttlingRetryOptions=RetryOptions{maxRetryAttemptsOnThrottledRequests=9, maxRetryWaitTime=PT30S} However when we encountered actual throttling ("statusCode\":429,\"subStatusCode\":3200) we see in the diagnostics values increasing in multiples of 4 \"retryAfterInMs\":4.0 x-ms-retry-after-ms=4, \"retryAfterInMs\":8.0 x-ms-retry-after-ms=8 and resulting in Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later. Can you please let me know the difference in behavior here(maxRetryWaitTime as shown in throttlingRetryOptions and retryAfterInMs in the diagnostics as seen above in the event pf throttling) ? I was expecting in the event of throttling the request will be retried after 30 seconds only based on throttlingRetryOptions setting? This is having a compounding effect in case of concurrent requests which affects overall throughput. We need to customize based on our requirement the retry no of times and interval in the event of throttling. Which parameter should we use for that? With Regards, Nitin RahimNumber fields rounding off in Cosmos Db from Azure Portal
Hi Azure Cosmos Db Team, We are seeing an issue from Portal in Cosmos Db. When we enter a numeric field in cosmos db from portal for eg "digittest": 123456789123456789 it is rounding of to "digittest": 123456789123456780 Saw this behavior after 16 characters. Thought the issue is with portal ( related to java script).So tried using the sdk. When using JAVA SDK we saw we were able to retrieve the same value we created with sdk. However when we update another attribute in document in the portal and then retrieve the same document from sdk we are seeing the portal saved value for the number even though we didn't update the LONG number field . Can you please confirm for LONG fields and INT fields we can save and see the same way in portal and from sdk irrespective of length? This can be very misleading. With Regards, Nitin RahimNOT IS_DEFINED in Comsos Db
Hi Azure Cosmos Db team, We need to use NOT IS_DEFINED to evaluate a property below NOT IS_DEFINED(c.TestLocation['South Central US'] per the results analyzed NOT IS_DEFINED is not utilizing Index and is doing a full scan. There was an update from Cosmos Db Team that NOT IS_DEFINED can utilize index now. Below is the blog pertaining the same. https://devblogs.microsoft.com/cosmosdb/april-query-improvements/ Can you please provide an alternative available if we cannot use NOT IS_DEFINED to evaluate the same property and to utilize index without a data model update? With Regards, Nitin RahimPagination in Cosmos Db (Maxitemcount or pagesize)
Hi Azure Cosmos Db Team, Is there an equivalent in Cosmos Db for MaxItemCount parameter( present in dotnet sdk) in JAVA SDK? We wanted to test with MaxItemCount to -1 in JAVA SDK? Where is this exposed? In feed options or Cosmos queryrequestoptions? I see a pagesize parameter in JAVA SDK but we cannot set the same to -1 it seems so that the SDK can optimize the same? We are using SQL API. We are currently using pagesize ==100 for cross partition queries.We are seeing high latency for queries exceeding 2000 results from SDK even though from the portal the RU and latency seems to be less. https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.documents.client.feedoptions.maxitemcount?view=azure-dotnet With Regards, Nitin RahimAzure Cosmos Db Materialized View general availability
Hi Azure Cosmos DB Team, Can you please confirm if materialized view for Cosmos Db is general available now and can be recommended for production workloads? Also lag for materialized view to catchup is dependent on SKU allocation and provisioned RU in source and destination container only? Does consistency have any impact when querying the materialized view query or for the materialized view catcup in case of heavy writes and updates in source container? If the account is set up with bounded staleness consistency materialized view querying will also have bounded staleness consistency associated with them when using cosmos JAVA sdk for querying? We are using SQL API. With Regards, Nitin RahimControlling an Application’s Throughput Consumption in Cosmos DB with ThroughputControlGroup
Hi Azure Cosmos Db Team, Was referring to the below document link to limit the throughput consumption per requests. https://the9090rule.medium.com/controlling-an-applications-throughput-consumption-in-cosmos-db-with-throughputcontrolgroup-247cc0eafa92 How will the application behave in case the operations in the group exceed the set throughput? Will it get throttled? With Regards, Nitin RahimReplace Operation Cosmos Db
Hi Team, We have noticed that during millisecond interval sequential operations (create, read, replace operations happening in very small interval) for some records even though we don't get an error we see the cosmos database didn't do a successful replace or update on the item. Is there a limitation in the replace operation with high volume requests with respect to atomicity? With Regards, Nitin RahimConnection Pooling
Hi Azure Cosmos Db Team, Is there any best practices to identify if there is connection pooling happening with Cosmos Db JAVA SDK? I see for pointread call the average duration in ms is well around 10 ms from comsos db diagnostic logs but in app insights for the Microsoft.DocumentDb call duration for the cosmos db interaction ( for doing pointread) varies considerably and sometimes reaches around 600 ms especially as number of threads( 5-600 threads step by step in our case) are increased in load testing ? I am suspecting this is happening as the sdk is trying to open a new connection as threads increases and this results in increased latency for "some" requests? I see that for the first call there is a login.microsoftonline.com call happening before doing the actual read in cosmos db but cannot see this call in all the ones with higher latency from app insights probably due to sampling in app insights? We are using direct connection mode in sdk. The direct connection configuration parameters currently used are based on the below documentation. Connection configurations for Azure Cosmos DB Java SDK v4 | Microsoft Learn With Regards, Nitin Rahim