Forum Widgets
Latest Discussions
Azure Synapse: What’s Next?
With the recent introduction of Microsoft Fabric, which aims to unify various data and analytics workloads into a single platform, how will this impact the future of Azure Synapse Analytics? Specifically, will Azure Synapse Analytics become obsolete, or will it continue to play a significant role alongside Microsoft Fabric? Additionally, what are the recommended migration paths and considerations for organizations heavily invested in Azure Synapse Analytics?”
Azure Sql Database Linked Service Lookup Activity Error
Started receiving this error when previewing results in both lookup and copy activities that are connected to an Azure SQL Database. My production pipeline (that is triggered) fails on the Azure Database lookup activity. Does anyone hav any insights on what is causing this issue? And/or how to resolve? Thanks!Error 403 on random occasions
Getting the following error on random occasions since yesterday. It may be related to other issues I am seeing at the same time, such as the Spark pool suddenly not starting up and the splash screen getting stuck with 401 (unauthorized) error. This is not related to our configuration of access. This has been working before. It also works when I try some times, and other times not. It seems completely random. Azure Service Health is just showing green (as always).Synapse Studio stuck on loading the splash screen with '401 unauthorized'
Synapse Studio has been stuck on loading for two days in a row. On a few occasions, I have been able to access Synapse, but most often I am not able to. It's just getting stuck on the splash screen when starting styudio. On yesterday evening, I was able to work as normal, but in the morning the issue was back again. This happens on different instances and resource groups where Synapse is installed. I have tried both Edge and Chrome. Same issue both places We have tried on different accounts where we know we have working access I have checked Azure health status, and Synapse shows up falsely as green. When I look in the console log for my web browser, I see 'Failed to load resource: the server responded with a status of 401 ()'. Meaning there is some kind of authorization issue for URL https://management.azure.com/subscriptions/....?api-version=2025-05-01 (I have deliberately removed the hexadecimal subscription ID).
Nested if statement in the expression builder (dynamic editor)
Hello I am trying to check activity steps failed and if so, get Target (activity) value, something like this below @{ "FailedStepAt": "@{if(not(empty(activity('lkp_tables_list').erro.message)), activity('lkp_tables_list')?.erro.target, if(not(equals(activity('lkp_GetPipelineIDByName').error.message, '')), activity('lkp_GetPipelineIDByName').error.target, if(not(equals(activity('Export_table_to_NTADLS').error.message, '')), activity('Export_table_to_NTADLS').error.target, '')))}", "ErrorMessage": "@{concat(activity('lkp_tables_list')?.error?.message, activity('lkp_GetPipelineIDByName')?.error?.message, activity('Export_table_to_NTADLS')?.error?.message } But it doesn't work . However, to check the target value if activity fails, used set variable to capture the target value but it supports with one condition something like ." "@{if(not(empty(activity('lkp_GetPipelineIDByName').error)), activity('lkp_GetPipelineIDByName').error.target, 'no err') }" I want to achieve using nested if within dynamic edition Or within set variable activity Any help would be appreciatedData Flow textbox size increase
Hi Community, Is there a way to increase the textbox size to Data Flows Select mapping fields (or request a change)? The Destination field (Name As) is way smaller than the source field, and therefore can't see the complete text - there is plenty of space to the right to increase this size...
Query insights
Question: How can I identify unused data in a modern data platform built with Azure Synapse and the medallion architecture using Log Analytics? I’m working with a client who has built a modern data platform based on the medallion architecture, leveraging Azure Synapse and Azure Storage Accounts. Users access the data in various ways within Synapse workspaces: some through Python scripts, others through serverless SQL endpoints, and others via dedicated SQL pools (utilizing views and stored procedures). We log a significant amount of information via Log Analytics, which means that all select statements executed on the data are essentially logged. The client now wants to identify which data is not actively used, in order to reduce storage costs by removing unused datasets. In a traditional SQL data warehouse, the Query Store could be used for this, but in this platform, we only have access to the log data stored in Log Analytics. My question is: How can we, based on the logs in Log Analytics, determine which data (tables, views, etc.) is processed through the various layers of the medallion architecture but not actually used? The goal is to remove unused data to save costs. Some additional questions: Is there a way to analyze usage patterns of datasets based on raw logs in Log Analytics? Are there any existing tools or KQL queries that could help identify which datasets have been inactive over a certain period? Could a metastore tool, such as Azure Purview, play a role in identifying unused datasets? If so, how can this be integrated with our existing platform? Any suggestions or insights would be greatly appreciated!SQL DB(serverless sql pool) access grant to end user.
What is the best way to grant access to users for SQL DB(serverless sql pool) build on top of synapse azure lake db. How should SQL DB (views or external tables) access the lake database (storage container). Should we use RBAC, Managed Identity, ACLS, etc. Please tell me the best way/used in industry to handle it.
Intermittent Binding Error when accessing Dataverse / Data Lake.
Dear All, We have a Dataverse created via an Azure Synapse Link. The data is synching with D365 Finance & Operations every 15 mins and is using the Parquet file format. We have created a separate DB (that basically is just a bunch of Views over the Tables in the Dataverse - call it VIEW_DB) which we expose to other users rather than going directly to the Dataverse DB. Once we have this data in the Dataverse, we then need to copy some data to a different target for reporting and use traditional SSIS to do this. As such, also every 15 mins we run this process to extract data from the VIEW_DB that is being synch'd. This mostly works OK (99% success). However, every so often and at irregular times, we receive the following error message ... Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.". Description: "Could not use view or function 'db_datareader.TM1_vwJournal' because of binding errors.". Description: "Invalid object name 'dataverse_xxxx_unqxxxxxxxxx.dbo.LedgerJournalTable'.". Description: "OLE DB Source" failed validation and returned validation status "VS_ISBROKEN". Description: One or more component failed validation. Has anyone experienced anything like this? Why would this run OK for most of the time but fail once or twice every couple of days or so? Would it have anything to do with the Sync process disrupting our OLEDB connection? Any help much appreciated. Basil
