synapse spark
69 TopicsSynapse – Data Lake vs. Delta Lake vs. Data Lakehouse
As a data engineer, we often hear terms like Data Lake, Delta Lake, and Data Lakehouse, which we might be confusing at times. In this blog we’ll demystify these terms and talk about the differences of each of the technologies and concepts, along with scenarios of usage for each.52KViews14likes0CommentsRead & write parquet files using Apache Spark in Azure Synapse Analytics
Azure Synapse Analytics is analytical solution that enables you to use Apache Spark and T-SQL to query your parquet files on Azure Storage. In this article you will see the sample that can help you to do basic basic read/write operation on parquet files placed in Azure Storage.46KViews2likes0CommentsUnleashing the capabilities of Azure Synapse Analytics for a healthcare customer
This blog post aims to provide you with insights into the best practices for optimizing performance in data ingestion and transformation. You will learn how to efficiently use available resources, compute capacity, and workload management in Azure Synapse Analytics workspace. While the solution discussed in this blog pertains to a healthcare industry customer, the optimization techniques presented here are applicable to other industries as well.38KViews5likes6CommentsThe Data Lakehouse, the Data Warehouse and a Modern Data platform architecture
There are two contradictory themes about how to build a modern data platform being proposed to data architects today. This article discusses why is there such a big disparity between two approaches, how we make sense of these competing patterns and why the modern data warehouse architecture provides a flexible and pragmatic approach.38KViews11likes3CommentsThe best practices for organizing Synapse workspaces and lakehouses
While designing the Lakehouse solution, you should carefully organize your databases and tables based on the underlying folder structure. In this article, you will find some best practices and recommendations that can help you to organize your lakehouses if you are using Synapse Analytics workspace to implement them.38KViews16likes3CommentsNotebook - This request is not authorized to perform this operation. , 403
This a quick post about this failure and how to fix: Error: org.apache.spark.sql.AnalysisException: java.lang.RuntimeException: The operation failed: 'This request is not authorized to perform this operation.', 40335KViews0likes17Comments