Query serverless SQL pool from an Apache Spark Scala notebook
Apache Spark notebooks in Azure Synapse Analytics workspace can execute T-SQL queries on a serverless Synapse SQL pool. This way you can leverage load data from some SQL table or view into your Apache Spark data frames apply some advanced data processing. In this article you will learn how to call SQL code form spark notebook.How to – Advanced Properties of Linked Services
Azure Data Factory and Synapse Pipelines have a wealth of linked service connection types that allow them to connect and interact with many services and data stores. The Workspace UI provides the most important properties that are needed for the connection. However, at times we need more control that the UI doesn’t offer. In this tutorial, we will focus on customer scenario to see how you can find all the available properties for the Linked Service you are working with, then how to incorporate that property into your Linked Service directly from within the UI.
Microsoft® and the .NET Foundation announce the release of version 1.0 of .NET for Apache® Spark™
It is our pleasure to announce the release of version 1.0 of .NET for Apache® Spark™, an open source package that brings .NET development to the Apache® Spark™ platform. .NET for Apache® Spark™ is available as an OSS project on the .NET Foundation’s GitHub and can be downloaded from NuGet. It is built into Azure Synapse Analytics and Azure HDInsight where you can enjoy an out-of-the-box experience. Version 1.0 will be released in these products in the next major release. .NET for Apache® Spark™ can also be used in other Apache Spark cloud offerings including Azure Databricks and as well as AWS EMR Spark. For on-prem deployments, it offers is multi-platform support for Windows, MacOS, and Linux.
