Orb - A tool to run Live Site using Azure Data Explorer
Orb (Object-based Resource Browser) is a tool used by several internal Azure engineering and support teams to run Live Site for their services – I’m happy to announce that this tool is now available on GitHub. Orb enables you to reduce the time to investigate Live Site issues in your services by providing the ability to rapidly browse through information, based on an object hierarchy. Object hierarchies are user-defined and backed by Azure Data Explorer (ADX) queries - ADX integration allows rapidly searching for objects and discovering hierarchies dynamically based on event logs from your services. Internal Azure services use Orb to map the layout of Azure data centers and cross-service interactions. For example, a user Virtual Machine (VM) is mapped to a host server. A host server is further mapped to a rack, network switch and so forth. This post provides more information on how our internal services emit these event logs. Orb also allows streamlined access and discovery of common resources like queries, dashboards, web links and scripts across different teams. Within a large organization, it’s easy for these resources to get fragmented on different team specific wiki pages and notebooks. With Orb, the entire organization can have a shared and consistent view of these resources. The resources are organized by Object – think of Objects as directories and resources as files in a directory. In a file system, you’d have directories linked together by metadata stored on disk – in Orb, directories or Objects are linked together dynamically based on ADX queries (or PowerShell scripts). When a resource under an object is clicked, all necessary context about the object is injected into the resource. For example, if you click a saved web link for a VM object and the URL contains a VM Id, the VM Id is automatically injected into the query. If there were more object properties in the URL that needed to be populated (say, VM region), those would also be added in by Orb. In addition to web URLs, this parameter injection works across different resource types like saved ADX queries, PowerShell scripts/terminals and more. The combination of the object hierarchy and shared resources is what enables the rapid information exploration. When a Live Site incident is raised to our team in Azure Compute, we use Orb to quickly navigate from a VM object to the host server. From there, we can look at all the various host resources and dashboards that determine the host server health. Since Azure Networking has also modeled their objects in Orb, my team can jump to the network switch object the host is running under and then look at all the shared dashboards that determine network switch health. Without this shared resource model, we might have previously engaged another engineer on the Networking team to get access to the same information. Under the covers, Orb uses a git repository to store the shared resources and object hierarchy definitions. This allows organizations to control who was read/write permissions on the shared view, as well as audit and rollback capabilities. Users can update the shared view and submit pull requests from within Orb. Orb is available to download and use for free on GitHub. Windows is the only supported Operating System at this point. Orb ships with a sample ADX cluster and object definitions - to use Orb with your real service data, take a look at this guide.Azure Data Explorer Support for Virtual Network (VNet) Deployment
Azure Data Explorer now supports deploying a cluster into a subnet in your Virtual Network (VNet), enabling you to: Enforce Network Security Group (NSG) rules on your ADX cluster traffic. Connect your on-premise network to ADX cluster's subnet. Secure your Data Connection sources (EventHub/EventGrid) with service endpoint. The feature is still in private preview, please fill this form to deploy ADX cluster into your VNet.
Check out what we announced at //Build 2019
Here is a roundup of all the product news at //Build 2019 related to Azure Data Explorer - https://techcommunity.microsoft.com/t5/Azure-Data-Explorer/What-s-new-in-Azure-Data-Explorer-at-Build-2019/ba-p/547984#.XNMXJ1x_lBs.twitter What could we do better or more of? Tell us in your comments!Analyze data in Azure Data Explorer using KQL magic for Jupyter Notebook
We are excited to announce KQL magic commands which extends the functionality of the Python kernel in Jupyter Notebook. KQL magic allows you to write KQL queries natively and query data from Microsoft Azure Data Explorer. You can easily interchange between Python and KQL, and visualize data using rich Plot.ly library integrated with KQL render commands. KQL magic supports Azure Data Explorer, Application Insights, and Log Analytics as data sources to run queries against. Learn more about KQL magic commands and using Azure Data Explorer with Jupyter Notebooks on the Azure blog.Kusto .NET and .NET Standard SDKs are public on NuGet.Org
We are thrilled to announce that Kusto .NET SDK has been released to Nuget.Org. It includes .Net Standard SDK that is currently available for preview. What’s new? Kusto SDK is now officially available from https://nuget.org. It includes .NET (release) and .NET Standard (preview) packages: [Release] Microsoft.Azure.Kusto.Data 4.0.3 [Release] Microsoft.Azure.Kusto.Ingest 4.0.3 [Preview] Microsoft.Azure.Kusto.Data.NETStandard 4.0.3 [Preview] Microsoft.Azure.Kusto.Ingest.NETStandard 4.0.3 [Release] Microsoft.Azure.Kusto.Management 1.0.7 [Release] Microsoft.Azure.Kusto.Tools 1.1.5 (LightIngest and Kusto.Cli) See Quickstart: Ingest data using the Azure Data Explorer .NET Standard SDK for ingestion using .NET SDK. What’s next? .NET Standard SDK released today is a preview version. We will continue to invest in its quality and reliability in the following versions. Currently .NET Standard SDKs are published as stand-alone packages. We plan to repackage both flavors into a single package. Stay tuned!
