Forum Widgets
Latest Discussions
Could not find stored procedure 'OPTIMIZE' in Synapse
Hi Fellow Azure people! I'm trying to use the OPTIMIZE procedure on a delta table in Synapse but I'm getting a "Could not find stored procedure 'OPTIMIZE'" error. Delta's docs do mention that this feature is only available in Delta Lake 1.2.0 and above, I've double checked and we are running Delta 1.2 Below is an example of what I'm doing: OPTIMIZE '/path/to/delta/table' -- Optimizes the path-based Delta Lake table Does anyone know what this could be happening? I did notice that there was no reference to OPTIMIZE in the Synapse docs but it did exist in the Databricks docs. Perhaps the procedure hasn't been implemented in Synapse yet?
how to add custom field in "CommonSecurityLog" Table
Hi Guys, I am adding new column in CommonSecurity Table. But i am having issue in kql quey. Please help me. This is Palo alto related logs. As "cat" field is in both Threat and System. So it is going to overlap. thats why i have to run query with "OR" operator so Threat "cat" field will not overlap with System "cat" field. Please help me. . CommonSecurityLog | where Activity in ("TRAFFIC", "THREAT") | extend SessionEndReason_CF = extract('reason=([^;]+)',1, AdditionalExtensions) | extend ThreatContentName_CF = extract('cat=([^;]+)',1, AdditionalExtensions) | extend thr_category_CF = extract('PanOSThreatCategory=([^;]+)',1, AdditionalExtensions) combined with "OR" condition | where Activity == "SYSTEM" | extend EventID_CF = extract('cat=([^;]+)',1, AdditionalExtensions)
Can't write to custom logs
I am new to Intune. I have created a log analytics workspace. Using PowerShell I want to write to this location. My web address is always wrong, regardless of what I use. PowerShell gives me this error: Invoke-WebRequest : The remote name could not be resolved: I am trying to use this code that I found online. # Log analytics part $CustomerId = "7aa60a6a-1234-4c67-5678-93d0f5f997c8" $SharedKey = '' $LogType = "TestReport" $TimeStampField = "" #*********************************************************************************************************** # Log analytics functions Function Build-Signature ($customerId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { $xHeaders = "x-ms-date:" + $date $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) $keyBytes = [Convert]::FromBase64String($sharedKey) $sha256 = New-Object System.Security.Cryptography.HMACSHA256 $sha256.Key = $keyBytes $calculatedHash = $sha256.ComputeHash($bytesToHash) $encodedHash = [Convert]::ToBase64String($calculatedHash) $authorization = 'SharedKey {0}:{1}' -f $customerId,$encodedHash return $authorization } # Create the function to create and post the request Function Post-LogAnalyticsData($customerId, $sharedKey, $body, $logType) { $method = "POST" $contentType = "application/json" $resource = "/api/logs" $rfc1123date = [DateTime]::UtcNow.ToString("r") $contentLength = $body.Length $signature = Build-Signature ` -customerId $customerId ` -sharedKey $sharedKey ` -date $rfc1123date ` -contentLength $contentLength ` -method $method ` -contentType $contentType ` -resource $resource $uri = "https://" + $customerId + ".MyAzureCompany.onmicrosoft.com" + $resource + "?api-version=2022-11-13" # $uri = "https://" + $customerId + ".MyAzureCompany.azure.com" + $resource + "?api-version=2022-11-13" ############################################################################################################################## $headers = @{ "Authorization" = $signature; "Log-Type" = $logType; "x-ms-date" = $rfc1123date; "time-generated-field" = $TimeStampField; } $response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing return $response.StatusCode } $Current_User_Profile = Get-ChildItem Registry::\HKEY_USERS | Where-Object { Test-Path "$($_.pspath)\Volatile Environment" } | ForEach-Object { (Get-ItemProperty "$($_.pspath)\Volatile Environment").USERPROFILE } $Username = $Current_User_Profile.split("\")[2] $WMI_computersystem = gwmi win32_computersystem $Manufacturer = $WMI_computersystem.manufacturer If($Manufacturer -eq "Lenovo") { $Get_Current_Model = $WMI_computersystem.SystemFamily.split(" ")[1] } Else { $Get_Current_Model = $WMI_computersystem.Model } $Get_Current_Model_MTM = ($WMI_computersystem.Model).Substring(0,4) $Get_Current_Model_FamilyName = $WMI_computersystem.SystemFamily.split(" ")[1] $BIOS_Version = (gwmi win32_bios).SMBIOSBIOSVersion # Get Hard disk size info $Win32_LogicalDisk = Get-ciminstance Win32_LogicalDisk | where {$_.DeviceID -eq "C:"} $Disk_Full_Size = $Win32_LogicalDisk.size $Disk_Free_Space = $Win32_LogicalDisk.Freespace # Hard disk size percent [int]$Free_DiskPercent = '{0:N0}' -f (($Disk_Free_Space / $Disk_Full_Size * 100),1) # Hard disk size Log Anaytics format $Disk_Size = ([System.Math]::Round(($Disk_Full_Size) / 1MB, 2)) $Free_DiskSpace = ([System.Math]::Round(($Disk_Free_Space) / 1MB, 2)) # Bitlocker status $Bitlocker_status = (Get-BitLockerVolume -MountPoint "c:").protectionstatus If($Bitlocker_status -eq "On") { $Bitlocker_status = "Enabled" } Else { $Bitlocker_status = "Disabled" } # Create the object $Properties = [Ordered] @{ "Computer" = $env:computername "UserName" = $Username "Vendor" = $Manufacturer "DeviceModel" = $Get_Current_Model "BIOSVer" = $BIOS_Version "DiskSize" = $Disk_Size "FreeDisk" = $Free_DiskSpace "FreeDiskPercentage" = $Free_DiskPercent "BitlockerStatus" = $Bitlocker_status } $DeviceInfo = New-Object -TypeName "PSObject" -Property $Properties $DeviceInfoJson = $DeviceInfo | ConvertTo-Json $params = @{ CustomerId = $customerId SharedKey = $sharedKey Body = ([System.Text.Encoding]::UTF8.GetBytes($DeviceInfoJson)) LogType = $LogType } $LogResponse = Post-LogAnalyticsData @params No matter what I try for $CustomerID and URL it fails. For CustomerID I have tried my subscription ID, my tenant ID and the Workspace ID and nothing works. Question: Do I need to pre-create a custom log in Azure? If so how? It askes for a custom script and I don't know what to use. Any help would be great.
Looking for guidance on designing an Azure data analytics pipeline for reporting
I’m working on modernizing an old reporting workflow that currently runs on a few on-premises databases and scheduled scripts. The current process collects operational data from multiple systems, performs some basic transformation and aggregation, and then generates reports for different business teams. As the data volume is growing, the existing setup is becoming difficult to maintain and slow to refresh. I’m looking for an Azure-based architecture that can ingest data from different sources, store both raw and processed data, run scheduled transformations, and make the final datasets available for reporting tools like Power BI. Would appreciate any suggestions on the recommended architecture, especially around data storage, transformation, refresh performance, and cost control. Thanksretreiving security form d365 CE ?
Hi ! I wondered if someone found a way to retrieve the Row security level of D365 (both CE and F&O) in the data lake export. I don't see how to manage that. security in D365 CE (aka dataverse) is prety complex. Any record belong to one user, one Business unit and one Team. moreover it can be shared with other users. how can I be sure to get this security in my powerbi reports based on the export to data lake ? Finops seems a little bit easier as thez security is based on the legal entity the user belong to. Any welp would be greatly appreciated.
status code threshold
Trying to figure out how to find events where status code 200 goes above a certain avg threshold, say 10%. I think I have the avg figured out, but unsure how to get the rest written. Any help would be appreciated. W3CIISLog | where TimeGenerated > ago(7d) |where scStatus =="200" | summarize totalCount = count() by bin(TimeGenerated, 1h), scStatus |summarize avghits =(avg(totalCount))
Query EPS for specific log collector
Hi, i would like to ask for help for a specific kusto query to get the average eps for a specific log collector? i tried to search the community and tried one of the query i saw but to no avail. hope you can help me as i am still learning sentinel and kusto query. thank you in advance.
Looking for an Azure solution for time series validation solution
Hi folks, I am planning to replace one of my old on prem based time series based process validation tool with Azure based solution. Required solution should be able to perform validation of time series coming from various different sources like DBs or CSV file etc(eventually data is coming from different plants in those DBs or CSV files) and it should be able to perform certain calculations(based on some defined mathematical functions) on those time series. Post that it should be able to send it to some centralized Datawarehouse where reporing etc can be performed on that gathered data with tools like Power BI or Graphana. I found one similar sort of product which Azure timeseries but it is going to be out of support in 2025.Integrate Jenkins with Azure Databricks & GitHub into VSCode
Hello Team, Greetings of the Day!!! Hope you have a great day ahead!!! We have installed extension of Azure Databricks, GitHub & Jenkins in VSCode. Now the configuration parts come into the picture, so we have configured Azure Databricks & Logged in GitHub in VSCode. Now Turn comes of Jenkins. We want to know that how can we configure Jenkins with GitHub. All Notebooks from Azure Databricks will be version controlled in GitHub for doing that we want to use Jenkins. There is no documentation to do so. Can you guide us how to do it. Reference Link :- https://learn.microsoft.com/en-us/azure/databricks/dev-tools/ci-cd/ci-cd-jenkins Thank You in advance for any Support or Suggestion : ) Looking forward for your valuable input. Regards, Niral Dave.Send custom data to Log Analytics in Azure Functions
When I send custom biz data to Azure Log Analytics in an Azure Function(C# code), I want to use a OperationalInsightsDataColloector, but I can't find the class in Microsoft.Azure.Operationallnsight package from nuget. Anyone knows whehter it is obseleted?
