Forum Widgets
Latest Discussions
Effective Cloud Governance: Leveraging Azure Activity Logs with Power BI
We all generally accept that governance in the cloud is a continuous journey, not a destination. There's no one-size-fits-all solution and depending on the size of your Azure cloud estate, staying on top of things can be challenging even at the best of times. One way of keeping your finger on the pulse is to closely monitor your Azure Activity Log. This log contains a wealth of information ranging from noise to interesting to actionable data. One could set up alerts for delete and update signals however, that can result in a flood of notifications. To address this challenge, you could develop a Power Bi report, similar to this one, that pulls in the Azure Activity Log and allows you to group and summarize data by various dimensions. You still need someone to review the report regularly however consuming the data this way makes it a whole lot easier. This by no means replaces the need for setting up alerts for key signals, however it does give you a great view of what's happened in your environment. If you're interested, this is the KQL query I'm using in Power Bi let start_time = ago(24h); let end_time = now(); AzureActivity | where TimeGenerated > start_time and TimeGenerated < end_time | where OperationNameValue contains 'WRITE' or OperationNameValue contains 'DELETE' | project TimeGenerated, Properties_d.resource, ResourceGroup, OperationNameValue, Authorization_d.scope, Authorization_d.action, Caller, CallerIpAddress, ActivityStatusValue | order by TimeGenerated asc
MetricsQueryClient returning different results based on timespan
I'm using the Python MetricsQueryClient to list out how many tokens were used on certain days via the APIM policy "azure-openai-emit-token-metric". The problem is that when I call the query_resource() function with "timespan" set for the entire month of October, I get different results for token count usage for today's date than when I set the "timespan" to just the last 48 hours. For example, when setting the timespan to be from 10/20/2024 to 10/22/2024, I see 34 prompt tokens for today's date. But if I set the timespan to be 10/1/24 to 11/1/24, I see 0 prompt tokens for today's date. Is this a known issue? It is documented somewhere?
What do sign-in logs to Azure AD Identity Governance - Entitlement Management tell me
BLUF: I'm seeing a few user accounts (members and guests) that are showing up in sign in activity to Azure AD Identity Governance - Entitlement Management...but most other user accounts (members and guests) do not show up with sign-ins against that app, why would I be seeing sign-in logs for that, and what exactly am I seeing? We have about 50 internal users (members) and 500 external users (guests)...out of those typical sign-ins I mostly see then within the apps that they are utilizing, however there are a few user accounts (members and guests) where I'm seeing sign-in logs specifically to the application name = Azure AD Identity Governance - Entitlement Management. We do have 100+ apps in which we have setup access packages for external users, but I haven't seen any other sign in activity to apps that they haven't been given permission to access, this seemed to have started around first of June 2024. I'm just curious as to what I'm actually seeing when I see a user sign-in log showing up for Application = Azure AD Identity Governance - Entitlement Management.Ho w to correctly measure Bytes Received/sec &&&&& Bytes Sent/sec
I would like to correctly measure through log analytic and then in Grafana network traffic generated for one or more VMs. For test VMs I have enable Data collection rule and enabled collecting data every 60s for network Interface "Bytes Received/sec" and "Bytes Sent/sec". Inside metric is also enabled. Query that I use in log analytic is : Perf | where TimeGenerated between (datetime(2024-03-19) .. datetime(2024-03-20)) | where Computer == "***********" | where ObjectName == "Network Interface" and CounterName == "Bytes Received/sec" and InstanceName == "Microsoft Hyper-V Network Adapter _2" | summarize BytsSent = sum(CounterValue)/1073741824 by bin(TimeGenerated, 24h),CounterName InsightsMetrics | where TimeGenerated between (datetime(2024-03-19) .. datetime(2024-03-20)) | where Origin == "vm.azm.ms" | where Namespace == "Network" and Name == "ReadBytesPerSecond" | where Computer == "******" | extend NetworkInterface=tostring(todynamic(Tags)["vm.azm.ms/networkDeviceId"]) | summarize AggregatedValue = sum(Val) by bin(TimeGenerated, 1d), Computer, _ResourceId, NetworkInterface Results for Perf is 0,32339 GB/day and for InsightsMetrics is 14.7931 GB/day. If I go to network interface and select metric data for network interface is data that I get return from query in log analytic for Metric same/correct . I have now shorten sample period of data collection rule to 15s, I hope that this will ,give more accurate results. I’m I doing something wrong or I collect data the wrong way. I don’t want to activate inside metric for every VM, I want to activate any data that I’m interesting.
Data Collection Rule : XPath queries to filter 7036 without WMI etc
Hi, In PowerShell on server I’m trying to filter out some events from Event Id 7036 Service Control Manager Start stop services. I’m trying to filter out WMI Performance Adapter, so I don’t want to have those events imported in log analytic workspace with data collection rule. Can you help me what I’m doing wrong ? $XPath = 'System!*[System[(EventID="7036")]] and [EventData[Data[@Name="param1"]!="WMI Performance Adapter"]]' Get-WinEvent -FilterXPath $XPath Get-WinEvent : Could not retrieve information about the Security log. Error: Attempted to perform an unauthorized operation.. At line:3 char:1 + Get-WinEvent -FilterXPath $XPath + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Get-WinEvent], Exception + FullyQualifiedErrorId : LogInfoUnavailable,Microsoft.PowerShell.Commands.GetWinEventCommand Get-WinEvent : No events were found that match the specified selection criteria. At line:3 char:1 + Get-WinEvent -FilterXPath $XPath + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (:) [Get-WinEvent], Exception + FullyQualifiedErrorId : NoMatchingEventsFound,Microsoft.PowerShell.Commands.GetWinEventCommand $XPath = 'System!*[System[(EventID="7036")]] and [EventData[Data[@Name="param1"]!="WMI Performance Adapter"]]' Get-WinEvent -LogName 'System' -FilterXPath $XPath Get-WinEvent : The specified query is invalid At line:2 char:1 + Get-WinEvent -LogName 'System' -FilterXPath $XPath + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Get-WinEvent], EventLogException + FullyQualifiedErrorId : System.Diagnostics.Eventing.Reader.EventLogException,Microsoft.PowerShell.Commands.GetWinEventCommand
KQL Query to summerize session counts over time
Hello folks, I'm trying to find a "good" way to achieve what I think is a simple task but cannot think of a simple solution. I have logs with session information, one entry per session StartTime(datetime), EndTime(datetime), Duration(in seconds), Computer(string) I want to count how many sessions are active for each 5 minute interval and graph that. Keep in mind, the sessions will overlap. I included a graphic of what I'm trying to do. With the result below Which should return: Time + 5, 1 Time + 10, 3 Time + 15, 3 Time + 20, 2Consumer REST API for Azure Event Hub
Hello, We have an existing setup where we use Kafka and we have a Kafka client written in Java that talks to the Kafka server. We have a Java producer that sends data to consumer for a topic and a Java consumer that subscribes to this topic and gets data fro kafka. We are now trying to offer similar support for Azure Event Hub too. From the documentation and examples, I can see that we can add similar code in Java. Basically a Java producer and consumer that talks to Azure Event Hub server. We are now trying to do the same with REST API. I see that we have an API to send data to an Azure Event Hub server. But I don't see any field in that to include the topic. In addition, is there an API to consume data for a particular topic or subscribe for a particular topic? (Since the documentation mentions nothing about consumer API, I am assuming there is no support for consumer REST API in Azure Event hub. But wanted to confirm. Logically also, using REST API for this kind of system where producer and consumer behave in async fashion is not a good idea) Please let me know. Thanks, OmAzure Devops to Workbook visualization
Hi team, please help us on enabling the azure devops pipeline jobs status, log and relevant dashboard to integrate in azure workbook view rather checking from Azure devops portal. please share the procedure, documentation link on performing this activity. Thanks!Azure Monitoring Agent Virtual Machines not connecting to log analytics workspace
Hey there, i tried to rollout monitoring for Azure Virtual Machines. For Testing i created a basic DCR to collect general Performance Counters of the associated VM's. The DCR is in Terraform defined as following : resource "azurerm_monitor_data_collection_rule" "log" { name = "test_rule" location = azurerm_resource_group.test_group.location resource_group_name = azurerm_resource_group.test_group.name kind = "Windows" destinations { log_analytics { workspace_resource_id = azurerm_log_analytics_workspace.default_workspace.id name = azurerm_log_analytics_workspace.default_workspace.name } } data_flow { streams = [ "Microsoft-Perf" ] destinations = [azurerm_log_analytics_workspace.default_workspace.name] } data_sources { performance_counter { streams = [ "Microsoft-Perf" ] sampling_frequency_in_seconds = 60 counter_specifiers = [ "\\Processor Information(_Total)\\% Processor Time", "\\Processor Information(_Total)\\% Privileged Time", "\\Processor Information(_Total)\\% User Time", "\\Processor Information(_Total)\\Processor Frequency", "\\System\\Processes", "\\Process(_Total)\\Thread Count", "\\Process(_Total)\\Handle Count", "\\System\\System Up Time", "\\System\\Context Switches/sec", "\\System\\Processor Queue Length", "\\Memory\\% Committed Bytes In Use", "\\Memory\\Available Bytes", "\\Memory\\Committed Bytes", "\\Memory\\Cache Bytes", "\\Memory\\Pool Paged Bytes", "\\Memory\\Pool Nonpaged Bytes", "\\Memory\\Pages/sec", "\\Memory\\Page Faults/sec", "\\Process(_Total)\\Working Set", "\\Process(_Total)\\Working Set - Private", "\\LogicalDisk(_Total)\\% Disk Time", "\\LogicalDisk(_Total)\\% Disk Read Time", "\\LogicalDisk(_Total)\\% Disk Write Time", "\\LogicalDisk(_Total)\\% Idle Time", "\\LogicalDisk(_Total)\\Disk Bytes/sec", "\\LogicalDisk(_Total)\\Disk Read Bytes/sec", "\\LogicalDisk(_Total)\\Disk Write Bytes/sec", "\\LogicalDisk(_Total)\\Disk Transfers/sec", "\\LogicalDisk(_Total)\\Disk Reads/sec", "\\LogicalDisk(_Total)\\Disk Writes/sec", "\\LogicalDisk(_Total)\\Avg. Disk sec/Transfer", "\\LogicalDisk(_Total)\\Avg. Disk sec/Read", "\\LogicalDisk(_Total)\\Avg. Disk sec/Write", "\\LogicalDisk(_Total)\\Avg. Disk Queue Length", "\\LogicalDisk(_Total)\\Avg. Disk Read Queue Length", "\\LogicalDisk(_Total)\\Avg. Disk Write Queue Length", "\\LogicalDisk(_Total)\\% Free Space", "\\LogicalDisk(_Total)\\Free Megabytes", "\\Network Interface(*)\\Bytes Total/sec", "\\Network Interface(*)\\Bytes Sent/sec", "\\Network Interface(*)\\Bytes Received/sec", "\\Network Interface(*)\\Packets/sec", "\\Network Interface(*)\\Packets Sent/sec", "\\Network Interface(*)\\Packets Received/sec", "\\Network Interface(*)\\Packets Outbound Errors", "\\Network Interface(*)\\Packets Received Errors" ] name = "datasourceperfcounter" } } description = "General data collection rule for collecting windows performance counter rules" } Also i created the association of the DCR and my Virtual Machine using either Terraform, Policies and Portal. The Monitor Agent and identity is assinged in all cases properly. But the Connection of the DCR / DCR Associations doesn't seem to work in case of terraform or policy enrollment. For some reason the log analytic namespace neither receive an Heartbeat of the agent nor creating the tables for the performance counters. If i recreate the association between DCR and vm in those cases it works again. Is there any additional Step required when using the Policies or Terraform to setup the data collection rule or this a bug where some kind of required event is not raised properly ?Openmetrics on Azure observability
Team, I am trying to connect a couple of openmetrics endpoints to Azure Monitor allthough it does not seem to support out of the box, I am overseeing the documentation? is there a easy way to connect openmetrics endpoint ? It seems it might be able to connect the openmetrics endpoints if I deploy Azure Monitor managed service for Prometheus, but its not clear.
