User Profile
Dolev_Shor
MicrosoftJoined 4 years ago
User Widgets
Recent Discussions
Azure Virtual Machine: Centralized insights for smarter management
Introduction Managing Azure Virtual Machines (VMs) can be challenging without the right tools. There are several ways for monitoring, some of which extend beyond the platform's native capabilities. These may include options like installing an agent or utilizing third-party products, though they often require additional setup and may involve extra costs. This workbook is designed to use the native platform capabilities to give you a clear and detailed view of your VMs, helping you make informed decisions confidently without any additional cost. To get started, check out the GitHub repository. Why do you need this Workbook? When managing multiple VMs, understanding usage trends, comparing key metrics, and identifying areas for improvement can be time-consuming. The Azure Virtual Machine Insights Workbook simplifies this process by centralizing essential data into one place from multiple subscriptions and resource groups. It covers inventory to provide you with a clear overview of all your VM resources and platform metrics to help you monitor, analyze, compare, and optimize performance effectively. Scenarios to use this Workbook Here are a few examples of how this workbook can bring value: Management Centralized Inventory Management Easily view all your VMs in one place, ensuring a clear overview of your resources. Performance and Monitoring Performance monitoring Analyze metrics like CPU, memory, network, and disk usage to identify performance bottlenecks and maintain optimal application performance. Performance trends Examine long-term performance trends to understand how your VMs behave over time and identify areas for improvement. Comparing different VM types for the same workload Compare the performance of various VM types running the same workload to determine the best configuration for your needs. Virtual Machines behind a load balancer Monitor and compare the performance of VMs behind a load-balanced to ensure even distribution and optimal resource utilization. Virtual Machines farm Assess and compare the performance of VMs within a server farm to identify outliers and maintain operational efficiency. Cost Cost Optimization Detect and compare underutilized VMs or overprovisioned resources to reduce waste and save on costs. Analyse usage trends over time to determine if an hourly spend commitment through Azure savings plans is feasible. Understand the timeframes for automating the deallocation of non-production VMs, unless Azure Reservations cover them. Independent software vendors (ISVs) ISV managing VMs per customer Compare performance across all customer VMs to identify trends and ensure consistent service delivery for each customer. Trends and Planning Resource Planning Track usage trends over time to better predict future resource needs and ensure your VMs are prepared for business growth. Scalability Planning Utilize insights from trends and metrics to prepare for scaling your VMs during peak demand or business growth. Examples from the workbook Conclusion The Azure Virtual Machine Insights Workbook helps you manage your VMs by bringing key metrics and insights together in one place, using native Azure features at no extra cost. It lets you analyze performance, cut costs, and plan for future growth. Whether you are investigating performance issues, analyzing underused resources, or predicting future needs, this workbook helps you make smart decisions and manage your infrastructure more efficiently. For any queries or to contribute, feel free to connect via the GitHub repo or submit feedback!Azure Snapshots: Simplify Management and monitoring
Introduction Managing snapshots in Azure can be challenging when you have large a number of them across different Virtual Machines and subscriptions. Over time, snapshots can accumulate, leading to increased storage costs and making it harder to identify which ones are still needed. This was the trigger to build the Azure Snapshots Insights Workbook, designed to simplify monitoring and management of Azure Snapshots. To get started, check out the GitHub repository. Benefits of using this workbook Centralized Monitoring: Manage all your snapshots in one place across multiple resource groups and subscriptions. Cost Optimization: Reduce storage expenses by identifying and deleting outdated snapshots. Better Insights: Gain a clearer understanding of your snapshot usage patterns by the inventory dashboard. Key features Overview: Monitor important details like snapshot name, source disk, size (GiB), resource group, location, storage type, snapshot type (Full/Incremental), time created, public network access, etc. Inventory: Monitor the snapshot usage count by Subscription ID, Resource Group, Location, Storage type, Source disk size (GiB), Snapshot type, Disk state, Create Option, API Version, Public Network Access, Access policy, Data Access Auth Mode. Filtering: Filter snapshots by specific subscription/s, resource group/s and specific resource/s. Age-Based Filtering: View snapshots by age creation (1, 2, 3, 4, 5, 6 ,7, 14, 30, 60, 90 days ago), making it easy to identify old ones. Snapshot Deletion: Remove outdated or unnecessary snapshots with just a few clicks directly from the workbook. Age-Based Filtering Inventory Best practices for managing Azure Snapshots Cost Optimization Regularly review snapshot usage: Track the number and size of snapshots. This helps avoid unexpected costs and ensures you are not keeping unnecessary snapshots. Monitor snapshot age with the Age-Based filtering: Regularly review the age of your snapshots to identify and delete those that have surpassed your retention period to avoid unnecessary storage costs. Implement a retention policy: Define how long snapshots should be retained based on your organization data retention requirements. Delete older snapshots regularly to avoid accumulating unnecessary costs. Use Incremental snapshots: Whenever possible, use incremental snapshots, which only capture changes since the last snapshot. This approach is more cost-effective and efficient than full snapshots. Monitor snapshot costs: Regularly monitor the cost associated with snapshots and optimize them as needed. Evaluate snapshot storage options: Consider the type of storage account used for snapshots, especially when dealing with large volumes. Premium storage might be necessary for high-performance requirements, while standard storage can be more cost-effective for less critical data. Optimize snapshot frequency: Evaluate your snapshot frequency based on how often your data changes. For example, if your data changes infrequently, taking daily snapshots might not be necessary. Clean up orphaned snapshots: Identify and clean up orphaned snapshots that are no longer associated with any resources to optimize costs. Security Secure your snapshots: Use Azure Role-Based Access Control (RBAC) to restrict access to your snapshots. Ensure that only authorized users have permission to create, manage, or delete them. Performance and Efficiency Store snapshots in the same region: Ensure your snapshots are stored in the same region as the source disk to minimize latency and reduce costs when creating or restoring snapshots. Automate snapshot management: Automate the snapshot creation, deletion, and management processes. Data Protection and Recovery Test snapshot recovery: Periodically test the restoration process to ensure that your snapshots can be successfully used to recover data when needed. Take snapshots before major changes: Always create snapshots before performing significant updates or configurations on your Virtual Machines or managed disks, so you can quickly roll back if something goes wrong. Use Azure Backup for Long-Term Storage: If you need long-term retention, consider using Azure Backup instead of keeping snapshots indefinitely, as it provides more cost-effective and robust data retention. Governance and Management Apply Tags for better organization: Tag your snapshots with relevant information, such as Environment (e.g., "production", "development"), Application, or Owner, to improve management and cost tracking. Use Resource Locks: Apply resource locks to prevent accidental deletion of critical snapshots. This is particularly useful for snapshots that serve as backups for essential data. Check for dependencies: Before deleting a snapshot, ensure it’s not linked to any dependent resources or processes that could be disrupted if the snapshot is removed. Conclusion The Azure Snapshots Insights Workbook offers a streamlined approach to managing and optimizing your Azure snapshots, helping you stay organized and reduce unnecessary storage costs. By leveraging its centralized monitoring, age-based filtering, and deletion capabilities, you can maintain a more efficient cloud environment. Implementing these best practices will ensure your snapshots are always under control, ultimately enhancing your Azure resource management. To get started, check out the GitHub repository. For any queries or to contribute, feel free to connect via the repo or submit feedback!Maximizing Azure Resource Insights: Introducing the Azure Inventory Gateway
Introduction Azure Resource Graph (ARG) is an efficient tool for querying Azure resources, but it doesn’t always provide the full scope of information you need about them, especially when it comes to in-depth details. This is where Resource Inventory Gateway comes into play. ARG’s limitations - such as not retrieving all attributes or configurations for a resource – force users to make additional API calls to Azure Resource Manager (ARM). Instead of manually querying multiple resources, this gateway automates and consolidates those calls, returning a complete data set. TLDR; GitHub Repository Why we build the Azure Inventory Gateway? ARG (Azure Resource Graph) missing attributes: Azure Resource Graph is a powerful service that allows you to explore and query your Azure resources across multiple subscriptions quickly and efficiently, providing detailed insights without needing individual API calls but it is not includes all ARM attributes. Azure Resource Manager (ARM) Rest API capability ARM is designed to handle queries for a single resource at a time. To retrieve data from multiple resources, whether across a single subscription or multiple subscriptions, you need to execute multiple API calls. The Azure Inventory Gateway simplify the achievement of this capability. What is Azure Inventory Gateway? Azure Inventory Gateway is a .NET-based gateway that simplifies the management and aggregation of REST API calls. This solution provides an endpoint where you can send an API call along with variables. It processes the request and returns a unified response by aggregating the results from the APIs calls. Currently it is including 2 interface gateways: ARM Gateway – Interact with Azure Resource Manager (ARM) REST API. Cost Gateway – Interact with Azure Cost Management REST API. Azure Workbook Workbooks provide a flexible canvas for data analysis and the creation of rich visual reports within the Azure portal. They allow you to tap into multiple data sources from across Azure and combine them into unified interactive experiences. You can use the Azure inventory Gateway in Azure workbooks. To make a call to the Azure Inventory Gateway, you need to use the custom endpoint data source and provide the appropriate API call along with the necessary variables. You can find all details here. Examples Azure OpenAI Model Deployments List all deployment models across all my Azure OpenAI resources This provides a centralized view of your deployment models across subscriptions and resources. If you need to locate where a specific model is deployed, you can now easily search across all your instances. Azure OpenAI Models List all supported models across all my Azure OpenAI resources This provides a centralized view of your supported models, versions, lifecycle (Generally Available, Preview), creation date, etc. across subscriptions and resources. If you need to locate where a specific model or version is supported, you can now easily search across all your instances. Conclusion Azure Inventory Gateway provides a streamlined approach to accessing complete resource data, eliminating the need for multiple ARM queries and delivering unified results. This tool saves time, automates the complexity of resource management, and expands the capabilities of Azure Resource Graph. To get started, check out the GitHub repository, and integrate it into your resource management workflows today. For any queries or to contribute, feel free to connect via the repo or submit feedback!Simplify app management with the Azure App Service Insights Workbook
Introduction Developing, maintaining, and managing multiple applications on Azure App Service Plans can be complex. At scale, the challenge intensifies as you juggle different Subscriptions, Resource Groups, Operating Systems (Linux, Windows), Regions, Scale (instances), Pricing Plans, and Resource Allocations across various environments. To simplify this, I have developed an Azure App Service Insights Workbook. This tool provides a unified view to help customers easily track and compare different applications. It designed to empower developers, DevOps, and FinOps teams with insights and streamlined monitoring capabilities. In this blog post, I will explore the benefits of this Workbook for various scenarios and stakeholders, and explain how it can drive cost efficiency. TLDR: Get the Workbook on GitHub Monitor, Track and Compare Holistic View: This workbook provides a holistic view of all your Azure App Services and Azure App Service Plans. Managed Efficiency: A simple way to monitor, track and compare your application behaviors. Cross-App Comparison: Compare critical metrics across different applications or App Service Plans to identify trends and outliers. This helps you prioritize optimization efforts and ensure consistent performance across all apps. Smart Decisions: Whether you have a single app or multiple apps, being able to see the behaviors will help you be efficient and make smart decisions. This workbook is not intended to replace monitoring capabilities like Application Insights or 3rd party monitoring tools, it gives a holistic view of all Azure App Services with comparison capabilities, and you can dive into each of them with additional tools. Azure App Service Azure App Service is an HTTP-based service for hosting web applications, REST APIs, and mobile backends. Multiple languages and frameworks – App Service has first-class support for .NET, .NET Core, Java, Node.js, PHP, or Python. Publish – You can choose to publish as Code, Containers or Static Web app. Global scale with high availability - Scale up or out manually or automatically. Host your apps anywhere in Microsoft's global data center infrastructure, and the App Service SLA promises high availability. Operating System – Run your app on both Windows and Linux-based environments. Allocation - You can host multiple apps on a single App Service Plan. (Ref: App Service plans - Azure App Service | Microsoft Learn) Holistic View Filters The workbook supports filtering by Subscriptions, Resource Groups, App Service Plans, Apps. You can select multiple options (e.g., 2 App Service Plans). Time range - Viewing the metrics over time. Overview Overview of all your App service Plans, App Services and Staging Slots. App Service Plan: Location, Operating System, Tiers, Status, Number of hosted Apps, Instances, Maximum Scale. App Service/Slot: Location, Kind, Type, Tier, App Service Plan, State. Monitor Overview of all your App service Plans, App Services and Staging Slots Metrics. App Service Plan App Services and Staging Slots Additional Metrics Each resource type has different Metrics. Expanding the Additional Metrics will allow to view and compare all the Metrics supported by the resource type. App Service Plan: Data, Sockets, TCP, Queue Length App Service: Requests/Response, Data, HTTP, IO, Garbage Collections, Others Staging Slot: Requests/Response, Data, HTTP, IO, Garbage Collections, App Domains, Others App Service | Data Metrics App Service | Additional Metrics Cross-App/Plan Comparison The workbook supports comparing metrics between two or more resources. Resources can be an App Service Plan, App Service or Staging slot. App Service Plan App services Java vs .NET 6 Example Both App Services are in the Same App Service Plan. Different Runtime (Java vs .NET 6). Node.js vs .NET 8 Example Different App Service Plan. Different Regions. Different Runtime (Node.js vs .NET 8). App Service Plan App Services Inventory The Inventory dashboard provides a holistic view of all the App Services resources group by various categories, e.g. Subscriptions, Resource Groups, Tier, Status, etc. For Developers Developers often face the challenge of ensuring that a single or multiple applications run smoothly and efficiently. This workbook allows you to compare key metrics and can be invaluable in identifying trends, pinpointing issues, validating changes, and ensuring optimal performance across various scenarios. Scenario Use case How this Workbook helps? Development and Testing Profiling code, unit testing, and integration testing Compare metrics during different stages of development to ensure consistency and efficiency Application Performance Testing Load, stress and scalability testing Compare pre-test and post-test metrics to identify performance improvements or regressions Comparative Analysis Across Multiple Applications Developer manages multiple applications and needs to compare their performance to identify which ones require attention Unified View: Aggregate metrics from all applications into a single view for easy comparison. Benchmarking: Set benchmarks for key performance indicators and compare each application against these benchmarks. Identify Outliers: Quickly identify applications that are underperforming or experiencing issues, allowing the developer to prioritize optimization efforts. Diagnosing Performance Bottlenecks Investigating issues like slow response times, crashes, or memory leaks to find the root cause Detailed Metrics: Access detailed metrics on CPU usage, memory consumption, and request latencies to pinpoint performance bottlenecks. Correlation Analysis: Correlate and compare different metrics to pinpoint the root cause of issues. Historical Data: Review historical performance data to identify trends and recurring issues that may be affecting application performance. Third-Party Services Monitoring the impact of third-party libraries and SDKs on resource usage Ensuring that third-party integrations do not degrade overall performance Ensure third-party services meet performance expectations by comparing historical data Production Monitoring ** Monitoring metric performance Track historical trends metrics to maintain optimal performance Deployment and Scale ** Compare resource usage before and after deployments or scaling to ensure no performance degradation Pre and Post Deployment/Scaling Comparison: Compare key performance metrics before and after deployment or scaling to assess the impact. Custom Metrics: Monitor specific metrics related to the new features or changes introduced in the deployment. Rollback Indicators: Quickly identify any negative impacts, such as increased error rates or degraded performance, to make informed rollback decisions if necessary. System Upgrades ** Verifying performance after software upgrades (runtime, new release, new feature, etc.) Performance Validation: Validate that upgrades have not negatively impacted performance unexcepted. Rollback Indicators: Quickly identify any negative impacts, such as degraded performance, to make informed rollback decisions if necessary. ** In collaboration with the DevOps teams. For DevOps Engineers DevOps engineers have few performance challenges when managing and operating App Service plans. Scenario Use case How this Workbook helps? Production Monitoring Monitoring metric performance Track historical trends metrics to maintain optimal performance Scale Ensuring that the app services can scale effectively to handle varying loads without sacrificing performance. Track resource usage to ensure balanced resource allocation Performance and Cost *** Balancing performance and cost, especially when scaling up/out. Track resource usage to ensure efficiency. Resource Allocations *** Deciding the right SKU of the App Service Plans (e.g.: CPU, memory) and configuring auto-scaling to handle varying loads. Track resource usage to ensure resource right-sizing Multiple-Environments *** Managing configurations across different environments (development, staging, production) and multiple subscriptions. Monitor and compare resource usage across different environments for optimal performance Multiple-Regions *** Managing configurations across different Regions (e.g., West Europe, East US, etc.) and multiple subscriptions. Monitor and compare resource usage across different Regions for optimal performance *** In collaboration with the FinOps teams. For FinOps (Cost Efficiency) Effective cost management is a critical aspect of cloud operations. This Workbook provides several capabilities to help track and optimize your cloud spending: Scenario Use case How this Workbook helps? Resource Optimization / Over-Provisioning Optimizing application code and resource utilization Resource Utilization Insights: Track usage to identify underutilized resources that can be scaled down or repurposed. Cost Analysis: Understand the cost implications of resource usage and make data-driven decisions to optimize both performance and cost. Idle Resources Identification of unused resources Monitor App Service Plans that have not hosted any applications. Reservations and Savings Plan Identification of potential discounts Monitor App Service Plans to understand the utilization and identify potential to apply Reservations or Saving Plans. Conclusion The Azure App Service Insights Workbook is a simple tool designed to simplify application management for developers, DevOps and FinOps teams. By offering a unified view of key metrics it streamlines operations and enhances efficiency across the board. Whether you are a developer looking to optimize application performance, a DevOps engineer focused on reliability, or a FinOps engineer aiming to optimize and reduce costs, the Azure App Service Workbook has something valuable to use.
