How To Install Log Analytics Gateway Using Load Balancers In Azure
Published Jan 05 2020 03:01 PM 7,844 Views
Microsoft

Hello everyone,

after a long time, I am back with a new post. Today I will drive you through the configuration of a load balanced Log Analytics Gateway (formerly OMS Gateway and briefly referred below as LAGW) installed on Azure Virtual machines, using the Azure Load Balancer.

 

clipboard_image_0.png

 

In this post, I am not going to discuss capacity and sizing of the necessary virtual machines nor I am going to discuss the steps to install and configure them in an Availability Set.

Trust me :smile:, the size I went for is in line with the hardware requirements you can find in the Set up your system page and the availability set configuration was performed following the guidance and principles from the article Manage the availability of Windows virtual machines in Azure.

 

The topic:

Basically, the Log Analytics Gateway is an HTTP forward proxy dedicated to the outbound Microsoft Monitoring Agent communication with the Log Analytics workspace. Common circumstances in which you could use it are:

Despite the native redundancy capabilities offered by Azure, installing a single virtual machine acting as LAGW, represents a single point of failure hence you might want to make your LAGW highly available. Of course, Azure gives the ability of configuring load balancing as well as the On-premises does. Since the Network Load Balancing (NLB) configuration is documented in the Configure network load balancing page, below I will only focus on the Cloud part shortly summarizing what’s necessary to get to this configuration.

 

Planning the configuration:

Implementing this configuration requires you plan for the below items:

  • Creation of 1 Availability Set to configure the virtual machines with. Should you already have the virtual machines installed, you can configure them to belong to an availability set using the script attached to this post (Thanks to Samir Farhat who created the previous versions located at https://gallery.technet.microsoft.com/Set-Azure-Resource-Manager-f7509ec4)
  • Creation of, at least, 2 virtual machines on which you have installed the LAGW
  • Creation of 1 Azure Load Balancer of type Internal (also referred as ILB) since you are making the internal IP Address highly available
  • Find #1 available IP Address for the VIP in the virtual network

 

Configuring the Load Balancer:

Once you’ve finished the virtual machine configuration and installation (including the LAGW for which the installation procedure can found on page Connect computers without internet access by using the Log Analytics gateway in Azure Monitor) you can go ahead with the ILB configuration.

 

NOTE: the LAGW configuration (TCP port to be used for the gateway, proxy address and user for the company proxy the LAGW needs to communicate through) should be the same

 

First thing first, you need to create an ILB instance and configure the necessary settings.

  1. From the Azure Portal, write Load Balancers in the search box on the top of the page and click on the related result (see the blue arrow)

clipboard_image_1.png

  1. Click Add to create a new ILB object

clipboard_image_2.png

  1. Enter the necessary information as per your infrastructure making sure to select, for this specific purpose, the Internal type and Basic As far as the IP address assignment goes, you could go either Static or Dynamic. I would recommend Static

NOTE: Choosing a Static IP address, will require less changes on the agent configuration later on, making it also easier to associate a DNS Record to it so you can use the FQDN during agents configuration.

 

Click Review + Create

clipboard_image_3.png

  1. Make sure to pass the validation check and to have entered the correct information and click Create

clipboard_image_4.png

  1. It will take a while to deploy the new Load Balancer. Once finished you will get a blade similar to this in which you can click on the Go to resource button

clipboard_image_5.png

  1. After the load balancer is created, a backend pool needs to be created, which distributes traffic to one or more gateway servers. From the left side menu, select Backend pools and click Add.

clipboard_image_6.png

  1. Enter a name, associate it the availability set you created previously

clipboard_image_7.png

  1. Click on + Add a target network IP configuration and select the IP corresponding to the virtual machines you want to balance and click OK

clipboard_image_8.png

  1. Next you need to configure the Health probes. The health probe dynamically adds or removes the gateway servers from the load balancer rotation based on their response to health checks. On the left menu select Health probes and then click Add. Enter the necessary information according to the LAGW port you configured during the installation (this must be the same port the agents will connect to) and click OK.

clipboard_image_9.png

  1. Configuration of Health probes is followed by the definition of a Load Balancing On the left menu, select Load balancing rules and then click Add. Enter the required information bearing in mind the following and the click OK:
    1. Port and Backend port must be the same
    2. Session persistence could be set to either Client Ip or Client Ip and protocol (see Configure source IP affinity settings)
    3. Since this is a dedicated rule and we create only one, Floating IP (direct server return) must be set to Disabled

clipboard_image_10.png

  1. Last but not least, configure your agents to use the VIP or (Frontend IP Address) in the Proxy Settings tab and click Apply.

clipboard_image_11.png

  1. If the configuration was done correctly, you should get the green check indicating that the agents is connecting to the Log Analytics workspace with no issues.

clipboard_image_12.png

 

Hope that this post will make Log Analytics Gateway High Availability configuration smooth and easier.

 

Thanks,
Bruno Gabrielli

 

Disclaimer
The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.

 

2 Comments
Copper Contributor

@Bruno Gabrielli Great article, but I'm having a hard time understanding what the LAGW source limitations for the gateway are. How many source system or events per hour/minute/second can a single gateway support?

 

In my situation, I have 160k hosts to collect security audit logs from (ideally, powershell, some limited sysmon, and a few other channels too) but I'm not sure of the size of the back end infrastructure to support it to propose to the customer.

 

Any recommendations?

Microsoft

Hello Joe,

you could look at the sizing part which is in the Supported number of agent connections paragraph inside the Connect computers without internet access by using the Log Analytics gateway in Azure Monitor page. You could also think about splitting the load among several load balanced gateways installments.

 

Thanks,

Bruno

Version history
Last update:
‎Dec 13 2019 07:27 AM
Updated by: