Forum Discussion

FahadAhmed's avatar
FahadAhmed
Brass Contributor
Oct 15, 2021

OMS Agent on Azure Sentinel Log forwarder not receiving and forwarding logs to sentinel workspace

Hello,   We have observed that we no longer are receiving Syslog and CEF logs from the Azure Sentinel Log forwarder that is deployed on client premise. I have performed the following steps:   net...
  • FahadAhmed's avatar
    FahadAhmed
    Brass Contributor
    Oct 21, 2021

    GaryBushey  Thank you for the prompt response. We have got the issue resolved through microsoft support. Apparently for some reason the OMI Agent was in a zombie/stuck state. Restarting the agent didnt work, had to manually kill the process and start the agent again. 

     

    Killing and starting the agent again resolved the issue. As per MS teams , one of the possibility of this behavior may be that the disk space got full earlier at some point in time, which was then resolved however may be that disk space issue might have caused the agent to go into such a state.

     

    Anyways, the issue has been resolved, we are still monitoring to see if it remains stable or not.

     

    Sharing the above for the benefit of all.


    Thanks once again for your support, much appreciated.

     

    Fahad.

    • alazarg's avatar
      alazarg
      Copper Contributor
      Feb 04, 2023
      FahadAhmed - I am having the same issue here. how did you prevent it from getting disk full again?
      • KennethML's avatar
        KennethML
        MCT
        Apr 14, 2023

        Hi alazarg 

        Check your /etc/rsyslog.d/50-default.conf file. It has a section like this:

        #
# First some standard log files.  Log by facility.
#
#auth,authpriv.*                        /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
#daemon.*                       -/var/log/daemon.log
kern.*                          -/var/log/kern.log
#lpr.*                          -/var/log/lpr.log
mail.*                          -/var/log/mail.log
#user.*                         -/var/log/user.log

        The problem is line 5 where any facility (*.*) is written to /var/log/syslog file. Solution is to either comment it out (using #) or even better:

        #
# First some standard log files.  Log by facility.
#
if ($fromhost-ip == '127.0.0.1') then {
  #auth,authpriv.*                        /var/log/auth.log
  *.*;auth,authpriv.none          -/var/log/syslog
  #cron.*                         /var/log/cron.log
  #daemon.*                       -/var/log/daemon.log
  kern.*                          -/var/log/kern.log
  #lpr.*                          -/var/log/lpr.log
  mail.*                          -/var/log/mail.log
  #user.*                         -/var/log/user.log
}

        Then only local generated logs will be written to the filesystem and the disk will not be filled.

         

        Good luck.

         

        /Kenneth ML

         

Resources