Automatic Log Upload using Docker on Ubuntu in Azure

[simcpk]

A few weeks back, my perfectly functional log collector (Docker on Ubuntu in Azure) stopped receiving syslogs from both of my sources.  The reason for this sudden breakage is still not completely understood.  After messing around with the log collector and trying to reconfigure my Docker instance to start receiving logs again, I simply couldn't get it to work.  I decided to spin up a new Ubuntu instance in Azure using the direction provided here -- https://docs.microsoft.com/en-us/cloud-app-security/discovery-docker-ubuntu-azure

In short, things still aren't working and I'm not good enough with Linux/Docker to know how to troubleshoot this.  I do have concerns that the commands generated when adding new log collectors in the CAS portal is incorrect though.  The sources that I'm trying to receive logs from are sending to UDP/514 and UDP/515.  The command that is generated to configure my Docker instance is the following --

(echo be178dc0a43dea3b51697e7369d616054fb260e47bb93146e5da0ccf37c96394) | docker run --name LogCollector_Azure -p 21:21 -p 20000-20099:20000-20099 -e "PUBLICIP='10.3.2.20'" -e "PROXY=" -e "SYSLOG=false" -e "CONSOLE=mycompanyname.us2.portal.cloudappsecurity.com" -e "COLLECTOR=LogCollector_Azure" --security-opt apparmor:unconfined --cap-add=SYS_ADMIN --restart unless-stopped -a stdin -i microsoft/caslogcollector starter

It seems strange to me that it seems to only be configuring for FTP but ignoring my UDP-based syslog ports.  Am I wrong in thinking that it should look something like the following?

(echo e8f2683d346b4cb90e3184b7de7fd464841358808b6ff6fe19fde25b18e78a1) | docker run --name LogCollector_Azure -p 21:21 -p 514-515:514-515/udp -p 20000-20099:20000-20099 -e "PUBLICIP='10.3.2.20'" -e "PROXY=" -e "SYSLOG=false" -e "CONSOLE=sawdustinvestments.us2.portal.cloudappsecurity.com" -e "COLLECTOR=LogCollector_Azure" --security-opt apparmor:unconfined --cap-add=SYS_ADMIN --restart unless-stopped -a stdin -i microsoft/caslogcollector starter

Regardless, I've run it both way and it doesn't work.

I don't have any records of exactly what commands I ran when I initially configure this many months back, but I did have all of this running successfully at one point.

[#2]

Phillip,

I've done a few Ubuntu/Docker on prem installs without issue. I'll try setup an Azure instance and let you know if I get any issues. Did you raise this with MS support?

Matt

[#3]

I started a support case and by the time they got back to me I'd sorted it out.  I changed my command as follows and then it worked --

(echo e8f2683d346b4cb90e3184b7de7fd464841358808b6ff6fe19fde25b18e78a1) | docker run --name LogCollector_Azure -p 21:21 -p 514-515:514-515/udp -p 20000-20099:20000-20099 -e "PUBLICIP='10.3.2.20'" -e "PROXY=" -e "SYSLOG=true" -e "CONSOLE=sawdustinvestments.us2.portal.cloudappsecurity.com" -e "COLLECTOR=LogCollector_Azure" --security-opt apparmor:unconfined --cap-add=SYS_ADMIN --restart unless-stopped -a stdin -i microsoft/caslogcollector starter

I think the commands being generated by the portal during the setup of the collector are wrong when configuring syslogs.  I'd used them successfully in the past, but I don't have any record of what they looked like when I last set up a collector successfully.  The support engineer said he'd pass my comments back to the team so that they could check on the issue.

Phil

[#4 · Accepted solution]

It appears they made some changes based on my comments because the portal generated command now appears as follows --

(echo e8f2683d346b4cb90e3184b7de7fd464841358808b6ff6fe19fde25b18e78a1) | docker run --name LogCollector_Azure -p 514:514/udp -p 515:515/udp -p 21:21 -p 20000-20099:20000-20099 -e "PUBLICIP='10.3.2.20'" -e "PROXY=" -e "SYSLOG=true" -e "CONSOLE=sawdustinvestments.us2.portal.cloudappsecurity.com" -e "COLLECTOR=LogCollector_Azure" --security-opt apparmor:unconfined --cap-add=SYS_ADMIN --restart unless-stopped -a stdin -i microsoft/caslogcollector starter