These logs and traces are stored in the ContainerLog table. You can navigate to an autogenerated query from inside the Container Insights blade from the Azure Portal.
During an engagement, we faced an issue with viewing these logs and traces. We could view the logs in the “Live Logs” tab inside the Portal, but none of these got sent to Log Analytics workspace and we were unable to retain and run Kusto queries against that data.
As it turned out, after a troubleshooting session with a colleague, the reason the Nginx Ingress Controller logs and traces were not sent in Log Analytics workspace was the fact that the controller had been deployed to the kube-system namespace, which happens to be excluded by default from the container log collection, based on the below note.
Taking it to the next level, we wanted to do this with zero downtime. For that, a secondary Ingress Controller needs to be created. Then, the ingress routes need to be duplicated to use the secondary ingress controller. Finally, you should do a swap in the Firewall Appliance to redirect traffic to the new Ingress IP address.
Below you can find the detailed steps to deploy and configure a secondary ingress controller:
Create an additional ingress controller on a different / appropriate namespace, providing a custom ingressClass (e.g., “nginx-v2”), given the default “nginx” ingressClass will already be in use by the existing ingress controller. The helm install command is simplified for readability, you will probably have additional parameters as per your script, but the highlighted ones are required: