Feb 20 2020
12:32 AM
- last edited on
Apr 08 2022
10:18 AM
by
TechCommunityAP
Feb 20 2020
12:32 AM
- last edited on
Apr 08 2022
10:18 AM
by
TechCommunityAP
Hello everyone,
I'm trying to extract exceptions within our logs using regular expression. And it happens that I need to perform a negative lookahead to ignore a specific string. The query looks like below:
let pattern = @'(?!Exception: System.Web.HttpUnhandledException)Exception: (.+)\s+Message: ([\S\s]+)\s+Source: ([\S\s]+)';
let standardize = (msg:string) {
let msg2 = replace(@'"(\S+)"', '"xxx"', msg);
replace(@"'(\S+)'", '"xxx"', msg2)
};
WebsiteLogs
| where ExceptionMessage contains "System.Web.HttpUnhandledException" or ExceptionMessage contains "[Extra Exception from config]"
| where TimeGenerated > ago(7d)
| project URL,
Exception=extract(pattern, 1, ExceptionMessage ),
Message = standardize(extract(pattern, 2, ExceptionMessage )),
StackTrace = substring(extract(pattern, 3, ExceptionMessage ), 0, 500)
| summarize Errors_Count = count() by Message, URL, StackTrace
| sort by Errors_Count desc nulls last
| limit 1000
Exception: System.Web.HttpUnhandledException
Message: An unhandled exception occurred.
Source: Sitecore.Mvc
at Sitecore.Mvc.Pipelines.MvcEvents.Exception.ShowAspNetErrorMessage.ShowErrorMessage(ExceptionContext exceptionContext, ExceptionArgs args)
at Sitecore.Mvc.Pipelines.MvcEvents.Exception.ShowAspNetErrorMessage.Process(ExceptionArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain)
at Sitecore.Mvc.Pipelines.PipelineService.RunPipeline[TArgs](String pipelineName, TArgs args)
at Sitecore.Mvc.Filters.PipelineBasedRequestFilter.OnException(ExceptionContext exceptionContext)
at System.Web.Mvc.ControllerActionInvoker.InvokeExceptionFilters(ControllerContext controllerContext, IList`1 filters, Exception exception)
at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
at System.Web.Mvc.Controller.ExecuteCore()
at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
at Sitecore.Mvc.Controllers.ControllerRunner.ExecuteController(Controller controller)
at Sitecore.Mvc.Controllers.ControllerRunner.Execute()
at Sitecore.Mvc.Presentation.ControllerRenderer.Render(TextWriter writer)
at Sitecore.Mvc.Pipelines.Response.RenderRendering.ExecuteRenderer.Render(Renderer renderer, TextWriter writer, RenderRenderingArgs args)
Nested Exception
Exception: System.ArgumentNullException
Message: Value cannot be null.
Parameter name: item
Source: Website.Module.Product
Feb 20 2020 04:36 AM
Well, I managed to get over it without using regex negative lookhead by using another custom function to skip the non-relevant information. My solution looks like below in case someone else is facing similar case
let pattern = @'Exception: (.+)\s+Message: ([\S\s]+)\s+Source: ([\S\s]+)';
let standardize = (msg:string) {
let msg2 = replace(@'"(\S+)"', '"xxx"', msg);
replace(@"'(\S+)'", '"xxx"', msg2)
};
let getException = (msg: string) {
case(msg contains "System.Web.HttpUnhandledException", substring(msg, indexof(msg, "Nested Exception") + 17), msg)
};
WebsiteLogs
| where ExceptionMessage contains "System.Web.HttpUnhandledException" or ExceptionMessage contains "[Extra Exception from config]"
| where TimeGenerated > ago(7d)
| project URL,
Exception=extract(pattern, 1, ExceptionMessage ),
Message = standardize(extract(pattern, 2, getException(ExceptionMessage) )),
StackTrace = substring(extract(pattern, 3, getException(ExceptionMessage )), 0, 500)
| summarize Errors_Count = count() by Message, URL, StackTrace
| sort by Errors_Count desc nulls last
| limit 1000
Here, the getException() function just skip the ExceptionMessage till the first Nested Exception for logs containing the System.Web.HttpUnhandledException.
Nevertheless, I'm still wondering if regex lookahead is supported or not
Feb 24 2020 06:59 AM
Feb 25 2020 04:06 AM - edited Feb 25 2020 04:09 AM
Apparently the full explanation is here. Quoting from it:
"Kusto uses the re2 library: https://github.com/google/re2/wiki/Syntax, as mentioned here: https://docs.microsoft.com/en-us/azure/kusto/query/re2
What you're trying to achieve isn't supported by that library - see https://github.com/google/re2/wiki/WhyRE2:
'As a matter of principle, RE2 does not support constructs for which only backtracking solutions are known to exist. Thus, backreferences and look-around assertions are not supported.'"
Jan 26 2021 05:37 AM
@Noa Kuperberg thanks for the information on the RE2 library. Negative lookahead would be very useful in some cases. :\ would be great if this could be added somehow. Why not use System.Text.RegularExpressions instead of re2? :p