The leaver who never left: tuning a Sentinel detection from 103 rows to one alert

Most account compromise detections look for something odd: a new country, an impossible journey, or a sign-in at 3 AM. I wanted to catch a case where nothing looks odd at all.

Someone leaves the organisation. HR knows, their manager knows, but the request to disable their account sits in a queue or never gets raised. A week later, they sign in from the same laptop, on the same Wi-Fi, into the same apps they used every day. To Entra ID, that is a normal sign-in. Location rules have nothing to catch because nothing has changed.

I started from a threat model for a hospital running Microsoft 365. Of the threats I scored, this one stood out because it requires no advanced skill. The person already has a password that works. The only signal is that they are not supposed to be there anymore, and that fact lives with HR, not in the logs. So, the detection has to bring that context into Microsoft Sentinel.

Writing it in plain words first

Before writing any KQL, I defined exactly what the rule should do:

• When an account signs in successfully

• AND it is on HR's leavers list

• AND the sign-in is after their leave date

• THEN raise one High-severity alert per person

• EXCEPT when IT is already closing the account, or the person has been rehired

The sign-ins are in SigninLogs. For the leavers list, I used a Sentinel watchlist: a CSV with two columns, UserPrincipalName and LeaveDate. To test it, I put my own account on the list with a leave date in the past, so every sign-in after that date should trigger.

Version 1

let Leavers = _GetWatchlist('Leavers')
    | project UPN = tolower(tostring(SearchKey));
SigninLogs
| where TimeGenerated > ago(7d)
| where ResultType == "0"
| where tolower(UserPrincipalName) in (Leavers)
| project TimeGenerated, UserPrincipalName, AppDisplayName, IPAddress, Location

It worked, and that was the problem. It returned 103 rows for one person: every token refresh, every app. An analyst opening that incident would scroll through a hundred near-identical lines to learn one thing: this person is still getting in. It also ignored the leave date, so it would fire on sign-ins from before someone left as well.

Version 2

I made two changes: only count sign-ins after the leave date, and summarise per person instead of listing every event. The result was one row: 103 sign-ins, the first and last time seen, the 4 apps used, and the 2 IP addresses. That is exactly what an analyst needs to decide what to do next.

The bug that took the longest

Version 2 failed with this error:

Failed to resolve scalar expression named 'LeaveDate'

The column was clearly there. I could see it in the watchlist. I rewrote the query more than once before I looked at the raw Watchlist table and saw the column was actually called " LeaveDate", with a leading space. The CSV had been saved with a space after each comma. The header looked fine to me, and Sentinel took it literally.

The fix was to read the Watchlist table directly, use coalesce() to accept either spelling of the column, and trim() to strip spaces from the values. The bigger lesson: lists from other teams will not always be clean, and a detection should cope with that rather than break.

The final query

let Leavers = Watchlist
    | where TimeGenerated > ago(90d)
    | where WatchlistAlias == "Leavers"
    | summarize arg_max(TimeGenerated, *) by _DTItemId
    | where _DTItemStatus != "Delete"
    | extend RawDate = coalesce(tostring(WatchlistItem["LeaveDate"]), tostring(WatchlistItem[" LeaveDate"]))
    | project UPN = tolower(trim(@"\s", tostring(SearchKey))), LeaveDate = todatetime(trim(@"\s", RawDate));
SigninLogs
| where TimeGenerated > ago(7d)
| where ResultType == "0"
| extend UPN = tolower(UserPrincipalName)
| join kind=inner Leavers on UPN
| where TimeGenerated > LeaveDate
| summarize SignIns = count(),
            FirstSeen = min(TimeGenerated),
            LastSeen = max(TimeGenerated),
            Apps = make_set(AppDisplayName),
            IPs = make_set(IPAddress)
            by UPN, LeaveDate

arg_max keeps only the latest version of each watchlist row, and the _DTItemStatus filter drops rows that have been deleted, so someone taken off the list stops alerting.

What I would add before using it for real

• Feed the list from the HR system instead of uploading a CSV by hand. A list that is two weeks out of date has the same gap as no list.

• Build in the exceptions: accounts IT is already closing, and rehires.

• Make it a scheduled rule: hourly, 7-day lookback, High severity, with the user mapped as an account entity so the incident links to anything else that account has done.

• Check the data is arriving. In the same lab, I found three log sources that had been silent for a month with no error anywhere. A detection is only as good as the data under it.

The queries and notes are available on my GitHub: github.com/RabiatuMohammed/sentinel-detections

Update: catching the re-enable, not just the sign-in

After publishing this, I tested a worse version of the problem. I disabled a test account, watched it get refused at sign-in, then switched it back on. Two minutes later, it was signed in again.

13:11 - Account disabled (AuditLogs: Disable account)

13:15 - Sign-in refused (SigninLogs: 50057, account disabled)

13:29 - Account re-enabled (AuditLogs: Enable account)

13:31 - Signed in again (SigninLogs: 0, success)

The rule above only fires at 13:31, after they are back in. The moment that actually matters is 13:29. Disabling the account was the right ending, so switching it back on needs a reason.

So, I wrote a second rule that watches for "Enable account" on anyone on the leavers list, and shows who did it:

AuditLogs
| where OperationName == "Enable account"
| extend UPN = tolower(tostring(TargetResources[0].userPrincipalName)),
         EnabledBy = coalesce(tostring(InitiatedBy.user.userPrincipalName),
                              tostring(InitiatedBy.app.displayName))
| join kind=inner Leavers on UPN
| where TimeGenerated > LeaveDate
| project TimeGenerated, UPN, LeaveDate, EnabledBy

Against the test, it returned one row at 13:29, two minutes before the sign-in.

One thing surprised me: the sign-in screen told the user the account was locked, but the log said disabled (50057). Locked would have been 50053. I now go by the error code, not by what the user says they saw.

The full query is available on GitHub as DET-LEAVER-02.