Hey Folks,
This blog is meant to describe what a good, healthy HTTP request flow looks like when using Windows Authentication on IIS. But first, let's go over some of the basics.
Foundation
IIS is a user mode application. It sits on top of HTTP.sys, which is the kernel mode driver in the Windows network stack that receives HTTP requests. IIS picks up requests from http.sys, processes them, and calls http.sys to send the response.
IIS, with the release of version 7.0 (Vista/Server 2008), introduced Kernel Mode authentication for Windows Auth (Kerberos & NTLM), and it's enabled by default on all versions. This feature offloads the NTLM and Kerberos authentication work to http.sys. Http.sys, before the request gets sent to IIS, works with the Local Security Authority (LSA, lsass.exe) to authenticate the end user. IIS just receives the result of the auth attempt, and takes appropriate action based on that result.
Before diving into both Kerberos and NTLM request/response flows, it's worth noting that the vast majority of HTTP clients (browsers, apps, etc.) don't send any credentials on their first request for a resource. This means that first request is anonymous, even if credentials have been configured for that resource. This anonymous request, when Windows Auth is enabled and Anonymous Auth is disabled in IIS, results in an HTTP 401 status, which shows up as "401 2 5" in the normal IIS logs. Both request flows below will demonstrate this with a browser, and show that it is normal.
The NTLM and Kerberos exchanges occur via strings encoded into HTTP headers.
Kerberos
Request & Response #1
This is the initial anonymous request by the browser:GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Encoding: gzip, deflate, peerdist
Accept-Language: en-US, en; q=0.5
Connection: Keep-Alive
Host: server
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299
I've configured Windows Authentication to only use the "Negotiate" provider, so these are the headers we get back in the HTTP 401 response to the anonymous request above:HTTP/1.1 401 Unauthorized
Cache-Control: private
Content-Length: 6055
Content-Type: text/html; charset=utf-8
Date: Tue, 13 Feb 2018 18:57:03 GMT
Server: Microsoft-IIS/8.5
WWW-Authenticate: Negotiate
X-Powered-By: ASP.NET
We can see this response has been sent from IIS, per the "Server" header. During the course of processing the request and generating the response, the Windows Authentication module added the "WWW-Authenticate" header, with a value of "Negotiate" to match what was configured in IIS. This tells the client how the server expects a user to be authenticated.
This response gets logged as a "401 2 5" in the IIS logs:
sc-status = 401: Unauthorized
sc-substatus = 2: Unauthorized due to server configuration (in this case because anonymous authentication is not allowed)
sc-win32-status = 5: Access Denied
Side note: the "Negotiate" provider itself includes both the Kerberos and NTLM packages. These can be discerned by looking at the encoded auth strings after the provider name. NTLM and its auth string is described later in this post.
Side note 2: The default settings for Windows Authentication in IIS include both the "Negotiate" and "NTLM" providers. This means the standard HTTP 401 response to the anonymous request will actually include two "WWW-Authenticate" headers - one for "Negotiate" and the other for "NTLM." Clients generally choose the one listed first, which is "Negotiate" in a default setup.
Request & Response #2
The client browser has received the HTTP 401 with the additional "WWW-Authentication" header indicating the server accepts the "Negotiate" package. The client will prefer Kerberos over NTLM, and at this point will retrieve the user's Kerberos token. The browser then re-sends the initial request, now with the token (KRB_AP_REQ) added to the "Authorization" header:GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Encoding: gzip, deflate, peerdist
Accept-Language: en-US, en; q=0.5
Authorization: Negotiate YIIg8gYGKwY[...]hdN7Z6yDNBuU=
Connection: Keep-Alive
Host: server
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299
Notice the encoded auth string starts with "YII.." - this indicates it's a Kerberos token, and is how you can discern what package is being used, since "Negotiate" itself includes both NTLM and Kerberos.
Side-note: The client device will reach out to Active Directory if it needs to get a token. This communication takes place after the server sends the initial 401 (response #1), and before the client sends request #2 above. Of course, if the client has a cached Kerberos token for the requested resource already, then this communication may not necessarily take place, and the browser will just send the token it has cached.
Side-note 2: Troubleshooting Kerberos is out of the scope of this post. This post shows a healthy, successful, working authentication flow, and assumes there were no problems retrieving a Kerberos token on the client side, and no problems validating that token on the server side.
Once the server has received the second request containing the encoded Kerberos token, http.sys works with LSA to validate that token. If everything is good, http.sys sets the user context on the request, and IIS picks it up. At this point, the response gets built and the requested resource delivered to the browser:HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 608
Content-Type: text/html
Date: Tue, 13 Feb 2018 18:57:03 GMT
ETag: "b03f2ab9db9d01:0"
Last-Modified: Wed, 08 Jul 2015 16:42:14 GMT
Persistent-Auth: true
Server: Microsoft-IIS/8.5
WWW-Authenticate: Negotiate oYG3MIG0oAMKAQChC[...]k+zK
X-Powered-By: ASP.NET
We can see this request was ultimately serviced by IIS, per the "Server" header. This also means we'll see this particular request/response logged in the IIS logs with a "200 0 0" for the statuses.
We can also see an additional "WWW-Authenticate" header - this one is the Kerberos Application Reply (KRB_AP_REP). This is so the client can authenticate if the server is genuine.
It's certainly not obvious here that http.sys took care of user authentication for the 2nd request before IIS got involved - just know that it did, as long as Kernel Mode is enabled :)
NTLM
Request & Response #1
This is the initial anonymous request by the browser:GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Encoding: gzip, deflate, peerdist
Accept-Language: en-US, en; q=0.5
Connection: Keep-Alive
Host: server
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299
I've configured Windows Authentication to only use the "NTLM" provider, so these are the headers we get back in the HTTP 401 response to the anonymous request above:HTTP/1.1 401 Unauthorized
Cache-Control: private
Content-Length: 6055
Content-Type: text/html; charset=utf-8
Date: Tue, 13 Feb 2018 17:57:26 GMT
Server: Microsoft-IIS/8.5
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
We can see this response has been sent from IIS, per the "Server" header. During the course of processing the request and generating the response, the Windows Authentication module added the "WWW-Authenticate" header, with a value of "NTLM" to match what was configured in IIS. This tells the client how the server expects a user to be authenticated.
This response gets logged as a "401 2 5" in the IIS logs:
sc-status = 401: Unauthorized
sc-substatus = 2: Unauthorized due to server configuration (in this case because anonymous authentication is not allowed)
sc-win32-status = 5: Access Denied
Request & Response #2
The browser sees the server has requested NTLM authentication, so it re-sends the original request with an additional Authorizationheader, containing the NTLM Type-1 message:GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Encoding: gzip, deflate, peerdist
Accept-Language: en-US, en; q=0.5
Authorization: NTLM TlRMTVN[...]ADw==
Connection: Keep-Alive
Host: server
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299
Side note: we can tell this is NTLM because the base64-encoded auth string starts with "TlRM..." - this will also be the case when NTLM is used with the Negotiate provider.
At this point, the server needs to generate the NTLM challenge (Type-2 message) based off the user and domain information that was sent by the client browser, and send that challenge back to the client. This is where the IIS/http.sys kernel mode setting is more apparent. Instead of the HTTP request with the encoded auth string being sent all the way up to IIS, http.sys makes a call to the Local Security Authority (LSA -> lsass.exe) to retrieve the NTLM challenge. Once it has been received, http.sys generates the next HTTP response and sends the challenge back to the client. This is another 401:HTTP/1.1 401 Unauthorized
Content-Length: 341
Content-Type: text/html; charset=us-ascii
Date: Tue, 13 Feb 2018 17:57:26 GMT
Server: Microsoft-HTTPAPI/2.0
WWW-Authenticate: NTLM TlRMTVN[...]AAA
Note the "Server" header now - this indicates the response was generated and sent back to the client by http.sys, not IIS.
We've also got another "WWW-Authenticate" header here, containing the "NTLM" provider indicator, followed by the base64-encoded NTLM Type-2 message string.
Since this request never made it to IIS, so you will not see it logged in the IIS logs. It's not logged by http.sys, either. It, along with the other requests shown here, can be observed by using an HTTP message tracer, such as the Developer Tools built into all major browsers, Fiddler, etc.
Request & Response #3
At this point, the browser has received the NTLM Type-2 message containing the NTLM challenge. Now, it needs to send the original request one more time, and add the challenge response (NTLM Type-3 message):GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Encoding: gzip, deflate, peerdist
Accept-Language: en-US, en; q=0.5
Authorization: NTLM TlRMTVN[... much longer ...]AC4A
Connection: Keep-Alive
Host: server
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299
This completes the client-side portion, and now it's up to the server to finish the user authentication.
Just like before, http.sys takes care of parsing the "Authorization" header and completing the authentication with LSA, before the request is handed over to IIS. Once authentication is complete, http.sys sets the user context to the authenticated user, and IIS picks up the request for processing. In other words, when IIS receives the request, the user has already been authenticated.
If all went well, then the appropriate response is generated by IIS and the hosted page/app/etc., and the response is sent back to the user. In our case below, the response had a status of HTTP 200:HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 608
Content-Type: text/html
Date: Tue, 13 Feb 2018 17:57:26 GMT
ETag: "b03f2ab9db9d01:0"
Last-Modified: Wed, 08 Jul 2015 16:42:14 GMT
Persistent-Auth: true
Server: Microsoft-IIS/8.5
X-Powered-By: ASP.NET
We can see this request was serviced by IIS, per the "Server" header. This also means we'll see this particular request/response logged in the IIS logs with a "200 0 0" for the statuses.
Thoughts/Observations
This post shows what good, working HTTP requests and responses look like when Windows Authentication using Kerberos and NTLM is used successfully. The Kernel Mode aspects aren't as obvious at this level, with the exception of the NTLM Type-2 Message (the challenge) sent in the response from http.sys.
You can also see that HTTP 401 statuses are completely normal in these scenarios, with Kerberos auth receiving just one 401 (for the initial anon request), and NTLM receiving two (one for the initial anon request, the second for the NTLM challenge). If you've stumbled across this post looking to understand why you're seeing 401s when nothing is actually wrong, hopefully this helps clear at least some of the smoke.
All current browsers, at least that I know of, handle these authentication processes with no need for user intervention - the browser does all the heavy lifting to get this done. Generally, browsers will only prompt the user for credentials when something goes wrong with the flows shown above. The same goes for many applications using various kinds of frameworks, like .NET.