Cannot authenticate IWA services with HTTP/2 enabled

[dotMorten]

Description

Using SocketsHandler, If I make a web request to a service that has IWA authentication, the request will fail with 401 if HTTP/2 is enabled. The response is as if the request was made without setting the credentials, leading me to believe there is no authentication sent with http/2.

Reproduction Steps

Run the following code (update URL to an IWA secured endpoint):

var handler = new SocketsHttpHandler();
handler.PreAuthenticate = true;
handler.Credentials = CredentialCache.DefaultCredentials; // Use default Windows credential
// handler.Credentials = new NetworkCredential("username", "password"); // Use specific credential (for instance other non-Windows platforms)

HttpClient httpClient = new HttpClient(handler);

string url = "https://uri-to-IWA-secured-service";
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Get, url);
msg.Version = HttpVersion.Version20; // Set this to 1.1 and it'll be OK

HttpResponseMessage response = await httpClient.SendAsync(msg);
response.EnsureSuccessStatusCode(); // Throws 401  if http/2

Expected behavior

Request correctly authenticates regardless of http version.

Actual behavior

System.Net.Http.HttpRequestException: 'Response status code does not indicate success: 401 (Unauthorized).'

>	http2test.dll!Program.<Main>$(string[] args) Line 46	C#
 	[Resuming Async Method]	
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<Program.<<Main>$>d__0>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.HttpClient.SendAsync.__Core|83_0(System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationTokenSource cts, bool disposeCts, System.Threading.CancellationTokenSource pendingRequestsCts, System.Threading.CancellationToken originalCancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.SocketsHttpHandler.SendAsync.__CreateHandlerAndSendAsync|115_0(System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.SocketsHttpHandler.<<SendAsync>g__CreateHandlerAndSendAsync|115_0>d>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.SocketsHttpHandler.<<SendAsync>g__CreateHandlerAndSendAsync|115_0>d>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.SocketsHttpHandler.<<SendAsync>g__CreateHandlerAndSendAsync|115_0>d>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.RedirectHandler.SendAsync(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.RedirectHandler.<SendAsync>d__4>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.RedirectHandler.<SendAsync>d__4>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.RedirectHandler.<SendAsync>d__4>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.DiagnosticsHandler.SendAsyncCore(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.AuthenticationHelper.SendWithAuthAsync(System.Net.Http.HttpRequestMessage request, System.Uri authUri, bool async, System.Net.ICredentials credentials, bool preAuthenticate, bool isProxyAuth, bool doRequestAuth, System.Net.Http.HttpConnectionPool pool, System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.AuthenticationHelper.<SendWithAuthAsync>d__17>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.AuthenticationHelper.<SendWithAuthAsync>d__17>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.AuthenticationHelper.<SendWithAuthAsync>d__17>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(System.Net.Http.HttpRequestMessage request, bool async, bool doRequestAuth, System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__52>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__52>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__52>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.Http2Connection.SendAsync(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.Http2Connection.<SendAsync>d__111>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.Http2Connection.<SendAsync>d__111>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.Http2Connection.<SendAsync>d__111>.MoveNext()	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject)	Unknown
 	System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult>.TrySetResult(System.Threading.Tasks.VoidTaskResult result)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetExistingTaskResult(System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult> task, System.Threading.Tasks.VoidTaskResult result)	Unknown
 	[Completed] System.Net.Http.dll!System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(System.Threading.CancellationToken cancellationToken)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<System.Net.Http.Http2Connection.Http2Stream.<ReadResponseHeadersAsync>d__74>.ExecutionContextCallback(object s)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread threadPoolThread, System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<System.Net.Http.Http2Connection.Http2Stream.<ReadResponseHeadersAsync>d__74>.MoveNext(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<System.Net.Http.Http2Connection.Http2Stream.<ReadResponseHeadersAsync>d__74>.ExecuteFromThreadPool(System.Threading.Thread threadPoolThread)	Unknown
 	System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()	Unknown
 	System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()	Unknown
 	System.Private.CoreLib.dll!System.Threading.Thread.StartCallback()	Unknown

Regression?

No, Reproduces on .NET8, 9 and 10 and Windows, MacCatalyst, iOS and Android (might be more but these are the platforms I observed the problem).

Known Workarounds

None that I could find. I can't tell the difference between a normal authentication error (ie missing or invalid network credential), and a valid credential but caused by using HTTP/2.

Configuration

.NET 8-10. Windows, iOS, Android, MacCatalyst

Other information

This prevents us from moving to HTTP/2. since we're required to support IWA secured services, but can't authenticate.

/cc @karelz

[dotnet-policy-service]

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[MihuBot]

I'm a bot. Here are possible related and/or duplicate issues (I may be wrong):

• HTTP2: SocketsHttpHandler doesn't downgrade to HTTP/1.1 during Windows authentication #1582
• HTTP2: Support downgrade to HTTP/1.1 for NT auth #26925

[wfurt]

HTTP/2 spec does not allow session based authentication because of the multiplexing nature of H/2. e.g. this is likely #1582 .... that can be done manually as you noticed @dotMorten

[dotMorten]

@wfurt Thanks. I just commented on the linked issue, since this was a duplicate (I swear I searched but didn't find it - that bot is amazing). Closing here to continue in the other issue.

[wfurt]

BTW what is IWA? If you could set up some site for testing it would likely go long way towards the fix @dotMorten . The linked issues are pretty old. Partially because not many users hit this but also this is something our tests infrastructure does not support very well.

[wfurt]

Also you do set PreAuthenticate. That does not really work in this case either because of #33964

[karelz]

From offline chat with @dotMorten -- for a temporary workaround, it would be sufficient to detect that authentication failed, so they can retry manually.
@dotMorten why is 401 not sufficient here?

(reopening to explore a workaround)

[dotMorten]

@dotMorten why is 401 not sufficient here?

Because it is identical to a 401 caused by an invalid credential that wasn't trigged by IWA.

Basically these two cases render the exact same error:

handler.Credentials = CredentialCache.DefaultCredentials; // Assuming user's windows credentials ARE allowed to access resource
msg.Version = HttpVersion.Version20;

and

handler.Credentials = CredentialCache.DefaultCredentials; // Assuming user's windows credentials AREN'T allowed to access resource
msg.Version = HttpVersion.Version11;

If a request gets a 401 response, we'll prompt the user to enter their credentials, but for the Windows scenario, they shouldn't be prompted at all if they do have access to the service using their Windows credentials.

Also we don't want to downgrade to 1.1 if the failure isn't because of IWA, but could be any other number of reasons, like digest authentication. In some of our cases http/2 can give us up to a 100x performance boost (no that's not an exaggeration).