0x1F is an invalid start of a value

Recently I ran into the following error when working on integration of Identity Server into an Asp.Net Core application.

This error occurred when the application tried to get Access Token as part of OAuth code workflow. The code uses IdentityModel library to perform Open Id operations. The following code returned the mentioned error.

await httpClient.RequestAuthorizationCodeTokenAsync(....)

This error makes no sense in the context of this function per se. This drove me nuts for good part of my day. The bad part was the code did not throw any error on local machine. Then I compared the local execution with production deployment. The production deployment uses CloudFlare WAF. All requests are proxied through CloudFlare. I turned on HTTP Logging for the application by adding following two items in the code and app settings JSON file.

application.UseHttpLogging();

"Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning",
      "Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware": "Information"
    }
  }

I could see in the log file that Identity Server generated the Access Token correctly and was contained in response as well. Now, when I looked at the error, I could see that request did return a response but IdentityModel library could not read it. Now look at the HEX number in error message. 0x1F is starting signature of a compressed file. This was clue to the exact problem. The server returned compressed response and IdentityModel was not expecting it. Then I looked at the request headers. I had following line of code present.

HttpClient.DefaultRequestHeaders.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));

Even though Identity Server did not return compressed response, it was CloudFlare that performed compression on response. By default, CloudFlare performs request inspection and if it finds that the request has indicated that it can accept compressed response, CloudFlare will perform compression.

Short term solution was to remove request header. And it worked fine. Long term solution will be to enhance IdentityModel library to make it inspect response and then De-serialize uncompressed response.