Share via

Proxy-Authentication breaks many applications

When I first joined Office, I worked on the team responsible for delivering Help, Templates, and ClipArt into the client applications. As we were testing our work in various simulated customer environments, we found a big problem. At least one big customer (tens of thousands of licenses) had a network environment in which their users were forced to enter a username and password in order to authenticate to the proxy server. Without authenticating to the proxy, all HTTP/HTTPS requests were blocked.

Now, this was a fairly uncommon architecture, even then, and is perhaps more so now. In most environments, either the proxy server doesn’t require authentication, or the proxy relies upon the NTLM/Kerberos authentication schemes which permit users’ Windows logon credentials to be automatically used to respond to challenges from the proxy server. Environments that relied upon BASIC or DIGEST authentication require that the user explicitly submit their credentials, typically once per process (because most networking components, e.g. WinINET would cache these credentials for the lifetime of the process).

The problem with my features in Office was that they all passed the INTERNET_FLAG_NO_UI flag to WinINET, or ran atop WinHTTP, which explicitly doesn’t include any user-interface, including dialogs. The result of this was that in an environment with a BASIC/DIGEST proxy, all requests failed. In order to work properly in such environments, the application must itself supply the needed credentials to the network stack (e.g. for WinINET, call InternetSetOption, passing the INTERNET_OPTION_PROXY_PASSWORD and INTERNET_OPTION_PROXY_USERNAME option flags) to avoid the need to prompt the user.

I added a new rule to Fiddler that made it simple to test products for this problem: 

image

When the Require Proxy Authentication box is checked, Fiddler automatically responds to any request lacking a Proxy-Authorization header with a HTTP/407 response containing a Proxy-Authenticate header specifying the authentication scheme required:

GET /ua.aspx HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Host: www.enhanceie.com

HTTP/1.1 407 Proxy Auth Required
Connection: close
Proxy-Authenticate: Basic realm="FiddlerProxy (username: 1, password: 1)"
Content-Type: text/html

<html><body>[Fiddler] Proxy Authentication Required.<BR> </body></html>

A client that supports manual proxy authentication will then prompt the user for the username and password:

image

The client will then reissue the same request, supplying the provided credentials (base64-encoded) in the Proxy-Authorization header:

GET /ua.aspx HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Proxy-Authorization: Basic MTox
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Host: www.enhanceie.com

If the client fails to collect the credentials, it will typically treat the HTTP/407 response as fatal and will show an error message or fail silently.

When you try this, you can find broken scenarios all over. For instance, when I tried to post this blog using Windows Live Writer, the following error message was shown:

image

Afterward, I was prompted to re-enter my credentials for the web server—there was no way to supply the credentials required by the proxy!

Sometimes, an otherwise failing scenario may pass depending on what happens earlier in a process. For instance, if you enable the Fiddler rule, then launch IE to about:blank you will find that Search Suggestions from the Address bar don’t work, showing “An error occurred.”

image

Notably, if you subsequently navigate the tab to a web page, IE will prompt you for proxy credentials using the CredUI dialog box shown above. After you supply those credentials, the Search Suggestions feature starts working—that’s because the proxy credentials are cached for the lifetime of the process.

In other cases, failure are silent and there’s no notice to the user. For instance, many background updaters are based on BITS/WinHTTP and will fail silently when a HTTP/407 is encountered. Similarly, Windows’ CAPI component’s Certificate Revocation Checks will fail because the svchost.exe process doesn’t have the required proxy credentials.

If you need to sell your software into an enterprise that uses proxies, or just want to make your software robust against even uncommon network configurations, be sure to test manual proxy authentication scenarios!

-Eric

Comments

  • Anonymous
    August 23, 2012
    The comment has been removed

  • Anonymous
    August 23, 2012
    @Maximilian-- Indeed, that idea has arisen periodically for the last decade or so. As I mentioned, manually-authenticating proxies have always been rare (and some of those who use them would not want this "automatic" behavior). These are two of the key reasons why this investment has never been made.

  • Anonymous
    November 09, 2012
    Having the proxy at the application level causes very many issues in the IT world... moving it below layer 7 would solve so many headaches. The key problem is that every application needs to deal with the "system default proxy", and if it doesn't then there is often no solution for the end user (other than asking the vendor to "fix" their program). Case in point: Most video players running in Microsoft Silverlight will fail when behind any kind of proxy (we're behind an authenticating proxy that transparently uses the user's Windows credentials).  Our company has several sites with Silverlight training videos, and we need to give users static routes to the video hosts as the Silverlight Player does not respect the system default proxy. It's a ridiculous situation really... and if Microsoft is capable of fixing it, then it would make a lot of people very happy.  (Looking at this as an issue that only affects manually authenticating proxies ignores a huge chunk of the problems with proxies at the application level.)

  • Anonymous
    November 26, 2013
    New KB Article: "Using authenticated proxy servers together with Windows 8" - support.microsoft.com/.../2778122 Summary: Don't.

  • Anonymous
    November 26, 2013
    For my Windows 7, I ended up with using Squid to act as an intermediate proxy server, put parent proxy server's basic authentication info into Squid configuration file, redirected the whole internet connection to it, and it just works without the annoying prompts. Haven't validated it yet against Windows 8/8.1, but I think it should just work as well.

  • Anonymous
    September 14, 2014
    If you have a .NET application that fails, you may be able to update its config file: <system.net>    <defaultProxy useDefaultCredentials="true" >    </defaultProxy> </system.net> chentiangemalc.wordpress.com/.../case-of-the-system-net-webclient-407-proxy-authentication-required