Many mobile apps now use an internet connection of some kind. Coding HTTP interactions is perhaps the trickiest of the common mobile software development tasks. The HTTP protocols are costly to code from stratch, and (more importantly) to get right. There are different versions of the protocol, different server implementations, firewalls and routers to consider. It's a minefield out there!

So anything to avoid reinventing that particular set of wheels is to be welcomed. The Apache HttpComponents suite is a great toolset for handling all the common http interactions in Java apps, and a lot of the more specialized ones too. It can tackle WebDAV & Authentication extensions, and even HTTPS connections.

Blackberry developers have been hamstrung for years by the more rudimentary net.rim.device.api.io.http package, or even plain old java URLConnection, and have to deal with more limitations (not mention the issues of routing through BES/BIS and MDS!). Luckily, Android had the Apache HttpComponents included in the SDK by default since day one, so there's no need to bloat out the size of your app with extra jars if you want the convenience of HttpClient.

However, the classic tradoff in deciding to use any software toolset is that, while it may save time and cost, it can also be a 'black box' that obscures some of the details and makes it harder to track down problems. A well-designed toolset will help is this regard.

For example, I had an Android app interacting perfectly with a test server, then switched it over to access Yahoo's web server on the internet. Instant problem: HTTP 502 Proxy Error.

So why didn't it work? In the end, the solution is simple (but finding it was less simple).

This is where the design of the HttpComponents library really helps. HttpClient allows logging of all HTTP chatter through the standard Java logging api. Here's an example of setting-up logging to the standard Java error console. I had to do this with a java console app because commons logging won't configure in Android via the system properties - that's a job for another day!

System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.defaultlog","trace");
System.setProperty("org.apache.commons.logging.simplelog.showdatetime", "true");
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire.header", "trace");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.commons.httpclient", "trace");

Here's the trace it produces:

[DEBUG] headers - >> REPORT /dav/user/Calendar/MyCal/ HTTP/1.1
[DEBUG] headers - >> Depth: 1
[DEBUG] headers - >> Authorization: Basic Ymh3aGVs34$52byj5jb206a29zaGthcg==
[DEBUG] headers - >> Content-Type: application/xml; charset="UTF-8"
[DEBUG] headers - >> Content-Length: 296
[DEBUG] headers - >> Host: caldav.calendar.yahoo.com
[DEBUG] headers - >> Connection: Keep-Alive
[DEBUG] headers - >> Expect: 100-Continue
[DEBUG] wire - >> "<C:calendar-query xmlns:D="DAV:" ...
[DEBUG] wire - << "HTTP/1.1 502 Proxy Error[EOL]"

HttpClient was sending the request fine, but the server was immediately returning the 502 Proxy Error and no data. The request headers show that HttpClient was sending an Expect: 100-Continue header. (The Expect-Continue mechanism was introduced for HTTP 1.1 [RFC 2616] to make HTTP interaction more efficient).

And here we get to those issues I mentioned that the big bad internet can throw up.

Proxy Servers are very common on the internet. The buffer and insulate hosted environments from attack, balance loads, etc. The thing is - a proxy server uses the lowest common denominator of HTTP protocols (because it can never assume the client understands HTTP 1.1, it always falls back to HTTP 1.0). Since Expect-Continue is a mechanism of the 1.1 HTTP prortocol, a proxy server will reject it, even if the web server behind it can handle HTTP 1.1. It's to do with authentication (which I won't get into here). For now, it's enough to realise that the third-party server must have been using a proxy server.

One solution to this is to force HttpClient to use HTTP 1.0, but then you lose all the performance benefits of HTTP 1.1, which are especially important on a mobile device.

Yet again the excellent design of HttpClient comes to the rescue though. There is a configuration that you can set to prevent HttpClient from sending the Expect-Continue header over a Http 1.1 connection:

HttpClient httpclient = new HttpClient();
httpclient.getParams().setParameter("http.protocol.expect-continue", false);

It works! Isn't HttpClient great? You can see why Google chose it for Android.