Here's one of those obsure problems I came across in Torquebox that might help some other developer; it sure would have saved me some time!

One of the things I like about using the Java flavour of Ruby (JRuby) is that it's pretty easy to invoke and access EJBs. Here's an example:

require 'java'
require '~/Applications/torquebox-1.0.1/jboss/lib/jboss-common-core.jar'
require '~/Applications/torquebox-1.0.1/jboss/client/jbossall-client.jar'
require '~/ExampleEAR/ExampleEAR-ejb/dist/ExampleEAR-ejb.jar'
require '~/Applications/netbeans/enterprise/modules/ext/javaee-api-6.0.jar'

include_class 'java.util.Properties'
include_class 'javax.naming.InitialContext'
include_class 'com.hypermatix.session.ExampleSessionBean';
include_class 'com.hypermatix.session.ExampleSessionBeanRemote';

class EJBExampleController < ApplicationController
  
  def index
    properties = Properties.new
    properties.put(Context::INITIAL_CONTEXT_FACTORY,"org.jnp.interfaces.NamingContextFactory")
    properties.put(Context::PROVIDER_URL, "jnp://localhost:1099")
    properties.put(Context::URL_PKG_PREFIXES, "org.jboss.naming:org.jnp.interfaces");

    context = InitialContext.new(properties)

    sessionBean = context.lookup("ExampleEAR/ExampleSessionBean/remote")

    result = sessionBean.someBusinessMethod(parameters)
  end
end

Torquebox is a great platform that combines JBoss AS 6 and JRuby into one server, and adds a lot of very powerful integration additions too. However, when I deploy this JRuby test app and the EJB on the same Torquebox JBoss instance, I get a name not bound message with the JNDI context.lookup for the EJB. Looking further down the stack trace (getting a stack trace from a JRuby NativeException is another issue!):

Can not find interface declared by Proxy in our CL + org.jboss.web.tomcat.serice.WebCtxLoader

Further again down the stack trace, it turned out that a ClassNotFoundException was causing this. My own ExampleSessionBean EJB class was not being located by the Torquebox JBoss instance. When I tried to invoke it in a standalone Java app, it worked fine. I couldn't find any similar problem reported for JRuby, however a similar error can occur in Apache web apps. It turns out to be the way the JBoss class loading mechanism works. The solution is to ensure that the web app uses the same class loading domain as the EJB. To do this, just add a jboss-classloading.xml file into the config directory of the JRuby app and into the EAR you use to deploy the EJBs:

<?xml version="1.0" encoding="UTF-8"?>
<classloading xmlns="urn:jboss:classloading:1.0"
              domain="DefaultDomain"
              parent-domain="Ignored"  
              export-all="NON_EMPTY"   
              import-all="true">
</classloading>

It's been a busy couple of months. I've transferred to other projects, so unfortunately CalDAV Sync for Android hasn't been getting the attention it deserves, or the attention requested. I should get back to work on it soon. In the meantime, the latest version of CalDAV Sync up the Android Market has had its time-lock removed completely.

The trial period time-lock initially served two purposes: to prevent unstable Beta releases from being out there too long by forcing upgrades to more stable later releases, and also to support the possibility that the app could be charged for at some future point, to cover development costs. The few donations that were received during development were very much appreciated - thank you to those who thought the app was worth contributing to, and to those who e-mailed such supportive comments about the app. However, the few numbers of donations (compared to the number of actual app installations) would hardly address any actual development costs. I started developing CalDAV Sync as a side-project out of a need myself to synchronize with my work calendar. As it turns out, the lack of native CalDAV support in the Android calendar meant that I wasn't the only one who needed it, and still do!

So the intended future plan for the app is as follows:

The current Sychronization app is relatively stable now, but is still missing one or two crucial features. The app will remain free, but mostly unsupported, except for development of one or two of those outstanding features that are important (mostly to do with recurring events and extra time zones). However, most of the development efforts will be focussed now on a new app (explained below) which will build on the functionality already developed for CalDAV Sync. The current app will be made available for download outside the Android Market for those who can't access it on the market. The free sync app is also the only one that will allow CalDAV sync on older Android OS (i.e. 1.6). The new app will use the newer API features of Android 2.1. The majority of Android users are now using Android 2.1 and later.

Synchronizing with the existing calendar on the Android device was a short cut that saved development time, but it creates a few problems: the inability to create new calendars, duplicate events under some upgrade scenarios, and also the problems inherent in sychronizing with several calendar code branches by handset manufacturers that are outside of my control - I can't charge for an app that might cease to function after the user has paid for it, and where I couldn't do anything about it. Such breaks in functionality have already happened when handset maufacturers issue new versions of their calendar app.

The way forward has slowly become apparent; build our own customized version of the Android calendar app that can be expanded as a CalDAV client. This will address all the remaining issues that can not be dealt with in the current app (because of its paradigm of synchronizing with third-party undocumented APIs), and also means it can be expanded in future with all the facilities unique to CalDAV clients (such as To-Do lists, attendees, etc.) Such an app (which *can* be supported) would need to be a paid app, which (given the number of daily downloads of the current app) should cover development costs.

Watch this space in the new year!

The original expiry deadline for Beta Calendar Sync for Android is nearly here. There's been a lot of good feedback (and of course bugs) since it first went up on the market. A lot of those popular feature requests will be available in the next major release v0.9.4 - multiple calendars, better sync options, etc. It will also address some things that have been on the known issue list for a while, even since the app was first released (such as time zone support). There will still be some bugs and issues to be addressed, but better to get all the major features out, extend the Beta phase a little longer, and test with as much major functionality together while it's still in beta.

After 31st July, synchronization will be prevented in any installed versions of the app and users will get a message to download a new version. Because so much is happening for the next release, unfortunately that means it won't be ready by the current expiry date. However, to support all those who now use it on a daily basis, an update of the existing app will be put up on the market that will extend the expiry date by a month.

The new major version, v0.9.4, will be released during August 2010.

UPDATE 14/4/10:
This blog post has been superceded by changes in version 0.9.2.0 of Calendar Sync You can now accept certs without using the method below simply by checking the option "Accept SSL Certificates from untrusted sources" in the advanced connection settings. However, note that enabling this option could expose you to man-in-the-middle attack. You must ensure for yourself that you can trust the server/certificate.

Calendar Sync can connect to servers via secure SSL connections. This works fine for 'big domain' calendar servers if they are signed by one of Android's supported CAs (Certificate Authorities). However private calendar servers' SSL certificates are usually signed by the server itself, or perhaps by a less well-known CA that Android doesn't support by default. In such cases, the error "Not trusted server certificate" appears.

We need a way to make Calendar Sync accept these SSL connections, since many CalDAV calendars are run securely from private servers. Fortunately, there is a way, but it's not pretty. Thanks to Bob Lee for his great blog post about trusting SSL certs in Android. I've reproduced the relevant steps here, and the extra steps necessary to make it work with Calendar Sync.

In short, you do this from your computer, where you have to create an encrypted keystore file which contains the public cert of the CA that issued your server SSL certificate. For self-signed certs, that normally means the CA is your own server's public cert.

1: Download Pre-requisites

Download the Java Runtime if you don't already have it installed.

Download the Bouncy Castle Provider JAR

For this example, let's assume we downloaded bcprov-jdk16-145.jar into the directory c:\temp on the computer. You should download the latest jar, or whichever best matches the version of Java you have installed.

2: Get the CA cert

Get the public certificate for your server or CA. It needs to be in PEM format. A PEM format certificate is just encoded text in a file, wrapped in BEGIN and END CERTIFICATE like this:

-----BEGIN CERTIFICATE----- 

    [...Some Encoded Text...]

-----END CERTIFICATE----

Where and how to get this file depends on your server or CA.

For a CA, there should be a PEM file of their certificate available to download.

If the certificate is self-signed by the server, you may need to generate a server certificate PEM file. The way you do this depends on your server operating system and the program you used to generate the cert, which is beyond the scope of this article. For example, here's a guide on how to do it with OpenSSL.

Save the PEM cert into a local directory on our computer, e.g. c:\temp\myCACert.pem

3: Create a keystore for the cert in BKS format

Open a command terminal window on your computer and run Java's keytool. This will create a file called CA.bks.

c:\temp> keytool -import -v -trustcacerts -alias 0 
                 -file myCACert.pem 
                 -keystore CA.bks -storetype BKS 
                 -provider org.bouncycastle.jce.provider.BouncyCastleProvider 
                 -providerpath bcprov-jdk16-145.jar 
                 -storepass myCAPassword

[...]
Trust this certificate? [no]:  yes
Certificate was added to keystore
[Storing CA.bks]

C:\temp>

4: Copy your root cert keystore to the Android device

Connect your Android phone to the computer in USB Drive mode.
Create a directory on your phone's SD card and copy the keystore file you just created into it: /CalDAVSync/CA/CA.bks

Note: The directory and file name must match this exactly or Calendar Sync will not find it.

5: Tell Calendar Sync to use the cert in the keystore file

Open Calendar Sync and navigate to More->Advanced->Advanced Connection.

Check to Enable use of the Self-Certified Cert Authority.

Enter the password you used when creating the keystore (which is myCAPassword in the example above).

Save the new settings and run Synchronization to test connection to your calendar server.

Note: You must turn off USB drive if connected to the computer before synchronizing again. Calendar Sync cannot read the SD card while the phone is in USB drive mode, and the "Not trusted server certificate" will persist.

Note: If you ever change your CalDAV Calendar connecion to a different SSL server location, remember to uncheck Use Self-certified Cert Authority again, or it may not connect.

That's it. It's not straightforward. Wouldn't it be great if Android could add trusted certificates system-wide? Maybe in a future release Google?

Many mobile apps now use an internet connection of some kind. Coding HTTP interactions is perhaps the trickiest of the common mobile software development tasks. The HTTP protocols are costly to code from stratch, and (more importantly) to get right. There are different versions of the protocol, different server implementations, firewalls and routers to consider. It's a minefield out there!

So anything to avoid reinventing that particular set of wheels is to be welcomed. The Apache HttpComponents suite is a great toolset for handling all the common http interactions in Java apps, and a lot of the more specialized ones too. It can tackle WebDAV & Authentication extensions, and even HTTPS connections.

Blackberry developers have been hamstrung for years by the more rudimentary net.rim.device.api.io.http package, or even plain old java URLConnection, and have to deal with more limitations (not mention the issues of routing through BES/BIS and MDS!). Luckily, Android had the Apache HttpComponents included in the SDK by default since day one, so there's no need to bloat out the size of your app with extra jars if you want the convenience of HttpClient.

However, the classic tradoff in deciding to use any software toolset is that, while it may save time and cost, it can also be a 'black box' that obscures some of the details and makes it harder to track down problems. A well-designed toolset will help is this regard.

For example, I had an Android app interacting perfectly with a test server, then switched it over to access Yahoo's web server on the internet. Instant problem: HTTP 502 Proxy Error.

So why didn't it work? In the end, the solution is simple (but finding it was less simple).

This is where the design of the HttpComponents library really helps. HttpClient allows logging of all HTTP chatter through the standard Java logging api. Here's an example of setting-up logging to the standard Java error console. I had to do this with a java console app because commons logging won't configure in Android via the system properties - that's a job for another day!

System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.defaultlog","trace");
System.setProperty("org.apache.commons.logging.simplelog.showdatetime", "true");
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire.header", "trace");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.commons.httpclient", "trace");

Here's the trace it produces:

[DEBUG] headers - >> REPORT /dav/user/Calendar/MyCal/ HTTP/1.1
[DEBUG] headers - >> Depth: 1
[DEBUG] headers - >> Authorization: Basic Ymh3aGVs34$52byj5jb206a29zaGthcg==
[DEBUG] headers - >> Content-Type: application/xml; charset="UTF-8"
[DEBUG] headers - >> Content-Length: 296
[DEBUG] headers - >> Host: caldav.calendar.yahoo.com
[DEBUG] headers - >> Connection: Keep-Alive
[DEBUG] headers - >> Expect: 100-Continue
[DEBUG] wire - >> "<C:calendar-query xmlns:D="DAV:" ...
[DEBUG] wire - << "HTTP/1.1 502 Proxy Error[EOL]"

HttpClient was sending the request fine, but the server was immediately returning the 502 Proxy Error and no data. The request headers show that HttpClient was sending an Expect: 100-Continue header. (The Expect-Continue mechanism was introduced for HTTP 1.1 [RFC 2616] to make HTTP interaction more efficient).

And here we get to those issues I mentioned that the big bad internet can throw up.

Proxy Servers are very common on the internet. The buffer and insulate hosted environments from attack, balance loads, etc. The thing is - a proxy server uses the lowest common denominator of HTTP protocols (because it can never assume the client understands HTTP 1.1, it always falls back to HTTP 1.0). Since Expect-Continue is a mechanism of the 1.1 HTTP prortocol, a proxy server will reject it, even if the web server behind it can handle HTTP 1.1. It's to do with authentication (which I won't get into here). For now, it's enough to realise that the third-party server must have been using a proxy server.

One solution to this is to force HttpClient to use HTTP 1.0, but then you lose all the performance benefits of HTTP 1.1, which are especially important on a mobile device.

Yet again the excellent design of HttpClient comes to the rescue though. There is a configuration that you can set to prevent HttpClient from sending the Expect-Continue header over a Http 1.1 connection:

HttpClient httpclient = new HttpClient();
httpclient.getParams().setParameter("http.protocol.expect-continue", false);

It works! Isn't HttpClient great? You can see why Google chose it for Android.