A date with Java's URL API

We’re currently working on a project that involves signing parameters with HMAC+SHA1. In the Java sense, that is no big deal, you can simply use the Mac API.

After creating the classes and the appropriate unit tests, I went ahead and ran a full test suite, but to my horror I received at least a dozen failed unit tests.

However, when I went back and ran the test suite for the individual classes, the suite passed with flying colors. Now I knew that the issue lied with the interaction between multiple classes and something in the more complicated world of multi-threading and race conditions.

Summary

There is an issue where if Mac.getInstance is called prior to the Smock JaxWsPortClientInterceptor URL creation is made, then you will end up using the live http handler rather than the mocked out one. This will cause it to connect to http://localhost:8080 which will most likely fail.

A little background

The new class that was created served the purpose of signing a set of parameters with the HMAC+SHA1 algorithm.

The failing test suite was a web service class that was being tested using the Smock, which ultimately utilizes spring-ws to do a lot of the heavy lifting.

The stack trace of the failing tests had something along the lines of

org.springframework.remoting.RemoteAccessException : Could not access remote service at [ApiService]; nested exception is javax.xml.ws.WebServiceException: java.net.ConnectException: Connection refused
at org.springframework.remoting.jaxws.JaxWsPortClient Interceptor.doInvoke(JaxWsPortClientInterceptor.ja va:504)
at org.springframework.remoting.jaxws.JaxWsPortClient Interceptor.invoke(JaxWsPortClientInterceptor.java :481)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)

At first glance, the error seems to be related to Spring’s ApplicationContext somehow getting dirty and causing the remote service to connect to an endpoint that does not exist.

Debugging the tests

Where is the bug?

To first determine if the error was due to the injection of faulty code on my part to somewhere in the Spring configurations, I went ahead and ran the test suites independently. The result was a bit infuriating, they both passed. When I ran the two classes sequentially, the error was reproduceable. So it seems that the issue is somewhere in the code I have written.

The first step to finding out where exactly is causing the issue was to try to clean up the Spring context. Spring has a pretty nifty feature if you’re using the SpringJunit4ClassRunner that allows you to reset the application context after it has been dirtied. By using the @DirtiesContext annotation, Spring will clear up the application context either after every method invocation or after the class has completed running.

Unfortunately for my case, even with the @DirtiesContext annotation, the unit tests were still failing with the same stack trace.

Setting up breakpoints

At first thought, the best place to start debugging would be to check where we are attempting to connect to. Not knowing exactly how the spring-ws testing framework is set up, I decided to debug the connection at the JaxWsPortClientIntercepter. When I ran the tests, I found that we were attempting to connect to http://localhost:8080. When I did a curl http://localhost:8080, curl returned to me that the connection was indeed refused.

Now that I found out that was the source of the connection refusal, I checked to see if any process was running that uses port 8080. I ran the test suite standalone (the one that works), and no process ever spawned that listens on 8080. I then reran the failing test combination and still no process ever spawned. So, somehow the framework is actually connecting to the endpoint without having to spawn a process that listens to port 8080!

A little experiment with the Mac class

My attention now turned to the Mac class that was the source of the issue. When I attempted to load the source code into my IDE, Intellij, I was unable to view the source since my JDK wasn’t compiled with debug symbols. After finding a more appropriate JDK, I was able to load the source code. However, nothing was of interest in the method I was calling, Mac.instanceOf(). Therefore, I decided to take a closer look at how JaxWsPortClientInterceptor works.

A closer look at Java’s URL API

In JaxWsPortClientInterceptor, I discovered that the way it connects to an endpoint is by loading up the WSDL and then uses the Java URL API to connect to it.

Thinking that the issue may be in the way URL is being used, I loaded up the source in a debugger. After stepping through the code several times, I found that the way URL works is that whenever it finds a new protocol schema, it will add it to a HashTable of protocol handlers. This protocol handler list is static and thus accesible to all instances of URL.

Knowing this, I loaded up what the HashTable contains on both the failing unit test and the passing ones.

What I found was that in the passing test, the handler for HTTP is ThreadLocalMockHttpUrlConnection whereas when the test fails, it is sun.net.www.protocol.http.Handler. It seems that whenever the failing test combination runs, something is causing the Handler to be set as the Sun implementation of HTTP Handler! This of course causes a live attempt to connect to http://localhost:8080 which will cause the Connection Refused!

Knowing this, I went back to the Mac class and lo and behold, I found where the Handler was being added. Take a look here at JceSecurity. It seems that due to security, they want to statically connect to http://null.sun.com. Due to this, the HTTP Handler gets added and will later on be used to connect to our fake endpoint.

To get around this, there were a couple options.

  1. Somehow clear the handler HashTable
  2. Workaround the whole issue by changing the endpoint in my WSDL from http to https

Since URL didn’t provide any means to clear the table in any clean fashion, I decided that since this is a mere test, I’d simply change the http to https and be done with it. The tests then passed.

So it took about a good day’s worth of work to figure out this out only to realize you can fix it by adding one single character. It’s just one of those days.