My Digital Neuron

While developing J2ME game “Arcadia” at Playscape Games, I have had to grapple with the question of how best to unit test J2ME code.

My eventual conclusion has been: don’t test in J2ME, test your code in normal J2SE instead. Its been a journey of discovery with some mistakes along the way.

J2ME Intergration Testing Phase

I started out 18 months ago with the open source J2MEUnit, which I then heavily modified. J2ME doesnt have reflection, so to invoke lots of tests in a row, I made them all implement a common interface, and dynamically loaded each test class in turn, using a config file. Clunky, no integrated IDE UI, but it works.

Three aspects of this approach, and one personal factor, pushed me towards coarse grained integration tests:

• J2ME emulators are slow, so running many tests gets slow
• Theres no Mock object support, so testing parts in isolation is more difficult
• Adding each test was a bit of work: write the test in its own file, then register it in the config
• I didnt understand that while integration tests are efficient to detect failure, they are brittle and don’t indicate why/where the failure has occurred.

I followed the approach for 15 months, accumulating about 35 integration tests. They caught lots of bugs, but I also noticed that they

• broke easily when I added features (ie changed the code), soaking up masses of time to keep them all passing
• often failed without giving me much idea why, and I would need long followup sessions with a debugger to zoom in to the problem.

I’ve had similar reinforcing experiences in my work for IBS, to the point where I rather hate Integration tests, and in future will try to get by with the bare minimum of them.

J2SE Unit Testing Phase

It was obvious thatI was much more productive when unit testing with J2SE:

• Fast running fine grained tests and especially the excellent Eclipse IDE JUnit support
• Access to quality debuggers – J2ME emulator debuggers are slow, unstable and lack features
• Use of Mocking frameworks where needed

But running J2ME in J2SE is not easy! J2ME is not a clean subset of J2SE; many of its classes (eg user interfaces and record stores) are unique to it. Furthermore, when these classes are instatiated in J2SE, they blow up because they make native calls that dont resolve.

I played with relying on model-view separation to unit test just my model in J2SE, but they are too coupled for some good reasons.

Instead, an effective solution was to create a set of stub classes that shadow the key J2ME API objects: Canvas, Display, Font, etc, but do very little or nothing. When the unit tests run, the shadow classes come first in the classpath, causing the game code to run headless but correctly. Its even capable of exersizing the screen paint routines, although nothing ever gets drawn.

The effort involved was the kind of signficant investment you make only after trying the easier options and discovering their unpalatable aftertaste.

Stub files are available

Following several requests, I have prepared a zip file containing the stub versions of J2ME classes. WordPress wont let me host a zip file for download, so until I organize something better, leave your (encoded) email address in a comment and I’ll send the archive to you.

Notes

• It is not the complete set, just the classes I needed to stub for my tests.
• If you end up using and adding to it, please contribute your enhancements back to me.
• The Display class is the only class needing explanation, because its more than a passive stub. It provides a simulation of the task execution facility provided by Display.callSerially() in the MIDP API. Call Display.start() if you want to activate the task queue.