How to unit test J2ME applications?

While developing J2ME game “Arcadia” at Playscape Games, I have had to grapple with the question of how best to unit test J2ME code.

My eventual conclusion has been: don’t test in J2ME, test your code in normal J2SE instead. Its been a journey of discovery with some mistakes along the way.

J2ME Intergration Testing Phase

I started out 18 months ago with the open source J2MEUnit, which I then heavily modified. J2ME doesnt have reflection, so to invoke lots of tests in a row, I made them all implement a common interface, and dynamically loaded each test class in turn, using a config file. Clunky, no integrated IDE UI, but it works.

Three aspects of this approach, and one personal factor, pushed me towards coarse grained integration tests:

  • J2ME emulators are slow, so running many tests gets slow
  • Theres no Mock object support, so testing parts in isolation is more difficult
  • Adding each test was a bit of work: write the test in its own file, then register it in the config
  • I didnt understand that while integration tests are efficient to detect failure, they are brittle and don’t indicate why/where the failure has occurred.

I followed the approach for 15 months, accumulating about 35 integration tests. They caught lots of bugs, but I also noticed that they

  • broke easily when I added features (ie changed the code), soaking up masses of time to keep them all passing
  • often failed without giving me much idea why, and I would need long followup sessions with a debugger to zoom in to the problem.

I’ve had similar reinforcing experiences in my work for IBS, to the point where I rather hate Integration tests, and in future will try to get by with the bare minimum of them.

J2SE Unit Testing Phase

It was obvious thatI was much more productive when unit testing with J2SE:

  • Fast running fine grained tests and especially the excellent Eclipse IDE JUnit support
  • Access to quality debuggers – J2ME emulator debuggers are slow, unstable and lack features
  • Use of Mocking frameworks where needed

But running J2ME in J2SE is not easy! J2ME is not a clean subset of J2SE; many of its classes (eg user interfaces and record stores) are unique to it. Furthermore, when these classes are instatiated in J2SE, they blow up because they make native calls that dont resolve.

I played with relying on model-view separation to unit test just my model in J2SE, but they are too coupled for some good reasons.

Instead, an effective solution was to create a set of stub classes that shadow the key J2ME API objects: Canvas, Display, Font, etc, but do very little or nothing. When the unit tests run, the shadow classes come first in the classpath, causing the game code to run headless but correctly. Its even capable of exersizing the screen paint routines, although nothing ever gets drawn.

The effort involved was the kind of signficant investment you make only after trying the easier options and discovering their unpalatable aftertaste.

Stub files are available

Following several requests, I have prepared a zip file containing the stub versions of J2ME classes. WordPress wont let me host a zip file for download, so until I organize something better, leave your (encoded) email address in a comment and I’ll send the archive to you.

Notes

  • It is not the complete set, just the classes I needed to stub for my tests.
  • If you end up using and adding to it, please contribute your enhancements back to me.
  • The Display class is the only class needing explanation, because its more than a passive stub. It provides a simulation of the task execution facility provided by Display.callSerially() in the MIDP API. Call Display.start() if you want to activate the task queue.

9 Comments

  1. claes said,

    July 17, 2008 at 6:16 pm

    Hi,

    I was just wondering, are those stub classes available for download somewhere?

    cheers

    /claes

  2. krishna said,

    December 2, 2008 at 12:50 pm

    hey, first of all.. nice work done. And pretty nice of u to be sharing the hard work u have done. Thanks in advance for the stub classes. I will definitely send u back my addition to them. Please send them to my mail id: krishna797979@gmail.com. Also, let me know if you want to start a free source forge kind of a project on this which will be helpfull to all.

  3. Achim said,

    January 17, 2009 at 1:18 am

    Hi Ben,

    I’m using JMUnit and the Hammock mock objects which are quite useful:
    http://hammingweight.com/modules/hammock/
    but are missing a RecordStore mock class.
    Do you have a stub class for RecordStore? Please send me your stub classes by mail (to the address entered in the comment form).

    Cheers
    Achim

  4. Vicenzo said,

    January 21, 2009 at 2:23 am

    HI, I´m try to learn how to Unit Test J2ME applications and I would like to try out your method, since i know people that work with J2SE testing already.

    Thanks a lot
    (email included in the comment form)

  5. ShaneG said,

    February 17, 2009 at 1:38 pm

    Hi Ben,

    Thanks for the nice article. I’ve been looking at going down the same road myself. I would love to have a copy of the stub classes (my email is in the comment field).

    I’d like to try and integrate them into a simple test framework I’ve developed (the goal is to be able to run the tests both on the device/emulator and under a J2SE environment for build testing).

    Thanks,
    Shane

  6. Markus said,

    February 17, 2009 at 11:37 pm

    Hi Ben,

    sounds great !!!
    Would you be so kind to send me the stub to blowinaway@hotmail.com

    Thanks
    Markus

  7. Puddy said,

    February 18, 2009 at 7:54 am

    Hey Ben,
    I was wondering, have you hear of the Jmunit testing framework. It’s really cool and provides an on device test result page. Also, the developer of this framework also created a mocking framework called hammock for the javaME platform. You can download it at http://sourceforge.net/projects/hammockmocks

  8. Julius Martin Dadis said,

    July 29, 2013 at 12:49 pm

    Hello Ben,

    Thank you for your article. Kindly send me the stub. I want to try it. jmart.wiz32@gmail.com

  9. Karl said,

    January 23, 2015 at 7:35 pm

    Hi Ben,
    Thank You for making this article. Kindly send me the stub classes. Thank a lot.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: