How to unit test J2ME applications?

While developing J2ME game “Arcadia” at Playscape Games, I have had to grapple with the question of how best to unit test J2ME code.

My eventual conclusion has been: don’t test in J2ME, test your code in normal J2SE instead. Its been a journey of discovery with some mistakes along the way.

J2ME Intergration Testing Phase

I started out 18 months ago with the open source J2MEUnit, which I then heavily modified. J2ME doesnt have reflection, so to invoke lots of tests in a row, I made them all implement a common interface, and dynamically loaded each test class in turn, using a config file. Clunky, no integrated IDE UI, but it works.

Three aspects of this approach, and one personal factor, pushed me towards coarse grained integration tests:

  • J2ME emulators are slow, so running many tests gets slow
  • Theres no Mock object support, so testing parts in isolation is more difficult
  • Adding each test was a bit of work: write the test in its own file, then register it in the config
  • I didnt understand that while integration tests are efficient to detect failure, they are brittle and don’t indicate why/where the failure has occurred.

I followed the approach for 15 months, accumulating about 35 integration tests. They caught lots of bugs, but I also noticed that they

  • broke easily when I added features (ie changed the code), soaking up masses of time to keep them all passing
  • often failed without giving me much idea why, and I would need long followup sessions with a debugger to zoom in to the problem.

I’ve had similar reinforcing experiences in my work for IBS, to the point where I rather hate Integration tests, and in future will try to get by with the bare minimum of them.

J2SE Unit Testing Phase

It was obvious thatI was much more productive when unit testing with J2SE:

  • Fast running fine grained tests and especially the excellent Eclipse IDE JUnit support
  • Access to quality debuggers – J2ME emulator debuggers are slow, unstable and lack features
  • Use of Mocking frameworks where needed

But running J2ME in J2SE is not easy! J2ME is not a clean subset of J2SE; many of its classes (eg user interfaces and record stores) are unique to it. Furthermore, when these classes are instatiated in J2SE, they blow up because they make native calls that dont resolve.

I played with relying on model-view separation to unit test just my model in J2SE, but they are too coupled for some good reasons.

Instead, an effective solution was to create a set of stub classes that shadow the key J2ME API objects: Canvas, Display, Font, etc, but do very little or nothing. When the unit tests run, the shadow classes come first in the classpath, causing the game code to run headless but correctly. Its even capable of exersizing the screen paint routines, although nothing ever gets drawn.

The effort involved was the kind of signficant investment you make only after trying the easier options and discovering their unpalatable aftertaste.

Stub files are available

Following several requests, I have prepared a zip file containing the stub versions of J2ME classes. WordPress wont let me host a zip file for download, so until I organize something better, leave your (encoded) email address in a comment and I’ll send the archive to you.

Notes

  • It is not the complete set, just the classes I needed to stub for my tests.
  • If you end up using and adding to it, please contribute your enhancements back to me.
  • The Display class is the only class needing explanation, because its more than a passive stub. It provides a simulation of the task execution facility provided by Display.callSerially() in the MIDP API. Call Display.start() if you want to activate the task queue.
Advertisements

J2ME Memory Profiling

Recently during the development of J2ME app Arcadia at Playscape Games, we have bumped up against the memory limits of our test handsets, which have heap sizes starting at 800Kb. I don’t believe in premature optimization, and so previously I had done little to measure & conserve memory beyond following a simple but crucial heuristic:

Ensure the objects that are most numerous in your app use memory most efficiently

I’ve now spent some time learning more about Arcadia and general J2ME memory consumption.

Findings from the WTK Memory Profiler

  • The Memory Profiler provided in Sun’s Wireless Toolkit 2.5 (WTK) is invaluable! It shows instance counts, instance consumption, and also allocation code sites on the right-hand side (a nice surprise).
  • Primitive types boolean, byte and short use 4 bytes in the VM (same as an int), even when grouped together, making them useless from a heap conservation standpoint.
  • Some objects sizes in bytes (ignoring contained objects), as reported by the Emulator (but very likely similar in most JVMs)
    • Object 12
    • String 24
    • Integer, Boolean 16
    • Vector, Hashtable 24 (plus internal arrays ?16+ contents?)

Fragmented & Unreclaimable Memory

Memory fragmentation or unreclaimable memory (for Image objects on Samsung handsets) has previously been reported on KVM-Users list and J2ME.org, whereby some J2ME app cannot effectively recover freed memory.

I havent yet found any evidence of this on our test handsets (SE K500, Nokia 6233, Motorola V3, Sansung D500), but its hard to tell definitively.

According to this paper, Suns KVM (presumably the basis/inspiration for many handset VM impls) provides a compacting Mark-Sweep garbage collector.

Load & Save

  • There is a transient spike in memory usage while levels are loading and saving. I suspect this will tend to be a recurrent architecural issue.
  • Data must be loaded from a MIDP RecordStore as byte[], which is then parsed and allocated into objects. During this process, the level data is effectively duplicated twice in memory, and only after the level is fully loaded is the byte array discarded. A potential solution is an improved DiscardingByteArrayInputStream, which discards its contents once they have been read(). It would need multiple chunked byte[] buckets internally to do this.

    Arcadia-specific optimizations

    • Locations in the world map are the most numerous objects and the biggest consumer of memory. I added a lazy-initialisation mechanism so that empty locations dont eagerly allocate space for potential occupants. Big win.
    • Unloading unused sprites has also helped. I introduced a lazy-load mechanism there as well.
    • Condensing multiple boolean and byte fields into ints
    • Reducing the number of audio Player objects in the game by discarding low value sound effects.

    Game Developer Confererence 2007 Roundup

    I gave at roundup of this year’s Game Developer Conference at Melbourne gamedev event Dissecta a fortnight ago. See the attached slides [game-developer-conference-07.ppt] from the talk for the details, but here are my picks for this years trends in the game industry in 30 seconds.

    • User-created content and user-driven gameplay (eg Spore, Little Big Planet). See also Web 2.0.
    • Social, connected gaming (XBox Live, Sony Home, Little Big Planet).
    • Casual, Browser and Downloadable games (eg BigFish, XBox Live, Nintendo Wii, Steam).
    • Multithreading (eg Valve Source engine)