J2ME Memory Profiling

Recently during the development of J2ME app Arcadia at Playscape Games, we have bumped up against the memory limits of our test handsets, which have heap sizes starting at 800Kb. I don’t believe in premature optimization, and so previously I had done little to measure & conserve memory beyond following a simple but crucial heuristic:

Ensure the objects that are most numerous in your app use memory most efficiently

I’ve now spent some time learning more about Arcadia and general J2ME memory consumption.

Findings from the WTK Memory Profiler

  • The Memory Profiler provided in Sun’s Wireless Toolkit 2.5 (WTK) is invaluable! It shows instance counts, instance consumption, and also allocation code sites on the right-hand side (a nice surprise).
  • Primitive types boolean, byte and short use 4 bytes in the VM (same as an int), even when grouped together, making them useless from a heap conservation standpoint.
  • Some objects sizes in bytes (ignoring contained objects), as reported by the Emulator (but very likely similar in most JVMs)
    • Object 12
    • String 24
    • Integer, Boolean 16
    • Vector, Hashtable 24 (plus internal arrays ?16+ contents?)

Fragmented & Unreclaimable Memory

Memory fragmentation or unreclaimable memory (for Image objects on Samsung handsets) has previously been reported on KVM-Users list and J2ME.org, whereby some J2ME app cannot effectively recover freed memory.

I havent yet found any evidence of this on our test handsets (SE K500, Nokia 6233, Motorola V3, Sansung D500), but its hard to tell definitively.

According to this paper, Suns KVM (presumably the basis/inspiration for many handset VM impls) provides a compacting Mark-Sweep garbage collector.

Load & Save

  • There is a transient spike in memory usage while levels are loading and saving. I suspect this will tend to be a recurrent architecural issue.
  • Data must be loaded from a MIDP RecordStore as byte[], which is then parsed and allocated into objects. During this process, the level data is effectively duplicated twice in memory, and only after the level is fully loaded is the byte array discarded. A potential solution is an improved DiscardingByteArrayInputStream, which discards its contents once they have been read(). It would need multiple chunked byte[] buckets internally to do this.

    Arcadia-specific optimizations

    • Locations in the world map are the most numerous objects and the biggest consumer of memory. I added a lazy-initialisation mechanism so that empty locations dont eagerly allocate space for potential occupants. Big win.
    • Unloading unused sprites has also helped. I introduced a lazy-load mechanism there as well.
    • Condensing multiple boolean and byte fields into ints
    • Reducing the number of audio Player objects in the game by discarding low value sound effects.

    Lord of the Rings Online: My experience

    I played Lord of the Rings Online (LotRO) for about 2 weeks during the free beta. Overall, I enjoyed the experience, yet felt the game fell short of what I hoped for. Over 2 years after Blizzard’s World Of Warcraft, Turbine have created something that feels like a clone of WoW’s world, interface and gameplay, yet with less humour and slightly clunky in the little details.

    The Good

    • LotRO copies WoW’s user-interface, and that works well. Consistency amongst game interfaces is a good thing; there’s no need to re-invent the wheel. I found it very easy to immediately learn the basics of the game.
    • The environment of the Shire is quite cute, humourous and fun. Beforehand, I thought it would be a boring region of the game, but Ive enjoyed it.
    • I tried a night’s Monster Play (playing a monster on Sauron’s side) as an Uruk in the Ettinmoors and greatly enjoyed the stylistic and gameplay contrast.

    The Bad

    • I have a level 10 Elf Loremaster. Loremasters abilities seem to suck. They have the armour and weapon restrictions of a magic-using class, but at level 10 Ive only got 2 main combat-orineted spells, Burning Embers & Summon Raven, which I use endlessly. The other abilities are limp Buffs and Heals that have little attraction.
    • My first hour of gameplay was poor: despite choosing to play an Elf Loremaster I was corralled into Thorins courtyard running errands for Dwarves.
      • When entering my first monster battle in minute 3, entirely scripted and unavoidable, I pointed-and-clicked my default weapon at the appointed opponent, and we stood there hacking until I died. Dying in your learn-to-fight battle is not fun. If you’re going to have a compulsory first learning battle, make winning near certain. (Likely explanation: difficulty was not calibrated for Loremaster weaklings)
      • After first battle, all characters and places I’d learned about were scrapped, and I got teleported to big empty elf ruins, containing one elf, who told me to leave ruins and go somewhere else. There seems little point of having the ruins if you do nothing but leave them…
    • I find the terrain ridiculously and needlessly hilly: in Ered Luin I repeatedly fell off cliffs and had to take boring back-tracks to return to the road. The creases in the terrain force you to take tiresome round-about loops when trekking cross-country.
    • What dissapopnted me the most was the heavily constrained geography of Eriador. Unlike Frodo and co, in LOTR Online, it’s not possible to slip out of Hobbiton at dusk to go tramping over hill and dale. The Shire is a walled zone with 2 exits, one to Bree and one to Ered Luin (in the Beta, at least). All other directions are impassable. Just think how it would have made those Black Riders jobs easier; post a wraith at both exits and wait! Turbine’s middle earth has a graph topology: nodes connected by edges. Its a sharp contrast with the solid earth, open skies and freedom of 4 compass points that one feels while reading the travel sequences of the novel.

    Game Developer Confererence 2007 Roundup

    I gave at roundup of this year’s Game Developer Conference at Melbourne gamedev event Dissecta a fortnight ago. See the attached slides [game-developer-conference-07.ppt] from the talk for the details, but here are my picks for this years trends in the game industry in 30 seconds.

    • User-created content and user-driven gameplay (eg Spore, Little Big Planet). See also Web 2.0.
    • Social, connected gaming (XBox Live, Sony Home, Little Big Planet).
    • Casual, Browser and Downloadable games (eg BigFish, XBox Live, Nintendo Wii, Steam).
    • Multithreading (eg Valve Source engine)

    The mysterious Speed of Gravity

    How fast does gravity travel? When I was young I thought “instantly” while once I had some univeristy-level physics I thought: “at the speed of light” (like everything does, it seems sometimes). It’s actual speed has never yet been definitively measured by science.

    If the gravity force was instant then if we were clever we could devise a machine to sense the gravity waves given off by a distant moving object, a gravitational radio. With one, we could send information faster than light (FTL), and from there its a slippery slope to full FTL travel.

    I was recently startled to learn of some scientifically uncontested observations that naively suggest that gravity is instant, or near enough. Read the rest of this entry »