Annoying Java Bug/Limitation: Generic methods don’t combine well with static imports

package test;
import java.util.*;

import static test.StaticallyImported.*;

public class Test {

  public static void main(String[] args) {

    //passing Type Argument works fine
    HashMap m1 =
      StaticallyImported.<HashMap>aStaticallyImportedGenericMethod();

    //same thing, with static import syntax: compiler error
    HashMap m2 = <HashMap>aStaticallyImportedGenericMethod();
  }
}

class StaticallyImported {
  static <A extends Map> A aStaticallyImportedGenericMethod() {
    return null;
  }
}

Sun’s rejection of Delegates for Java

In 1998 Microsoft introduced a variant of Java called Visual J++ that included delegates, basically type-safe function pointers. Anders Hejlsberg was very involved with their development, and he went on to add them to C#.

This occurred against backdrop of a war between Sun and Microsoft, much of it over the non-compliance of the MS Java implementation with Sun’s license terms.

Sun rejected the idea of delegates, and a whitepaper attributed to “The Java Language Team”  [http://java.sun.com/docs/white/delegates.html]  was published justifying this decision on technical grounds, in no uncertain terms. Here the gist of its message:

“The newest version of the Microsoft Visual J++ development environment
supports a language construct called delegates or bound method
references
. … It is unlikely that the Java programming language will
ever include this construct. … bound method references are unnecessary
and detrimental to the language…. bound method references are
harmful because they detract from the simplicity of the Java
programming language and the pervasively object-oriented character of
the APIs. “

Java’s Alternative: Interfaces and Inner-classes?

The whitepaper describes what became the “Java-way” of representing pointers to functions

  • One interface is defined for each function signature, eg String => Integer
  • One inner class adapter is written that implements the interface and bridges the call to one underlying method being invoked.

The problem was, delegates (and similar ideas – functions-as-values, closures, blocks, lamdas) have proven to be a much more flexible and elegant than the Java-way. They are supported in virtually every language to become successful in the last decade. Except  Java, the worlds most popular language, still lacks them, much to many peoples frustration.

So, how did Sun get it wrong? They were presented a working implementation of with what’s now recognized as a good solution, but spurned it in favor of an inferior alternative.

The power of group allegiances to affect human judgment and clear thinking

Anders proposed a good idea for Java, but he worked at Microsoft.  It seemed the Sun-MS rift prevented technical people at Sun seeing, or perhaps admitting, that the delegate concept of a typesafe function-pointer had genuine merit.

The underlying tendency to unconsciously prefer ideas that originate from people in allied groupings, organizations or cultures, regardless of actual content, is very widespread (certainly not confined to Sun, as at minimum it afflicts me too). History is full of examples. While it encourages intra-group harmony and consistency, it interferes with our ability to distinguish truth from untruth, and thus slows our learning and progress, and ultimately, reduces our intelligence.

In this case, Java was a casualty of polarised group-think between two warring corporations.

Beautiful Music: Empire of the Sun

Aussie indie electro-pop band Empire of the Sun are making some beautiful music…

An indie game developer’s journey: from beta 1 to beta 2 and beyond…

This is the second of a series, here is Part 1.

After Heroes of Arcadia Beta 1 (Sept 2007), it took 13 months of development to reach Beta 2 in October 2008.

Towards a Story-driven RPG style…

We remade the game, essentially, in a lighter, more streamlined single avatar model where story counted for more and resource management for less.

Instead of requiring the player to manage the whole village concurrently, by default we focused on a party of adventurers led by a single Hero lead character.  The village became an NPC faction managed by the AI but allied to the player, that serves as the backdrop to scripted interactions with NPCs, each leading to quest sequences.

Scripted Content

The scripting itself is one of strongest innovations in the Arcadia engine. It supports a Domain Specific Language (expressed as a fluent API) for describing the logic and content that drives character interactions and game quests.

This language is encoded in a sequence of executable objects that are run in response to event triggers that fire within the Arcadia world model. Script objects are also Serializable, so they can be saved to- and loaded from- both the downloadable game package and the mobile device’s memory.

Single-Avatar Control

During beta 1, while watching play-testers try to play Arcadia, I observed a key problem that turn-based games with many concurrent avatars suffer from. The more avatars that a player controls concurrently, the longer it takes to take a full turn among them, and thus the slower of overall game goes.

So the intended reward of growing the number of units under player control can in fact become a curse, because the per-turn effort to play becomes unsustainable.

Our response to this challenge was to introduce a simple level-of-detail system into the player interface, so that they could switch between

  • Quest Mode: Controlling just their Hero, with the rest of the party running in an automatic “- Assist Mode -”, but with the option for the player to manually control of other party members when desired.
  • Tactical Mode: Controlling their whole Party in round robin fashion by default.

Removal of Resource Rules and Growth

We also removed the village operation, food production and human reproduction aspects of the game from player control. It just didn’t seem to fit in an adventure-oriented game where zones of in-world locations can be loaded into and out of memory, as the player travels through them.

If a player controlled the village unit’s, what would happen when his or her attention moves to another zone, and the village gets unloaded from memory? Presumably it waits fairly passively until the player re-enters the zone, at which time the player-aligned units indigenous to the zone re-join the player’s party. These are certainly big ideas to explore in a future game, but the the first release I wanted to keep it simple and focus on a small group of mobile adventurers.

Zones

Zones have been a key idea in Arcadia since the early days. Each is a named rectangular region of the game world, complete with inhabitants, quests and scripting, as large as can be comfortably fitted into the memory available to the game at any one time. Beta 2 saw us develop a robust method for traveling between them, while maintaining a history of past events previously visited zones..

Getting the player’s transition between Zones to work properly was some of the most difficult design and programming I have yet had to do in my career.The main challenge was determining what objects in the current zone’s World Object-Graph are player owned- and which ones are zone local-, and devising a  method to separate them when a transition occurs. A transition can result in dangling references where zone-local objects (eg a monster) were referencing a player-owned object (eg party member), that was then suddenly removed. Ensuring such situations were handled gracefully was one of the trickiest problems.

Zone transitions also required:

  • A PlayerProfile record to store the player’s current and on-going transferable state, including the Hero character and their traits, inventory and effects, all other party members likewise, XP and Monetary wealth, and the state of any active or completed multizone Quests.
  • Re-combining the PlayerProfile and the incoming Zone’s local- objects when a transition occurs, by injecting the wealth, units and quest state into the appropriate points in the world Domain Model — though this is much easier than the extraction process.
  • A reasonable process of logging and error recovery for the transition process, which involves 2 large -reads and 2 large -writes to the mobile device’s store, as well as a major memory reclamation and re-allocation, and is thus prone to failure.

Beta 2 Process

For beta 2, I again used something like a dozen workmates, friends and acquaintances who had expressed interest in the game.

As a general heuristic, I found that I tended to get good feedback from a given play-tester only on one release version / trial, though there were exceptions. Feedback fatigue seems to take place when you ask the same tester to evaluate multiple editions of the game, if they are at-all similar.

Something I tried for this release was a web survey form to collect responses, via SurveyMonkey.com. I felt the survey helped to harvest good constructive feedback, though observing people play your game, and listening to their verbal accounts of the experience, are probably the most important activities. I did need to remind people to complete the form though, and even then only about half filled it in.

The road ahead…

  • Improving the experience in the first 5-minutes of the game.
    • Popup tutorial/info boxes that teach the player the game concepts and interface unobtrusively as they play, drip feeding them enough info to progress without forcing them to learn everything upfront.
    • A 60-120s scripted “Woodhaven Village Raid” scene at game start, which ends with heroine Eleanor hiding in the bushes from the raiding orcs.
  • Better player character growth through expandable unit abilities, purchased from earned XP.

An indie game developer’s journey: The road to Arcadia’s first beta release

Inception       Jan 2005

I had a month off between jobs, and spent much of it in the beautiful and inspring surrounds of Sherbroke in the Dandenong Ranges, a hilly region of cool temparate rainforest 40 kms east of Melbourne. During the time of relaxation, I amused myself by learning to work with J2ME, and chose to build a toy game to drive my own discovery.

Arcadia was born, through when it began it was simply a motionless grid of self-made terrain tiles.

Go for Funding!     February 2005

From February my wife Louise starting trying to convince me to make an application for funding from Film Victoria’s Digital Media Fund.

May 2005          Putting a Funding Application together

Fortunately she prevailed and by May we were both hard at work putting an application document together.

Here’s what we said we were going to try to do:

“Playscape Games is making this application to fund the development of the first episode of mobile phone game-series Arcadia to a feature-complete prototype stage. Our vision for Arcadia is to create a Java mobile game of unusual depth and richness, which innovates in its game design, usability, and technical engineering. It will be one of the first mobile titles developed with a turn-based strategy/adventure format, in a fantasy/mythic setting. It will appeal to gamers who want to play a more deeply engaging game over a longer time period than the current staple of short-duration action games can provide.”

July 2005   Our primary artist Albert Sandaja (aka Alex Huang) signs on to start designing sprites for us.

September 2005     You’ve Got Funding!

…made especially sweet by the fact that the message arrived while I was having a lovely time on holiday staying with Louise’s uncle Phillip on Bribie Island.

November 2005    The first tranche of our funding, approximately $11k, gets paid to us.

Jan 2006      Leave Goldman Sachs JB Were behind to work a 3-day-week at Internet Business System as my day job. Turned out well, as Im still there now almost 3 years later. This freed up 2 days a week to put towards Arcadia (at least in theory – distractions were and are a major problem).

Feb 2006 Attend GDC Mobile in San Jose. An eye opener.

April 2006  Milestone 1

Deliver 40 page Business Plan to Film Vic. Main objective is to become a niche provider of very high quality and innovative RPG & Strategy content, distributed through a mixed aray of channels, into a variety of global markets, including direct online sales. In response, the Full funding program confirmed, and another trnache paid.

January 2007 Milestone 2

This was a tough milestone to pass. We demoed our playable prototype to Amelia King of Film Vic. It had lots of rough spots and while it technically worked, broken game mechanics and poor usability made it only barely playable. We agreed on some improvements to target for the final milestone.

Feb 2007   GDC in San Francisco.

I learned more at this second conference. I started to feel the strength of the competition in the space, but I also saw alot of techniques I realsed I could apply in Arcadia.

August 2007    Final milestone, which led onto…

Sept 2007     Beta 1

I used friends, family and work mates as the beta testers.

Went Well

* Shipped a working mobile game that ran decently without crashing much on a number of different phones, including Nokia, SE, Motorola and Samsungs.

* Main map view looked good and clearly showed what was happening.

* Fantastic bestiary of animal and monster sprites, from snakes and spiders through to tribal orc bands, huge bears, wyverns and cyclops. Thanks are due to Alex Huang, the talented artist who drew the set.

* People able to use the menu system fairly intuitively.

* MIDI Music worked well. Very nice musical selection created by James Langford.

* Interesting resource system based around farming to produce food that allows your village population to grow and evolve.

Do Differently

* Confusing to play: lots of controls, not enough help, steep learning curve.

* The game involved controlling the whole village’s units; an uneasy mixture of Hero characters and their party members on one side, and peaceful farming civilians on the other.

* Control of multiple units concurrently involved lots of flicking back-and-forth as each unit was moved once per turn in a round-robin fashion. This was hard for people to learn. A tendency made stronger, I feel, by the small size of a mobile screen and limited controls.

* The problem of concurrent unit control was addressed by way of an Automation feature, which allowed units to be put on a program The most useful programs were Escort, which sent a unit following and fighting with a designated leader, and Tend Fields, which keep the food supply ticking over back home. This released the need for constant manual control.

However, most beta players were not able to discover and use Automation feature, because of unintuitive controls.

After Beta 1, it took 13 months of development to reach Beta 2 in October 2008. Having finished our Digital Media Fund program at Beta 1, this period was marked by greater fiscal dicipline. I’ll try to cover the Beta 2 release and the present state-of-play in a future post.

At least five things got us to Beta 1

  • Talent: do what you love, do what you’re good at, do it well and with passion.
  • Help from others, eg my lovely wife, and the Digital Media Fund.
  • Luck, eg meeting Albert at the start of the project
  • Hard Work,  lots of evenings and weekends spent on the project.
  • Tenacity:  You have to Suffer for your art. When things get tough, just Keep Going. Dont give up.

Wiki Software for Java and Tomcat: The search continues

I eventually want to create a Blog/Wiki hybrid residing at http://www.benhutchison.com A JAMWiki (see below) site is running at the URL now, but it is temporary.

This need drove me to consider Java-based Wiki software that would run on Tomcat.

I installed, configured and tested 3 packages:

Of the three, JAMWiki seems to be the best, but sadly, still not adequate for my needs. On the plus side, it was easy to install and use for everyday Wiki stuff, and is MediaWiki compatible. Its technology stack is based on Spring 2.5 and JSP, and the code is code is neat and java-doc’ed. The 0.6.7 version tag understates its maturity.

However, when I dug deeper I found that internally, it uses a very old-fashioned DB-centric (anemic) domain model. A fair bit of the business logic is encoded into SQL queries. Plenty of code duplication and verbosity. No ORM. Poorly abstracted – lots of individual servlets take care of rendering particular pages, making the system hard to re-compose or re-configure. Ultimately, dissappointing.

I have used it for a while for http://www.benhutchison.com , but am going to move on once I find/build something better /more flexible.

XWiki has a sizeable community but I found it was too big, heavy and complex for my taste – the 45Mb War consumed masses of memory but was seemed short on useful, useable features.

JSPWiki has amateurish UI & docs (and internal code, I suspect)  – I would advise steering clear of it.

So sadly, I dont know of any Java -based Wiki engine I would recommend right now, if you want a degree of customization.

Understanding Type Parameters, Covariance and Contravariance

Parametrizing code with types will be part of almost every future statically typed language. I think any programmer serious about working with static typing needs to have an understanding of the conceptual territory:

  • What does Type Variance mean?
  • Covariance versus Contravariance
  • Definition-site versus Use-site Variance

I recommend Barry Kelly’s post as an nice summary of the issues:

http://barrkel.blogspot.com/2006/07/covariance-and-contravariance-in-net.html

How to Configure Tomcat root context

Tomcat has a horrible gotcha around the seemingly simple task of deploying a web-app to the so called “root context”, ie at <hostname>/<webapp>.

Either

  1. Define a ROOT.xml context file in conf/Catalina/localhost
  2. Name your webapp WAR “ROOT.war” or containing folder “ROOT”

(Note: I mean, literally, it must be named “ROOT”. Case sensitive)

Note that advice given in Tomcat docs and elsewhere on setting the context in <webapp>/META-INF/context.xml can be misleading: you cannot deploy to root context simply by using this setting.

Cause

The path attribute, which configures deployment to root context, is ignored outside conf/server.xml. See the docs for path attribute, especially 2nd paragraph. I think this was introduced with Tomcat 5.5.

Why is this poor design?

Well, it seems to run misdirect the “Convention over Configuration” principle that has proven so powerful in recent years. While inferring the context from the filename is a convention, the principle states that configuration should be optional and when present override the convention. In Tomcat’s case, the configuration (ie a “path” attribute) is silently ignored when specified.

The impact is that you must change your War filename or deployment folder, with resultant impacts on build processes & tools, just to vary its deployment URL – a form of inappropriate coupling. Even though there is a setting defined that would allow the artefact name and deployent URL to be decoupled.

I dont know what they were thinking…

Update: Is Glassfish v2 a better alternative?

July to October 2008, I trialed using Glassfish v2 instead of Tomcat, to run 3 small-medium Java webapps. Glassfish was pretty good, but I am now going back to Tomcat, as it seems the better overall option.

While I found the Glassfish Admin user-interface very convenient, the app-server proved slow and memory hungry compared with Tomcat. I also had problems getting virtual hosting to work properly under Glassfish.

I think the situation could change with Glassfish 3, currently in beta, as it is much more modular and can run webapps in a much more lightweight configuration.

Retooling for Scala

As an undergraduate at Melbourne Uni in the mid 90s, we mainly used C with vi on slow, ugly unix terminals. I thought it sucked.  When Java appeared on my radar in 96 I was in 3rd year, and I immediately jumped ship to get onto a higher-level, OO language. 12 years later, and I’d call myself a “native speaker” – Im familiar with almost every corner (and almost every shortcoming) of the language.

Anyway, Java’s weaknesses are well documented elsewhere, so I wont bore you. Ive been searching for the exit for some years, and have finally found my ticket into the next generation, something to get excited and inspired by, something worth comprehensively rewiring my brain for, something to love. Its Scala.

If at first you dont succeed…

I only got into Scala on my second attempt, and even then, the realisation of how good a programing language it is came only slowly. Now it rushes like a torrent in my head, opening new vistas and resetting expectations about how easy programming should be.

The first time, a year ago, its weird swiss-academic accent put me off; eg the name: type syntax, underscore wildcards._, Bottom Types, and the inability of the Eclipse plugin to auto-import like Java does. “Not for me”, I thought, “too strange and theoretical!”

So I followed Groovy, JRuby, C# and Boo instead. But none of them were right for me. C# and Boo dont run on the JVM. And the weak type systems in Groovy and JRuby is something I cannot comfortably swallow, because deep down I’m convinced types represent a fundamental programming truth, and thus should be modelled by the language.

…try, and try again

So I came back and made a second attempt at Scala. I bought Programming in Scala (which is a great beginner book) and I listened to an inspiring talk by Scala creator Martin Odersky. And gradually, I was able to see beyond the strange accent and appreciate that Scala delivers a cutting-edge, ultra-high-level, performant language thats widely usable in the real world, because its so thouroughly compatible with Java and the JVM.

If you’re considering Scala, realize its a very deep, very intelligently designed language. Like me, you might not grok it without considerable effort… but once you do, the rewards are worth it.

Convert Java into C#

Rodrigo Di Olivera is a powerfully creative man, of relatively few words, who perhaps speaks loudest through the software he creates (including Boo, Prevayler).

Now he’s released a nice piece of programming alchemy: a tool that converts Java into C#.

One day I hope to port my Arcadia RPG engine to Windows Mobile (.NET Compact Framework). I expect this tool has just made that task almost trivial, given that Arcadia makes very few demands of its host environment’s APIs.

« Older entries Newer entries »