James Roper presents the Play Framework at Melbourne Scala group

James Roper, Typesafe’s technical lead for the Play Framework demonstrates building a webapp in 45 minutes, at the Melbourne Scala user group. April 2013.

Designing with Types in Scala

Design issues when using Static Typing in Scala

This was given at the Melbourne Scala User Group, in August 2012.

To get the most out of it, I recommend viewing the code that accompanies the talk:
https://github.com/benhutchison/ScalaTypeSystemTutorial/blob/master/src/main/scala/typestalk/TotalFunctions.scalaub.com/benhutchison/Scala…

Further wrinkles in The Lord of the Rings

Following on from my last post, here’s some more oddities in The Lord of the Rings:

Where are the all people?

From my understanding of history, most areas of our earth that were agriculturally viable we’re populated to some degree, once humans arrived. But most of Middle Earth is inexplicably empty. Even in the Shire, one of the most densely populated places, they didn’t meet a single traveller on the road the whole way to the Brandywine, save for Merry. Most of Eriador is a vast Wilderness. Another example is Rohan: Aragorn/Legolas/Gimli cross this supposedly populated nation without ever bumping into anyone.

This de-population of the world actually works well the first couple of times you read the book. It ensures that each and every person you meet is relevant to the story (with the unfortunate exception of Bombadil). But from about the third reading, you start to notice the emptiness of the place.

…and what do they eat?

Apart from the Shire, the Pelennor Fields, and a tiny bit of land around Isengard, no one in Middle Earth seems to grow any food or plough any fields. Where is the food grown that feeds the household of Rivendell? Or Bree, for that matter.

 

For some absurd reason, I became fascinated by these two flaws in the book while I was recently reading it to my son.

I now have this fantasy (in an alternate lifetime) where I do a machinima [http://en.wikipedia.org/wiki/Machinima] interpretation of the Lord of the Rings – my version. In many ways, it will follow the book’s plotline much more faithfully than Peter Jackson did. But in subtle ways, it will be different: there will be inhabitants, huts, farms, fields. There will be side roads, lanes and paths. Shepherds and livestocks.

And everyone will need to grow, gather or hunt what they eat, somehow. Not just the humans, but subterreaneans like Orcs and Dwarves too. Edible fungi anyone? ~Termite Style~

Even the too-good-for-mortals Elves won’t e able to conjure lembas from starlight. They’ll have to grow Oats, or Barley or Corn, to bake it from. That means they’ll have to keep livestock to plough the earth. And even Arwen’s pretty little hands might occasionally be rostered on for mucking out the oxen’s barn.

The Lord of the Rings: the Boring Bits

This is one for Tolkien fans, amongst which I number.

I’ve been reading the Lord of the Rings, aloud, to my 8 year old son the past 6 months or so. A little bit at a time, perhaps 5 nights a week. It’s been alot of fun, an out-of-world experience that we’ve shared together.

But it’s undeniable that Lord of the Rings has some boring bits and some weak points:

Fellowship of the Ring, the finest of the three books, has the best signal-to-noise ratio with only 2 slow downs:

  • Tom Bombadil. This is the only part I omitted when reading to my son, because I find Bombadil such a meaningless, senseless blemish on the book. He has no relevance to the plot. I don’t like the Barrow Downs much either, for similar reasons.
  • Farewell to Lorien. All right, we’ve been in Lorien for 3 chapters now. Time to move on.
  • From a scan of the net, it seems not uncommon for people to say that Shadow of the Past, or Council of Elrond are boring. Very strange…. such readers must simply be reading the wrong book. Those 2 chapters are some of Tolkien’s finest work, and lie at the heart of Lord of the Rings.

The first half of The Two Towers is very well paced, with only one boring bit.

  • Treebeard, come on down. Yes, you might be a lovable good hearted ol’ tree-troll-thing, but the fact is, my man-plant, you and your interminable entmoot go on for too long.

..but in the second half things go less well:

  • While I don’t find Taming of Smeagol / Passage of the Marshes boring, is it somehow a come down after the gallant deeds of the western front. Frodo is about as much fun to be around as washing the dishes, and just as virtuous. Our hero Sam is good value, however, and holds the story together.
  • But the chapter that was truly whacked with the boring stick is Journey to the Crossroads. Um, exactly what happens? “The hobbits travel south and the sky gets darker. The end.” Now expand that out to 10+ pages please with some irrelevant descriptive text.

But, as any Tolkien fan knows, it’s Return of the King where the real nuggets of boredom are to be found. It’s as if, as the thing was conceived, the sheer enormity of what he had created began to get out of control. Whereas in Fellowship, we travel with the characters, at their shoulder, and share their meals, by the Return of the King, they are more distant. The language becomes more formal, spontaneous dialogue lessens.

  • Things start to get the boredom wobbles fast in chapter 1, Minas Tirith. Then the first half of The Passing of the Grey Company plods along.
  • Muster of Rohan rates a big score of the boreometer. A whole chapter devoted to one Red Arrow, home delivered, and Merry being told he’s got to stay behind.
  • Fortunately, the first book picks up once we reach the Siege of Gondor, and holds its pace through until The Black Gate Opens.

In the second half…

  • Ironically, the climatic chapter, Mount Doom, can get a little tedious: they walked til they couldn’t go a step further; but they did. They crawled. They groaned. They parched. They got carried. Maybe the intention is for the reader suffer alongside them…
  • I don’t know any other book with so many chapters after the climax: The Field of the Cormallen, The Steward and the King, and Many Partings. These chapters have a too much celebration, too much grandeur, too many trumpets and heralds and flaming jewels. Too much huffy Eowyn. And not enough plot tension.

 

  • … but then in Homeward Bound, just as you’re wondering why the book bothered to go on, there is the gentle but unmistakable feeling of plot tension re-appearing as we approach the excitement of The Scouring of the Shire. Its a great plot twist, although they have been away only 1 year, so the transformation of the shire into some totalitarian state has progressed awfully quickly.
  • I find the end of Saruman a bit goofy. First, Frodo’s limp-wristed pacifism suggests the author thinks killing Saruman would be wrong. But then he has Wormtongue kill him, and be killed himself, with little justification, suggesting that he, Tolkien, really wanted the pair of them dead all along. All goes to show that complete pacifism is an untenable moral position.
  • And a nice little surprise resolution at the end in the Grey Havens, that made me cry when I first heard the story as a child. I guess because Sam was separated from both Frodo and Gandalf, rather suddenly.
  • Back in those days, I hadn’t read the Appendices, and so didn’t know that Sam eventually get’s his ticket to Valinor. When I finally discovered that, it gave me a feeling of sweet relief something like when Gandalf re-appears in The White Rider. In Lord of the Ring, the good guys always win in the end.

 

 

 

 

Starcraft and Me

 

I play alot of Starcraft, perhaps 8 hours a week or so. It’s a popular evening recreation after my kids have gone to bed; a TV substitute. I much prefer to play 2v2 and 3v3 teams allied with friends, with voice chat over skype, but I play 1v1 when there’s no one I know online.

The social aspect is a big part of it’s appeal to me. Its brought me closer to the friends I regularly play with, both because we catch up on news while we’re playing, and because we have to work together to win games.

The level of play on the open leagues is very high. I’ve been regularly playing for nearly 2 years, on the Australia / SE Asia server, as a Terran under the name ‘bunge’, and I’m about Silver league standard. It must be daunting for new players to begin.

The game is very mentally stimulating and gets my adrenaline flowing. For 10-30 minutes you are required to constantly make decisions, act, react, outwit another human’s mind. In team games, there can be up to 8 humans minds contributing to the game, with concurrent battles going on all over the map. I like to think I’ve become a better thinker under time-pressure as a result of playing.

For the first 12 months of playing, I would have trouble sleeping for hours afterwards; not good, when you play in the evenings. It no longer has such an effect on me, but I do find myself reviewing what went wrong in a game, as I’m lying in bed going to sleep.

A Winning Streak, a Winning Strategy

Currently, I’m on my longest winning streak ever in Starcraft 2 league play, 11 of the past 12 games. The matching algorithm pairs you with opponents of similar skill level, so it’s hard to sustain wins for long periods.

In this case, the streak is associated with a particular strategy: fast cloaked Banshees, which I used in every one of those winning games. In higher leagues, players know how to defend them, but in my adopted home of Bronze & Silver leagues, they cause havoc.

Starcraft 2: on a winning streak!

My build:

  • Almost always wall off my base with a Barrack and 2 Supply Depots, to prevent scouts seeing my Banshee build
  • 13 Gas
  • 15 Orbital Command
  • Circa 16, Gas 2
  • Factory on 100 Gas
  • Continual marine and SCV production. Send a marine out to the towers and look around for anything unusual or vulnerable: Overlords, pylons, hidden expansions, proxy Barracks, etc. Place a marine in any dark corners of your base, to spot any incoming Drops or Warp-ins.
  • I pre-build a few Depots, so that once my Banshees are out I have some excess supply
  • Starport right after Factory
  • Factory builds a tech lab, then swaps off it so its ready for the starport to use.
  • Build a banshee as soon as Starport is hooked to the tech lab, and also I begin Cloaking research.
  • I carefully setup a multi-step rally flight path for my Banshees, so that they rally to my opponents mineral line. Aiming to avoid enemy held Xelnaga towers, hazards, etc.
  • Usually, by now you are building up minerals but are low on gas. So, once Banshee’s are out, start on
    • An expansion
    • Another Barracks
    • Engineering Bay
    • Turrets, if Mutalisks, Banshees or Dark Templars threats are suspected.
  • Produce 3-4 Banshees in all, while at home…
  • Transitioning out of Banshees into 2-base Marine-Marauder-Medivac

Using Banshees Well

  • Target workers as first priority
  • …except for Zerg, where a single queen can be killed off without needing cloaking
  • Turn on Cloaking when needed, turn off when no threats present
  • Target SCVs building turrets
  • Almost never take on anti-air structures with Detection: Turrets, Cannons or Spore Crawlers
  • Move Banshees around to find angles of attack that are not covered by defence
  • In team games, spread the Banshee love around between different opponents. I find that players who have not yet come under Banshee attack, will often be unprepared even if their team mates have been hard hit.
  • Preemptively pull wounded or energy depleted Banshees back into dead airpace to rest, before they are destroyed. Let the victim forget about them by waiting a few minutes, or attack elsewhere with a different Banshee. Then move back in again.
  • Cloaked Banshees continue to work very well late game in large team games. Spread them around, exploit the chaos and confusion.
  • Cloaked Banshees can also do duty clearing seige-tank clusters.
  • Even if your Banshees are not doing heavy damage directly, its remarkable how much they can distract and harras opponents, unhinging their own strategy and putting them into a reactive mode.
  • Run away from air-to-air units like Vikings and Mutas. They suppress banshees pretty well.

 

Reinventing Maybe/Option in Guava: get the basics right, please!

Google’s Guava open source library provides a class Optional<T> that holds zero or one value, as a superior alternative to null references. Its a certainly good idea, but also a very mature, well understood idea. Haskell has Maybe, Scala has Option.

For some inexplicable reason, Guava’s designer turn there back on prior art and wisdom with a lame dismissal: Optional is not intended as a direct analogue of any existing “option” or “maybe” construct from other programming environments, though it may bear some similarities.

“Some similarities”? Sure does! Its the same damn abstraction. How many abstractions can you get that consist of exactly 0..1 values of some type T? One, that’s how many.

And one of the most useful capabilities of this abstraction is that can be a monad; essentially, support a flatMap (of flattenTransform in Guava terms) operation. The flattening part is useful when you have an Optional<T> value, and a function from T to another Optional<S> value. If you simply transform the input, you get a Optional<Optional<T>> container as a result. Typically, you want to flatten any number of Optional wrappers down into a single Optional<S> layer. That is, either I have a result value of type S, or I don’t.

Sadly, Guava hobbled their Optional class by leaving out a flatten operation.

I find myself speculating on why they left it out, and suspect that it resulted from the kind of ignorance of the outside world that seems to common in Java programming culture. An attitude that its OK to reinvent a existing abstraction that has been widely used and well understood, and yet  ignore or dismiss any wisdom that comes from outside the Java ecosystem.

“Scala’s Type System: a tutorial” – Video’s from my Feb 2012 talk on Youtube

A tutorial on Scala’s Type System, presented by Ben Hutchison at the Melbourne Scala User Group in Feb 2012.

 

Part 1 covers structural vs nominal typing, type inference, parametric polymorphism, and type bounds.

 

Part 2 covers type members, compound and singleton types, type projection, type lambdas, going beyond inheritance hierarchies..

 

The code used in the talk is at:https://github.com/benhutchison/ScalaTypeSystemTutorial/blob/master/src/main/…

Heroscape 2.0 Slides from Melbourne Scala user group presentation

HeroscapeMkIICaseStudy

Attached are the slides from my April 2011 Melbourne Scala user group presentation, on “Heroscape 2.0:  A case study in using Scala for complex HTML document generation”

State of Scala 2010 slides from OSDC

StateOfScala

Here are the slides from a “State of Scala” talk I gave at OSDC in November 2010, summarizing where Scala is currently at.

http://2010.osdc.com.au/

In search of something better than Test-Driven Development

My previous post was perhaps a bit symbolic (or plain smart-arse) and in need some some explanation.

In simple terms, it is a lament about the shortcomings off Test- (aka Behavior) Driven Development (TDD or BDD), a testing practice that is widely considered current best practice for software engineering, or least my experience of doing it.

In this method, software is constructed by first building executable tests, or specifications, that function somewhat like moulds in the casting of metals or plastics; they determine form of the software by specifying very precisely how it must behave. This takes more of the time and effort; once the specifications are constructed, it is usually less effort to construct the software to satisfy them.

Its been discovered, by myself empirically, but also by many others, that using many fine-grained, non-overlapping, unit tests yields software and tests that are easier/cheaper to build and maintain. However, an uncanny thing happens to unit tests as they become fine-grained: they start to look eerily similar to, or perhaps mirror images of, the code they specify.

So, the mirrors halves of photo in the previous post represents the two sides of a system (in this case, the text on the sign): the code (on the left), and the tests (on the right). What I was seeking to highlight is how the information content of the system has not been increased by adding the tests. We have said the same thing twice: once, as code, and again, as tests over that code.

What we have added is redundancy, which serves to detect errors, just as redundancy does in error correcting memory or RAID disks.

So, to me, the essence of the TDD practices is actually to ensure redundancy in a system’s specification. When the 2 copies of the specification, Code and Tests, do not agree, we have a test failure that is easily detected.

My problems with present day TDD are that

  1. This redundancy is currently achieved by enormous manual effort. In the most efficient cases Ive seen where system behavior is 100% specified by tests, test code is typically 200% of the bulk of the code it specifies, and it can be far worse, 300-400% is not uncommon. In my experience, that consumes a great deal of effort/time/money.
  2. It seems very difficult to measure or control the level of redundancy in the system introduced by the TDD practices. There seems to be a tendency to prescribe a “one-size-fits-all” approach of 100% unit test redundancy (ie every line, every branch, every data case in code covered by a unit test) as being appropriate for every project.
    In contrast, when we use redundancy to control errors for information storage or transmission, the amount of redundancy is a parameter that we can freely vary, to achieve a trade-off between bulky robustness and lightweight delicateness. We need to find a similar variable dial for software testing practices.
  3. When we develop software in the TDD style, we are writing down the same pieces of information twice in 2 ledgers as we work: Write a test. Write the matching code. Execute and verify matching. Rinse and Repeat.
    For me, that feels tedious and unnatural. My instinct is to write it once, and see it execute, to engage in a flowing conversation with the compiler and the runtime.

Ive been thinking alot lately about what could replace TDD, to give us reliable software at more lightweight redundancy levels, in the 10-50% of code size range, rather than the 100%+ redundancy of textbook TDD. Some early ideas:

  • Static Typing
  • Dependent Types
  • Design by Contract (which seems partly isomorphic to the very powerful Property-Based Testing approaches)
  • Automatic type-driven generation of function inputs, so that function can be executed without explicitly writing tests (The other half of Property-Based Testing).
  • Inline executable Examples, expressed as annotations, that describe valid sample function inputs and corresponding outputs, much as tests do, but much closer to the code.

« Older entries