Scala and Lift – Status after six months

There are many choices to be made when starting a new project. In this post I’ll try to explain our technology choices and the experience we’ve had so far.

Background

One of the nice things about starting a new project in a startup is the freedom of choice when it comes to selecting platform and tools. You can spend an awful lot of time mulling between the different choices, but in the end, it is usually not the choice of programming language that kills a startup.

First, a little background. I’ve been programming for more than 15 years in C++,VB,C#, Java,Perl & PHP. The last 5-6 years it has been mostly enterprise Java. When I started at my last job (another startup) we had a product implemented in Java that we turned into a SaaS platform. The core remained in Java  and much of the web frontend was implemented in PHP. I really liked the productivity we got out of PHP as compared to the Java code. Very fast turnaround times. But somehow the language/platform doesn’t really  turn me on.

So many tools, so little time

So I knew I didn’t want to use pure Java or PHP,  in the end I ended up choosing between

• Ruby + Rails
• Python + Django
• Scala + Lift

Despite the success of Rails in the past few years, it never really clicked with me. Granted I haven’t used it for more than a few weeks, but it never really felt natural.

Same with Django. I’ve used python on several occasions for scripting tasks, but never really for webapps.

So I ended up with Scala and Lift. I looked briefly at Scala in 2007, liking what I saw, but didn’t use it for anything serious. In the end, what made the difference was:

1. Well known development and deployment platform (JVM, Build tools, IDEs, app servers etc)
2. Java interop, both for use of existing libraries but also as a fallback plan if everything failed.
3. Lift’s clean templating. We didn’t really need any of the real-time features (ie. comet support) in Lift, although the Ajax support looked nice.

Status

So here we are, 6 months after development began. How has it been? In short: Not bad. We have a platform up and running, productivity is pretty good and getting better all the time.

When compared with Java code, our Scala/Lift code is not very verbose, but compared to Rails & Django code I still feel it is (slightly) more verbose. I think this is because of the static typing that (despite type inferencing) requires quite a few type annotations. Otoh, when something type checks, it usually works 🙂

Scala

Pros

Scala the language has a lot of benefits, many of which have already been described in other articles on the net. Here are some others  that I find important:

• Very concise, you can skip most types, parentheses, semicolons etc. Except when using parameterized types (generics). Here’s Hello World in Scala (using the Application trait is considered bad mojo for various reasons but ignore that for this example :-))

  object HelloWorld extends Application {
    println("Hello World")
  }
  

• You can start coding as you would in Java and gradually ease into the Functional Programming paradigm (if you want)
• You can use all existing Java libraries
• Very nice testing frameworks such as Specs, ScalaTest & ScalaCheck
• You can write very concise, readable code. E.g we have a dataset to be charted. It has a number of dataseries each having a number of values. To get the total of all values, we can just write this method

  def sum = series.map(_.sum).foldLeft(0.0)(_+_)

which says, for each series, calculate the sum and then add all the results

• Lazy parameters make logging code (which I think is very important, but is responsible for a lot of boilerplate) very readable. In Java you usually guard calls to logging to avoid the expensive computation of the string to be logged:

if (log.isDebugEnabled() {
  log.debug("Here is the XML blob:"+xml)
}

in Scala (or rather, Lift) you can, without loss of performance, just write

  log.debug("Here is the XML blob:"+xml)

The string to be logged will not be computed if debug log statements should not be logged.

• Very easy to write meaningful DTOs. In Java, a lot of boiler plate is needed to make a class wih e.g. 3 properties. Often I would use an array (which will be untyped) or a tuple. Both suffer from the problem that you’ll have to know that the 2nd field is the customer id and the 3rd is the total amount. In Scala you can simply write a case-class and get named fields with typed access:

   case class MyRecord(id:Int, custNo:String, amount: Double)
  

Cons

There are also a number of drawbacks to using the Scala language

• Tooling. We’re using the Scala 2.7.x plugin for Eclipse. While it works, there are a lot of bugs that means you’ll often have to rebuild the entire project for your changes to be seen. The 2.7.x version is basically abandoned and all work is going on in the 2.8 version which is supposedly much better. Unfortunately, we can’t switch to 2.8 until all the other tools we use are running on 2.8 (which is not in beta yet)
• While type inference works, it’s not as powerful as in ML (and probably never will be. Scala’s type system has inheritance making it much harder to inference the correct types). It frequently requires you to add some type info. After a while you figure out where this is needed though.
• The Scala compiler is rather slow. Fortunately, the Eclipse incremental compiler works most of the time.
• The Scala book is very well written, but the API documentation is rather sparse. With most Java code I seldom have to use the source, but for the scala code it seems easier to start with the source.
• The syntax can get quite ugly until you recognize the patterns. This is a problem since the documentation is rather bad. Parameterized types can can be especially bad I think. This is an example from the scala-user mailing list:

implicit def TraversableBind[M[X] <: Traversable[X]] = new Bind[M] {
  def bind[A, B](r: M[A], f: A => M[B])(implicit w: CanBuild[B, M[B]]): M[B] = r.flatMap(f)(breakOut)
}

• No binary compatibility between Scala versions. This means that code compiled with Scala 2.7.3 is not (in general) compatible with code compiled with 2.7.5 and needs to be recompiled. This is not a big deal for our own code, but it does make it difficult to move to another version since all the Scala libraries used needs to be released first. I think this is also what has slowed down use of the forthcoming 2.8 release: people with sizable apps are waiting for the entire tool chain to be available and since 2.8 is not yet in beta, you need to agree on a version of 2.8 for everything.

Lift

Overall, I like the Lift framework and how it utilizes the Scala language. The fundamental approach to request handling seem very well thought out and makes it easy to handle both traditional web apps, Ajax, REST APIs etc. The focus on Lift seems to get things done, not so much to create the perfect web framework abstraction that has all corner cases covered. This means most code has been battle tested, but sometimes you’ll wander along an untrodden path and strange things will happen.

Pros

• Without a doubt, the best thing about Lift is the community. Very friendly people, fast response to most questions. Eager to get newbies up to speed.
• The Eclipse incremental compiler, Jetty & the JRebel plugin makes for fairly rapid turnaround times even if it’s not in the same league as e.g. PHP & Python. But many changes can be reloaded on the fly without having to restart the server.
• Very fast to get a real app up and running. Prebuilt archetypes for a basic app with authentication and DB access.
• Focus on security. Much effort has been put into making Lift apps secure. E.g. you cannot repost a form since the field ids are unique
• The Lift ORM, Mapper, is fairly simple, but does provide a very quick way to get CRUD functionality up and running.
• Very easy to add Ajax functionality (I can’t speak on the Comet functionality)
• Pretty good i18n support. Strings, templates etc. can be locale dependent and the locale can be computed for each request if needed.
• The clean separation between UI and code (it is impossible to have code in templates) makes it easy for designers to modify the layout and styling with breaking anything.
• The focus on getting things done means if there’s a real use case, new features can be added fairly quickly. Same goes for bugs in production code.

Cons

• API Documentation is very scarce. Usually, I use the source to figure out how stuff is working, but sometimes you need the big picture in order to get a good understanding. There is a free book available, but I usually don’t use this on a day to day basis.
• Form handling is very basic. There isn’t really any form specific code in Lift, so making a real, usable form (with i18n, validation, etc) takes some effort.
• Lift apps are very stateful. This is not a problem for us, but for apps that needs to be very horizontally scalable, this is a showstopper. The problem is not easily solved, since it requires serialization of closures.
• The Lift ORM, Mapper, is fairly simple, which means that for more complex scenarios it is somewhat difficult to extend, loosing some of the RAD capabilities
• The focus on getting things done means that some areas are not really consistent or complete.
• While you can’t get code into your templates, it’s easy to get UI into your code, which is (almost) just as bad. The dynamic part of the UI is done by snippets and they of course need to emit HTML. But it is easy to put all kinds of style, class attributes as well as other things which belong in the template, into this dynamically generated code. This makes it difficult for designers to modify the layout and styling without touching the Scala code.