Scala and Lift – Status after six months

There are many choices to be made when starting a new project. In this post I’ll try to explain our technology choices and the experience we’ve had so far.

Background

One of the nice things about starting a new project in a startup is the freedom of choice when it comes to selecting platform and tools. You can spend an awful lot of time mulling between the different choices, but in the end, it is usually not the choice of programming language that kills a startup.

First, a little background. I’ve been programming for more than 15 years in C++,VB,C#, Java,Perl & PHP. The last 5-6 years it has been mostly enterprise Java. When I started at my last job (another startup) we had a product implemented in Java that we turned into a SaaS platform. The core remained in Java  and much of the web frontend was implemented in PHP. I really liked the productivity we got out of PHP as compared to the Java code. Very fast turnaround times. But somehow the language/platform doesn’t really  turn me on.

So many tools, so little time

So I knew I didn’t want to use pure Java or PHP,  in the end I ended up choosing between

  • Ruby + Rails
  • Python + Django
  • Scala + Lift

Despite the success of Rails in the past few years, it never really clicked with me. Granted I haven’t used it for more than a few weeks, but it never really felt natural.

Same with Django. I’ve used python on several occasions for scripting tasks, but never really for webapps.

So I ended up with Scala and Lift. I looked briefly at Scala in 2007, liking what I saw, but didn’t use it for anything serious. In the end, what made the difference was:

  1. Well known development and deployment platform (JVM, Build tools, IDEs, app servers etc)
  2. Java interop, both for use of existing libraries but also as a fallback plan if everything failed.
  3. Lift’s clean templating. We didn’t really need any of the real-time features (ie. comet support) in Lift, although the Ajax support looked nice.

Status

So here we are, 6 months after development began. How has it been? In short: Not bad. We have a platform up and running, productivity is pretty good and getting better all the time.

When compared with Java code, our Scala/Lift code is not very verbose, but compared to Rails & Django code I still feel it is (slightly) more verbose. I think this is because of the static typing that (despite type inferencing) requires quite a few type annotations. Otoh, when something type checks, it usually works :-)

Scala

Pros

Scala the language has a lot of benefits, many of which have already been described in other articles on the net. Here are some others  that I find important:

  • Very concise, you can skip most types, parentheses, semicolons etc. Except when using parameterized types (generics). Here’s Hello World in Scala (using the Application trait is considered bad mojo for various reasons but ignore that for this example :-))
    object HelloWorld extends Application {
      println("Hello World")
    }
    
  • You can start coding as you would in Java and gradually ease into the Functional Programming paradigm (if you want)
  • You can use all existing Java libraries
  • Very nice testing frameworks such as Specs, ScalaTest & ScalaCheck
  • You can write very concise, readable code. E.g we have a dataset to be charted. It has a number of dataseries each having a number of values. To get the total of all values, we can just write this method
  •   def sum = series.map(_.sum).foldLeft(0.0)(_+_)
    

    which says, for each series, calculate the sum and then add all the results

  • Lazy parameters make logging code (which I think is very important, but is responsible for a lot of boilerplate) very readable. In Java you usually guard calls to logging to avoid the expensive computation of the string to be logged:
  • if (log.isDebugEnabled() {
      log.debug("Here is the XML blob:"+xml)
    }
    

    in Scala (or rather, Lift) you can, without loss of performance, just write

      log.debug("Here is the XML blob:"+xml)
    

    The string to be logged will not be computed if debug log statements should not be logged.

  • Very easy to write meaningful DTOs. In Java, a lot of boiler plate is needed to make a class wih e.g. 3 properties. Often I would use an array (which will be untyped) or a tuple. Both suffer from the problem that you’ll have to know that the 2nd field is the customer id and the 3rd is the total amount. In Scala you can simply write a case-class and get named fields with typed access:
     case class MyRecord(id:Int, custNo:String, amount: Double)
    

Cons

There are also a number of drawbacks to using the Scala language

  • Tooling. We’re using the Scala 2.7.x plugin for Eclipse. While it works, there are a lot of bugs that means you’ll often have to rebuild the entire project for your changes to be seen. The 2.7.x version is basically abandoned and all work is going on in the 2.8 version which is supposedly much better. Unfortunately, we can’t switch to 2.8 until all the other tools we use are running on 2.8 (which is not in beta yet)
  • While type inference works, it’s not as powerful as in ML (and probably never will be. Scala’s type system has inheritance making it much harder to inference the correct types). It frequently requires you to add some type info. After a while you figure out where this is needed though.
  • The Scala compiler is rather slow. Fortunately, the Eclipse incremental compiler works most of the time.
  • The Scala book is very well written, but the API documentation is rather sparse. With most Java code I seldom have to use the source, but for the scala code it seems easier to start with the source.
  • The syntax can get quite ugly until you recognize the patterns. This is a problem since the documentation is rather bad. Parameterized types can can be especially bad I think. This is an example from the scala-user mailing list:
  • implicit def TraversableBind[M[X] <: Traversable[X]] = new Bind[M] {
      def bind[A, B](r: M[A], f: A => M[B])(implicit w: CanBuild[B, M[B]]): M[B] = r.flatMap(f)(breakOut)
    }
    
  • No binary compatibility between Scala versions. This means that code compiled with Scala 2.7.3 is not (in general) compatible with code compiled with 2.7.5 and needs to be recompiled. This is not a big deal for our own code, but it does make it difficult to move to another version since all the Scala libraries used needs to be released first. I think this is also what has slowed down use of the forthcoming 2.8 release: people with sizable apps are waiting for the entire tool chain to be available and since 2.8 is not yet in beta, you need to agree on a version of 2.8 for everything.

Lift

Overall, I like the Lift framework and how it utilizes the Scala language. The fundamental approach to request handling seem very well thought out and makes it easy to handle both traditional web apps, Ajax, REST APIs etc. The focus on Lift seems to get things done, not so much to create the perfect web framework abstraction that has all corner cases covered. This means most code has been battle tested, but sometimes you’ll wander along an untrodden path and strange things will happen.

Pros

  • Without a doubt, the best thing about Lift is the community. Very friendly people, fast response to most questions. Eager to get newbies up to speed.
  • The Eclipse incremental compiler, Jetty & the JRebel plugin makes for fairly rapid turnaround times even if it’s not in the same league as e.g. PHP & Python. But many changes can be reloaded on the fly without having to restart the server.
  • Very fast to get a real app up and running. Prebuilt archetypes for a basic app with authentication and DB access.
  • Focus on security. Much effort has been put into making Lift apps secure. E.g. you cannot repost a form since the field ids are unique
  • The Lift ORM, Mapper, is fairly simple, but does provide a very quick way to get CRUD functionality up and running.
  • Very easy to add Ajax functionality (I can’t speak on the Comet functionality)
  • Pretty good i18n support. Strings, templates etc. can be locale dependent and the locale can be computed for each request if needed.
  • The clean separation between UI and code (it is impossible to have code in templates) makes it easy for designers to modify the layout and styling with breaking anything.
  • The focus on getting things done means if there’s a real use case, new features can be added fairly quickly. Same goes for bugs in production code.

Cons

  • API Documentation is very scarce. Usually, I use the source to figure out how stuff is working, but sometimes you need the big picture in order to get a good understanding. There is a free book available, but I usually don’t use this on a day to day basis.
  • Form handling is very basic. There isn’t really any form specific code in Lift, so making a real, usable form (with i18n, validation, etc) takes some effort.
  • Lift apps are very stateful. This is not a problem for us, but for apps that needs to be very horizontally scalable, this is a showstopper. The problem is not easily solved, since it requires serialization of closures.
  • The Lift ORM, Mapper, is fairly simple, which means that for more complex scenarios it is somewhat difficult to extend, loosing some of the RAD capabilities
  • The focus on getting things done means that some areas are not really consistent or complete.
  • While you can’t get code into your templates, it’s easy to get UI into your code, which is (almost) just as bad. The dynamic part of the UI is done by snippets and they of course need to emit HTML. But it is easy to put all kinds of style, class attributes as well as other things which belong in the template, into this dynamically generated code. This makes it difficult for designers to modify the layout and styling without touching the Scala code.
About these ads

30 responses to this post.

  1. Posted by Ittay on January 18, 2010 at 23:22

    Why didn’t you consider Grails? Have you looked at the Play framework?

    Reply

    • Posted by Jeppe on January 18, 2010 at 23:55

      I didn’t look closely at Grails. When I looked at Groovy some years back I didn’t really like it. I’ve used it in a smaller scale with the Gradle build system, and it’s not to bad so I’m sure things have improved. In the end, I’m probably (due to old habits?) inclined towards statically typed languages for moderately complex systems. I’ve only recently discovered Play, looks interesting. Still Java though, but can probably be used from Scala.

      It never ceases to amaze me the sheer number of web frameworks that are available.

      Reply

      • Posted by Ittay on January 19, 2010 at 13:02

        Play is moving to Scala.

      • Posted by Jeppe on January 19, 2010 at 20:07

        Interesting. Will it still be Java compatible? One of Lift’s strengths imo is the functional approach to many things. E.g. provide a partial function to rewrite requests. Akka is another framework written in Scala, which provides both Java & Scala interfaces for concurrent programming. If it catches on, it will be interesting to see the Java/Scala ratio of client apps.

      • I can confirm that Play is a really great framework.

        Current stable version is 1.0.1. The 1.1 version will be fully scala compatible (currently in developpement but available on launchpad).

        Just watch the screencast exemple at http://www.playframework.org/ .

        Doing a web application is so easy and fast with play ;)

        Best regards,
        Gaetan

      • Posted by bendanpa on January 20, 2010 at 22:00

        Hi Gaetan,

        Would you provide some experiences about The Play Framework? How is its support for Google App Engine? Since the Groovy is used in the template engine does it have the performance issue on GAE?

        Thanks,
        Qingshan

      • Play support GAE
        You can find more information on : http://www.playframework.org/documentation/1.0.1/gae

        Best regards,
        Gaetan

      • Regarding Play and GAE, in recent week I’ve been converting a project to run on GAE using Play. It works fine, performance wise the Groovy templates are acceptable. The main reason not to use GAE would probably be the long warmup time for your initial request (up to 30 seconds!). This is BTW also true for most frameworks and GAE, so it’s really a GAE issue IMO.

        Play! is great though, first framework that actually makes writing webapps in Java fun!!

        -andy

  2. Posted by Anthony on January 19, 2010 at 11:03

    Thanks for writing this so that I didn’t have to… My experience is similar.

    After years of Java, I am liking Scala. Nice to get functional programming features, closures, and traits solve OO design so well. I truly think this is the future of Java. But the biggest weakness is the lack of compatibility. Coming from Java, I don’t understand that code compiled in 2.7.x isn’t compatible with 2.7.y, but… I don’t see how the language adequately supports libraries with that limitation.

    For Lift, I am still pondering the “solution” to the con about having UI code bleed into the snippets. I definitely don’t like that these two aspects are strongly coupled. My problem is that I can’t just change the view to get a new look without changing the code. I specifically have this problem in that I want multiple views of the same information (one for a desktop, and another view for a mobile web browser). This results in code duplication… But the answer to this problem isn’t so obvious to me.

    The aspect of security is probably Lift’s greatest strength.

    Documentation! Documentation! DOCUMENTATION!!! I think this is a weak area for both Scala AND Lift. It really pales in comparison to Javadocs for Java from Sun. But I think this area needs some very serious attention! (One strategy I have pondered is to link the ScalaDocs/JavaDocs to its unit test. The unit test would demonstrate how the function would work. This would promote unit testing, as well as avoid redundant documentation. I’ll keep dreaming.) I think this is a major impediment for adoption.

    If you were to do it all over again, would you still choose Scala + Lift? Or is something else more enticing?

    Reply

    • Posted by Jeppe on January 19, 2010 at 12:58

      Would I choose this again? Probably. I still love the fast turnaround times of e.g Python (just did a few things in Google App engine and the fact that you can just reload after every change is quite nice :-). But I really like Scala as a language, seems to have the right mixture of FP & OO.

      I’ve also dabbled a little with Scala 2.8 and the Eclipse plugin has been much improved. But, alas, it’s still not ready for real use with Lift.

      Reply

      • Posted by fedesilva on January 23, 2010 at 08:42

        Why not use sbt? code.google.com/p/simple-build-tool/
        sbt compiles across multiple scala versions if needed and can
        compile changed files on demand. It also run tests on demand a’la
        ruby’s autotest, launches jetty to test and redeploys on recompiles.

        If you are “married” with eclipse you could try this: http://github.com/frank06/sbt-eclipse-plugin
        and eventually run sbt via launchers?

        regards!

  3. Well written clear and concise. Greatly appreciate your effort. I recognize myself very much into your experience using Scala + Lift.

    Reply

  4. Posted by Roy on January 19, 2010 at 17:48

    I picked up Scala a few months back and have been porting my personal projects into Scala 2.7.x. It is a great language, if you avoid the cryptic syntax-sugar and build out your Functional programming skills carefully.

    For my web project I built a prototype using Lift and Scala+Wicket. I wanted Lift to win, but decided on Wicket, as I found it difficult to customize the out-the-box functionality in the ways I wanted. In the end, I went with the more mature/familiar framework, and I’ll use Scala+Ibatis for my database layer.

    Reply

    • Posted by Jeppe on January 19, 2010 at 20:10

      I’ve only used Wicket in a POC. Any significant benefits of using Scala? Same question for IBatis :-) I’ve never used it, but always felt it was more appropriate in many cases than e.g. Hibernate which is quite complex on it’s own.

      What did you miss from Lift? Personally, I find the lack of decent form handling the most troubling…..

      Reply

  5. Posted by Dan S. on January 19, 2010 at 20:01

    Nit: Scala’s type inference is actually rather more powerful than ML, but it is handicapped by having to work over a far richer set of types. :)

    Reply

  6. Great analysis! I really love the concise code you get to write in Scala when compared to the same feature written in Java. Also, the fact that you can call Java code from the Scala code and vice-versa makes the development even more better, both work on the same JVM! I was attempting to write the JTA using Scala sometime soon. But now my interest is towards Ruby. I am learning this and I am quite happy with Ruby so far. Thanks for sharing your experiences! :-)

    Reply

  7. Posted by bendanpa on January 20, 2010 at 21:34

    Hi,

    Very nice write down. I am choosing a framework for a upcoming project. The project will be host on google app engine first (free starter). I was looking at Grails but the support for GAE is pretty bad and looks won’t get better in near future. I turn to Lift and still look at other frameworks. The Play framework looks pretty nice and it has good GAE support through Siena to access the BigTable. One more sale point is it is turning to Scala. But one weak point is it uses the Groovy as the template engine which is causing slow performance. Otherwise it is perfect framework for starter project on GAE.

    Can you provide any comments on Lift one GAE and Play on GAE?

    Thanks,
    Bendanpa

    Reply

    • Posted by Jeppe on January 21, 2010 at 10:24

      Don’t know about Play. Lift works on GAE, but with some limitations. I.e. you cannot start threads in GAE so Comet support doesn’t work (yet. There has been talk of making it work without threads). Also, the Lift ORM needs JDBC which is not supported on GAE, but JDO works. Search the Lift mailing list, and you can find a few examples of how Lift works on GAE.

      Reply

  8. […] Scala and Lift – Status after six months There are many choices to be made when starting a new project. In this post I’ll try to explain our technology […] […]

    Reply

  9. […] Scala and Lift – Status after six months – interesting observations about my favorite future programming language… I really need to spend some more time on Scala… […]

    Reply

  10. Just wondering about the horizontal scalability issue with Lift that you mention, as long as one is using sticky session load balancing, isn’t this largely mitigated?

    Reply

  11. I enjoyed reading this post. You gave an honest assessment of how difficult it can be to choose a framework with so many options out there. From the comments above, I found a framework (Play) that I think I might seriously give a closer look.

    Thanks.

    Reply

  12. Posted by Andy on September 17, 2010 at 06:21

    Have you thought about a pure Javascript approach, maybe using Node.js or Axiom? Javascript in the entire stack seems like a pretty good strategy, especially if your app fits the NoSQL world, where CouchDB could give you Javascript on the backend too.

    Reply

    • Posted by Jeppe on September 17, 2010 at 10:18

      Andy, I haven’t considered Javascript on the backend. Call me old fashioned, but I prefer a statically typed language for the major business functionality :-)

      Reply

    • Posted by Mulleteer on October 26, 2010 at 07:51

      Dynamically typed languages are not good choice for complicated projects with several developers. Moderately complicated software with good regression test suite and 1 or 2 developers is doable, but already shows the issues.. data and logic starts mixing, interfaces get mangled, etc. In the end you need to define interfaces as statically as with static language in the first place.

      Reply

      • if you have dumb developers, it almost does not matter which language you choose !! … choose good developers and even with a “no-typed” language like JavaScript, they will create and do great things … and of course with languages like Scala, they will fly !! ;) :)

  13. Thanks for the article, I found myself in this point of choice and I think I will go for Scala + Lift, I really like the idea of View first and the focus on security.

    Reply

  14. […] we started creating fleetdna.com, we iterated quickly using the ORM built into Lift: Mapper. It is a fairly simple (in a good way!) […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: