Building a social site using Scala, Lift & MongoDB

So, it has been quite a while since the last update on this blog. Time to do something about it and build a real-time social application using lots of new and shiny technology!

Rapid Lift application development with Eclipse and JRebel

In this article I’ll describe the setup I use to do develop Lift applications. While more heavy-weight than if an interpreted language is used, I find this setup provides fairly decent turnaround times.

Lift quick start: Look Ma! No Maven

For the last year or so, I’ve been using the Lift web framework to develop our B2B SaaS application (my experiences can be found here). I’ve enjoyed this very much, especially the community, so when David Pollak asked me to be become a committer I couldn’t say no

One area where Lift is lacking is in documentation, so I’ll try to write a few posts here that can help people getting started using Lift. I’ll be writing about the workflow we use, since

1. It works for us
2. It is not using Maven, a source of some ….. frustration for many

Scala and Lift – Status after six months

There are many choices to be made when starting a new project. In this post I’ll try to explain our technology choices and the experience we’ve had so far.

Background

One of the nice things about starting a new project in a startup is the freedom of choice when it comes to selecting platform and tools. You can spend an awful lot of time mulling between the different choices, but in the end, it is usually not the choice of programming language that kills a startup.

First, a little background. I’ve been programming for more than 15 years in C++,VB,C#, Java,Perl & PHP. The last 5-6 years it has been mostly enterprise Java. When I started at my last job (another startup) we had a product implemented in Java that we turned into a SaaS platform. The core remained in Java  and much of the web frontend was implemented in PHP. I really liked the productivity we got out of PHP as compared to the Java code. Very fast turnaround times. But somehow the language/platform doesn’t really  turn me on.

Making libmp3lame work with ffmpeg on OS X

I wanted to convert some youtube footage to mp3 by using the Firefox addon DownloadHelper together with ffmpeg.

Ok, installed ffmpeg on OS X using MacPorts:

sudo port install ffmpeg +gpl +lame +x264 +xvid

Tried to convert a file. Got this error:

[libmp3lame @ 0x1804600]lame: output buffer too small (buffer index: 8360, free bytes: 1432)

Using twitter to maintain an operations log

Whenever you’re dealing with a system that changes over time, at some point you’ll find it useful to go back and see what changes were made 3 weeks ago.

And then you wish you belong to the (small?) group of people who write down every change to a log file, so you can answer this question.

I’m not in this group but have had this need often enough that I’ve tried to automate at least some of the logging.

Beware of Scala’s lazy Range in for/yield

I’m currently using Scala for most of our development and like it a lot. Coming from a mostly Java background, it is nice with a familiar execution environment and tools. But there a few things that has bitten me, one which I just spend almost an hour trying to nail. So I’m posting it here hoping it might save somebody in the future.

Scala has a nice mechanism for creating lists with a for comprehension:

val l = for (...) yield computeResult(...)

Basically, l will become the list of return values from computeResult. Cool.

Scala also has Range class that, together with the for comprehension mimics a standard for-loop construct:

for(i <- 1 to 20) {
 ... 
}

Problem

I had the following case. I needed to create some objects in a database and return the results in a list. So I created something that looked like the following:

val databaseObjects = for(i <- 1 to 20) yield {
  val o = CreateObject
  o
}

When I ran this code, nothing happened?! I checked the DB, no records to be found anywhere. I tried printing the object o inside the loop. Nothing got printed. Tried the construct in the Scala REPL. Everything worked fine. Then I tried printing databaseObjects.length after the loop. Shows that 20 objects were created. Hmmmm. Then I tried printing all elements of databaseObjects. They were created just fine. Hmmmmm.

I recalled  a discussion about the Range being created by the expression “1 to 20″ is a lazy data structure which is not evaluated before the results are needed. Turns out this lazyness is contagious (more details can be found here)  Ahhhh. Everything makes sense now, since nothing happened until I tried to actually fetch the objects from the list (and the REPL implicitly retrieves the objects to print out the results!)

Solution

The fix is easy, you need to force the lazy range:

val databaseObjects = for(i <- (1 to 20).force) yield {

EC2 Continuous Deployment: Wrapping up

Note: This is the fifth (and last!) article in the series on continuous deployment. If you want to start with the overview, go here.

Finally, we’re coming to the end of the road. The only things left in order to finish the continuous deployment cycle are these steps:

• Run integration tests
• Shutdown instance
• Add new nightly job to Hudson that runs the integration tests

EC2 Continuous Deployment: Cooking with Chef and a stint of Gradle

Note: This is the fourth article in the series on continuous deployment. If you want to start with the overview, go here.

What we’ve done so far is a pretty basic Continuous Integration setup. To take this to the next level, we’ll add a new job, to be run at night, that provisions and configures a new EC2 instance, deploys the application and runs the integration tests.

The steps that needs to be executed are

• Start a new EC2 instance
• Configure the instance with all the software needed to run the application (in our case Jetty & nginx)
• Deploy the application
• Run the integration tests
• Shutdown the instance

EC2 Continuous Deployment: Managing the build process

Note: This is the third article in the series on continuous deployment. If you want to start with the overview, go here.

We can now build our application manually, but need to automate the actual build process. For this I’ll be using the Hudson Extensible continuous integration engine. I’ve been using CruiseControl for many years, and while it does the job, I’ve been annoyed enough that I’ll try something new.