Using twitter to maintain an operations log

Whenever you’re dealing with a system that changes over time, at some point you’ll find it useful to go back and see what changes were made 3 weeks ago.

And then you wish you belong to the (small?) group of people who write down every change to a log file, so you can answer this question.

I’m not in this group but have had this need often enough that I’ve tried to automate at least some of the logging.
Continue reading

Advertisements

Beware of Scala’s lazy Range in for/yield

I’m currently using Scala for most of our development and like it a lot. Coming from a mostly Java background, it is nice with a familiar execution environment and tools. But there a few things that has bitten me, one which I just spend almost an hour trying to nail. So I’m posting it here hoping it might save somebody in the future.

Scala has a nice mechanism for creating lists with a for comprehension:

val l = for (...) yield computeResult(...)

Basically, l will become the list of return values from computeResult. Cool.

Scala also has Range class that, together with the for comprehension mimics a standard for-loop construct:

for(i <- 1 to 20) { ... } [/sourcecode]

Problem

I had the following case. I needed to create some objects in a database and return the results in a list. So I created something that looked like the following:

val databaseObjects = for(i <- 1 to 20) yield { val o = CreateObject o } [/sourcecode] When I ran this code, nothing happened?! I checked the DB, no records to be found anywhere. I tried printing the object o inside the loop. Nothing got printed. Tried the construct in the Scala REPL. Everything worked fine. Then I tried printing databaseObjects.length after the loop. Shows that 20 objects were created. Hmmmm. Then I tried printing all elements of databaseObjects. They were created just fine. Hmmmmm. I recalled  a discussion about the Range being created by the expression "1 to 20" is a lazy data structure which is not evaluated before the results are needed. Turns out this lazyness is contagious (more details can be found here)  Ahhhh. Everything makes sense now, since nothing happened until I tried to actually fetch the objects from the list (and the REPL implicitly retrieves the objects to print out the results!)

Solution

The fix is easy, you need to force the lazy range:

val databaseObjects = for(i <- (1 to 20).force) yield { [/sourcecode]

EC2 Continuous Deployment: Wrapping up

Note: This is the fifth (and last!) article in the series on continuous deployment. If you want to start with the overview, go here.

Finally, we’re coming to the end of the road. The only things left in order to finish the continuous deployment cycle are these steps:

  • Run integration tests
  • Shutdown instance
  • Add new nightly job to Hudson that runs the integration tests

Continue reading

EC2 Continuous Deployment: Cooking with Chef and a stint of Gradle

Note: This is the fourth article in the series on continuous deployment. If you want to start with the overview, go here.

What we’ve done so far is a pretty basic Continuous Integration setup. To take this to the next level, we’ll add a new job, to be run at night, that provisions and configures a new EC2 instance, deploys the application and runs the integration tests.

The steps that needs to be executed are

  • Start a new EC2 instance
  • Configure the instance with all the software needed to run the application (in our case Jetty & nginx)
  • Deploy the application
  • Run the integration tests
  • Shutdown the instance

Continue reading

EC2 Continuous Deployment: Managing the build process

Note: This is the third article in the series on continuous deployment. If you want to start with the overview, go here.

We can now build our application manually, but need to automate the actual build process. For this I’ll be using the Hudson Extensible continuous integration engine. I’ve been using CruiseControl for many years, and while it does the job, I’ve been annoyed enough that I’ll try something new.

Continue reading

EC2 Continuous Deployment: Building the software

Note: This is the second article in the series on continuous deployment. If you want to start with the overview, go here.

The app we’ll deploy will just be a simple “Hello World” application, written using Lift, a very nice Scala web framework. I will not go into any Scala or Lift specifics in this article, but will  show how to build it.

Continue reading

EC2 Continuous Deployment: Hello world

I’m a strong believer in agile development and one of the biggest benefits I see is to get working software in front of the customer as fast as possible. The term ‘working’ may mean different things to different people but it does imply that there’s some level of quality that needs to be  achieved.

Many developers consider their job done when they’ve committed a feature and the build is successfull (you do run some sort of CI don’t you?). But this does not bring the software in front of the customer. In many places, doing a production deployment is a large, manual process with lots of opportunities for mistakes, which means that deployments either take longer than necessary or happen infrequently.

Timothy Fitz wrote an interesting entry about Continuous Deployment, basically deploying to production after each commit. While I don’t think deploying to production after every commit is workable for everybody, the idea of actually deploying the latest code automatically is very valuable. This article series will cover the basics on how to get a simple software product deployed automatically.

Continue reading