I’ve been reading Ship It! off of my “Pragmatic Bookshelf” over the last week, and have picked up a number of great ideas. As I have gone through the book I have been dusting off the cobwebs and thinking about the projects I have been involved in over my career, and applying the candle of thought to them: what worked? what didn’t? what could I have done differently?
One of the things that I now look back on and realize I could have done better on was Continuous Integration. This is something that I’ve known about since 2001, when I first read Kent Beck’s seminal eXtreme Programming eXplained, but have only applied it in a couple projects, always as a Daily Build, not the usual as-fast-as-you-can-build-it velocity.
One thing that I noticed with the daily builds that both books talk about is the quicker turn-around for integration problems. We just didn’t have those silly “oops I forgot to check that file in a two weeks ago lets hope its still on my machine” problems.
So, it wouldn’t be worth thinking about all of those past projects without at least looking at a current project I’m working on. And who woulda thunk it: we have some issues. The one that leaped out at me first was that we have not had a clean build for as long as I can remember. And while my memory, could argueably be shot (just ask my wife after the next time I forget to take out the trash:-), I am pretty sure it has been at least a month since my sandbox did not have at least one compile error in it. I was just ignoring those pesky errors, as I knew so-and-so was in that code, and that it would be fixed eventually.
Reading the book helped kick me out of my complacency, and on Monday I took the first steps of bringing continuous integration to my dev team by downloading and setting up CruiseControl on my machine. Now, I have known about CC for a long time, but had not played with it before. It took about two hours to setup, split over Monday and Tuesday mornings. Cake. Buttery, Creamy Cake. If I had known just how easy it would be, I would have been using this tool for all of my previous projects. I will be from now on. Here’s why:
Tuesday morning I configured the script to start emailing the entire group. Of course, the first email everyone gets has a subject of “Build Failed!”. I sent out an email explaining that I had set it up for a 15 minute cycle, and that they (the rest of the team) would only see the “Build Failed”‘ messages, unless they desired to see the successful ones as well.
The most interesting response I got was that “It should be 2-3 hours, 15 minutes is not enough time for the developers to fix their bugs”. I kindly reminded him that compile errors are not bugs, and should not get into the build in the first place, but since we are all human, this just helps us catch those types of errors in about 15 minutes, versus trying to remember what we did yesterday or worse: last week.
The result:
Within one hour ALL of the compile errors were taken care of, and over the course of the rest of the week we only had a couple of incidents. I am really looking forward to seeing the benefits of a growing functional and unit test suites in conjunction with the continous build.
Brandon Keepers | 15-Apr-06 at 7:09 am | Permalink
Hey Matt, great post! I love hearing stories that remind me that we’re not alone in the “I’ve known about this great thing but just haven’t gotten around to doing it” experience.
We actually started using CruiseControl at UNO about a year ago. As suggested in Pragmatic Automation, we went with a short (5 minute) build cycle, and that worked great. Even if your build takes longer than that, it’s ok because it won’t start another build if one is in process. And a build it only kicked off if code was committed. What I found with a longer build cycle, like 15 or 30 minutes is that often I would commit at the end of the day before I went home, and then my build would end up being broken overnight. Where as, with a 5 minute build cycle, I wait 5 minutes after I commit my code to make sure I didn’t break anything, then go home. And usually 5 minutes is enough time to fix it, since a broken build is *usually* from forgetting to commit a file.
The other great benefit we got from CruiseControl is automated deployments. We set it up to deploy to a test server after the build every time, then we could tell the project manager and testers that the latest is always available on the test server.
Matt | 15-Apr-06 at 8:03 am | Permalink
Brandon, yeah I’ve already had that happen to me. I was meeting my wife and kids for lunch, and had just checked in some code that had a new Jar dependency. Forgot to check in the jar. Fortunately the CruiseControl server is on my machine, so I was able to cheat a little and restart it with a 30 second window…
Otherwise my wife would have been waiting for at least 10 minutes for the build to run. I am hoping to start pushing it to 5 minutes as soon as I figure out how to keep the fifteen minutewindow for failed builds.