Disclaimer: I’m probably being a complete dolt on this topic… please chime in and let me know what you think.

The other night I helped my wife dig out what will one day be a patio and retaining wall from the side of a gentle slope in our backyard. While digging out hard-packed clay I started to think about a talk that Ted Neward gave at the Des Moines NoFluffJustStuff conference called “The Fallacies of Enterprise Systems”.

The primary point of the talk was that developers can and should build their systems keeping in mind that parts of the system do fail. The network fails. Harddrives fail. Every part of the system could and sometimes do fail. While making his point that we don’t have to fall for this and other fallacies, he quoted Pat Helland (a quote which for the life of me I cannot remember). I responsed with a quote from another Microsofty, Nathan Myhrvold : “Software sucks because users demand it to” , and went on to say that sometimes we are forced into a position that we do not want to be in. To which Ted responded succinctly: “Thats disturbing.” Both comments got laughs.

The thing is, I could’nt get Ted’s response out of my head. Bottom line: he’s right (don’t tell him I said that:-) It is disturbing. And unfortunately it is still true.

Software sucks because:

• Tight deadlines with unrealistic feature sets
• Pain-in-the-ass management that push with no take
• Teammates not willing to work together
• Developers who are willing to say “its good enough” when they know it isn’t
• Customers who just don’t “Get it”
• Users don’t know what they want
• Bob/Alice/whoever is a real and won’t help us!
• etc etc

We all know that software does not have to suck. We’ve heard anecdotal evidence of successful projects. Many of us have been on such projects. Yet somehow, software projects still seem to fail at an alarming rate.

To which I ask: Who’s fault is that? Who is responsible for these problems? Or better, who is willing to take responsibility for these problems?

This is a tough question, and I have a feeling that sometimes the reason projects fail is precisely because no one is willing to step up and take responsibility. I cannot answer for anyone else, but I feel it is my responsibility, as a professional developer, to make the system the best it can be, regardless of the challenges in the way of that. Being a professional is hard, especially in the software industry where the maturity level is barely a 2 in most organizations. It is the professional’s job to communicate the consequences of the problem to the people who can resolve it and try to remedy the situation. Now, that takes a lot of bravery, and probably a little stupidity (I know I have a lot of the latter). It is hard to stand up to your boss and make him/her/it aware of the problems. It is hard to tell the customer that you need this or that from them, especially if it is because you horked up and didn’t do the job right in the first place, or your on your last warning before getting booted.

Maybe I am being an idealist. I probably am. I feel that if you care about what your doing than you should do your best to alleviate the problems.

• If the customer wants too much too quickly say “We can only get these features to you by X date if we try to get all of these features in there will be many(pick a high number) more bugs”.
• If the boss is demanding more hours, google for reports that show the loss in productivity that is experienced with overtime in the software industry. There is plenty of research to back you up.
• If there is a problem developer on the team, call an intervention. Explain to them what the problems are and try to help them get over them. (I’ve been at my job for two months now… I’m due)
• If the user doesn’t know what they want, create mockups to see if you can center their thinking.

Sometimes this won’t work, sometimes it will, sometimes it may even get you fired. Communication is key. We do not all work in a job where we can walk up to the boss or customer and tell them whats going on. Some of us have really nasty jobs with really nasty bosses and even worse customers. But let me ask you this: If the boss/customer is nasty now, what is going to happen when you cannot deliver what they want? My bet is that they will get even more upset than if you had told them when you knew. My mind floats back to those Shark Tank stories where the boss doesn’t want to know up front… only when there is a “problem” (i.e. its too late) and then blames the developers for not telling in time to change anything.

If the problems are not being handled, and you’ve done your best to resolve them, its time to find a new job. I’m not saying quit immediately, but find someplace else. We are not forced to stay at a job, and even if you have become attached to the project (most all of us do at some point), you know the likeliness that you get hurt in the end can be pretty high.

New(er) Projects

So, what can we do for new/near new/slightly used projects? (I’m shopping for a car… sorry) This is where we as developers can try to make the rest of the project team understand the complexities involved in software development, and offer up solutions to better facilitate the successful management of the project. The goal of everyone on the team should be the successful completion of the project. Go to agilealliance.org, drink the Kool-Aide. Get religion. Whatever. Just take a look at how successful people-oriented, fast feedback, iterative processes are and see if they make sense for your project.

Software development can be very much like building a retaining wall. You first must have a plan, otherwise your in deep sh!t already. In the case of our retaining wall, my wife’s plan was to “start digging and see where I end up”. At first that really annoyed me… how can she not know where she is going?!?!. But she had a plan. That night I was helping her we realized that if we kept on her initial trajectory we would run into the roots of tree. What should we do? Fill in what we’ve already dug out? Scrap the project altogether? No, we changed direction. My wife was being suprisingly agile: She knew approximately where she wanted the wall, but wasn’t quite sure. So, she started down the path anyway, confident that she would be able to correct herself, and kept a close eye on the progress she was making.

Why can’t software projects be like this? We know they can. Just look at how much success XP and Scrumm are having by embracing change. The problem is, can we all embrace change? I hope so. I think that the concept of agile development is an empowering one. It tries to make everyone accountable for their share, but with open and honest feedback, so that everyone is on the same page and does not need to be worried that so-and-so is not doing their part.

So, the positive side of me (call him Mr. Happy) has finally knocked down the door to the padded cell. Busted it clean off the hinges. Software does not have to suck. But the only people who are going to make it not suck are the people who work in it everyday; the developers.

Ted, if your at the Omaha NFJS, I still owe you a drink.

---

Test Driven Development, by Kent Beck

Books in the queue:

---

On Intelligence, by Jeff Hawkins and Sandra Blakeslee
Finished:
The Best Software Writings volume I, edited by Joel Splosky.

Matt Secoske

---

Woot! New Theme! WordPress, as with most blogging software, allows you to plugin themes easily. I was using the default 1.5 theme, which completely stunk… I could not get the text to display the way I wanted it to, regardless of the changes I made to the CSS files. This theme is almost exactly what I want… something light and clean.

I’ve been thinking alot about Agile Development (AD) since getting back from NFJS. Listening to Venkat Subramaniam talk about Test First programming and AD has rekindled a desire to express myself in the medium of bits and bytes.

I have been following the AD movement since eXtreme Programming eXplained came out. At the time, I thought the concept was awesome, but not very practical - it seemed like a mountain would need to be move to convince my team that this was a better way. But, one of the nice things about AD is that you can do most of it by yourself. So, lets see how I compare:

Testing:
I have always done programmatic testing (except for a few bleak years where I dealt with PROCEDURE DIVISION statements all day), though I had a tendancy to toss the code after the test. Don’t ask me why. It just didn’t click that “Hey… all this can be reran to ensure the correctness of the code!”. Every once in a while I would find a useful test program for things like JSSE code, and would move them to a directory, usually never to be used again.
I think that very rarely has testing driven any aspect of my designs though… I’ve always had an idea of what I wanted, then coded the test to work with it.

Pair Programming:
Most code that I have written has been alone, usually late at night. However, I have worked as part of a pair before, and would say that the code written by two people is almost always better than that written by the solitary coder. Recently Kyle and I sat down for a day and refactored that JSF application I just blogged about to get rid of IBM’s insidious “codepage” from inbetween our Managed beans. I would say that the two of working side by side on that made us a lot more productive then if we had each just taken parts and worked on them seperately. By “more productive” I am really saying that together we produced fewer errors, and were able to talk through the problems at hand an come up with a more elegant solution together.

User Stories / Planning Game
I have never done anything like this, and am very interested to see how it works. I have only witnessed the “upfront” planning style, with little to no pushback allowed from the developers end as far as dates/features went. I have always disliked this, but have not found the voice to say that there is a better way, mainly because I do not “know” any other way.

There are obviously other tenets to XP, and Agile Development in general. The three above are the ones I am most familiar with. I think the most important tenet for me right now is the concept of Test First Development, which, in my mind, is a more disciplened version of Test Driven Development. You write out every test before you write the code, even the method signature!

So, I have volunteered to speak at the next OJUG meeting regarding Agile Develpoment… kindof a intro to Agile, with a more indepth look at TFD. I am starting out in Venkat’s foot steps: I’ve written, test first, a Peg Game engine. I’ve mentioned this game at least a couple of times on my site, and most people who know me have probably seen a printout of a solution to the game from a solver I wrote many years ago. I still have this code, but forced myself to stay away from it… knowing the terrible path that lies there.

I’ve written the engine, and now am trying to put together a paper/presentation the steps and missteps I took while writing it. One thing I can say for sure is that I am much happier with the code this time around. I think the main reason for that is that I’ve written it for general purpose use… it is no longer a Game Theory model for a recursive solver program, though it could be used for that (and knowing my fascination with this game, it probably will sooner or later).