I recently came across a Nov. 25/09 post on Davy Brion’s blog, The Inquisitive Coder, that enumerates his recommended reading list for software developers who are wanting to step up their game.  This is a common theme on coding blogs, and there have been literally a galaxy of postings with some great and not so great recommendations guaranteed to perplex and flummox the average coder.

The implied line with all of these articles is that you ignore these tomes at your peril – you just won’t get to the “A” levels without them.

I like how Brion has broken his recommendations into three strata according to developer experience:  newbie, sophomore and grizzled veteran.  Well, sort of.  I mean, learning agile/iterative/lean software development really should occur at the start of your career as you’re going to have to unlearn the BDUF/waterfall crap that was loaded into your head by your CS/SoftEng profs who last “delivered” software (if at all) when 8–bit CPUs were da bomb.

What I don’t like is the complexity of some of the volumes that Brion shows – some of them are just so dense as to be inaccessible and not immediately applicable, eg. Eric Evans’ Domain-Driven Design.

Read/Master These Books First

If you want to get immediate benefits, try this shorter reading list:

1. The Pragmatic Programmer:  From Journeyman to Master (Hunt & Thomas)
2. Pragmatic Unit Testing in Java/C# (Hunt & Thomas)
3. Refactoring:  Improving the Design of Existing Code (Fowler)
4. Practices of an Agile Developer (Subramaniam & Hunt)
5. Agile Software Development with Scrum (Schwaber & Beedle)

These books will help you begin to develop good habits – and in some places, possibly fired for wanting to implement.  That’s ok, because you don’t want to waste your life there, anyway.   It should take you 6–12 months to get through these and really internalize the practices and make them second nature.  I’ve recommended #5 because, as I mention above, you should get exposure to agile practices like Scrum sooner rather than later.

Read/Master These Books Next

You’ve got a couple of years under your belt and have seen the good, the bad and the ugly in the industry.  You’ve likely been on some bad projects and you’re beginning to question the wisdom of your chosen career path.  Take solace in these books to notch your game up further:

1. Design Patterns (Gang of Four – Gamma, Helm, Johnson, Vlissides) – Alternatively, any book that explains patterns in your language of choice will help you “grok” them.
2. Clean Code: A Handbook of Agile Software Craftsmanship (Robert “Uncle Bob” C. Martin)
3. Agile Estimating and Planning (Cohn)

By the time you get through these (another 6–12 months) you will be truly cynical about the state of the industry, as well you should.  But you’re not wanting to be a “mort” or a “501” kinda coder – you’re wanting more.  So, you should supplement the above list with these titles:

1. Death March 2^nd Edition (Yourdon)
2. The Mythical Man-Month (Brooks Jr.)
3. Agile Project Management with Scrum (Schwaber)

These titles will give you the intellectual “legs” to position your strategies and arguments (yes, arguments) for implementing best practices with peers and managers.

Ongoing Professional Development

After you’ve reached your fifth or sixth year in the industry, you should have a pretty good idea about who you are professionally and where you think you’re headed:  Leadership, Consulting, Management.  Depending on your path, different books will influence your thinking.  You will likely be wanting to show what you’re capable of – perhaps in a new role or firm.  Some of these titles will help:

1. Fearless Change:  Patterns for Introducing New Ideas (Manns & Rising)
2. Agile & Iterative Development:  A Manager’s Guide (Larman)
3. Implementing Lean Software Development:  From Concept to Cash (Poppendieck)

Conclusion:  Just Do It

In my experience, not very many folks who have a lot of “paper” experience have actually ready many of these books.  They’ve heard of them, but never read them or applied their lessons.  I’ve come across consultants who are really smart and with scads of experience who have yet to write a single unit test – or worse, think that a tool can do it all for them.  Common sense ain’t so common.

If you master 60% of the above list, you will be a better coder and professional than 3/4 of the people you’ll come across over your career.  It will take time, some personal commitment and in some cases risk – but it will be immediately worth it.  Follow the advice of Ken Schwaber (co-creator of Scrum) who urged attendees at a conference to actively begin improving their software delivery experience by implementing agile techniques by getting out there and doing it:  “Don’t procrastinate; do something – no matter how small.” 

Recently, I was asked to make a return visit to a customer for whom I had built a customized SharePoint web app to do “refresher” round of knowledge transfer to folks who would be entrusted with support and deployment.  It was only for an hour, and in that time I did a soup-to-nuts flyby of the highlights along with some deep-dives into code when asked about particular features.

While describing the schema for an external database we’re using to gather data, one of the attendees asked me why I opted for a single flat-file and not a series of relational tables, etc. as this would alleviate “maintenance headaches” down the road.

I reminded the fellow of the theme (which I had explained in a prior session) that guided my development on this solution :  The simplest thing that works.  Using this as a “knothole” test, I triaged and pared features and design down to keep the project achievable within the 4.5 week timeframe I was alotted.  Had I chosen to modify the schema as he suggested, I’d actually have complicated my solution on the reporting end, which uses a series of pivot charts in Excel connected to the datastore.

(I should also hasten to mention that another influence on my adherence to simplicity for this project was the failure of a similar, prior solution developed by another consultant that ended up gutted and a shadow of its former self because the customer’s support team found it “too complex” and failed to perform as advertised.)

This episode brought to mind a couple of the rules of Unix design philosophy:

2. Rule of Clarity:  Clarity is better than cleverness.

6. Rule of Parsimony:  Write a big program only when it is clear by demonstration that nothing else will do.

Beyond the example of the simple, flat schema for the data store, I employed these rules throughout my design and development of the solution, leveraging as much OOTB functionality from SharePoint as I could and only treading into custom code where I couldn’t avoid it – and even then, using low-impact techniques.

Where I couldn’t build functionality in a reasonable amount of time, I recommended purchasing pre-built web parts.  Where I couldn’t find ready-made web parts for a report user interface, I built one using client-side techniques (HTML + jQuery + AJAX + SharePoint web services) that avoided creating heavy-weight, server-side code.  Where we needed adhoc reporting capabilities because the customer was unsure of how they wanted to drill into data, I recommended using Excel with a SQL connection instead of SQL Server Reporting Services.

Now, I didn’t come to these solutions immediately;  in fact, as I know from years of experience, good design emerges iteratively.  No amount of clever Big Design Up-Front planning ever yields this quality of results.  In-line with just about every engagement I’ve undertaken within MCS, had I stuck to the original plan, I’d have delivered a solution that would have been complex and substantially off-the-mark.  However, by breaking my time down into weekly iterations, I was able (with the customer’s guidance) to zero-in on the highest value features they needed while making key compromises with their blessing.

Thus, a fundamental tool for communicating these design decisions to the customer and garnering their support were my weekly demonstrations each Friday where they could see the working solution come to life and have direct input:  “Yes, I like this;  no, that totally misunderstands what that number is for;  I am disappointed we can’t do this other thing, but I now understand why – maybe we can look at this for v2.00”.

In this way I was able to enforce the Rules of Clarity and Parsimony:  “The simplest thing that works.”

Ed.: I've made some minor changes to this post to improve flow and revise the conclusion.

A few weeks ago (October 16 to be precise…) I happened across an intriguing blog post by developer David Christiansen entitled Is there REALLY any rigor in Waterfall?  Far from being a standard pro-agile polemic, Christiansen describes his own intellectual journey from a self-professed waterfall-neutral “fence-sitter” to agile/iterative-friendly convert through firsthand experience and research.  Rather than accept prima facie the rhetoric on the problems with phased delivery, he researched the model’s pedigree and coupled it with his own experience and had his own revelation:

“For the last couple of years I’ve sort of ridden the fence on where I stand on project management. I’ve conceded that waterfall might have a place in the IT world in the past, that it’s a tool we should have in our toolbox, even if we don’t use it very often…

The waterfall approach to managing software development projects is one of the most costly mis-diagnoses ever. It structures projects for failure. It creates a cost-change curve that is unacceptably expensive (you know, the cost of a change grows exponentially as the project progresses) and is largely responsible for the high failure rate of IT projects.  Waterfall creates a project structure that is rigid and forces you to make decisions and predictions when you know the least, then abide by them no matter what realities you expose in the process.

I am off the fence. Waterfall doesn’t belong in any IT project manager’s toolbox. It is based on and littered with myth.”

While I do enjoy reading about another developer’s Conversion on the Way to Damascus, David’s piece stands out to me because outside of Craig Larman, he’s the only person I’ve come across who makes reference of Winston W. Royce’s role in the story of the waterfall model.  While Royce isn’t really responsible for the waterfall model – that came about as a result of our industry’s 30 year failure to RTFM – knowing who he is and the history behind phased delivery is vitally important to putting the rationale for agile/lean/iterative models into their proper context. 

Is there a One True Way?

A week later, David’s post was picked up in a feedback thread to a rather pedantic advice column on InfoWorld that was responding to a student’s query on the value of requirements vs. testing (btw, I think both the instructor and the student are wrong) with the ensuing fracas prompting David to write a follow-up post where he expresses some disbelief about the fervor that seems to pervade the pro/con waterfall debate, ie. that there’s only “one true way” to deliver software: (emphasis mine)

Frankly, I’m perplexed by the belief among many technology practitioners in the ideal, the “one true process,” that will solve any problem, whether it is Waterfall, Spiral, Agile, or whatever comes next. It is a fixation I believe is unique to IT - I don’t remember seeing this type of zealotry when I worked as a manufacturing R&D engineer at Bell Helicopter. Engineers there had a much more experiment, do-what-it-takes sort of attitude that couldn’t be captured, analyzed, and then reduced to a simple prescription for project success.

With all due respect (and a wink and a nod), David’s probably not thinking of some of the holy wars that are waged in other disciplines like physics, quantum mechanics, climatological science and medicine when it comes to “one true way” zealotry.  Hell, look at the trouble Robin Warren and Barry Marshall had to overcome in gaining acceptance of their evidence that a bacterium and not spicy foods were responsible for the vast majority of gastric and peptic ulcers.  Intransigence and zealotry can be found just about anywhere.

But I digress:  I also have to disagree with David's observation that the "experiment, do-what-it-takes" attitude of the Bell R&D engineers can't be captured and turned into a prescription for better project outcomes.  In fact, this is the very essence of the models used in manufacturing (ie Toyota Production System, Lean) that agilists have adapted into a variety of methodologies from Scrum to XP to Crystal Clear to Lean.

Thus, while I agree that there isn’t an out-of-the-box “silver bullet” that will make every project successful, I am of the opinion that practices which belong to the family of empirical process controls have the highest probability for successfully delivering working software than any other process because they have significantly better visibility into project activities and an internal feedback-loop to support self-correction based on the realities of the situation at hand rather than a six-month-old plan.

Agile Values

David concludes his post by observing that in the final analysis it may be better to rely on human characteristics such as “brilliance, bravery, strength, and intelligence” to solve our problems than a naive and idealistic belief in a singular “right” way to build software.  I think David’s got this half right, because I think he’s not fully aware of the values that back-up agile/lean/iterative practices – and I’ll grant that the zealots probably don’t know them, either. 

For example, take the first value statement of the Agile Manifesto:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools.

Next, consider the Four Values that Kent Beck defines for project success under eXtremeProgramming practices:

We will be successful when we have a style that celebrates a consistent set of values that serve both human and commercial needs:  communication, simplicity, feedback and courage. 

Finally, consider Scrum creator Ken Schwaber’s comments on the challenges and rewards for implementing agile practices:

Of all organizations that attempt to become a world class software organization (by adopting agile practices like Scrum) only one third will make it.

The inspect and adapt practices of Scrum is where the pain comes in, because when you start doing that you discover everything in your organization that’s not working.

Agile is hard work;  it’s a long process and it is extraordinarily satisfying

These aren't the words of idealistic zealots, but reasoned pragmatists, and they demonstrate a closer affinity to the core "human" values that David seems to cherish.  While he may be dissuaded or alienated by the zealots who proclaim "one true way", the fact of the matter is that agile/lean/iterative disciplines (and that is what they are) work because they are built upon the fundamental understanding that building software is about people and how they can work best in a very complex and creative enterprise.

Indeed - I had no idea this existed and from what Jeremy Miller notes, neither did a roomful of Microsofties:  Another secret from within the Halls of the Mountain King, I suppose - and a bit of the left hand/right hand syndrome. ;)

First glance, I like what I see:

• P&P "dudes" are actively involved in the site and development of materials, labs, articles and the like;
• The Agile Manifesto is mentioned front-and-centre - wow, I NEVER thought I'd see that happen. 
• Other community-driven tools are prominently featured on the front page, including CC.NET, NAnt, NUnit, NMock, RefactorPro!, etc. - Now, they do have to dedicate some space to shoehorning TFS into doing some continuous integration, which I'd never do - but it is a MSFT site;
• There's a link to material written by Venkat Subramaniam on agile design (he's the co-author of Practices of an Agile Developer: Working in the Real World) which is a positive vote for the Pragmatic Programmer community;
• Scrum and XP are featured prominently - and well they should be, beyond the fact that Ken Schwaber has authored a book for MSFT Press on Scrum which is in its second edition.

This is pretty cool to see - I'll have to sift through the materials before giving a final thumbs up or down, but if first looks are any indication it's a positive step in the right direction.  What I sincerely hope to find is a broader acceptance of Scrum, XP and the agile community-driven practices, and not the first step in trying to co-opt and coerce bastardized flavours of them into the MSF (yuck!) or somesuch.

Finally, I find it gratifying to see this on a personal level as it vindicates the tools and practices I (and many others) have advocated for years in the face of hostile opposition.  Memo to my former colleagues and bosses:  I hate to say "I told you so", but... :D