Interesting piece in yesterday’s NYTimes Business section by Mary Tripsas on how 3M partners with its customers to drive innovation and new ideas, as opposed to driving their innovation into them with a blunt instrument:  Seeing Customers as Partners in Innovation.  While 3M should be immediately familiar to most folks, it’s even doubly-so for software development professionals who employ lean delivery techniques since one of the acknowledged thought-leaders in the space is Mary Poppendieck, a former 3M employee.

The article references a Harvard Business School professor, Ranjay Gulati, who has recently completed a book I need to pick up:  (Re)(Organize) for Resilience: Putting Customers at the Center of Your Business.  His approach is definitely in simpatico with the agile/iterative/lean philosophy:

“Being customer-driven doesn’t mean asking customers what they want and then giving it to them,” says Ranjay Gulati, a professor at the Harvard Business School. “It’s about building a deep awareness of how the customer uses your product.”

This stands in stark contrast to how many firms perceive their customers, developing a homogenous view of their base through their product lines and offerings.  In this environment, innovation becomes an ugly stepchild – something I’ve witnessed more and more in the industry.  Gulati has seen this as well, and shares my cynical perspective:

The terms “customer driven” and “solutions” seem to be in every manager’s lexicon. But as Professor Gulati notes, “it’s an execution problem.” Companies, he says, “aren’t generally structured to access, absorb or utilize customer insights since they are organized by product, not by customer.”

And so we have the defining challenge of our industry which I am flummoxed to explain:  Why we are retreating to view software development as the end-product of a manufacturing vs. creative process, especially in light of the failures the former view has wreaked.

Within MCS, I am seeing this increasingly coming to the fore with a shift toward fixed-bid projects that limit the ability of consultants to drive the kind of rich innovation Tripsas’ article describes or the deep partnerships that Gulati promotes.  When we view customers as fitting or shoe-horned in to categories, we begin to constrain our thinking around our products which shutters our view of more exciting opportunities that would otherwise come to the fore.

In my opinion, if we adopt an “Innovation Center” approach as Tripsas describes, we may in fact develop a defining competitive advantage and so break out of the constraints of being organized by product and not by customer.

I recently came across a Nov. 25/09 post on Davy Brion’s blog, The Inquisitive Coder, that enumerates his recommended reading list for software developers who are wanting to step up their game.  This is a common theme on coding blogs, and there have been literally a galaxy of postings with some great and not so great recommendations guaranteed to perplex and flummox the average coder.

The implied line with all of these articles is that you ignore these tomes at your peril – you just won’t get to the “A” levels without them.

I like how Brion has broken his recommendations into three strata according to developer experience:  newbie, sophomore and grizzled veteran.  Well, sort of.  I mean, learning agile/iterative/lean software development really should occur at the start of your career as you’re going to have to unlearn the BDUF/waterfall crap that was loaded into your head by your CS/SoftEng profs who last “delivered” software (if at all) when 8–bit CPUs were da bomb.

What I don’t like is the complexity of some of the volumes that Brion shows – some of them are just so dense as to be inaccessible and not immediately applicable, eg. Eric Evans’ Domain-Driven Design.

Read/Master These Books First

If you want to get immediate benefits, try this shorter reading list:

1. The Pragmatic Programmer:  From Journeyman to Master (Hunt & Thomas)
2. Pragmatic Unit Testing in Java/C# (Hunt & Thomas)
3. Refactoring:  Improving the Design of Existing Code (Fowler)
4. Practices of an Agile Developer (Subramaniam & Hunt)
5. Agile Software Development with Scrum (Schwaber & Beedle)

These books will help you begin to develop good habits – and in some places, possibly fired for wanting to implement.  That’s ok, because you don’t want to waste your life there, anyway.   It should take you 6–12 months to get through these and really internalize the practices and make them second nature.  I’ve recommended #5 because, as I mention above, you should get exposure to agile practices like Scrum sooner rather than later.

Read/Master These Books Next

You’ve got a couple of years under your belt and have seen the good, the bad and the ugly in the industry.  You’ve likely been on some bad projects and you’re beginning to question the wisdom of your chosen career path.  Take solace in these books to notch your game up further:

1. Design Patterns (Gang of Four – Gamma, Helm, Johnson, Vlissides) – Alternatively, any book that explains patterns in your language of choice will help you “grok” them.
2. Clean Code: A Handbook of Agile Software Craftsmanship (Robert “Uncle Bob” C. Martin)
3. Agile Estimating and Planning (Cohn)

By the time you get through these (another 6–12 months) you will be truly cynical about the state of the industry, as well you should.  But you’re not wanting to be a “mort” or a “501” kinda coder – you’re wanting more.  So, you should supplement the above list with these titles:

1. Death March 2^nd Edition (Yourdon)
2. The Mythical Man-Month (Brooks Jr.)
3. Agile Project Management with Scrum (Schwaber)

These titles will give you the intellectual “legs” to position your strategies and arguments (yes, arguments) for implementing best practices with peers and managers.

Ongoing Professional Development

After you’ve reached your fifth or sixth year in the industry, you should have a pretty good idea about who you are professionally and where you think you’re headed:  Leadership, Consulting, Management.  Depending on your path, different books will influence your thinking.  You will likely be wanting to show what you’re capable of – perhaps in a new role or firm.  Some of these titles will help:

1. Fearless Change:  Patterns for Introducing New Ideas (Manns & Rising)
2. Agile & Iterative Development:  A Manager’s Guide (Larman)
3. Implementing Lean Software Development:  From Concept to Cash (Poppendieck)

Conclusion:  Just Do It

In my experience, not very many folks who have a lot of “paper” experience have actually ready many of these books.  They’ve heard of them, but never read them or applied their lessons.  I’ve come across consultants who are really smart and with scads of experience who have yet to write a single unit test – or worse, think that a tool can do it all for them.  Common sense ain’t so common.

If you master 60% of the above list, you will be a better coder and professional than 3/4 of the people you’ll come across over your career.  It will take time, some personal commitment and in some cases risk – but it will be immediately worth it.  Follow the advice of Ken Schwaber (co-creator of Scrum) who urged attendees at a conference to actively begin improving their software delivery experience by implementing agile techniques by getting out there and doing it:  “Don’t procrastinate; do something – no matter how small.” 

Mini-Microsoft is a blog written by an anonymous MSFT employee in Redmond, WA (I think…) who makes some rather interesting, biting and pithy observations about what’s going on inside one of the largest software companies in the world.  He’s known for his unvarnished opinions (which are easy to make when anonymous) on management and hiring, eg. would like to see 10,000 more jobs cut to restore profitability and efficiency to the corp.

I find it hard to argue with some of his comments – and I’ve found some simpatico between his observations and what I’ve seen in Canada, so he’s not just shooting from the hip.  So, I’m a fan of sorts – and apparently he has readers right up to the top ranks in the company.

However, it was his July 12/09 post that caught my attention – it highlighted a suspicion I’ve had:

For efficient product development: Yahoo!'s Carol Bartz has a good point when she swears like a sailor over having way too many program managers vs. actual developers (overloaded with one program manager for every three developers). <<edit edit edit - this went quickly into the weeds - let me sum up some quick thoughts>> Looking across groups, I still see exceptionally inefficient use of broad, front-loaded thinking and design locked into a 1970s waterfall model that leads to reality and focus coming way too late and a bunch of frantic, mediocre consensus driven crap floating like chunks into an end product. Kaizen. Kaizen. Kaizen. Efficiency is not going to happen as long as we continue rewarding people for this status quo. Shedding a respectable chunk of the company would bring an exceptional amount of upfront focus to our teams and result in high-quality features end-to-end, vs. what we see in misshapen compromise that we can fit in.

From what I’ve seen inside MCS, this is particularly the case – almost every large scale project is locked into a model that is predicated upon the Microsoft Solutions Framework (v3 or v4, depending on who you work with) which is, despite words to the contrary, very much single-pass/phased.  This said, I do know that my experience runs contrary to what I’ve heard from friends in product groups who have embraced the Schwaber/Beedle Scrum text as their “Bible” for project delivery.  Cripes, Schwaber himself has been to the Microsoft campus and has published books for Microsoft Press!

So, I really find it challenging to reconcile why we do things so bass-ackwards within MCS.  From my training last year through all the projects I’ve engaged on, we have continual disconnects such as those Mini observes that enforce the worst of bad practices.  For example, almost all projects are estimated and scoped up-front by folks who likely won’t be doing the delivery.  This leads to predictable schisms when a new resource is assigned and they have to reconcile the commitments made by their colleague who’s now on another project.

The reaction has been predictable – more effort to try and improve consultants’ estimating skills so as to try and homogenize their view of any project.  A fool’s errand, I fear.  To expand on Mini’s quote above, the Kaizen principles most needed to understand and embrace are those of Muri, Mura and Muda – uneven process flow, overburdening systems and waste respectively.  We could accomplish a great deal if we looked to the east.

Every so often, when I think I understand everything that there is to know about a subject, I read a really interesting insight that causes me to totally rethink my assumptions.  This is the case with a recent blog post by agile estimating and planning guru, Mike Cohn: How do story points relate to hours?

This is often a stumbling block for those new to agile estimation techniques that de-emphasize precise or padded time frames in favour of relative effort.  After an iteration or two, some folks may begin to notice a correlation between points and hours, but this is a false-positive conclusion to reach.

Why?  I’ll leave this crystal clear revelation to Cohn:

If the one-point stories are centered around a mean of x, ideally the two-point stories will be centered around a mean of 2x. This will never be exactly the case, of course, but a team that does a good job of estimating will be sufficiently close for reliable plans to be made from their estimates.

…[T]he relationship between points and hours is a distribution. One point equals a distribution with a mean of x and some standard deviation. The same is true, of course, for two-point stories, and so on…

A distribution is, of course, is a statistical term for the frequency or probability that variables take on in a sample.  In this case, the variables are actual time records to finish (done-done) a one point story across a discrete time scale.  Typical distributions, as they grow, tend to coalesce around a mean value, giving us a familiar graphical shape.

Cohn illustrates this with two distribution curves that will look familiar to most folks who survived high school math:

The overlap between the two distribution curves describes situations where teams’ estimates of a one point story on the high-end of effort converge with low-end two point stories.  If you’ve observed how teams estimate using story points, this is an obvious (yet unstated) conclusion.

Excellent post – it totally revitalizes my interest in the topic of agile/iterative/lean project development.

As promised to the attendees (all 10 or so of them…!) to my riveting and speculative session at the Agile 2008 conference yesterday, below is the link to my slide deck in PowerPoint 97–2003 format:

Agile 2008 – Who’s Teaching Best Practices?

I’ll write more on the feedback I received – it was definitely an engaging conversation that was lively and went a half-hour over the “official” closeout time.  The majority (all?) of attendees were either professors or masters/phD students who worked with professors at institutions from across Canada, the US and Europe.  Basically things broke down into two broad categories:

a) Methodology – I hung a lantern on this one:  Yes, looking at course catalogs or calendars is highly speculative and not “scientific”.  What was more interesting, however, were comments that suggested that descriptions are deliberately and intentionally vague in many institutions so as to afford instructors and professors flexibility to teach topics not specifically outlined.  It was also mentioned that the process for changing course descriptions was so difficult (read: bureaucratic) that it was best to go with the most bland and inoffensive wording so as to avoid dealing with the gatekeepers.

b) Role of the University as Developer of Leaders not Technique Instructors – This one caught me completely off-guard as I didn’t expect it from folks who are idenitifying as “pro-agile”.  There were a lot of comments in this area which essentially distill to “it’s not our job to prepare graduates for working with a specific technology or technique – we teach them how to think and lead”.  The corollary to this is that this is what colleges and “practical” technical institutes should do.

Although I understand where this is coming from, I’ve got a few issues with this response, as I am sure many readers might:  There were no apologies made for the elitist tone.  While I agree there is a need for CS/SE students to be able to learn how to think in abstractions (Joel Spolsky writes on this in his blog), there is also a compatible reality that students should be able to apply the abstract to some concrete tasks – very much what agile/iterative/lean processes do and their corresponding best practices.

I found it completely foreign to hear that there was absolutely no interest in teaching courses specific to best practices.  They could be taught as a component of a course, but it was felt that to add this on to the typical coursework would be unworkable.

Time to think things through…

I acknowledged that my methods were imperfect, my conclusions very rough – however, my personal experiences with university grads as interns suggested to me that there was a potentially interesting connection between what students were learning and what was actually expected of them in the real world.

Attendees posited to me that if industry wants this to change, they need to pony up more cash and reinstate or bolster their own internal training courses.  In effect, the responsibility for teaching agile/iterative/lean SDLC and best practices like refactoring code so that it’s readable and of value is not in the domain of the university.  Never should be, never will be.

Despite this reaction, I’ve been given a lot of good feedback, and I have to take time to process it and begin to address some points more thoroughly, perhaps in a future posting.