In my inbox today was a copy of Dr. Dobb’s Update Newsletter (Dec 10/09) with the Editor’s Note on the recent SEMAT Initiative that’s been launched by the likes of Ivar Jacobsen (UML and Rational Unified Process), Bertrand Meyer (Eiffel language and Design by Contract paradigm) and Richard Soley (Chair/CEO of Object Management Group – Standards around distributed OO systems, CORBA). 

SEMAT is an acronym for Software Engineering Method and Theory, and has been pitched as a community of professional, leading software industry researchers and computer scientists “united” with the aim of providing a unified theory to re-establish “software engineering as a rigorous discipline based on a solid theory, proven principles and best practices.”  Already, my eyes were rolling and my BS sensor was tingling.

However, in an attempt to woo folks to their island, they list some of their signatories, including folks like Scott Ambler, Barry Boehm, Alistair Cockburn, Larry Constantine, Erich Gamma, Capers Jones, Robert C. Martin, Ken Schwaber and the like.  Very impressive.  Also a dead giveaway that this effort will be largely symbolic and not bound to produce anything substantive.

What I find troublesome about SEMAT is its aims – I really do not subscribe to the notion that there is a single theory to describe software development and engineering because of the almost infinite variability of its primary inputs:  Human creativity and collaboration.  This is why I subscribe to agile/iterative/lean practices that emphasize incremental delivery with frequent inspect/adapt feedback loops.  It’s the best way to manage the chaos and drive value.

Not good enough for SEMAT – they see the need for a revolution:

Software engineering is gravely hampered today by immature practices. Specific problems include:

• The prevalence of fads more typical of fashion industry than of an engineering discipline.
• The lack of a sound, widely accepted theoretical basis.
• The huge number of methods and method variants, with differences little understood and artificially magnified.
• The lack of credible experimental evaluation and validation.
• The split between industry practice and academic research.

What a poor choice of words overall to build a case for a call to action to come up with some rigid specification and theory about how people come together on a complex endeavour to produce working software.  And of course, what we need is more research and theory rather than practical, readily-applicable strategies to build software.

They go on:

We support a process to refound software engineering based on a solid theory, proven principles and best practices that:

• Include a kernel of widely-agreed elements, extensible for specific uses
• Addresses both technology and people issues
• Are supported by industry, academia, researchers and users
• Support extension in the face of changing requirements and technology

And other mom & apple pie things. 

Lessons Learned from ALT.NET

While these a laudable goals, they’re impractical and not likely to be achieved, if for no other reason than the folks involved (personality clashes and the like).  However there’s another reason I think this will not go far, and it’s because I’ve seen it all before in the ALT.NET initiative.

ALT.NET was (is?) a loose confederacy of software development professionals, founded by NY developer Dave Laribee in early 2007.  The intent was to provide a community for “alternative” .NET professionals: Like-minded folks who used .NET technologies, but weren’t slavishly dedicated to a Microsoft-centric view.  The mantra of “the best tool for the job” was frequently repeated in the early days.

They scored some early wins with Microsoft’s Scott Guthrie announcing the release of ASP.NET Model-View-Controller (MVC) components at their premier open space meeting in October 2007, and there were lofty plans to help open chapters around the world.  A wiki was set up and the community (me included) began to help build a repository of best practice reference material.

Things went well until later that year and into 2008 when the community began to descend into infighting and a lot of rather harsh criticism that they were becoming to “cliquish” – some of the founding members ended up walking away and today, after taking the development world by storm and spawning dozens of podcasts, ALT.NET languishes in relative obscurity. 

The chief problem became the perception that ALT.NET was issuing decrees about what it was to be truly “ALT.NET”.  As a result, it became quasi-religious and so began to take on a spectrum of adherents with varying degrees of dogmatism.  You were either “with us” or “agin us”, a nary the twain shall meet.  Thus, I foresee the same problems hampering SEMAT:  It will collapse under its own weight as the signatories and signers-on get spun out in debates over whose methodologies or practices are better and what should be tested and what should not, and how and for how long.  In the end, it will winnow down to a small cadre of dogmatists who will have forced a good many people away.

And for what?  A fantastical “unified theory” of software development that doesn’t exist, nor will it ever.

Footnote:

For an excellent critical treatment of the out-of-the-gate problems with SEMAT, see Jorge Aranda’s Nov 29/09 blog post, Against SEMAT.  He raises some excellent points that I won’t bother to reiterate here.

In the post, I set out to test a hypothesis that the majority of computer science schools in Ontario were not teaching these best practices, focusing instead on the "worst practices" that are contributing to the industry's significant problem in meeting targets like cost, scope, schedule and satisfaction - something I collectively refer to as "BDUF/waterfall" or "Big Design Up Front / waterfall" processes, which define the prevailing view of how software should be developed.

I tested the hypothesis by performing a rough comparative analysis of the course calendars from each school - in much the same way a prospective student might when shopping around for a school to attend.  Unsurprisingly, I found my hypothesis held true with only four Ontario schools out of sixteen teaching best practices courses.

A Review of Canadian CompSci Schools

Naturally, I wanted to extend this analysis to the rest of Canada, so I set about gathering and sifting through twenty-five course calendars for schools from the Pacific to the Atlantic.  Below is the summary of my findings (click to enlarge).   As with the previous review, my raw research notes can be found via a shared Google Notebook I've created.

As with the previous iteration, this table illustrates three significant points of comparison between the schools:

1. Does the CS department offer a course exclusively in the best practice(s) ?
2. Does the CS department offer a course that teaches a best practice as a constituent component of a course ?
3. How many "software engineering" vs. "best practices" courses are required for a degree (H.BSc.)

With respect to the first point, cells are highlighted in orange, with contiguous "blocks" aggregated together with a border to indicate multiple practices that are taught in a single course - there are some clarifications for certain schools that I'll explain below.  With respect to the second point, cells are highlighted in blue.

I've also added two new comparison columns to break down the summaries: Object Oriented Design / UML, to indicate those courses that are definitive in the discipline, and Requirements Engineering to highlight this "bad practice", which I previously assimilated as part of BDUF/waterfall.

Highlights

As with the Ontario analysis, best practices aren't a uniform part of computer science curriculums across Canada:

• Of all practices, Object Oriented Development and Design Patterns make up the bulk of topic offerings for almost half the departments analyzed;
• Agile/lean/iterative practices are still in the minority, being offered by only six schools (U of Alberta, U of Calgary, U of Manitoba, McGill, Memorial University and Saint Francis Xavier), three of which are in a dedicated course offering (U of Alberta, U of Calgary and U of Manitoba), and of these only two (2) or 8% are degree requirements (Alberta and Manitoba);
• Refactoring is only offered by four schools (U of Alberta, U of Saskatchewan, U of Manitoba and McGill), three of which make it a part of a dedicated best practice course.  Of these schools, only two (2) offer courses that are within degree requirements.
• Agile Requirements are pretty much unheard of, being offered only by two schools (U of Alberta and McGill), one of which is within core degree requirements.
• BDUF/Waterfall processes are overwhelmingly pervasive, comprising the majority of "software engineering" course offerings and degree requirements.

Exemplary Schools

As with Ontario, there are a few schools in Canada that stand out from the miasma with some top-notch offerings that others would do well to emulate - if you or someone you know is interested in taking Computer Science, these are schools to include in your shortlist:

University of Alberta @ Edmonton

• CMPUT 301 Introduction to Software Engineering:  This course is a clear winner that rivals its Ontario competition if only because the modules are targeted at 5/6 best practices backed up by recognizable industry names.  I was immediately impressed that the work of Mike Cohn on agile requirements gathering was specifically mentioned.  Only criticism: No TDD - but that's minor considering how much else is in the curriculum.
• CMPUT 201 Practical Programming Methodology:  This course caught my attention because it takes a pragmatic approach to imparting professional skills to students.  One quote in particular blew me away: "Like the best carpenters and artists, the best programmers combine both a solid theoretical foundation with "best practices" and tools refined by years of experience... CMPUT 201 is a "doing art" and not an "art appreciation" course; it requires hands-on, active learning. For a programmer, the end product is a complete, working program."
  
  A definite winner, even though the course substance isn't totally within the realm of what I define as best practices.

University of Calgary

• Software Engineering 515 Agile Software Engineering:  While there wasn't a more detailed course outline, this non-required course stands out as a unique, dedicated offering.  Unfortunately, the prevailing thinking in the department is definitely biased toward BDUF/waterfall, as evidenced by the remaining courses.

University of Manitoba

• COMP 3350 Software Engineering:  This degree core requirement course offering is on-par with Alberta's CMPUT 301, covering test driven development, refactoring, design patterns and a gentle introduction to "appropriate agile processes" for a team project.  The textbook, Sustainable Software Development - An Agile Perspective by Kevin Tate is worth having on the bookshelf and hits all the right notes.
  
  I really liked how this course tied in TDD - even if it is missing the agile requirements gathering that Alberta's course has.
• Unfortunately, all the good of COMP 3350 may be totally undone by COMP 4050 Project Management which emphasizes a litany of worst practices, including project definition using PERT, Gantt charts and critical path analysis.  I say unfortunate because 4050 is also a core required course for the software engineering stream.

McGill - Quebec

• COMP 303 Software Development:  The sole exemplary offering from La Belle Province of best practices follows the formula set by U of Alberta, U of Calgary and U of Manitoba with this required course for CompSci/Software Engineers.  Design patterns, refactoring, unit testing and design-by-contract and top-notch OOD practices (eg. separation of concerns) are all covered.  While the required text is Java-biased, it does cover design patterns and using the JUnit unit test suite.
• COMP 304B Object Oriented Design:  While not a required course, this should be taken in conjunction with COMP 303 as it focuses exclusively on OO design fundamentals, including Gang of Four design patterns, which incidentally, is also a required text for the course.  Unit testing with Python is also done - a little weird, but it's good to stretch the mind beyond the C/C++/Java curly-brace languages.
• COMP 335 Software Engineering Methods:  Another course that teaches an agile/incremental process for guiding software projects - this one focusing on eXtreme Programming along with unit testing, teamwork and project scoping.  The required text isn't that great, but hopefully the profs can relate the material well enough to avoid relying on it.  Again, since this isn't a required course, it would be good to take along with the two previously mentioned courses to limit the damage done by other BDUF/waterfall offerings.

Honourable Mentions

University of British Columbia

• CMPT 275-4 Software Engineering:  Covers unit testing and OO analysis, which unfortunately get mired in some BDUF/waterfall practices.  It earns an Honourable Mention because of the texts in its recommended book list:  The Mythical Man Month by Fred P. Brooks, Professional Software Development and Code Complete by Steve McConnell.  While I disagree with Steve on a lot of points, he's a recognized name in the industry that students should be aware of - and Fred Brooks?  Obviously.
• CMPT 475-3 Software Engineering II:  Designed to pick up where 275-4 left off, this one I mention again because the course text gives me hope that the course isn't as bad as I think it is:  Facts and Fallacies of Software Engineering, by Robert L. Glass.

Bishop's University - Quebec

• CSC328 - Object-oriented Software Construction:  This course has a lot of potential from the outline to expand to cover a lot of best practices.  In its current incarnation only design patterns and UML are covered, but it does so through coding in four OO languages:  C++, Smalltalk, Eiffel and Java.  Of these, Smalltalk and Eiffel have a long pedigree of spawning best practices like test driven development, refactoring, etc.

Memorial University - Newfoundland & Labrador

• COMP 3716 Software Methodology:  Covers iterative software development with an eye toward comparing it critically with traditional BDUF/waterfall processes - this is rather unique across the board - alongside design patterns and UML.  The course text isn't the best, but the author, Craig Larman, is a recognized name in the industry in pro-agile circles.
• COMP 3718 Programming in the Small:  This course stuck out to me from its title as it implied an attention to detail - and that pretty much sums it up.  Backed by Brian Kernighan and Rob Pike's book, The Practice of Programming, this course looks to impart some fundamental skills.  Not so much "best practice" as de facto rules of thumb that contribute to professionalism.  Should be a required course!

Acadia University - Nova Scotia

• COMP 3783 Advanced Object Oriented Application Development with Smalltalk:  Smalltalk didn't win the war of OOD language supremacy - that went to Java and C# - however, a great deal of work in best practices was done using Smalltalk, and for that reason I recommend this course over the identical offering COMP 3773 which focuses on C++ and is required.

Personal Software Process

A few schools in my analysis have been promoting Watts Humphrey's PSP as part of their software engineering stream.  Far from being about personal development as a developer or a better way to manage small projects, PSS is a forerunner component to Team Software Process and the Capability Maturity Model (CMM), each of which emphasize the notion of coercing agile tenets within a heavy process framework.  I recommend avoiding any PSP offering as its adoption is near zero in the industry and is predicated on the notion of increased up-front planning yielding a better product without a customer feedback loop.

Conclusions

Across both the Ontario and Canadian school analyses, while a complete offering of best practices is still not supported by the majority of computer science departments, the ones that do are leading the way with some top-notch efforts which go a significant way toward putting the lie to the naysayers who often trumpet that these efforts "can't be done" and "don't believe" in their worth.

Unfortunately, the majority view of BDUF/waterfall practices continues to hold sway and degree required courses vary wildly without consistency, rhyme or reason:  Some departments require six software engineering courses, others two and still yet others none.  In terms of best practice courses, the number drops dramatically with the norm being zero.

As with the Ontario analysis, there are a myriad of potential reasons for these discrepancies which can be attributed to each school setting their own curriculum.  While this is their prerogative, it's unfortunate that there isn't even an accepted baseline to work against that promotes best practices. 

Additionally, the same discrepancy between "computer science" and "software engineering" stratifies who's required to learn best practices - and this doesn't help graduates at all.

In sum, there's still more work to be done to promote best practices that can be affected by business (who can demand it from schools) and by students themselves to pressure their departments to improve their offerings.  I've listed four schools who can serve as a starting point - I'd love to see four more join the list in four years!

Let me know your thoughts and observations.

Update: I've amended my observations under point #2 below to take into consideration some feedback I've been receiving.

Update:  I've revised the results table to highlight the best practices that are offered as an exclusive course and as a potential constituent unit, with the former in orange and the latter in blue.  Additionally, I've adjusted the totals for each best practice column to reflect the differences.

Update 2:  Dr. Gregory Wilson, Adjunct Professor of Computer Science at the University of Toronto recently contacted me to point out that they have a specific course offering in agile best practices (CSC301).  While I do observe this in my table of results and raw research notes, I didn't make it clear in the bulleted highlights.  CSC301 is exemplary in that it not only instructs on agile, but also design patterns AND refactoring in addition to other topics.

Update 3:  I've released my findings for CS schools across Canada here.