Update: I've amended my observations under point #2 below to take into consideration some feedback I've been receiving.
Since releasing the results for Ontario in my previous post, I've been applying the same methodology to the remaining universities in Canada. Working from west to east, I've just finished with Quebec, and have the Atlantic provinces to finish up with today.
In the interim, I wanted to post some observations from my rough research so far that I found interesting, and in some cases indicative of what is thought of as "best practices" in CS departments.
1) Requirements Engineering
A number of departments offer courses in "requirements engineering" which in every case equates to "worst practices" as they instill the false conviction that the software development practice is improved if we push more effort into precisely specifying and freezing system requirements up-front.
Oftentimes you might hear that people want to “freeze” requirements... You can't freeze requirements any more than you can freeze markets, competition, learning, evolution, or growth. And even if you tried, you'd almost certainly freeze the wrong ones.
If all you do is take [your customer's]... initial requirements and implement them, you will certainly not be anywhere close to satisfying their requirements by the time of delivery -- the requirements will have changed. You're exposing yourself to one of the biggest risks in software development: you've produced what they asked for, not what they've come to want. The result? Surprise, shock, and disappointment instead of satisfaction.
2) Computer Science vs. Software Engineering
As I sorted through all of the calendars to piece together their programs and requirements, I noticed a curious pattern where some schools made a distinction between a degree in computer science with no specialization and one that focuses on software engineering. The difference between the two fell squarely on whether any "best practices" courses were required, and even then the requirements for "software engineers" varied wildly from one best practice course to four!
This struck me as quite short-sighted: In effect, it was entirely possible to get a graduate who could develop AI systems, write a compiler and analyze complex algorithms yet be completely dysfunctional in a software development team - all due to whether they took "Computer Science" or "Software Engineering".
This distinction is damaging, as it leads to some misconceptions about how software is developed, ie. that it is possible to "engineer" software as much as it is any physical object or machine. In 2005 I wrote about this disconnect under a post called That Damned Construction Analogy. At the time, I referenced Martin Fowler's essay, The New Methodology where he states:
In software all the effort is design, and thus requires creative and talented people. Creative processes are not easily planned, and so predictability may well be an impossible target.
In my opinion, when it comes to best practices there should be no distinction between "Computer Science" and "Software Engineering"; in fact, I'd go a step further and remove the distinction entirely as the notion of "engineering" software has very little to do, as Fowler observes, with how software is actually constructed.
3) Those that do teach best practices, do it well
Overall, the departments that have gone out of their way to create courses that focus exclusively on a particular best practice or basket of practices appear to do it really well. From content to labs to textbooks, it's a course worth taking - and unfortunately, isn't usually a required one.
Some examples include:
McMaster's SFWR ENG 3S03 Software Testing and Management;
Ryerson's CPS 845 - ExtremeProgramming and Agile Processes;
Queen's CISC 327 - Software Quality Assurance;
University of Toronto's CSC 207H1 Software Design and CSC 301H1 Introduction to Software Engineering;
York University's CSE 3311 Software Design and CSE 4313 Software Engineering Testing.
There are even more and better examples from other schools in Canada that I'll review in when I release the results of my expanded analysis next week.
4) Most online course calendars are abysmal
In the interests of "dogfooding", I think a lot of schools would benefit from having their students evaluate their department's website and building a new one. In almost every instance they are extremely frustrating and laborious to use: A perfect example of what "software engineering" is doing to the industry!
A final thanks:
I see that Mike Gunderloy published a link to my post on his blog, Larkware.com - and that's put it in front of a good number of eyes. Thanks for that, Mike!
Also, thanks to all who have contacted me about the article - it's great to get feedback on it and hear other opinions and observations.
Next Week: The results from my Cross-Canada "Who's Teaching Best Practices?" CompSci School Review
Update: I've revised the results table to highlight the best practices that are offered as an exclusive course and as a potential constituent unit, with the former in orange and the latter in blue. Additionally, I've adjusted the totals for each best practice column to reflect the differences.
Update 2: Dr. Gregory Wilson, Adjunct Professor of Computer Science at the University of Toronto recently contacted me to point out that they have a specific course offering in agile best practices (CSC301). While I do observe this in my table of results and raw research notes, I didn't make it clear in the bulleted highlights. CSC301 is exemplary in that it not only instructs on agile, but also design patterns AND refactoring in addition to other topics.
Update 3: I've released my findings for CS schools across Canada here.
A sad fact of the software industry today is that almost every IT project initiated will fail in one way or another. Delivery schedules will be blown, budgets exceeded, customers frustrated and developers burned out. The combined costs of these failures are estimated in the billions of dollars, and in some cases lives risked and lost.
Indeed, it's been well-understood for decades what factors contribute to project failure, and many astute minds have made significant advances on the matter from Fred P. Brooks to Martin Fowler, Robert C. Martin, Ron Jeffries, Kent Beck, Ken Schwaber, Jeff Sutherland, Mary Poppendieck and more. They have all contributed to a body of work that is collectively referred to as "best practices" that help steer projects away from the rocky shoals of failure as best as possible.
The obvious question arises: "Why are failures so high, when there's been so much work done to help avoid them?" While the answer is definitely a combination of factors, I decided to test a hypothesis that computer science departments are failing to teach "best practices", resulting in legions of graduates who are armed with antiquated or theoretical knowledge that help perpetuate the cycle of project failure.
The Methodology:
In order to test my hypothesis, I decided to do a comparative analysis of the computer science programs offered by the sixteen major universities in Ontario. Specifically, I wanted to look at their course calendars to see how many offered instruction in best practices and if it was possible to earn a degree without exposure to any best practices.
Best Practices Defined:
For the purposes of my investigation, I focused on six areas of discipline for best practices based on my own experience and generally agreed upon in the industry:
-
Agile / Lean / Iterative software development - any form of incremental process that emphasizes repeated cycles of development and feedback loops;
-
Refactoring of code to improve design (per Martin Fowler);
-
Design Patterns as popularized by the Gang of Four (Gamma, Helm, Johnson, Vlissides) or others;
-
Test Driven Development (unit testing), Continuous Integration, Acceptance Testing;
-
Performance testing through code profiling, metrics and load tests;
-
Agile requirements specifications - ie. writing user stories, relative weight estimation, task breakdowns, etc.
In addition, I wanted to know about courses offered that did not fit the above criteria, which I defined as Big Design Up Front / Waterfall "bad practices". Often these fall under the rubric of "software engineering".
The Results:
Below is a table that I constructed from the raw data I was able to glean from each university's online course calendar for the current 2007-08 academic year. At a very high level, I wanted to answer the question of whether the topics were covered, and have indicated with "Y" and "N" if the outline specifically mentioned the best practice in question. Those departments that offered instruction exclusively on a given practice (as opposed to a constituent unit) are highlighted in orange.
The last two columns in the table indicate how many "software engineering" vs. best practices courses are required to obtain a computer science degree.
Highlights:
-
Of the sixteen schools, less than one third offer any instruction in best practices with McMaster, Ryerson, University of Toronto and York alone offering exclusive instruction in agile software development practices;
-
While it is possible to earn a computer science degree at three institutions without exposure to any "software engineering" instruction (Laurentian, Ottawa and Western), less than one third require best practices instruction for their degree programs (McMaster, Queen's, Ryerson, Toronto and York)
-
Courses in "software engineering" at all schools overwhelmingly emphasize BDUF/waterfall/phased "worst practices";
-
No school offers a full compliment of best practices courses as either electives or degree requirements.
Perhaps most surprising in the findings was that the University of Waterloo offered no best practices courses. This is surprising given Waterloo's vaunted status as the go-to school for IT/software and happy hunting ground for Microsoft.
Conclusions:
Based on this very rough assessment, it seems that post secondary schools in Ontario are failing to impart recognized industry best practices to their students. I imagine that similar findings can be gleaned from other provinces and perhaps most U.S. schools - I hope to expand this research in the days ahead.
The net effect is that students are paying for knowledge that is a serious detriment to their profession and industry, and that a lot of post-graduate effort needs to be expended by employers (unlikely) and the students themselves (more likely) to become aware of and proficient with industry best practices - to go, as Andrew Hunt and David Thomas observe, from journeyman to master.
Unfortunately, as long as there is no demand from business to have better, more professional graduates who are skilled in these recognized best practices, software failures will continue to pile up, while those of us who are still in the game become increasingly cynical about an industry that seems indifferent to success.
It's time for a change, n'est-ce pas?
 This past Labour Day I took a trip to my local chain bookstore to pass an afternoon, get a cup of wonderful Starbucks and see what's new on the shelves. As a reformed computer text junkie who once had a floor-to-ceiling collection of O'Reilly, Wrox and Microsoft Press books, I have to admit it's a bit of a voyeuristic thrill.
I was perusing the software development racks when my eyes alighted on three books side-by-side that I thought neatly illustrated the differences between best practices and worst practices. Here are the texts in the order I found them:
Let's review these in turn, with the caveat that I only skimmed these texts for about 10 minutes each - which, in my experience is plenty of time to get the gist of the book as intended by the author.
Practical Software Estimation
It's beyond me how this book saw the light of day in 2007 when it's thinking is firmly rooted in the dark ages of software planning and estimation. In brief, this book's thesis revolves around the Function Point Method of information systems estimation that Alan Albrecht developed for IBM in 1979. It presumes, in good BDUF fashion, that to provide practical software cost/time estimates, we only need to know, up-front, all the functions that will go into the system.
Besides the fact that this is sheer fantasy, the clincher that this book should be relegated to the dustbin was a diagram where the famous "phases" of software development were laid out as the "ideal software delivery model" - ie. BDUF through to testing at the end.
Suffice to say, if you pick this book up and become swayed by the poppycock about how the methods are used at InfoSys and decide that it's how your next project will be run, well, you'll get what you paid for - which ain't much.
SCORE: 0/5
Continuous Integration
CI is, sadly, still an emerging best practice in the industry that throws a spanner in the works for BDUF adherents because it is predicated upon the heretical belief that systems should be continually integrated, compiled and tested. At least once a day. Whenever anyone checks in code. Automatically.
The lads who put this text together have some experience working with best of breed tools to make CI happen, ie SubVersion, JUnit and CruiseControl. Now, if your team is using different tools, you may need to do as the Marines and improvise, adapt and overcome!
All this said, they do provide a lot of practical information and snippets on how to set up a CI solution and have it up and running quickly. It's a practical guide for pragmatic developers. Semper fi!
SCORE: 4/5
Effective Prototyping for Software Makers
This is a book that every software professional should have on their shelf, in my opinion (yes, I'm getting a copy!) because it espouses the very important principle that in software development, design matters. This idea was impressed on me many times by a colleague at imason whose discipline was focused on usability, and has stayed with me because of the conceptual simpatico with agile/lean practices.
This text presents a very accessible way to model software applications using rapid prototypes that end users interact with early and frequently throughout the development process to keep refining the product. To what end? Why, a happy customer/end-user of course!
I was intrigued at the variety of strategies that the authors described, from traditional wireframes to mock-ups using Excel to the controversial Wizard of Oz mock-up where humans simulate how a system might behave once realized. All combined, they show how better systems can be developed through prototypes that help bring the actual system closer into line with customer expectations - a radical concept in the industry!
Ultimately, I believe Effective Prototyping is a must-have book because it "break the blinders" that have constrained developers into believing that they are engaged in a manufacturing discipline and not new product development. Once this distinction is made, however, there's no going back - so be sure that you want to "unplug from the Matrix" before reading this book!
Be sure to check out the book website for more info on Effective Prototyping and sample chapters.
SCORE: 4/5
Back in May I wrote about a disturbing cartoon that I noticed at the bottom of the MS TechNet Flash newsletter called "Scripting Guy". It featured a hapless white-jacketed, bespectacled and be-beakered "guy" who for some reason was being asked questions about how to automate some task using script - often by what looks like some of the Village People.
I was disturbed in a Haley Joel Osment "I see dead people" way about the strip which for some reason ended with "Scripting Guy" getting a dimensional flattening because he works with idiots. I think we can all relate to that in some way.
But I digress; my horror at the cartoon was because of how God-awful it was - I believe I called it the love-child of Ziggy and Dilbert. It was that bad.
Well, it seems that in this month's issue (Vol. 9, #16) the creators of "Scripting Guy" have finally conceded defeat and admitted that much like Hollywood, they're completely out of fresh ideas:
Astute observers will note that panels #2 and #3 are the same in every cartoon, with the slight modification that occurs to "Scripting Guy" which serves as the "punchline" where we all have a chortle about how sad his life is and how, for some continued inexplicable reason, he is forced from his real calling as an Evil Mad Chemical Engineer to respond to inane questions about how to script lookups for AD accounts or versions for DLLs. By field hockeystick-wielding madmen. Who embed their hockey balls in his forehead.
With this edition, however, I am heartened to finally get some indication from the strip's creators that yes, they really are getting desperate.
My advice: Let Scripting Guy die with some shred of dignity, such as he may have - it could be anything: A bizarre scripting accident, another hockey ball to the head, or maybe he loses it, climbs a clock tower with a high-powered rifle and starts taking out all those idiots who keep asking him inane scripting questions before injuring him in some awful physical way.
I remain ever-hopeful and vigilant for you hard-working consultants, developers and agilists out there against this kind of cartoon heresy!
Addendum: In addition to the "general resistance to change" numbers that the surveys relate, it's instructive to put them in the context of the damage that's been done in the industry in just a short period of time. For reference, see this table from Robert Charette's 2005 article for IEEE Spectrum.
Agile software development company, VersionOne, has just released their second industry survey on the adoption of agile practices with some illuminating results on how much the project delivery landscape is changing. The survey is the only one of its kind, and with over 1,700 respondents participating from over 70 countries (an increase of over 120%), the State of Agile Survey is providing an increasingly statistically relevant sampling of how practices are changing - albeit slowly and shallowly.
Most interesting to me were the results behind two key questions:
a) What specific improvements have you actually realized from implementing agile practices;
In 2006, 87% of respondents said that increased productivity was their top result, followed by accelerated time-to-market (86%), reduced software defects (86%), and reduced costs (63%).
This year 90% of respondents said that increased productivity was their top result, followed by reduced software defects (85%), accelerated time to market (83%), and reduced costs (66%).
What's interesting here is that adopters are strongly indicating, year-over-year, that they are benefiting their companies through increased productivity while reducing the number of defects in their products. This is a significant indicator that agile practices, be they scrum, XP or other, are delivering against their promises - which to me, is hardly surprising, but to some may seem like we've just tumbled with Alice through the looking glass.
You know who you are.
b) What are the barriers to further adoption of agile in your current organization.
In 2006 about the same number of respondents (20% vs. 21%) indicated that finding qualified personnel and resistance to change were holding back agile penetration. This year, the numbers jumped significantly, with 36% responding that resistance to change was the number one factor, while qualified personnel was not far behind at 34%.
I find these results interesting for two reasons: First, there's still a shortage of agile-qualified people to help facilitate change and coach projects. This means there's a lot of work out there I can do!
Second, the undeniable effect that resistance to change is having on the industry. I've written at length on how waterfall/BDUF is ruining software, costing billions, risking lives and destroying careers. Now I have statistics that show this in black and white.
Other results that piqued my interest included the increased adoption of Scrum over XP (37% vs. 12%) which bodes well for adoption rates as it is easier to understand and practice, and that the key agile practices of iteration planning, unit testing and continuous integration are being taken up.
Overall, this year's survey is showing that the trends toward adoption are moving in the right direction, even if at a glacial pace. I'd be more optimistic about this, but to me it's a case of Nero fiddling while Rome burns - there's a lot at stake while we continue to play the How Long/How Much game and it is readily apparent that education about agile needs to occur at the executive level as well.
Download the 2006 survey here. (PDF)
Download the 2007 survey here. (PDF)
|