I’m here in Rome this week, presenting a two-day seminar on  “Managing Death-March Projects” for Technology Transfer Institute. You should be there so you can hear whatever clever jokes may occur to me while I’m presenting my material, as well as the comments and questions from the other participants. But if you’re stuck in some other part of the world, or you couldn’t persuade your boss to send you to Rome, you can click here to download the (17MB) PDF version of the presentation, which has a whole  bunch of embedded links to other presentations, publications, books, articles, websites, etc.

 But after a week of watching a gut-wrenching roller-coaster ride on the New York stock market, and reading various gloom-and-doom predictions of bad economic times ahead, I thought it would be more appropriate to replace that talk with a presentation on “Death-March 3: Software Processes in the New Hard Times”:

If you click on either icon, you’ll download a 12.2-megabyte PDF file that actually contains both presentations. So you can look at either one of them, depending on whether you’re feeling optimistic or pessimistic. Enjoy … or don’t.  

Assume, for the moment, that you’ve got a software development project where the end-users (or customers, stakeholders, or whatever else you want to call them) and the developers (i.e., programmers, architects, database designers, business analysts, QA personnel, etc.) are all working in company X, here in the U.S. And imagine that you have another software development project, of equivalent size and complexity, where the end-users are in company X, but the development team is located in an offshore outsourcing company on the other side of the world (e.g., China, India, Singapore, eastern Europe, etc.) Why is it that Michael’s software metrics are showing that project #2 will have approximately 2.8 times as many software defects as project #1?

Two reasons, the first of which is familiar and predictable: precisely because the end-users and the developers are not “co-located” (in contrast to many of the “agile” or XP development projects, where a “customer proxy,” or end-user representative, is literally sitting in the same office cubicles, alongside the programmers), communication problems escalate; and if the two groups have different languages and different cultures, those problems become even more insidious and subtle. For the past several years, many of the top-notch outsourcing firms have pointed to the videoconferencing, Webex, email, and other collaboration tools that they’ve used to reduce these communication problems; and they’ve often co-located key developers in the midst of the end-user working environment to further reduce communication problems. But according to Michael, the communication problems still exist; and they contribute to higher defect levels. The world, as he likes to say, is not flat: you can’t just assume that software developers can be sprinkled randomly around the globe, where they’ll participate in a complex software development effort with no problems at all.

The second reason for the higher defect rate is one that I would not have predicted, though it’s not rocket science. Because offshore outsourcing firms typically offer an essentially unlimited number of technical personnel at a lower labor rate, many U.S. firms think they can achieve faster delivery schedules by hiring larger teams of offshore developers than they would if they staffed the project locally — and yet still save money. Thus, if the U.S. firm had previously staffed a project with 10 “expensive” local programmers, they figure they can hire 20 “cheap” Indian programmers, and still save money — while (in theory) cutting the schedule in half, because they’re blissfully unaware of Brooks’ Law. But aside from the schedule issue, Michael reminds us that software defects are strongly correlated to the number of “lines of communication” between the developers on the team; and that number increases combinatorially (i.e., N*(N-1)/2) as the size of the team increases (this is a variation on Conway’s Law, first articulated by computer programmer Melvin Conway in 1968, which says that the structure of a system is isomorphic to the structure of the organization that builds it). Hence, the number of defects in a software system increases in a manner proportional to the square of the number of developers; and as a result, he says, offshore projects are more likely to deliver buggy software.

All of this was part of a fascinating presentation that Michael gave at the New York City SPIN group this evening, to an audience of roughly a hundred software engineers and project managers who kept staring nervously out the window to see if a much-threatened blizzard was about to begin. The title of his presentation was “XP and Productivity Measures — What the Numbers Say,” and the bottom-line conclusion was that XP/agile projects deliver systems 50-100% (correction: after reviewing what I had written, Michael told me the correct figure was 25-50%) faster, with roughly half as many defects, and often with fewer developers, than “average” or “traditional” projects. This was illustrated with two case studies of U.S.-based development efforts — with internal/local development teams, and no outsourcing — whose results were compared against a massive database that Michael’s company, QSM Associates, has developed over the past 20 years. The metrics database contains information about roughly 7,300 projects representing 685 million lines of code, from 500 companies in 18 countries, written in 600 different programming languages. After he finished presenting the two case studies, Michael then presented a third case study involving a large offshore outsourcing effort, which culminated in the 2.8X offshore-to-average defect ratio discussed above.

The first company was a medium size project at a company called Follett Software, which Michael described as the “Amazon.com of the educational marketplace.” The project involved roughly 24 developers, 3 project leaders, and 7 testers; and the overall team produced approximately 1 million lines of code, with 7,000 automated unit-test cases, and 10,000 automated acceptance-test cases. What was significant about the project, from an extreme programming (XP) point of view, was the emphasis on the office environment and working conditions; Michael showed us several slides of the “high energy-level” single-room “bullpen” environment, and the dual-screen team-programming working environment.

“Core metrics” of size (lines of code), time (calendar months), effort (person-months), and defects were captured by Michael and his QSM colleagues after the project was finished, so that they could be compared against the industry averages of the 7,300-project database. (The four metrics mentioned above can be combined with a fifth, derived, metric of productivity to produce a total of “five core metrics,” about which QSM founder Larry Putnam has co-authored a book: Five Core Metrics: The Intelligence Behind Successful Software Management, which you can order from the friendly people at Dorset House Publishing.)

Bottom line: the Follett Software project showed staffing levels approximately the same as the average project in the QSM database; but the various “iterations” of XP deliverables occurred roughly 50-100% faster than average; and the number of defects was about half the QSM average. A second case study, from an unnamed company that builds medical instrumentation/testing equipment, had similar results. Indeed, Michael presented us with very detailed statistical summaries of the results, which I dutifully recorded; but just in case the figures might be considered confidential or sensitive, I’ll let you contact Michael directly if you feel you need to know about them.

As for the third case study, which involved a large outsourcing effort, it’s worth noting that the news was not all bad. As Michael explained, the “direct” cost of the offshore development was approximately 10% lower than that of the QSM average for a project of the same size. And the delivery schedule was 9.6 calendar months, in contrast to the QSM average of 12.3 months for a project of the same size — which represents a non-trivial improvement of approximately 21%. But this has to be balanced against the defects: an average project (of the same size) in the QSM database would have had 2,702 defects, but the outsourced project had 7,565 defects — i.e., 2.8 times higher. And as Michael pointed out, fixing all of those additional defects would ultimately have required more time, and more money — thus reducing, if not eliminating altogether, the initial benefits of lower labor costs and faster delivery of (buggy) software.

All in all, it was a thought-provoking presentation, with some provocative and intriguing insights into XP, agile development methods, and offshore outsourcing. Fortunately for all of us, the snow had just barely begun falling as Michael wrapped up his talk, and I scampered out of the building as fast as I could, so that I could get back to my office-at-home and jot down these notes before wrapping things up for the day.

So if you’re in the neighborhood, drop in and listen to the presentation. I’ll do my best to take notes and write up a blog posting later tonight or tomorrow. Stay tuned…