But after a week of watching a gut-wrenching roller-coaster ride on the New York stock market, and reading various gloom-and-doom predictions of bad economic times ahead, I thought it would be more appropriate to replace that talk with a presentation on “Death-March 3: Software Processes in the New Hard Times”:

If you click on either icon, you’ll download a 12.2-megabyte PDF file that actually contains both presentations. So you can look at either one of them, depending on whether you’re feeling optimistic or pessimistic. Enjoy … or don’t.  

I initially contacted two companies, based on recommendations that I found in Timothy Ferriss‘ The 4-Hour Workweek: Brickwork India, and GetFriday. The both responded within a day or two, and both quoted rates in the range of $10-20/hour for their services (with lower rates for higher volumes of work); however, I got the impression that both companies have become somewhat overwhelmed because of the attention they’ve gotten from books (e.g., the Ferriss book), newspaper articles (e.g., “Outsourcing Your LIfe,” in the June 2, 2007 Wall Street Journal) and blog articles all over the Internet. Indeed, I was unable to reach an agreement with Brickwork, because the smallest “chunk” of time they could deal with, for a one-shot outsourcing task, was 30 hours. Since the whole point of the initial task was to see how well it worked — if at all — I wasn’t comfortable risking several hundred dollars, especially since the nature of my initial task was almost certain to take far less time.

However, I did reach an agreement with GetFriday, and I sent off the details of my first task: go through one of the presentations on my Slideshare page, and test each of the hyperlinks I’ve embedded throughout the material, to ensure that it’s not broken (i.e., something that would generate the dreaded “404″ error message from a browser), and that the hyperlink actually points to the right page. You don’t have to be a rocket scientist or a brain surgeon to carry out such a task; and if I had been willing to take the time to export the presentations to HTML files, I could have tracked down a relatively inexpensive program that would have scanned for broken links automatically. But this would have taken a couple of hours of my time, and it would have cost a modest amount of money (the last time I tracked down such a program, which I’ve since lost, it cost about $50), and it would not have found the embedded hyperlinks that were “valid” (in the sense that they pointed to a real page out on the Internet) but nevertheless wrong.

So I sent off an email message describing the task to be done; and I’m pretty sure that I specified that only one of my presentations should be reviewed, with a “ceiling” of no more than 16 hours to be spent on the task. I had been told, when I first signed up with GetFriday, that they were so swamped with business that they wouldn’t be able to actually start working on my tasks for three weeks. I was a little disappointed, but figured I had nothing to lose; and I resigned myself to the likelihood that, with the Christmas holidays looming ahead, I probably wouldn’t actually see any work being done until the beginning of January.

During the ensuing six days after I sent off the description of my task, there was basically silence from GetFriday — except for a couple of cheery “Have a great day!” messages from time to time. But this morning, I got an email message with a lengthy Microsoft Word document detailing all of the errors they had found in my embedded hyperlinks. More interesting was the fact that instead of reviewing just one of my presentations, it appears they had reviewed all three that they found on my Slideshare page. And even more interesting was that the aggregate effort for doing all of this was 11 hours. To give you a sense of what’s involved, the largest of my three presentations (“Web 2.0, version 50“) is 122 pages long, with roughly 500 embedded hyperlinks.

I’ve done a quick spot-check of the work, and it seems to be accurate. I’ll know for sure when I do the work that, unfortunately, I can’t easily outsource: manually examining the broken links and figuring out whether the Web page I was linking to has expired or moved somewhere else, or whether I had an utterly incorrect URL (the default URL for embedded links in Apple’s Keynote program, which I used for these presentations, is www.yahoo.com — which is almost certainly not what I intended to use for the link). But without having outsourced this initial activity, I doubt that I would ever have gotten around to the second part of actually making the corrections.

Meanwhile, I’ve got a couple more small projects that I’m going to send GetFriday; these are tasks that I need to accomplish, and they’ll give me an opportunity to practice describing tasks more clearly and crisply, as well as refining the communication protocol with the “primary assistant” (PA) who coordinates the actual performance of my task by a “team pool” of anonymous workers.

This concept of a PA and a team pool, shown below (and described in more details here on GetFriday’s web site) is actually one of the reasons I think it makes sense for me to work with an offshore outsourcing firm, rather than simply hiring a local college kid at roughly the same rate of $15-20/hour. Eventually, I hope to outsource some work that will actually require the equivalent training and education of someone with a B.S. or M.S. college degree; but finding broken hyperlinks in a Powerpoint document could be done by any reasonably intelligent college kid — heck, maybe even a high school kid. But finding such a kid would be a nuisance; managing and babysitting them would also be a nuisance; and dealing with their unpredictable moods and vacation schedules (e.g., “hey, dude! It’s spring break and I’m going to Cancun; if the sex, drugs, and rock-n-roll is as good as my buddies are telling me, I might not ever come back!”) would be an even bigger nuisance. With GetFriday, I don’t have to worry about any of that; I don’t even have to know who’s doing the work, as long as I can communicate via email with my PA.

All in all, it’s been an interesting experience so far. I’m sure I’ll need some time and practice to make it work as smoothly and productively as possible, but I’m pretty optimistic. I’d recommend that you give it a try, but frankly, I’d be just as happy if you didn’t. After all, if I can use a low-cost, well-managed workforce to make myself more productive and efficient, then I’ll be that much more competitive than you… heh, heh, heh …

Well, both of them have now contacted me, and I’ve had a couple back-and-forth interactions. I’d still prefer not to give you their names, since I haven’t actually gotten started with either of them, and thus don’t know how good a job they do. But the initial interactions have been interesting, and worth passing on to you.

One of the companies responded right away, told me they could certainly handle the three modest tasks I had asked them to bid on, and suggested that I choose an ongoing, month-to-month “retainer” arrangement for either 10, 20, or 40 hours per week. The rates were in the $12-15/hour range, based on the volume of work. When I wrote back to ask whether it would be possible to arrange a single, non-continuing contract for a one-time task, I got a response suggesting that I could contract for a 40-hour block of time, for $800. Aside from the fact that this represents a higher $20/hour rate, it presents two other problems: (a) I risk wasting $800 if they do a lousy job, and (b) I’m not at all sure the initial tasks I had in mind will take anywhere near 40 hours of effort. So I’m still pondering this, and haven’t decided how to proceed.

Meanwhile, the second company responded with a request that I fill out a “membership form,” which included (among other things) a credit card number to which their services would be charged; this was to be faxed back to a toll-free “secure” fax number. Obviously, there’s some risk of credit card fraud here, but it’s a well-known company, and I can always shut down the credit card if things get out of control. So I filled out the form, faxed it back, and got a response the next day indicating that (a) my form had been “accepted,” but (b) the company was so busy they wouldn’t be able to start working on my tasks for another three weeks, at which point someone would contact me to discuss the details of what I want them to do. Well, that takes us through the Christmas holidays, so I won’t be hearing from them until the beginning of the New Year.

There are no great “life lessons” to be learned from any of this; it merely indicates that initiating a “personal outsourcing” initiative is not a trivial, instantaneous process. I tend to be a little (actually, a lot) impatient, and I was hoping I would have already gotten results from my initial tasks. But I’ll forge ahead, and see how things develop with these first two companies. Meanwhile, I’m planning to contact a couple of individuals — one in Mexico, one somewhere else — through a third “virtual outsourcing” firm.

Stay tuned … I’ll post updates from time to time …

It was my wife who first suggested to me, a few months ago, that I should consider getting an offshore “personal assistant” to help me with various work-related chores that take up far too much time, provide little or no intellectual benefit or financial compensation, and could be easily offloaded to anyone with an Internet connection. I thought it was a sufficiently interesting idea that I added it to my “undated” to-do list — along with a few dozen other vague, ambiguous projects that I may or may not ever get around to (“Climb Mt. Kilimanjaro before the snow melts from the peak”, “Visit South Pole while there still is a South Pole,” “Write Great American novel before people forget how to read,” and “Swim the English channel before it becomes so polluted that you have to walk all the way” have been on the list for years). But I had no idea where I would find such an offshore personal assistant, nor how I would go about setting up a working relationship … so the idea languished, and was destined to suffer the same feat as my Kilimanjaro project.

But then I started reading Timothy Ferriss‘ The 4-hour Workweek on my Kindle machine last week, and I found some of the specifics I needed, plus a bunch of straightforward, common-sense suggestions on how to get started. Thus far, I’ve contacted two Indian firms, asking for a price/schedule proposal on three relatively simple, modest projects; and I’ve got a list of more ambitious projects that I’ll consider outsourcing once I see how these first ones go. I’m also going to try a few other “suppliers” of outsourcing work, including one in Mexico and perhaps one in Canada; it can’t hurt to get some real-world experience with a few of them before I decide which one to continue working with.

At this point, I’m not going to give you the names of the firms I’ve contacted, for two reasons: first, you should get your own copy of The 4-hour Workweek, so that Ferriss gets a small royalty for the intellectual effort he has expended. But more important, I don’t know whether either of the first two that I’m contacting will turn out to be worth recommending. Indeed, the first one has not responded at all in the 24 hours since I sent my first enquiry; I didn’t expect a full-blown proposal that quickly, but I thought they would at least acknowledge having gotten my message.

What sort of things have I chosen as initial outsourcing projects? Here they are:

• I’d like to get some market research indicating the percentage of automobiles, worldwide, currently being built with onboard GPS navigation systems. Having just gotten such a navigation-equipped car for the first time about a year ago, and having finally figured out how to use the damn thing, I am enjoying what David Brooks called “The Outsourced Brain,” in a delightful Oct 26, 2007 New York Times op-ed piece. I want to write a blog article about the experience of no longer having to know, in advance, how to get from A to B in my automobile; but I’d like that market research data first.
• I want to get a fairly comprehensive list of all the Web 2.0 conferences being scheduled anywhere in the world during 2008.
• Because I’m a somewhat fanatical amateur (digital) photographer, I want to get a list of all the classes and seminars about digital photography in the metropolitan New York area in the 2008 — but not the product-specific classes that teach you a hundred obscure features in Adobe Photoshop, or a hundred advanced tricks with the Nikon D80.

Could I have done any of these tasks myself? Of course! Indeed, at least one of them — if not all of them — might have been easier to carry out myself, rather than outsourcing it to someone on the other side of the world. But the whole point of these first three projects is just to see how it works, on a project that’s small enough that it won’t waste much money even if they do screw it up.

If it works, then I’ve got several larger, more ambitious projects I’d like to consider outsourcing — things that require a reasonable amount of intelligence, experience, and “Internet literacy,” but which will be a bargain if I can get it done for five, ten, or even twenty dollars an hour.While none of these projects will make me instantaneously rich and famous, I think they may give me a competitive differentiation in today’s “noisy” world, where it gets increasingly difficult to get anyone to pay attention to you (unless you’re willing to do something illegal and/or obscene, which I’m not interested in). So I’m not going to describe the details of these ambitious projects … until after I’ve carried them out, if I turn out to be lucky enough to do so.

One of the outsourcing services I’ve contacted also takes on “personal” tasks, in addition to the more traditional business tasks. I don’t feel any pressing need for this kind of outsourcing at the moment, but again it’s something I feel I should experiment with. The project that I’m thinking of, as my initial experiment, is to contract with an outsourcer to send Christmas cards to everyone on my address list; but I think I should probably “scrub” that list first, to make sure I don’t send Christmas cards to competitors, enemies, people I don’t really know, and obnoxious people I never want to hear from again. And it may be too late for such a project; if nothing else, it will be a good test of just how far in advance such projects do need to be planned.

Interestingly, several of the most annoying things on my to-do list — like sorting through the piles of papers, magazines, and irrelevant documents in my office in order to pare things down to a less cluttered environment — are not things that I can easily outsource. Sure, I could track down a local freelancer — maybe a hungry college student — to tackle some of the tasks; but I’m more interested in pursuing offshore outsourcing at the moment, and that requires activities and tasks that can be carried out via the Internet.

Perhaps the most important thing that has occurred to me is that while there may be some incremental value to having a less-expensive person tackle some of the items that are already on my to-do list, the real value is probably associated with using outsourced intellect/labor to take on the projects that are not on the to-do list. Indeed, the “ambitious” projects I’m hoping to assign after my initial learning experience are things that never would have gotten onto my to-do list — because I instinctively knew that (a) I didn’t have anywhere near enough time or energy to do myself, and (b) it would have cost a small fortune (or a large fortune) to hire someone in the U.S. to perform the task.

Having said all of this, I’m also well aware of the overhead, communication problems, and other risks that have been reported about offshore outsourcing in the business environment these past several years. But there are obviously some companies that have figured out how to overcome those problems, and from my frequent travels to India, I know that an enormous amount of software development work has shifted overseas (just as I predicted would happen, somewhat prematurely, in my 1992 book Decline and Fall of the American Programmer). And if big companies like IBM and Microsoft (and a few thousand others) can figure out how to make it work, then maybe I can too.

In any case, stay tuned. I’ll let you know how it works out…

Mr. Yourdon,

I have been a consultant for the past twelve years, I specialise in trying to bring death march projects in the financial sector to some agreed end. In the past three years, I have noted a disturbing reason these types of projects are initiated. Please forgive me, as I am going to raise an issue which is normally unaccepted in the political framework of the US.

In one word, it appears to be a result of racism. Defined as, members of one “race” are intrinsically superior or inferior to members of other “races.”

In the financial domain, sub continent teams are being brought into the US in large numbers. As you know these team members, though on-shore, are paid substantially less then their American counterparts, are not paid over-time, have no access to labour laws in the US, and are considered a very cheap and easily available commodity. The lack of respect and outright agressiveness these professionals encounter upon arrive from the enterprise senior managers, project owners, American technical team shareholders, users, and project managers is shocking.

I have witnessed countless and repeatable unexceptable behaviours made to these teams, here are a few examples

- Agressive comments made about their accents

- Comments about their body odour and cooking smells on the floor

- Comments about their ignorance due to their inabillity to understand how Americans and “real-world” works

- Outright refusal to meet with team members until they learn to speak English correctly

- Isolation, some enterprises now keeps “contract” help in separate buildings. These areas are sometimes refered to as the “curry floors” by employees

- Yelling and intimidation

The view that these teams are sub-human and powerless in face of the enterprise because of skin colour and origin, encourages managers to deliberately compress timelines even more. Respect for any team member in current business environment is hard to find, but when it comes to this group there is no effort made at all.

As a consultant, I do not have a solution to the racism problem but I have well proven strategies I use when engaged in such a projects.

Best regards,

Lambert O’Neill

I wrote back to him, asking if I could publish his message on my blog — and if so, whether he wished to be identified by name. Within an hour, he sent the following response:

Mr. Yourdon,

Please feel free to post this message on your blog. You may also provide my name. The reason I wrote you is I believe this issue is something IT/IS/Business Partner community should face and identify/implement solutions.

Indian team members and I have openingly discussed this issue. It is one appraoch I use to identify death march issues and risks and implement some basic solutions to negotiate an end point to the death march.

As you know, speaking out against “authourity” is not culturally acceptable in India. They also seem to express a fear they will be sent back to India and/or fired from their companies. As their are limited employment opportunities at home and they often have extended families relying on their income it appears to me that this is a system of a bounded labourer (indentured servants). History books tell us this system was abolished in North America in the 19th century, they maybe inaccurate.

My last death march project just finished, and I will be travelling through Europe until January 2008. However, I will occasionaly read emails and your blog. I look forward to reading the responses to the posting, which according to modern corporate culture most likely will be – complete denial.

Much thanks for having the courage to open up this issue.

Warmest regards,

Lambert O’Neill

If I were publishing a newspaper instead of a blog, I would obviously be compelled to check out Mr. O’Neill’s background, and corroborate his story with independent sources. And you might feel that, even as an informal, unregulated blog, I have an ethical obligation to do so. You may be right; but I’ve seen enough similar situations in my own travels through IT shops throughout the U.S. to find his story sufficiently plausible and credible to warrant bringing to everyone’s attention. On the other hand, I have not personally experienced anything as blatant and extreme as what he describes, so it was a bit of a shock.

While I can’t claim to have done any kind of in-depth investigation of the situation, I did type “outsourcing racism” into my Google search engine, and found a few interesting results:

• “Racism is no answer to outsourcing,” in the January 25, 2005 issue of the San Jose Mercury News.
• “Outsourcing Racism,” in the October 26, 2005 posting of James McGovern’s blog, “Enterprise Architecture: From Incite comes Insight…”
• A 5-page PDF “fact sheet” from the SAALT Exchange, written sometime in 2005, entitled “Addressing and Responding to Bias Related to Outsourcing.”
• A November 17, 2005 article in the San Francisco Chronicle entitled “Outsourcing Outrage: Indian call-center workers suffer abuse.”

And there’s probably more: these four documents came from the first page of Google hits, out of a total of 1,720,000 hits on the search term. If you know of any examples, incidents, reports, or other information that confirms or rebuts Mr. O’Neill’s characterization of “racist” behavior in offshore outsourcing, I’d appreciate hearing from you. Obviously, if there is more documented, confirmed, objective information of either a positive or negative nature, that would help all of us better understand what we’re dealing with.

Assuming that Mr. O’Neill’s story is basically accurate, I don’t think it’s sufficient to react by saying, “Such behavior is unacceptable!”, and then just pretend that it doesn’t exist. It reminds me of all the young parents I’ve met over the years, who listen to my stories of the exploits and escapades of my own children (all three of whom have grown into wonderful, happy, remarkably successful adults), and who say to me, with great indignation and a touch of condescension, “Well, my child would never do X!” — where X is any combination of unacceptable behavior, ranging from misbehaving in fancy restaurants to (underage) drinking, smoking, drugs, sex, watching R-rated movies, visiting Internet porn sites, or lying about something while earnestly proclaiming otherwise. “Maybe your child did those things, and maybe you let them get away with it,” they’ll say smugly, “but my child would never do such a thing!” I used to get into heated arguments with these earnest young parents; I don’t bother any more … they’ll find out soon enough on their own.

While I obviously don’t accept, condone, or excuse the behavior that Mr. O’Neill describes, it doesn’t take a genius to understand some of the reasons it may be happening — and why it’s likely to continue happening as we continue outsourcing call centers, help desks, software development, and many other kinds of jobs to countries, cultures, and ethnic groups other than our own. At the very least, ignoring it and pretending that it doesn’t exist is going to reduce or eliminate some of the benefits typically associated with outsourcing; and at worst, it could cause a serious backlash.

Let me know what you think about all of this…

Assume, for the moment, that you’ve got a software development project where the end-users (or customers, stakeholders, or whatever else you want to call them) and the developers (i.e., programmers, architects, database designers, business analysts, QA personnel, etc.) are all working in company X, here in the U.S. And imagine that you have another software development project, of equivalent size and complexity, where the end-users are in company X, but the development team is located in an offshore outsourcing company on the other side of the world (e.g., China, India, Singapore, eastern Europe, etc.) Why is it that Michael’s software metrics are showing that project #2 will have approximately 2.8 times as many software defects as project #1?

Two reasons, the first of which is familiar and predictable: precisely because the end-users and the developers are not “co-located” (in contrast to many of the “agile” or XP development projects, where a “customer proxy,” or end-user representative, is literally sitting in the same office cubicles, alongside the programmers), communication problems escalate; and if the two groups have different languages and different cultures, those problems become even more insidious and subtle. For the past several years, many of the top-notch outsourcing firms have pointed to the videoconferencing, Webex, email, and other collaboration tools that they’ve used to reduce these communication problems; and they’ve often co-located key developers in the midst of the end-user working environment to further reduce communication problems. But according to Michael, the communication problems still exist; and they contribute to higher defect levels. The world, as he likes to say, is not flat: you can’t just assume that software developers can be sprinkled randomly around the globe, where they’ll participate in a complex software development effort with no problems at all.

The second reason for the higher defect rate is one that I would not have predicted, though it’s not rocket science. Because offshore outsourcing firms typically offer an essentially unlimited number of technical personnel at a lower labor rate, many U.S. firms think they can achieve faster delivery schedules by hiring larger teams of offshore developers than they would if they staffed the project locally — and yet still save money. Thus, if the U.S. firm had previously staffed a project with 10 “expensive” local programmers, they figure they can hire 20 “cheap” Indian programmers, and still save money — while (in theory) cutting the schedule in half, because they’re blissfully unaware of Brooks’ Law. But aside from the schedule issue, Michael reminds us that software defects are strongly correlated to the number of “lines of communication” between the developers on the team; and that number increases combinatorially (i.e., N*(N-1)/2) as the size of the team increases (this is a variation on Conway’s Law, first articulated by computer programmer Melvin Conway in 1968, which says that the structure of a system is isomorphic to the structure of the organization that builds it). Hence, the number of defects in a software system increases in a manner proportional to the square of the number of developers; and as a result, he says, offshore projects are more likely to deliver buggy software.

All of this was part of a fascinating presentation that Michael gave at the New York City SPIN group this evening, to an audience of roughly a hundred software engineers and project managers who kept staring nervously out the window to see if a much-threatened blizzard was about to begin. The title of his presentation was “XP and Productivity Measures — What the Numbers Say,” and the bottom-line conclusion was that XP/agile projects deliver systems 50-100% (correction: after reviewing what I had written, Michael told me the correct figure was 25-50%) faster, with roughly half as many defects, and often with fewer developers, than “average” or “traditional” projects. This was illustrated with two case studies of U.S.-based development efforts — with internal/local development teams, and no outsourcing — whose results were compared against a massive database that Michael’s company, QSM Associates, has developed over the past 20 years. The metrics database contains information about roughly 7,300 projects representing 685 million lines of code, from 500 companies in 18 countries, written in 600 different programming languages. After he finished presenting the two case studies, Michael then presented a third case study involving a large offshore outsourcing effort, which culminated in the 2.8X offshore-to-average defect ratio discussed above.

The first company was a medium size project at a company called Follett Software, which Michael described as the “Amazon.com of the educational marketplace.” The project involved roughly 24 developers, 3 project leaders, and 7 testers; and the overall team produced approximately 1 million lines of code, with 7,000 automated unit-test cases, and 10,000 automated acceptance-test cases. What was significant about the project, from an extreme programming (XP) point of view, was the emphasis on the office environment and working conditions; Michael showed us several slides of the “high energy-level” single-room “bullpen” environment, and the dual-screen team-programming working environment.

“Core metrics” of size (lines of code), time (calendar months), effort (person-months), and defects were captured by Michael and his QSM colleagues after the project was finished, so that they could be compared against the industry averages of the 7,300-project database. (The four metrics mentioned above can be combined with a fifth, derived, metric of productivity to produce a total of “five core metrics,” about which QSM founder Larry Putnam has co-authored a book: Five Core Metrics: The Intelligence Behind Successful Software Management, which you can order from the friendly people at Dorset House Publishing.)

Bottom line: the Follett Software project showed staffing levels approximately the same as the average project in the QSM database; but the various “iterations” of XP deliverables occurred roughly 50-100% faster than average; and the number of defects was about half the QSM average. A second case study, from an unnamed company that builds medical instrumentation/testing equipment, had similar results. Indeed, Michael presented us with very detailed statistical summaries of the results, which I dutifully recorded; but just in case the figures might be considered confidential or sensitive, I’ll let you contact Michael directly if you feel you need to know about them.

As for the third case study, which involved a large outsourcing effort, it’s worth noting that the news was not all bad. As Michael explained, the “direct” cost of the offshore development was approximately 10% lower than that of the QSM average for a project of the same size. And the delivery schedule was 9.6 calendar months, in contrast to the QSM average of 12.3 months for a project of the same size — which represents a non-trivial improvement of approximately 21%. But this has to be balanced against the defects: an average project (of the same size) in the QSM database would have had 2,702 defects, but the outsourced project had 7,565 defects — i.e., 2.8 times higher. And as Michael pointed out, fixing all of those additional defects would ultimately have required more time, and more money — thus reducing, if not eliminating altogether, the initial benefits of lower labor costs and faster delivery of (buggy) software.

All in all, it was a thought-provoking presentation, with some provocative and intriguing insights into XP, agile development methods, and offshore outsourcing. Fortunately for all of us, the snow had just barely begun falling as Michael wrapped up his talk, and I scampered out of the building as fast as I could, so that I could get back to my office-at-home and jot down these notes before wrapping things up for the day.

So if you’re in the neighborhood, drop in and listen to the presentation. I’ll do my best to take notes and write up a blog posting later tonight or tomorrow. Stay tuned…