I’m back in New York, after a 12-hour plane ride home from Tokyo. Piles of snail-mail, voice-mail, and e-mail waiting to be answered; and a new project to focus on next week. But before all of my impressions of Tokyo and the Japanese software profession fade away entirely, I thought I’d jot them down here.

More than anything else, I get the impression that the Japanese software industry has “grown up” since my visits in the late 1980s and early 1990s — a time when everyone I met exhibited a great deal of pride and optimism about their ability to develop sophisticated, high-quality systems that were finished on time, and within budget. Maybe some of those development efforts still exist today, but most of what I heard about was software projects that have grown too large, too complex, and too demanding. Death march projects are rampant; everyone is working long hours, and struggling to find ways of speeding up their development efforts, and cutting costs by a steady 20% each year.

Whether it’s automobiles, cell phones, or any one of a number of consumer appliances and electronics, many companies are focusing very heavily on embedded systems — i.e., a product that has one or more (in the case of automobiles, often several dozen) CPUs with complex software functions. Several of the people I spoke with told me about projects with 3-5 million lines of embedded code, and a relentless demand from their product marketing department to bring out new models (with lots of new features) at least once a year, and often twice a year. At least the software isn’t being written in assembler; but most of what I heard about involved C and C++, with a smaller amount in Java. Not much talk about frameworks, components, code generators, CASE tools, or various other ways of automating the development effort; so it sounds like there’s an enormous amount of heads-down, old-fashioned, manual programming activity going on.

One thing I found particularly interesting was the widespread concern about possible large-scale “software recalls” caused by undetected software bugs. Apparently there have been a couple of software recalls, sometime during the past couple of years, though I don’t recall any details and couldn’t find anything relevant with a Google search (this is not to be confused with the recall of defective laptop batteries a few months ago, which was a hardware problem). The concern, of course, is that unlike a bug in a one-of-a-kind software system, a bug in a commercial product could affect millions of units. And if it’s a bug in an embedded system, then (with rare exceptions), it can’t be fixed by downloading a “patch” in the fashion that’s become so familiar to users of laptop/desktop computer systems.

None of these problems are new or surprising, and they’re certainly not unique to the Japanese software industry; after all, we’ve been coping with the same issues in the U.S. for many years. But it seems like a more recent concern for the Japanese, and it seems to have made them fairly pessimistic about the short-term future. On the other hand, one of the journalists who interviewed me in Tokyo made an interesting observation: “The Japanese,” he said, “tend to swing back and forth between optimism and pessimism.” So perhaps I happened to catch them during an optimistic phase when I first began visiting the country in the early 1990s, and perhaps they’re just now going through a (temporary) pessimistic phase. As part of this pessimism, I heard from several people that enrollment in computer science and software engineering degree programs is dropping in Japan, just as it is in the U.S. And I heard that more and more software work is being outsourced by Japanese companies to China and India.

In any case, the obvious question is: how do the Japanese believe they’ll eventually solve their problems of ensuring quality in complex software systems? From what I could tell during this visit, they’re focusing primarily on software processes and tools — i.e., more formal methods of analysis, design, and testing, combined with a variety of testing tools and technologies. I think that’s all fine, but I was a little surprised to see so little awareness of the political, psychological, sociological, and cultural issues associated with software development. But I may have been exposed to only a narrow subset of the overall Japanese perspective on software development: most of my time was spent talking to people who were attending the Japanese Symposium on Software Testing (JaSST), and there may be other conferences and symposia that do focus on cultural and sociological aspects of software development.

But if there are such conferences, I’ll have to wait for some future visit to find out about them. For now, I’m back in America, where I’ll have my hands full focusing on our own successes and failures, trials and tribulations with software development.