November 19th, 2006
A couple weeks ago, I posted a blog article entitled “Why Hasn’t Software Development Gotten Any Better?” Hank Heath and Moshe Gotesman posted some thoughtful comments on the blog, which I encourage you to read; and I’ve also gotten some interesting offline comments sent via email. For example, a long-time colleague, Becky Winant, wrote to me and said:
“Your item on why software hasn’t got any better really resonated. My team at [a large insurance company] looks for way to improve methods, tools and practices. So, I have pondered this myself - particularly #12 (how do we explain this better?), #7, and number 5 (insurance is regulated at every level and with the advent of consumer heath plans - the envelope might be pushed)
“Here’s a few more that jumped out for me:
“13. We off shore our work and do not always specify quality. I was asked to join a conversation with our off shore company where we were asking them to re-structure some COBOL code to make it better. The conversation continued to talk about the amount of code and engagement dates until I stepped in and asked: How will you know you got back code restructured in a way that you wanted? Silence.
“14. We don’t do an effective job of re-use. Code (heck, requirements!) may be reconstructed for the same or similar functionality to 10 things we have already. Big organizations are not necessarily structured for effective software development. Even when they try - politics places a key group out under someone else’s control. (So, would you organize a very large group?) There are still a whole lot of siloed departments, and siloed projects where slices of work are rather small (cost concern) and focused (so we can get it quickly) and siloed directories with security so that if you are not in a very specific group (dept, project) you can’t get in. As a result there is precious little visibility in to what we have as an asset inventory. (And, yes, no $ incentives or scorecards incentives to do so).
“15. According to Myers-Briggs Personality Type Indicator tests, conceptual and abstract thinking is only about 10% of the population. So, principles would not be a natural mode of thinking for software folks (who often test in the “other” profile areas that prefer specificity and concreteness).”
Another colleague, Herb Krasner, wrote:
“On the question of “why haven’t we gotten any better?”, it is my belief that the question itself is based on a false expectation that software development is supposed to be easier than it really is.
It is my belief that software development was, is now, and in the future will be difficult and challenging. Turning concepts into artificial reality usually is. I still link back to the “No Silver Bullet” theory that the essence of software is in its complexity, changeability and conformity — that is its true nature.
“As I look at your dirty dozen explanations, I find that my beliefs mostly align with your reasons 3, 4, and 9 — but I just love your embedded reflection in item 10 (as you get older …). And you already know that a side point of my ROI talk is to solve the problem stated by your #12.”
Finally, software metrics guru Capers Jones wrote to me after he, Herb Krasner, and I participated in a “software best practices” conference in early November, and said:
“Thanks for your thoughts on the stasis in the software industry.
“Here is a tidbit from chapter 24 of the 2nd edition of “Estimating Software Costs” [scheduled to be published in 2007]. It summarizes an issue - too many mediocre choices.
“There is one other aspect of complexity that has a direct bearing on automated software cost estimating tools and also on benchmarks. As of 2007 the software industry uses some 600 different programming languages and creates about 120 different kinds of software applications each of which can contain about 23 different features. The software industry employs 90 kinds of specialists, creates 53 kinds of paper document, uses 43 different design methods, and measures with 38 different size metrics. Software applications are built utilizing 26 different development methods that include at least 35 different activities. About 25 international standards affect software projects. Software projects face 30 different kinds of risk factors. The software industry must also deal with 24 kinds of complexity, perform 23 different kinds of maintenance activities, and may perform 18 different kinds of testing. The combinatorial complexity of these topics is enormous. There is no way for either project managers or estimating tool developers to include all of these factors at the same time. It is obvious from the plethora of choices and alternatives in the software industry that we are groping for better ways of doing things, but have not yet found optimal approaches.”
I suspect there are still other explanations waiting to be articulated. And while all of this sounds rather gloomy, we should keep in mind that we have software today that simply didn’t exist 20 years ago. Yes, we had email and word-processors and spreadsheets back in the mid-1980s, but we didn’t have web browsers, or Google, or most of the graphics programs available on our PC’s today. But the bottom line is that we’re still encountering serious difficulties trying to develop large, complex software projects on time, within budget, and with an acceptable level of quality. And it probably won’t get much better anytime soon…
November 20th, 2006 at 9:15 pm
I’ve always been uncomfortable with the term, “Software Engineer.”
When I graduated with my shiny new MS and went to work for NCR I proudly called myself an engineer. But across the aisle from me was an old-school mechanical engineer who was working on print heads for very high speed printers. As I watched what he was doing, and compared it to what I was doing I had no choice but to start calling myself a “programmer” again.
When he needed high-density wire for the impact head, he pulled down a big catalog with various alloys available in a variety of forms, and chose one that fit his needs. I had to hand-code a binary tree package. When he had to deal with the deflection of the wire under impact, he had formulae in the CRC manual. I had to time my code with a wristwatch.
You get the idea - he was working from an engineering base of twenty centuries, but my field was less than fifty years old and we were mostly making it up as we went along. And I think that we’re entitled to that awkward adolescence and shouldn’t wail and beat our breasts quite so loudly. Sure, we need to get better, but we shouldn’t expect to reach the level of an engineering discipline overnight.
If we’re still stuck in fifty years, then I’ll concede that we’re all idiots.
November 27th, 2006 at 2:46 am
Bruce, you hit the nail on the head. Currently, most SW engineers are not really engineers. They are indeed programmers who work on building SW products. I also agree that SW engineering is young, so understandably not mature as an engineering discipline (which I think it should be). My concern is with the fact that the maturation process is very slow (at least seems slow to me). I think that the basic principles of SW engineering are now known but they are not getting into mainstream and not becoming standard practice.
I have covered this exact topic in my article: “Why do SW PMs need to know Software Engineering?” (http://mosgot.blogspot.com/2006/11/why-do-sw-pms-need-to-know-software.html).
May 31st, 2007 at 7:38 am
[…] we doing things that we know are good ideas — is something I’ve blogged about here and here. Take a look, if you’re […]