Food for Thought-An e-newsletter published by Software Quality Consulting, Inc.
September 2006, Vol. 3 No. 7
Are We There Yet? 

What topics would you like to see in this newsletter?

Each month, this newsletter tries to provide you with useful information.  This
is a two-way street and your feedback is important.  Please send your thoughts
and comments to [email protected].

--------------------------------------------------------------------------------

Welcome to Food for Thought(TM), an e-newsletter from Software Quality 
Consulting (http://www.swqual.com/index.html?Intro). I've created free
subscriptions for my valued business contacts. If you find this newsletter
informative, I encourage you to continue reading. Feel free to pass this
newsletter along to colleagues by clicking this Forward Email link at the bottom
of this newsletter. If you�ve received this newsletter from a colleague and
would like to subscribe, please click this Enter New Subscription link
(http://www.swqual.com/newsletter/Subscribe.htm?Newsletter). If you don't wish
to receive this newsletter, click the SafeUnSubscribe(TM) link at the bottom of
this newsletter, and you won�t be bothered again.

Your continued feedback on this newsletter is most welcome. Please send 
your comments and suggestions to [email protected].

*** In This Ussue ***

In This Months� Topic, I discuss the evolution of the software 
engineering discipline...

Regular features to look for each month are:

- Monthly Morsels
  Hints, tips, techniques and reference info related to this month�s topic 

- Calendar
  Conferences, workshops, and meetings of interest to software engineers, 
  QA engineers and anyone interested in software development 

--------------------------------------------------------------------------------

*** This Month�s Topic ***

Are We There Yet?

Every parent has heard this question a million times. When my kids were 
little, I would hear this question as we were backing out of the driveway. 
�No, not yet.� I would reply...

I have thought about this question with regard to software engineering. 
The software engineering discipline has experienced major changes since 
the late 1950�s. In contrast, changes to more mature engineering 
disciplines have been much more subtle. The journey has been anything but 
smooth � there have been many bumps along the road. To understand where 
software engineering is heading, we need to first understand from whence 
it came...

WHEN DID IT BECOME SOFTWARE ENGINEERING?

The term software engineering was was first used in the late 1950s. 
However, it wasn�t until 1968 at a NATO Science Committee conference that 
software engineering was recognized as an emerging engineering discipline.

  An interesting historical footnote - During the 1950�s and 60�s the job 
  of programming computers was performed mostly by women. Many women 
  mathematicians, led by Admiral Grace Hopper
  (http://ei.cs.vt.edu/~history/Hopper.Danis.html), were instrumental in 
  proving that computers could be used to solve real-world problems. 
  During this same time, men were primarily focused on hardware 
  engineering � which was then considered to be a more prestigious job...

Universities initially offered computer programming courses through the 
Mathematics or Physics Departments. This is partly a result of early 
computer users � who were primarily mathematicians and physicists. It 
wasn�t until the 1980�s that universities established departments and 
curricula specific to Computer Science at the undergraduate level.

Back in the day, developing programs involved using what now seems like 
stone age tools such as pencil and paper, flow chart templates, and coding 
sheets. Programs were �designed� by drawing detailed flowcharts. Programs 
were �written� by typing statements on computer punch cards using a 
keypunch machine.

After spending countless hours at the keypunch machine, the �card deck� 
was submitted to the computing center. Large mainframes were often hidden 
from view behind smoke-colored windows - accessible only to the first 
generation of computer nerds. Card readers would read your card deck along 
with dozens of others. Often, the card readers would mangle your cards and 
the computer operator occasionally got card decks mixed up. This required 
more hours of re-punching and re-ordering of your card deck. Once your 
card deck was successfully read, the first pass would often result in many 
compiler errors. After several iterations (which could take hours or even 
days), the program would finally compile cleanly. Only then would it be 
possible to begin the task of determining if it worked properly...

THE EMERGING SOFTWARE CRISIS

During the 1980�s, �programming� began its evolution into �software 
engineering�. But the fledgling software industry was facing its first 
significant challenge. By the end of the 1980�s, companies were spending 
more money on software maintenance activities than they were on new 
development. Quality was abysmal. Productivity was low. This was the 
infamous �software crisis� of the 1980�s � the first significant bump in 
the road. High profile software failures started appearing in the news. 
These failures resulted in economic losses and in a few cases, loss of 
Life (http://www.swqual.com/newsletter/vol2/no10/Therac-25.pdf).

The �software crisis� gave rise to the Methodology Wars
(http://www.swqual.com/newsletter/vol3/no7/MethodologyWars.pdf). Software gurus
of the day advocated their own software development paradigms that were sure 
to get us out of this mess � what Fred Brooks called the Silver Bullet
(http://www.swqual.com/newsletter/vol3/no7/No%20Silver%20Bullett%20Essence%20and
%20Accidents%20of%20Software%20Engineering%20Fred%20Brooks%20IEEE%20Computer%20A
pril%201987.pdf). They all promised, �if only those lazy, good-for-nothing
programmers would adopt their methodology, all of our problems would be solved!�
Ed Yourdon, James Martin, Tom DeMarco, and many others published their own 
methodologies and established cult-like followings.

Hindsight has shown that none of these methodologies ever lived up to 
their hype.

  Another interesting historical footnote: The object-oriented methodology 
  evolved from many sources including smalltalk
  (http://en.wikipedia.org/wiki/Smalltalk), Simula 67
  (http://en.wikipedia.org/wiki/Simula), and Modula
  (http://en.wikipedia.org/wiki/Modula-2). As O-O methods started to become
  popular, many developers learned O-O programming languages like C++ before
  learning O-O Analysis and Design techniques. As a result, many early C++
  programs thought to be �object-oriented� really were not because the
  developers had not learned to think differently.  

CHIEF PROGRAMMER TEAMS

In the 1970�s when Harlan Mills created the first Chief Programmer Team
(http://www.firstmonday.org/issues/special10_10/koch/index.html) [4], he
acknowledged a fact of human nature:

  all programmers are not created equal � some are more adept than others 
  at specific tasks such as architecture, design, coding, or testing...

(Do you see the similarity between Chief Programmer Teams and current 
Agile Methods?)

Another important contribution from Chief Programmer Teams was the 
notion of conceptual integrity (http://en.wikipedia.org/wiki/The_Mythical_Man-
Month). According to Fred Brooks:

  �I will contend that conceptual integrity is the most important 
  consideration in system design. It is better to have a system omit 
  anomalous features and improvements, but to reflect one set of design 
  ideas, than to have one that contains many good but independent and 
  uncoordinated ideas.� [2]

During the 1980�s some people believed that the Silver Bullet could be 
found in better tools. CASE Tool (http://en.wikipedia.org/wiki/Computer-
aided_software_engineering) companies developed and marketed all sorts of tools
that promised significant improvements in productivity and quality. Not
surprisingly, many of these tools never lived up to their hype either.

Others believed that more process was the solution. This eventually led to 
the establishment of the Software Engineering Institute (SEI �
http://www.sei.cmu.edu/) and the Capability Maturity Model � (CMM �
http://www.sei.cmu.edu/cmm/cmm.html)� one of the heaviest of the heavyweight 
processes.

The CMM � and CMMi � have been widely used for large government 
development projects. Outside of this arena, they are criticized as being 
overly bureaucratic and inefficient. This criticism begat the evolution of 
lightweight processes, commonly referred to as Agile Methods
(http://en.wikipedia.org/wiki/Agile_methods).

THE BUBBLE

During the 1990�s, the software engineering journey shifted into high 
gear. As Internet access spread across the globe, it created explosive 
growth in software development. While software was being applied to more 
and more problems (as well as to things that were not problems), it also 
caused more and more problems. High profile software failures were 
becoming commonplace (http://www.swqual.com/newsletter/vol2/no10/vol2no10.html).

The dot.com debacle of the late 1990�s negatively impacted the reputation 
of software engineering. As thousands of dot.com companies sprang up over 
night, they competed fiercely for a very limited number of experienced 
software engineers. Many dot.coms hired people to develop web applications 
who had little or no training in software engineering principles. As a 
result, many of these applications became major disasters...

The proliferation of software that was fueled by the Internet has created 
new problems (http://www.swqual.com/newsletter/vol3/no4/vol3no4.html) for
software engineers and new challenges for software engineering. Hackers, SPAM,
virtual terrorists, security leaks, identity theft, etc. are just some of the
problems that have resulted from the proliferation of software since 2000. 

SO, ARE WE THERE YET?

The Methodology Wars that started in the 1980�s are still raging. Today, 
it�s lightweight vs. heavyweight processes, Agile vs. traditional methods. 
One thing we should have learned by now is that...

There still is no silver bullet and there probably will never be...

As Roger Pressman said:

  �I contend that software engineering principles always work. It�s never 
  inappropriate to stress solid problem solving, good design, and thorough 
  testing (not to mention the control of change, an emphasis on 
  quality,...). A specific software process might fail because it is 
  overkill, or the work products it requires are unnecessary or 
  burdensome, or a person or team becomes overly dogmatic in the 
  application of the process. But history is on the side of a solid 
  engineering approach.� [1]

I couldn�t agree more. If software engineering is ever to become a 
respected engineering discipline, it has to be based on sound engineering 
principles � principles that have been shown to work. So what are some 
examples of such principles? Most of us probably have many examples... 

APPLY SOUND ENGINEERING PRINCIPLES...

There have been many outstanding software engineering books written in the 
past five decades. Of all the computer science and software engineering 
books that have been published in the past 50 years, only a few have 
published special anniversary editions � more than two decades after they 
first appeared. Why is that? It�s because their message still rings as 
true today as it did when they were first published. They embody many 
sound engineering principles...

What are these very special books?

  The Mythical Man-Month, by Fred Brooks [2]
  (http://www.awprofessional.com/bookstore/product.asp?isbn=0201835959&rl=1)

  A few examples of the lessons described by Brooks that he observed more 
  than 30 years ago and still apply today include:

  Adding staff to a late project will only make it later.

  Hard to believe that there are managers out there that still don�t get 
  this...

  Conceptual Integrity 

  This is probability the most important aspect of designing systems that 
  isn�t taught in colleges and universities. When I meet with young 
  software engineers, I often ask them about this and usually get a 
  puzzled look.

  Plan to throw one away 

  As the complexity of software has increased and the number of new 
  systems grows, companies need to recognize that product marketing is not 
  likely to get it right the first time. Planning to throw the first one 
  away means you trash the code not the knowledge gained. And, we need to 
  remember that the knowledge is far more valuable than the code.

  Learn to develop accurate project estimates and realistic schedules

  I can�t tell you how many times I�ve seen project teams give management 
  the answer Management wants rather than the truth. Haven�t these people 
  read Death March Projects (http://www.amazon.com/Death-March-Second-Edward-
  Yourdon/dp/013143635X/sr=1-1/qid=1158089440/ref=pd_bbs_1/103-2069278-
  6377461?ie=UTF8&s=books)? If you want to develop better estimates and 
  realistic schedules, you need to first train people how to do this
  (http://www.swqual.com/training/schedules.html), and then, require that they
  do it. Managers, you need to be prepared for schedules that may not fit with
  your goals. The answer is to not force the schedule to fit, but rather,
  negotiate with the project team to see what can reasonably be done with the
  given resources and requirements...

  Communication 

  Since software engineering is an inherently human process, communication 
  between humans developing software is probably pretty important. Yet, 
  software engineers are notoriously bad communicators. Ever read a good 
  requirements spec? They are few and far between. Product marketers are 
  just as bad at communicating as software engineers. The common thread � 
  English � one of the worst languages there is for stating requirements 
  for software. English is inherently vague and ambiguous. Software must 
  be deterministic. Do you see why there are so many communication 
  problems here? Read more...
  (http://www.swqual.com/newsletter/vol3/no3/vol3no3.html)

The other special book is from Gerry Weinberg and addresses some key 
behavioral characteristics of developers. He observed these issues over 35 
years ago and yes, they still apply today:

  The Psychology of Computer Programming, by Gerald Weinberg [3]
  (http://www.dorsethouse.com/books/psy.html)

  Weinberg identified several key personality traits that are critical for 
  success as a software developer. These traits include:

  - Ability to adapt to rapid change 
  - Organizational skills and the ability to keep track of lots of 
    important bits of information 
  - Humility � your code will have problems that will be found by others 
  - Some level of assertiveness � needed to get things done often in the 
    face of many obstacles 
  - A sense of humor, since the computer �doth make fools of us all�. 

The take away message from these books is that software development is a 
complex human activity that is influenced by many factors. To be 
successful, its my contention that software engineering requires three 
critical ingredients:

I�ve observed that many organizations have one of the three, a 
few have two of the three, and only best in class companies have all 
three.

SUMMARY

Lewis Carroll said that if you don�t know where you are going, any road 
will get you there. Software engineering is still a relatively young 
engineering discipline. The journey has had its ups and downs. I don�t 
know where things will end up but being a passenger on this journey has 
enabled me to make some observations and draw some conclusions. I�m 
enjoying the trip but sometimes it seems like We�re All Bozos on This Bus
(http://www.firesigntheatre.com/albums/album.php?album=bozos).

Until next time...

--------------------------------------------------------------------------------

*** Monthly Morsels ***

Every month in this space you�ll find additional information related to 
this month�s topic.

- References:

  [1] Pressman, R., �What A Tangled Web We Weave�, IEEE Software, Jan/Feb 
  2000, pp. 18-21.

  [2] Brooks, F., The Mythical Man-Month, Addison-Wesley, first published 
  1975. Silver Anniversary edition published 1995.

  [3] Weinberg, G., The Psychology of Computer Programming, Dorset-House, 
  first published 1971, silver anniversary edition published 1998.

  [4] H. Mills, "Chief Programmer Teams, Principles, and Procedures," IBM 
  Federal Systems Division Report FSC 71-5108, IBM, Gaithersburg, Md., 
  1971. 

- On-line Resources

  Read an interesting summary of the History of Software Engineering...
  (http://en.wikipedia.org/wiki/History_of_software_engineering) 

--------------------------------------------------------------------------------

Every month you�ll find news here about local and national events that 
are of interest to the software community ...

- Software Quality Calendar

  There are many organizations that sponsor monthly meetings, workshops, 
  and conferences of interest to software professionals. Find out what�s 
  happening...
  (http://www.swqual.com/links/upcoming.html) 

- Workshops Offered by Software Quality Consulting

  Software Quality Consulting offers workshops in many topics related to 
  software process improvement. Get more info... 
  (http://www.swqual.com/seminars/courses.html)

--------------------------------------------------------------------------------

*** About SQC ***

Software Quality Consulting provides consulting, training, and auditing 
services tailored to meet the specific needs of clients. We help clients 
fine-tune their software development processes and improve the quality of 
their software products. The overall goal is to help clients achieve 
Predictable Software Development(TM) � so that organizations can consistently 
deliver quality software with promised features in the promised timeframe. 

To learn more about how we can help your organization, visit our web site
(http://www.swqual.com/index.html?AboutSQC), or send us an email
([email protected]).

--------------------------------------------------------------------------------

I hope this newsletter has been informative and helpful. Your comments and 
feedback are most welcome. Send me your feedback ([email protected])...
Thanks,

Steve Rakitin
[email protected]

Food for Thought, Predictable Software Development, Act Like a Customer,
and ALAC are trademarks of Software Quality Consulting, Inc.
Copyright 2006. Software Quality Consulting, Inc. All rights reserved.
Graphic design by Sage Studio