|
Server : Apache/2.4.62 System : FreeBSD fbsdweb2.web.rcn.net 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64 User : www ( 80) PHP Version : 8.3.8 Disable Function : NONE Directory : /domains/srakitin/OLD/newsletter/vol6/no4/ |
Upload File : |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Food for Thought: Software Quality Assurance turns 50 - Part 1</title>
<link href="/newsletter/StyleSheet.css" rel="stylesheet" type="text/css">
</head>
<OpenTracking/>
<!-- Do NOT delete previous line if you want to get statistics on the number of opened emails -->
<body>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0">
<tr align="center" valign="top">
<td colspan="2"><img src="/newsletter/images/FoodForThoughtLogo.gif" alt="Food for Thought" width="600" height="105"></td>
</tr>
<tr class="Reference">
<td align="left" valign="top"><p>An e-newsletter published by<br>
Software Quality Consulting, Inc. </p>
</td>
<td align="right" valign="top"><p>June 2009 , Vol. 6 No. 4 <br>
[<a href="/newsletter/vol6/no4/vol6no4.txt" target="_blank">Text-only Version</a>]</p>
</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top">
<p>Welcome to <em><strong>Food for Thought™</strong></em>, an e-newsletter from <strong><a href="/index.html?Intro" target="_blank">Software Quality Consulting</a></strong>. I've created free subscriptions for my valued business contacts. If you find this newsletter informative, I encourage you to continue reading. Feel free to pass this newsletter along to colleagues by clicking this <strong><a href="http://ui.constantcontact.com/roving/sa/fp.jsp?plat=i&p=f&m=sctz69n6">Forward Email</a></strong> link. If you’ve received this newsletter from a colleague and would like to subscribe, please click this <strong><a href="/newsletter/Subscribe.htm?Newsletter" target="_blank">Enter New Subscription</a></strong> link. If you don't wish to receive this newsletter, click the <strong><a href="#bottom">SafeUnSubscribe</a></strong>™ link at the bottom of this newsletter, and you won’t be bothered again.</p>
<p>Your continued feedback on this newsletter is most welcome. Please send your comments and suggestions to <strong><a href="mailto:[email protected]">[email protected]</a></strong>.</p></td>
</tr>
</table>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/InThisIssue.gif" alt="In This Issue" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p>In <a href="#article"><strong>This Months’ Topic</strong></a>, I begin a discussion on the state of the software quality assurance profession...<br>
<br>
Regular features to look for each month are:</p>
<ul>
<li> <a href="#morsel"><strong>Monthly Morsels</strong></a><br>
Hints, tips, techniques and reference info related to this month’s topic</li>
</ul>
<ul>
<li> <a href="#calendar"><strong>Calendar</strong></a><br>
Conferences, workshops, and meetings of interest to software engineers, QA engineers and anyone interested in software development</li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<a name="article"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/ThisMonthsTopic.gif" alt="This Month's Topic" width="114" height="37"></td>
<td width="15"> </td>
<td width="471" align="left" valign="top" class="BodyText"><p align="center" class="Headline"><strong>Software Quality Assurance turns 50<br>
A critical look at the state of the profession<br>
<br>
Part 1 - History and Evolution</strong> </p>
<p> Software Quality Assurance (SQA) was used for the first time on a software development project about 50 years ago. Over the next several months, I plan to take a critical look at the state of the SQA profession as a way to recognize the significance of this milestone. In this month’s installment, I discuss the history and evolution of SQA. In subsequent e-newsletters, I will discuss some successes and failures, and the future of SQA.</p>
<p> During the past half-century, the software industry has gone through dramatic changes. Today, software is an integral part of daily life. Many software-based products that were beyond one’s wildest imagination five decades ago are now commonplace. The explosion of the Internet, digital gadgetry, and cheap hardware has resulted in software finding its way into millions of products and services, many of which are <strong><a href="/newsletter/vol4/no6/vol4no6.html" target="_blank">safety-critical or mission-critical</a></strong>. Today <strong>software plays an integral role in most every major segment of the global economy</strong>. The following are but a few examples:</p>
<ul>
<li><strong> Energy: </strong></li>
</ul>
<ul>
<ul>
<li> Electric power generation - nuclear and conventional power plants </li>
<li> National and regional power grids </li>
<li> Oil and natural gas distribution management systems </li>
</ul>
</ul>
<ul>
<li><strong> Transportation: </strong></li>
</ul>
<ul>
<ul>
<li> All kinds of vehicles </li>
<li> Urban mass transit systems </li>
<li> Railway signaling systems </li>
<li> Avionics and air traffic management </li>
</ul>
</ul>
<ul>
<li><strong> Healthcare: </strong></li>
</ul>
<ul>
<ul>
<li> Hospital patient monitoring systems </li>
<li> Life-supporting and life-sustaining medical devices </li>
<li> Electronic patient medical records </li>
<li> Research into diseases and development of new drugs </li>
</ul>
</ul>
<ul>
<li><strong> Banking and Finance: </strong></li>
</ul>
<ul>
<ul>
<li> On-line banking systems and ATM machines </li>
<li> International currency trading </li>
<li> Stock exchanges and brokerages </li>
</ul>
</ul>
<ul>
<li><strong> Defense: </strong></li>
</ul>
<ul>
<ul>
<li> Weapon systems </li>
<li> Command, Control, and Communications Systems </li>
<li> Satellite communications and imaging </li>
</ul>
</ul>
<ul>
<li><strong> Space exploration: </strong></li>
</ul>
<ul>
<ul>
<li> Hubbell Telescope </li>
<li> International Space Station </li>
<li> Space Shuttle </li>
</ul>
</ul>
<p>Everyday we become more and more dependent on software. As I have stated many times, <strong><a href="/newsletter/vol2/no11/vol2no11.html" target="_blank">all software is inherently defective</a></strong>. Unless you live in a third world country, a typical day involves using a significant amount of software - either implicitly (as in software embedded in a product) or explicitly (as in software applications). For people living in third world countries, there are <strong><a href="http://www.laptop.org/en/?gclid=CL--of2c_poCFRpN5QodGlWWdg" target="_blank">on-going efforts </a></strong>(some controversial) aimed at providing children with inexpensive laptops so they can connect to global virtual communities.</p>
<p> We begin this critical review by starting at the beginning...</p>
<p><strong> In the beginning...</strong></p>
<p> In the late 1950's, software first began to find its way into systems procured by US government agencies such as the Census Bureau and the Dept. of Defense (DoD). Not surprisingly, these projects were always behind schedule, over budget, and suffered from both technical and management problems. Frequently, software did not work as intended and many projects were cancelled before anything was delivered.</p>
<p> Software development contractors often gave overly optimistic assessments of the software development status to managers - the origin of “We’re 90% done!” Managers were frequently unaware of schedule, budget, and technical problems until very late into the program – when they were often unable to understand them, assess their impact or do anything to change the situation.</p>
<p> The Atlas Missile was the first operational intercontinental ballistic missile in America's nuclear arsenal and marked the beginning of the US space program. The <strong><a href="http://en.wikipedia.org/wiki/SM-65_Atlas" target="_blank">Atlas Missile Program</a></strong> was one of the first software projects to try to address these problems. The program manager hired an “independent software tester” to “perform additional, unbiased testing of the software”. [1] The program manager hoped to get a timelier, accurate and objective technical assessment of the project's status by employing someone<strong> independent </strong>of the software development contractor.</p>
<p> Around the same time, the first independent test team on a large software project was formed and led by <strong><a href="http://www.geraldmweinberg.com/Site/Home.html" target="_blank">Jerry Weinberg</a></strong> on <strong><a href="http://www.nasa.gov/mission_pages/mercury/index.html" target="_blank">Project Mercury</a></strong> - the first US manned space flight program.</p>
<p> During the 1960’s, the role of the independent software test team evolved from just focusing on testing to focusing on the entire software development life cycle. This role became known as <strong>Independent Verification & Validation (IV&V).</strong></p>
<p> Today, IV&V is a critical function contractually required on most large, mission-critical projects for US government agencies including DoD, NASA, FAA, HUD, EPA and DEA. The set of tasks performed by IV&V contractors is comprehensive and spans procurement, development and deployment.</p>
<ul>
<li> To learn more about IV&V see the <strong><a href="#morsel">Monthly Morsels</a></strong> section below... </li>
</ul>
<p>Much data has been collected to support the assertion that projects with IV&V perform much better than similar projects without IV&V. [2], [3] As a result of this data, NASA now requires IV&V to be applied on applicable NASA projects. [4]</p>
<p> Much of the success of IV&V is attributable to the fact that IV&V contractors are completely<strong> independent</strong> of the software development organization. Working for and reporting to the procuring entity, IV&V contractors <strong>provide an unbiased, objective technical and managerial assessment</strong> of a project. As a result, the procuring entity is in a much better position to identify and resolve issues that could otherwise easily be overlooked (intentionally or unintentionally) by the software development contractor. Raising these issues in a timely manner ensures that they are more likely to be resolved and not affect the project. </p>
<p><strong> When did it become Software Engineering? </strong></p>
<p> In the really old days (1940s-50s), people who worked with computers were usually mathematicians and were called <strong>programmers </strong>or<strong> data processing specialists</strong>. Computers of the time, such as the <strong><a href="http://en.wikipedia.org/wiki/ENIAC" target="_blank">ENIAC</a></strong>, were very clunky and were usually “programmed” by re-wiring patch panels that changed the sequence of operations the computer’s electromechanical relays performed.</p>
<p> Back then, designing hardware was much more prestigious and as a result, hardware engineers were mostly male. Women, most of whom were mathematicians like <strong><a href="http://en.wikipedia.org/wiki/Grace_Hopper" target="_blank">Admiral Grace Hopper</a></strong>, often performed the “lowly” job of programming the computers. </p>
<blockquote>
<p> Grace Hopper was a brilliant mathematician who worked at Harvard on the <strong><a href="http://en.wikipedia.org/wiki/Harvard_Mark_II" target="_blank">Mark II Aiken Relay Calculator</a></strong> – an early analog computer built from hundreds of electromechanical relays. She liked to tell a story about an event that occurred in late summer of 1947. It was before the advent of air conditioning so the windows in the computer lab were open most of the time. A technician solved a problem with the Mark II machine by pulling an actual insect (a moth) out from between the contacts of one of its relays. Admiral Hopper taped the moth to her lab notebook on September 9, 1947 and made the entry shown below: </p>
</blockquote> <p align="center"><img width="407" height="320" src="/newsletter/vol6/no4/vol6no4_clip_image002.jpg" alt="File:H96566k.jpg"></p>
<p> In the fall of 1968 and again in 1969, the <strong><a href="http://homepages.cs.ncl.ac.uk/brian.randell/NATO/index.html" target="_blank">NATO Science Committee</a></strong> sponsored technical conferences to bring together experts to discuss problems with the development of software. The conference organizers used the term <strong>software engineering</strong> as a way to provoke interest and discussion. The term wasn’t used prior to the conference and the birth of the software engineering profession is generally recognized to coincide with these conferences.</p>
<p> As observed by one of the conference attendees: </p>
<blockquote>
<p> “Although the term was not in general use at that time, its adoption for the titles of these conferences was deliberately provocative. As a result, the conferences played a major role in gaining general acceptance, perhaps even premature, for the term. The motivation for these conferences was that the computer industry at large was having a great deal of trouble in producing large and complex software systems. (Does that sound like dêja vu all over again?)” [5]</p>
</blockquote>
<p> The participants at the conference represented computer hardware manufacturers, computer users, representatives from a few small software development companies, and academia. Most of the people attending the NATO Conference acknowledged that there were many problems associated with producing large, complex software systems. (Note that systems called “large” and “complex” in 1968 terms would be considered “small” and “trivial” by today’s standards). In the summary of the NATO Science Committee report [6], it states: </p>
<blockquote>
<p> “... the report also contains sections reporting on discussions, which will be of interest to a much wider audience. This holds for subjects like the:</p>
<ul>
<li> problems of achieving sufficient reliability in the data systems which are becoming increasingly integrated into the central activities of modern society <br>
<br>
</li>
<li> difficulties of meeting schedules and specifications on large software projects<br>
<br>
</li>
<li> education of software (or data systems) engineers” </li>
</ul>
</blockquote>
<p><strong> </strong>The problems observed in 1968 are striking in how similar they are to problems we have today. </p>
<p><strong> The emergence of SQA </strong></p>
<p> The 1968 NATO report also used the term <strong>Software Quality Assurance</strong>. During the conference, the participants discussed the issue of SQA and raised several very interesting questions: [6]</p>
<ul>
<li> Is software quality assurance done by an independently reporting agency representing the interests of the eventual user? </li>
</ul>
<ul>
<li> Is the product tested to ensure that it is the most useful for the customer in addition to matching functional specifications? </li>
</ul>
<ul>
<li> Do software quality assurance test programs undergo the same production cycle and method (except Q/A) as the software they test? Are they defined and constructed concurrently with the software? </li>
</ul>
<ul>
<li> Is at least one person engaged in software quality assurance for every ten engaged in its fabrication? </li>
</ul>
<ul>
<li> Are there tests for overall system performance as well as for components? </li>
</ul>
<ul>
<li> Are software quality assurance tests a part of the general hardware acceptance test on the customer’s machine before it leaves the factory’? </li>
</ul>
<ul>
<li> Can software field release be held up if these tests are not passed? </li>
</ul>
<ul>
<li> Do the tests include a system logic exerciser? </li>
</ul>
<ul>
<li> Are tests provided to ensure matching of computational results with those of other equipment? </li>
</ul>
<ul>
<li> Is this test library applied upon issuance of each modification of the software system? </li>
</ul>
<ul>
<li> Is each customer’s system tape tested on the software production machine for a sufficient period of time, where feasible? </li>
</ul>
<p>These issues, raised more than 40 years ago, still resonate today.</p>
<p> During the 1970’s, software development activity expanded to commercial companies. These companies experienced the same poor results that US government agencies had seen a decade earlier. These companies had difficulty delivering software within the constraints of schedule, budget, and quality. Many projects undertaken in the 1980’s and 90’s were disasters. Several projects failed to deliver anything. The few projects that did deliver something were significantly over budget and years behind original schedules and delivered software of such poor quality that it was often unusable.</p>
<p> In the 1980’s, the software industry experienced what became known as the “software crisis” – the point in time when spending on software maintenance exceeded spending on creating new software products. The advent of the “software crisis” brought with it a host of changes - not the least of which was the emergence of SQA as a critical function to be performed on software development projects. Initially, SQA was viewed as sort of an internal IV&V function.</p>
<p> Drawing on its roots in IV&V, SQA evolved into an effective tool that software development companies have used to help identify quality problems earlier in the development process. While SQA was viewed as the “poor stepchild” of software development, many enlightened managers of the day saw measurable benefit from integrating SQA into the software development process.</p>
<p> By the 1990’s, many software companies had SQA functions within their organizations. Yet, high profile software failures continued to occur. (see [7, 8, 9]) Was SQA not living up to expectations? Hard to say. But there were several differences in the nature of software being developed during this time that are worth noting:</p>
<ul>
<li> Complexity of software developed during the 90’s increased significantly. </li>
</ul>
<ul>
<li> Competitive business pressures also increased significantly. </li>
</ul>
<ul>
<li> Software was being used in many new areas – especially areas that were life threatening. </li>
</ul>
<ul>
<li> Many people working in SQA received little formal training in SQA. SQA engineers were expected to learn their craft primarily from on-the-job training. </li>
</ul>
<ul>
<li> Universities failed to recognize that SQA is a legitimate discipline unto itself and that it requires specialized training. </li>
</ul>
<p><strong>The Bottom Line....</strong></p>
<p> Edsger W. Dijkstra was one of the distinguished participants at the 1968 NATO Conference. He made an incredibly insightful remark at this conference that, while important in 1968, is even more important today:</p>
<blockquote>
<p> “The dissemination of knowledge is of obvious value — the massive dissemination of error-loaded software is frightening.” [6]</p>
</blockquote>
Til next time...</td>
</tr>
</table>
<br>
<br>
<a name="morsel"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/MonthlyMorsels.gif" alt="Monthly Morsels" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p> Every month in this space, you’ll find additional information related to this month’s topic.</p>
<p><strong> References</strong></p>
<ol>
<li> Nelson, J. G., "Software Testing in Computer-Driven Systems," in <em>Software Quality Management</em>, ed. Fisher, Matthew J., and Cooper, John D., Petrocelli Books, 1979.<br>
<br>
</li>
<li> Arthur, J. D. and Nance, R. E., “Verification and Validation Without Independence: A Recipe for Failure”, Proc. 2000 Winter Simulation Conference, Orlando FL, 2000. <br>
<br>
</li>
<li> Wallace, D. R., and Fuji, R. U., “Software Verification and Validation: Its Role in Computer Assurance and Its Relationship with Software Project Management Standards”, National Institute of Standards and Technology, Special Publication 500-165, May 1989.<br>
<br>
</li>
<li> NASA Policy Directive NPD 8730.4A, Effective August 1, 2001. <br>
<br>
</li>
<li> Robert M. McClure, <strong><a href="http://homepages.cs.ncl.ac.uk/brian.randell/NATO/Introduction.html" target="_blank">Introduction to the 1968 NATO Software Engineering Conference</a></strong>.<strong><br>
<br>
</strong></li>
<li><strong><a href="http://homepages.cs.ncl.ac.uk/brian.randell/NATO/index.html" target="_blank">Software Engineering, Report on a conference sponsored by the NATO Science Committee Garmisch</a></strong>, Germany, October 7-11, 1968 <br>
<br>
</li>
<li> Glass, R., Software Runaways: <em>Lessons Learned from Massive Software Project Failures</em>, Prentice-Hall PTR, 1997.<br>
<br>
</li>
<li> Weiner, L., <em>Digital Woes: Why We Should Not Depend On Software</em>, Addison-Wesley, 1993. <br>
<br>
</li>
<li> Johnson, J., “Chaos: The Dollar Drain of IT Project Failures,” Application Development Trends, January 1995, pp. 41-47. </li>
</ol>
<p><strong> IV&V Resources </strong></p>
<ul>
<li> Lewis, Robert O., <strong><a href="http://books.google.com/books?id=DhrVOC0l7mUC&dq=IV&V+Books&source=gbs_summary_s&cad=0" target="_blank"><em>Independent verification and validation: a life cycle engineering process for quality software</em></a></strong>, Wiley-IEEE, 1992.</li>
</ul>
<ul>
<li> Schulmeyer, C. G. and Mackenzie, G. R., <strong><a href="http://www.amazon.com/Verification-Validation-Modern-Software-Intensive-Systems/dp/0130205842" target="_blank"><em>Verification and Validation of Modern Software-Intensive Systems</em></a></strong>, Prentice Hall-PTR, 2000. </li>
</ul>
<ul>
<li> Rakitin, S., <strong><a href="http://www.amazon.com/Software-Verification-Validation-Practitioners-Managers/dp/1580532969" target="_blank"><em>Software Verification and Validation for Practitioners and Managers</em></a></strong>, 2nd edition, Artech House 2001. </li>
</ul>
<ul>
<li> Arthur, J., et. al., “<strong><a href="/newsletter/vol6/no4/IVVComputerOct%2099.pdf" target="_blank">Evaluating the Effectiveness of Independent Verification & Validation</a></strong>,” <em>IEEE Computer</em>, October 1999. </li>
</ul>
<ul>
<li><strong><a href="/newsletter/vol6/no4/NASA.pdf" target="_blank">NASA IV&V Overview</a></strong>.</li>
</ul></td>
</tr>
</table>
<br>
<br>
<a name="calendar"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/Calendar.gif" alt="Calendar" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p> Every month you’ll find news here about local and national events that are of interest to the software community…</p>
<ul>
<li><strong> Software Quality Calendar</strong></li>
</ul>
<blockquote>
<p>There are many organizations that sponsor monthly meetings, workshops, and conferences of interest to software professionals. <strong><a href="/links/upcoming.html" target="_blank">Find out what’s happening…</a></strong></p>
</blockquote>
<ul>
<li><strong> Workshops Offered by Software Quality Consulting</strong></li>
</ul>
<blockquote>
<p>Software Quality Consulting offers workshops in many topics related to software process improvement. <strong><a href="/seminars/courses.html" target="_blank">Get more info…</a></strong></p>
</blockquote></td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/AboutSQC.gif" alt="About SQC" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p> Software Quality Consulting provides consulting, training, and auditing services tailored to meet the specific needs of clients. We help clients fine-tune their software development processes and improve the quality of their software products. The overall goal is to help clients achieve Predictable Software Development™ – so that organizations can consistently deliver quality software with promised features in the promised timeframe. </p>
To learn more about how we can help your organization, <strong><a href="/index.html?AboutSQC" target="_blank">visit our web site</a></strong> or <strong><a href="mailto:[email protected]">send us an email</a></strong>.</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top"><p> I hope this newsletter has been informative and helpful. Your comments and feedback are most welcome. <strong><a href="mailto:[email protected]">Send me your feedback…</a></strong></p>
<p>Thanks,</p>
<p> <img src="/newsletter/images/BusinessCard.gif" width="270" height="121" align="right"><img src="/newsletter/images/Signature.gif" width="90" height="68"><br>
Steve Rakitin<br>
<br>
<strong><a href="mailto:[email protected]">[email protected]</a></strong></p></td>
</tr>
</table>
<div align="center"><br>
<FONT class="Reference">Food for Thought, Predictable Software Development, Act Like a Customer,<br>
and ALAC are trademarks of Software Quality Consulting, Inc.<br>
Copyright 2009. Software Quality Consulting, Inc. All rights reserved.<br>
Graphic design by <a href="http://www.sarahcoledesign.com/" target="_blank"><strong>Sarah Cole Design</strong></a>.</FONT></div>
<a name="bottom"> </a></body>
</html>