|
Server : Apache/2.4.62 System : FreeBSD fbsdweb2.web.rcn.net 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64 User : www ( 80) PHP Version : 8.3.8 Disable Function : NONE Directory : /domains/srakitin/OLD/newsletter/vol2/no2/ |
Upload File : |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Food for Thought: To Release or Not To Release</title>
<link href="/newsletter/StyleSheet.css" rel="stylesheet" type="text/css">
</head>
<OpenTracking/>
<!-- Do NOT delete previous line if you want to get statistics on the number of opened emails -->
<body>
<strong> </strong>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0">
<tr align="center" valign="top">
<td colspan="2"><img src="/newsletter/images/FoodForThoughtLogo.gif" width="600" height="105"></td>
</tr>
<tr class="Reference">
<td align="left" valign="top"><p>An e-newsletter published by<br>
Software Quality Consulting, Inc. </p>
</td>
<td align="right" valign="top"><p>February 2005, Vol. 2 No. 2 <br>
[<a href="http://www.swqual.com/newsletter/vol2/no2/vol2no2.txt" target="_blank">Text-only Version</a>] </p>
</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top"> <p>
Welcome to <em><strong>Food for Thought™</strong></em>, an e-newsletter from <strong> <a href="http://www.swqual.com" target="_blank">Software Quality Consulting</a></strong>. I've created free subscriptions for my valued business contacts. If you find this newsletter informative, I encourage you to continue reading. Feel free to pass this newsletter along to colleagues by clicking this <strong><a href="http://ui.constantcontact.com/roving/sa/fp.jsp?plat=i&p=f&m=sctz69n6">Forward Email</a></strong> link. If you’ve received this newsletter from a colleague and would like to subscribe, please click this <strong> <a href="http://www.swqual.com/newsletter/Subscribe.htm" target="_blank">Enter New Subscription </a></strong>link. If you don't wish to receive this newsletter, click the <strong><a href="#bottom">SafeUnSubscribe</a></strong>™ link at the bottom of this newsletter, and you won’t be bothered again.<br>
<br>
Your continued feedback on this newsletter is most welcome. Please send your comments and suggestions to <strong><a href="mailto:[email protected]">[email protected]</a></strong>. </p>
</td>
</tr>
</table>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><div align="center"><img src="/newsletter/images/InThisIssue.gif" width="114" height="37"><br>
</div></td>
<td width="15"> </td>
<td align="left" valign="top"><p> In <a href="#article"><strong>This Months’ Topic</strong></a>, we look at Quality from a “Good Enough” perspective…<br>
<br>
Regular features to look for each month are:</p>
<ul>
<li> <a href="#morsel"><strong>Monthly Morsels</strong></a><br>
Hints, tips, techniques and reference info related to this month’s topic</li>
</ul>
<ul>
<li> <a href="#calendar"><strong>Calendar</strong></a><br>
Conferences, workshops, and meetings of interest to software engineers, QA engineers and anyone interested in software development</li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<a name="article"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><p align="center"><img src="/newsletter/images/ThisMonthsTopic.gif" width="114" height="37"><br>
</td>
<td width="15"> </td>
<td width="471" align="left" valign="top" class="BodyText"><p align="center"><span class="Headline">To Release or Not To Release</span></p>
<p>The cost of poor quality software is staggering. A recent report prepared for the National Institute of Science and Technology (NIST) found that:
</p>
<blockquote>
<p>“Estimates of the economic costs of faulty software in the U.S. range in the tens of billions of dollars per year and have been estimated to represent approximately just under 1 percent of the nation’s gross domestic product (GDP).” [1]</p>
</blockquote>
<p>Some of the reasons we produce so much “faulty” software are: </p>
<ul>
<li>We lack a business-appropriate definition of quality for many software products</li>
<li>We fail to identify and use quality indicators during the development process</li>
<li>We tend to rely solely on test results when making the release decision</li>
<li>Customers are not sufficiently outraged to force companies to change their ways</li>
</ul> </td>
</tr>
<tr>
<td align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/vol2/no2/Images/Scale.gif" width="110" height="144"></td>
<td> </td>
<td align="left" valign="top" class="BodyText"><p><strong>How do you make the decision to release software?</strong></p>
<p>Within most organizations, there is an implicit relationship between testing and quality. What we know to be true is that test results represent one indicator of quality. As pointed out by Steve McConnell:</p>
<blockquote>
<p>“Testing by itself does not improve software quality. Test results are only an indicator of quality. Trying to improve quality by increasing testing is like trying to lose weight by weighing yourself more often.” [4]</p>
</blockquote>
<p>There are other indicators of quality that can be used in addition to test results. These other indicators include such measures as:</p>
<ul>
<li>Requirements Stability</li>
<li>Design Stability</li>
<li>Code Stability</li>
<li>Defect Stability</li>
</ul>
<p><a href="http://www.swqual.com/newsletter/Indicators.pdf" target="_blank"><strong>Get more info on other indicators of quality…</strong></a></p>
<p>These other indicators can provide valuable additional information that Management can use to help make this important decision. If I were responsible for making the release decision, I would want as much factual information that could reasonably be collected.</p>
<p>Frequently though, test results are the only indicator of quality that software companies look at to make the release decision. But, as I pointed out in my <a href="http://www.swqual.com/newsletter/vol1/no4/vol1no4.html" target="_blank"><strong>December newsletter</strong></a>:</p>
<ul>
<li>If you don’t have a definition of quality, you can’t measure it. </li>
</ul>
<p>So the situation your company may be in is: </p>
<ul>
<li>Test results represent the only indicator of quality used to make the release decision<br>
<br>
</li>
<li>You don’t have a definition of quality and therefore you can’t measure it<br>
<br>
</li>
<li>You don’t have an understanding of the quality expectations of your customers</li>
</ul>
<p>The problem you face then is:</p>
<ul>
<li>How can you determine how much and what kinds of testing are sufficient?<br>
<br>
</li>
<li>How can you make a business-appropriate release decision?</li>
</ul>
<p><strong>Is your Software “Good Enough”?</strong></p>
<p>For software that is not safety-critical, mission-critical or life supporting, there is a school of thought that says this type of software doesn’t have to be perfect, it has to be “good enough”. Stated another way: </p>
<ul>
<li>“Our task is not to blindly eliminate all problems, but to understand the problems and benefits of a situation well enough to eliminate (or prevent) the right problems and also deliver the right benefits.” [2]</li>
</ul>
<p>One of the definitions of software quality I included in my <a href="http://www.swqual.com/newsletter/vol1/no4/vol1no4.html" target="_blank"><strong>December newsletter</strong></a> is based on the “Good Enough” principle as defined by James Bach. According to Bach, </p>
<blockquote>
<p>“To claim that any given thing is Good Enough is to agree with the following propositions:</p>
<ul>
<li>It has sufficient benefits.<br>
<br>
</li>
<li>It has no critical problems.<br>
<br>
</li>
<li>The benefits sufficiently outweigh the problems.<br>
<br>
</li>
<li>In the present situation, and all things considered, further improvement would be more harmful than helpful.</li>
</ul>
<p>Each point is critical. If any one of them is not satisfied, then the product, although perhaps good, cannot be good enough.” [2]</p>
</blockquote>
<p>An important point that Bach fails to mention in his “Good Enough” proposition is that, especially with respect to the first three points, it’s the customer’s perception that matters not the developers. </p>
<p>A problem with the “Good Enough” paradigm is that it results in a de facto definition of quality that may or may not be appropriate for your products and customers. It doesn’t take into account the customer’s perspective, nor does it take into account the other indicators of quality I identified above. </p>
<p>I believe that you need a business-appropriate definition of quality in order to apply the “Good Enough” paradigm. In fact, the NIST Report says that:</p>
<blockquote>
<p>“The problem is exacerbated because there is disagreement not only on how to define [good] enough, but also on what tests should be run to determine what is enough. For example, commercial software developers use a combination of the following non-analytical methods to decide when a software element is “good enough” to release:</p>
<ul>
<li>A ‘sufficient’ percentage of test cases run successfully. <br>
<br>
</li>
<li>Developers execute a test suite while running a code coverage analyzer to gather statistics about what code has been exercised.<br>
<br>
</li>
<li>Defects are classified into different severity categories and numbers and trends within each category are analyzed.<br>
<br>
</li>
<li>Beta testing is conducted, allowing real users to run a product for a certain period of time and report problems; then developers analyze the severity and trends for reported problems.<br>
<br>
</li>
<li>Developers analyze the number of reported problems in a period of time; when the number stabilizes or is below a certain threshold for a period of time, it is considered ‘good enough.’ ” [1]</li>
</ul>
</blockquote></td>
</tr>
<tr>
<td align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/vol2/no2/Images/Wanted.gif" width="110" height="127"></td>
<td> </td>
<td align="left" valign="top" class="BodyText"><p><strong>Wanted: Business-appropriate Models</strong></p>
<p>If we are to reduce the costs associated with what the NIST Report calls “faulty” software, we need business-appropriate models for making more informed and better business decisions about releasing software. Such models need to provide quality indicators throughout the software development cycle – not just during the testing phase. </p>
<p>With respect to quality, the die is cast by the time software is in test. The quality of software is influenced and determined by many factors including the quality of the: </p>
<ul>
<li>development process</li>
<li>requirements</li>
<li>design</li>
<li>code</li>
<li>tests</li>
</ul>
<p>as well as the skill and professionalism of the staff.</p>
<p>All of these factors contribute to the ultimate quality of the product. Using only one indicator of quality without considering others results in a biased view of the quality of your product.</p>
<p>Organizations need to work on producing higher quality software from the start. Not only will this reduce test time, it will reduce overall cost and enable companies to become more competitive.</p>
<p><strong>The “Good Enough Testing” Model</strong></p>
<p>In looking for a business-appropriate model, I came across Bach’s “Good Enough Testing” approach. He defines “Good Enough Testing” as:</p>
<blockquote>
<p>“… the process of developing a sufficient assessment of quality, at a reasonable cost, to enable wise and timely decisions to be made concerning the product.” [3] </p>
</blockquote>
<p>He further expands this notion to include the following four activities:</p>
<ul>
<li>“Assessment of Product Quality<br>
<br>
<ul>
<li>How accurate and complete is it?<br>
<br>
</li>
</ul>
</li>
<li>Cost of Testing<br>
<br>
<ul>
<li>How reasonable is this cost?</li>
<li> Is it within project constraints?</li>
<li>Is there are good return on investment, such as the extent of information gained per test?<br>
<br>
</li>
</ul>
</li>
<li>Decisions<br>
<br>
<ul>
<li>How well does the assessment serve the project and the business?<br>
<br>
</li>
</ul>
</li>
<li>Timing of all the above <br>
<br>
<ul>
<li>Is it soon enough to be useful?” [3]</li>
</ul>
</li>
</ul>
<p>All of these are critically important questions that need to be asked and answered if we expect to get a good return on the investment we make in testing.</p>
<p>While I think that Bach’s “Good Enough Testing” is a good start, more needs to be done if Management is to have a <strong>business-appropriate model</strong> for making better business decisions. </p>
<p>For now, chew on this and send me your thoughts and comments. Stay tuned for an example of what a <strong>business-appropriate model</strong> for making better business decisions might look like.</p></td>
</tr>
</table>
<br>
<br>
<a name="morsel"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/MonthlyMorsels.gif" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top">Every month in this space you’ll find additional information related to this month’s topic.
<ul>
<li><strong>On-line Resources</strong>:<br>
<br>
<ul>
<li><a href="http://www.satisfice.com/articles.shtml" target="_blank"><strong>Satisfice</strong></a> (James Bach’s company)
<ul>
<li>Articles on “Good Enough” quality and articles on testing<br>
<br>
</li>
</ul>
</li>
<li><a href="http://www.stickyminds.com" target="_blank"><strong>StickyMinds</strong></a>
<ul>
<li>Articles of interest to testing professionals<br>
<br>
</li>
</ul>
</li>
<li><a href="http://www.asq.org/pub/sqp/" target="_blank"><strong>Software Quality Professional</strong></a>
<ul>
<li>ASQ Software Division’s quarterly journal<br>
<br>
</li>
</ul>
</li>
</ul>
</li>
<li><strong>Testing Resources</strong>:<br>
<br>
Kaner, C., et. al, Testing Computer Software, 2nd ed, Thomson Computer Press, 1993<br>
<br>
Marick, B., The Craft of Software Testing, Prentice-Hall PTR, 1995<br>
<br>
</li>
<li><strong>References</strong>:<br>
<br>
[1] NIST Planning Report 02-3, “The Economic Impacts of Inadequate Infrastructure for Software Testing”, May 2002.<br>
<br>
[2] Bach. J., “Good Enough Quality – Beyond the Buzzword”, IEEE Computer, August 1997.<br>
<br>
[3] Bach, J., “A Framework for Good Enough Testing”, IEEE Computer, October 1998.<br>
<br>
[4] McConnell, S., Code Complete, Microsoft Press, 1993.</li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<a name="calendar"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/Calendar.gif" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top">Every month, you’ll find news here about local and national events that are of interest to the software community …
<ul>
<li><strong>Software Quality Calendar<br>
<br>
</strong>There are many organizations that sponsor monthly meetings, workshops, and conferences of interest to software professionals. <a href="http://www.swqual.com/links/upcoming.html" target="_blank"><strong>Find out what’s happening…</strong></a><br>
<br>
</li>
<li><strong>Workshops Offered by Software Quality Consulting<br>
<br>
</strong>Software Quality Consulting offers workshops in many topics related to software process improvement. <a href="http://www.swqual.com/seminars/courses.html" target="_blank"><strong>Get more info...</strong></a></li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/AboutSQC.gif" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top">Software Quality Consulting provides consulting, training, and auditing services tailored to meet the specific needs of clients. We help clients fine-tune their software development processes and improve the quality of their software products. The overall goal is to help clients achieve Predictable Software Development™ – so that organizations can consistently deliver quality software with promised features in the promised timeframe.
<p>To learn more about how we can help your organization, <a href="http://www.swqual.com" target="_blank"><strong>visit our web site</strong></a> or <a href="mailto:[email protected]"><strong>send us an email</strong></a>.<br>
</p>
</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top"><p> I hope this newsletter has been informative and helpful. Your comments and feedback are most welcome. <a href="mailto:[email protected]"><strong>Send me your feedback…</strong></a></p>
<p>Thanks,</p>
<p> <img src="/newsletter/images/BusinessCard.gif" width="270" height="121" align="right"><img src="/newsletter/images/Signature.gif" width="90" height="68"><br>
Steve Rakitin<br>
<br>
<strong><a href="mailto:[email protected]">[email protected]</a></strong></p></td>
</tr>
</table>
<div align="center"><br>
<FONT class="Reference"> Food for Thought and Predictable Software Development are trademarks of Software Quality Consulting, Inc. <br>
Copyright © 2005. Software Quality Consulting, Inc. All rights reserved.
Graphic design by <B><U><FONT color=blue><a href="mailto:[email protected] ">Sage Studio</a></FONT></U></B></FONT></div>
<a name="bottom"> </a></body>
</html>