|
Server : Apache/2.4.62 System : FreeBSD fbsdweb2.web.rcn.net 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64 User : www ( 80) PHP Version : 8.3.8 Disable Function : NONE Directory : /domains/srakitin/newsletter/vol9/no1/ |
Upload File : |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Food for Thought: Becoming Best-in-Class</title>
<link href="/StyleSheet.css" rel="stylesheet" type="text/css">
</head>
<OpenTracking/>
<!-- Do NOT delete previous line if you want to get statistics on the number of opened emails -->
<body>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0">
<tr align="center" valign="top">
<td colspan="2"><img src="/images/food_for_thought_big.gif" alt="Food for Thought" width="436" height="169"></td>
</tr>
<tr class="Reference">
<td align="left" valign="top"><p><br>
An e-newsletter published by<br>
Software Quality Consulting, Inc. </p>
</td>
<td align="right" valign="top"><p><br>
March 2012, Vol. 9 No. 1<br>
[<a href="/newsletter/vol9/no1/vol9no1.txt" target="_blank">Text-only Version</a>]</p>
</td>
</tr>
</table>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top">
<p>Welcome to <em><strong>Food for Thought™</strong></em>, an e-newsletter from <a href="/index.html?Intro" target="_blank">Software Quality Consulting</a>. I've created free subscriptions for my valued business contacts. If you find this newsletter informative, I encourage you to continue reading. Feel free to pass this newsletter along to colleagues by clicking on the <strong>Forward Email</strong> link at the bottom of this email. If you’ve received this newsletter from a colleague and would like to subscribe, please click this <a href="/e_newsletter.html" target="_blank">Enter New Subscription</a> link. If you don't wish to receive this newsletter, click the <a href="#bottom">SafeUnSubscribe</a>™ link at the bottom of this newsletter, and you won’t be bothered again.</p>
<p>Your continued feedback on this newsletter is most welcome. Please send your comments and suggestions to <a href="mailto:[email protected]">[email protected]</a>.</p></td>
</tr>
</table>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="145" align="left" valign="top"><img src="/images/in_this_issue.gif" alt="In This Issue" width="145" height="36"></td>
<td width="15" align="left" valign="top"> </td>
<td width="440" align="left" valign="top">In <a href="#article">This Month's Topic</a>,
I discuss what it takes to become a best-in-class software company…
<p>Regular features to look for each month are: </p>
<ul>
<li> <a href="#morsel">Monthly Morsels</a><br>
Hints, tips, techniques and reference info related to this month’s topic</li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<a name="article"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="145" align="left" valign="top"><img src="/images/this_months_topic.gif" alt="This Month's Topic" width="145" height="34">
</td>
<td width="15"> </td>
<td width="440" align="left" valign="top" class="BodyText"><div align="center" class="Headline">Becoming Best-in-Class</div>
<p> Today’s cars have a frightening amount of software. As I reported in my newsletters in <a href="/images/FoodforThought_Feb2010.pdf" target="_blank">February</a> and <a href="/images/FoodforThought_Mar2010.pdf" target="_blank">March</a> of 2010, today’s automobiles have upwards of <strong>100 million</strong> lines of code. By comparison, about <strong>6.5 million</strong> lines of code are required for the avionics and on-board support systems for the new Boeing 787 Dreamliiner. What does all of this automotive software do? As reported in IEEE Spectrum [4], software is used in the following areas:</p>
<table width="440" border="1" cellpadding="5" cellspacing="0" class="BodyText">
<tr align="center" valign="middle" class="BodyText">
<td width="155"><p> Air-bag system </p></td>
<td width="155"><p> Anti-lock Brakes </p></td>
<td width="170"><p> Automatic Transmission </p></td>
</tr>
<tr align="center" valign="middle" class="BodyText">
<td width="155"><p> Alarm system </p></td>
<td width="155"><p> Climate control </p></td>
<td width="170"><p> Collision avoidance system </p></td>
</tr>
<tr align="center" valign="middle" class="BodyText">
<td width="155"><p> Cruise control </p></td>
<td width="155"><p> Communication system </p></td>
<td width="170"><p> Dashboard instrumentation </p></td>
</tr>
<tr align="center" valign="middle" class="BodyText">
<td width="155"><p> Electronic stability control </p></td>
<td width="155"><p> Engine ignition </p></td>
<td width="170"><p> Engine control </p></td>
</tr>
<tr align="center" valign="middle" class="BodyText">
<td width="155"><p> Electronic seat control </p></td>
<td width="155"><p> Entertainment system </p></td>
<td width="170"><p> Navigation system </p></td>
</tr>
<tr align="center" valign="middle" class="BodyText">
<td width="155"><p> Power Steering </p></td>
<td width="155"><p> Tire pressure monitoring </p></td>
<td width="170"><p> Windshield wiper control </p></td>
</tr>
</table><br>
Ford recently introduced a touch screen control for several systems to its vehicles. Safety issues related to using a touch screen while driving aside, the software-intensive MyFord Touch system has been nothing but trouble for customers.
</td>
</tr>
<tr>
<td align="left" valign="top"><p><img width="145" height="108" src="/newsletter/vol9/no1/vol9no1_clip_image001_0000.jpg"></p>
<p align="center" class="Reference">MyFord Touch screen </p></td>
<td> </td>
<td align="left" valign="top" class="BodyText"><blockquote>
<p>“One customer reported the navigation system often malfunctioned, the rearview camera frequently stayed on while the vehicle was moving forward and the system randomly rebooted. The voice controls typically do not work until the vehicle has been on for five to 10 minutes, meaning short trips require dialing phone calls by hand, only to have the call cut off when the system finally starts up…” [5]</p>
</blockquote>
<p>Frustrated Ford owners paid about $395 for the MyFord Touch option only to find it exceedingly difficult to use and, in some cases, not usable at all. While Ford has been somewhat successful in trying to improve customers’ perception of quality, this recent software glitch is a significant setback.</p>
<p>The problem Ford is having with its MyFord Touch software [1] indicates how poor software can dramatically affect customer satisfaction. Ford dropped from 5th to 10th in the latest Consumer Reports annual ranking of auto manufacturers and dropped from 10th to 20th in the Consumer Reports annual reliability ranking. J.D. Power & Associates ranked Ford 23rd in its new car quality survey – down from 5th place. </p>
<p>Software problems can and do affect customer satisfaction which in turn have a dramatic impact on sales. How then can companies like Ford become best-in-class when it comes to software?</p>
<p><strong>What does best-in-class mean?</strong></p>
<p>Best–in-class companies develop products that have fewer customer-reported defects and, as a result, provide more value to customers. From a software perspective, best-in-class companies have very few customer-reported problems. Most best-in-class software companies use the <strong>Defect Removal Efficiency</strong> [2] metric to measure their performance along with frequent customer satisfaction surveys. </p>
<p>Defect Removal Efficiency measures how effective your tests are at removing defects your customers are likely to find based on actual customer use over a defined period of time. The metric is calculated as follows:</p>
<p align="center"><strong><u>Total defects we find prior to release</u><br>
Total we find + customer-reported defects</strong></p>
<p>Best-in-class software companies have defect removal efficiencies of 99.5% or higher. The higher the defect removal efficiency metric, the fewer problems your customers experience in using your software. Note that very high defect removal efficiency doesn’t mean that there aren’t any defects in the software – just that customers aren’t finding them. The defect removal efficiency for mediocre software is between 80-85%. </p>
<p>A survey performed by Capers Jones [3] of over 1,500 projects showed the following:</p>
<table width="440" border="1" cellpadding="5" cellspacing="0" class="BodyText">
<tr align="center" valign="middle">
<td colspan="2"><strong>Defect Removal<br>
Efficiency (%)</strong></td>
<td colspan="2"><strong>Number of<br>
Projects</strong></td>
<td colspan="2"><strong>Percentage<br>
of Projects</strong></td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><p align="center"> > 99</p></td>
<td colspan="2"><p align="center"> 6</p></td>
<td colspan="2"><p align="center"> 0.4%</p></td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><p align="center"> 95-99</p></td>
<td colspan="2"><p align="center"> 104</p></td>
<td colspan="2"><p align="center"> 6.9%</p></td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><p align="center"> 90-95</p></td>
<td colspan="2"><p align="center"> 263</p></td>
<td colspan="2"><p align="center"> 17.5%</p></td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><p align="center"> 85-90</p></td>
<td colspan="2"><p align="center"> 559</p></td>
<td colspan="2"><p align="center"> 37.3%</p></td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><p align="center"> 80-85</p></td>
<td colspan="2"><p align="center"> 408</p></td>
<td colspan="2"><p align="center"> 27.2%</p></td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><p align="center"> < 80</p></td>
<td colspan="2"><p align="center"> 161</p></td>
<td colspan="2"><p align="center"> 10.7%</p></td>
</tr>
</table>
<p>Low defect removal efficiency can result from a lack of understanding of how your customers are actually using your software. Watts Humphrey gave us the notion of the “testing footprint” (illustrated below) which exposes differences in how you test your software and how your customers are actually using your software. Best-in-class companies truly understand how customers are using their software and have focused testing efforts based on this understanding.</p>
<p> <img width="440" height="349" src="/newsletter/vol9/no1/vol9no1_clip_image001.jpg"> <br>
<strong>How to become best-in-class</strong></p>
<p>As you might expect, becoming best-in-class requires a strong commitment from management. Here then is my Top Ten List for becoming a best-in-class software company:</p>
<ol>
<li><strong>Balance defect prevention and defect detection activities.</strong></li>
</ol>
<blockquote>
<p>Management should provide incentives and rewards for preventing problems before they occur. Management needs to recognize the significant cost savings that can be realized by focusing on preventing problems rather than fixing problems. Management can do this by providing incentives and recognition for those who proactively prevent defects on project teams.</p>
<p>See my <a href="/newsletter/vol8/no4/vol8no4.html" target="_blank">Nov 2011 newsletter</a> for more details on this topic…</p>
</blockquote>
<ol>
<li value="2"><strong>Improve requirements writing skills.</strong></li>
</ol>
<blockquote>
<p>Poorly written requirements are the norm on many projects. Clearly written, testable requirements are the lynchpin for improving defect removal efficiency and becoming best-in-class. Project teams need to learn simple techniques for expressing requirements clearly and unambiguously to prevent problems associated with poorly written requirements.</p>
<p><a href="/training_mission_best_practices.html" target="_blank">More information on writing clear, unambiguous requirements…</a></p>
</blockquote>
<ol>
<li value="3"><strong>Involve SQA early to assess requirements testability.</strong></li>
</ol>
<blockquote>
<p>SQA engineers need to become involved early on to assess requirements and identify testability issues early when these requirements are less costly to change.</p>
</blockquote>
<ol>
<li value="4"><strong>Identify Root Cause of every customer-reported defect.</strong></li>
</ol>
<blockquote>
<p>Every customer-reported defect represents an aspect of customer use that is not well understood. Each customer-reported defect should be considered for root cause analysis in order to determine why this defect escaped and how did we miss it?</p>
</blockquote>
<ol>
<li value="5"><strong>Perform Pareto Analysis and take corrective action.</strong></li>
</ol>
<blockquote>
<p>Once root causes are identified, a pareto analysis should be performed to determine what aspects of your process are causing the largest number of defects – this is the proverbial 80/20 rule. Fix that part of your process that is causing these defects in order to improve defect removal efficiency.</p>
</blockquote>
<ol>
<li value="6"><strong>Use an effective Peer Review process.</strong></li>
</ol>
<blockquote>
<p>Many companies claim to perform peer reviews but few do them well. Peer reviews have been proven time and time again to be extremely effective in finding problems that would be far more difficult and costly to find in test. Work on improving the effectiveness of your peer review process by requiring participants to come to review meetings prepared!</p>
<p><a href="/training_mission_peer_reviews.html" target="_blank">More information on creating an effective peer review process…</a></p>
</blockquote>
<ol>
<li value="7"><strong>Everyone must take responsibility for his or her work.</strong></li>
</ol>
<blockquote>
<p>Becoming best-in-class requires a strong personal commitment from software developers and SQA engineers. Software developers must be willing to say:</p>
<blockquote>
<p>“I believe my code is as good as it can be and correctly implements defined requirements. I challenge anyone to prove me wrong.”</p>
</blockquote>
<p>Similarly, SQA should be willing to say:</p>
</blockquote>
<ol>
<blockquote>
<p>“I believe my tests are as good as they can be and accurately reflect defined requirements and how our customers use our software. I challenge anyone to prove me wrong.”</p>
</blockquote>
<li value="8"><strong>Use a combination of testing perspectives.</strong></li>
</ol>
<blockquote>
<p>From experience, we know that using a combination of several testing perspectives or stages increases the ability to identify defects. Examples of testing perspectives or stages include unit testing, integration testing, user testing, system testing, etc.</p>
<p>As reported by Capers Jones [3]</p>
<table border="1" cellpadding="5" cellspacing="0" class="BodyText">
<tr align="center" valign="middle">
<td><p align="center"><strong>Number of Testing Stages</strong></p></td>
<td><p align="center"><strong>Percent of<br />
Effort Devoted to Testing</strong></p></td>
<td><p align="center"><strong>Cumulative<br />
Defect<br />
Removal Efficiency</strong></p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">1 testing stage</p></td>
<td><p align="center">10%</p></td>
<td><p align="center">50%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">2 testing stages</p></td>
<td><p align="center">15%</p></td>
<td><p align="center">60%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">3 testing stages</p></td>
<td><p align="center">20%</p></td>
<td><p align="center">70%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">4 testing stages</p></td>
<td><p align="center">25%</p></td>
<td><p align="center">75%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">5 testing stages</p></td>
<td><p align="center">30%</p></td>
<td><p align="center">80%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">6 testing stages*</p></td>
<td><p align="center">33%*</p></td>
<td><p align="center">85%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center">7 testing stages</p></td>
<td><p align="center">36%</p></td>
<td><p align="center">87%</p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center"><strong>8 testing stages</strong></p></td>
<td><p align="center"><strong>39%</strong></p></td>
<td><p align="center"><strong>90%</strong></p></td>
</tr>
<tr align="center" valign="middle">
<td><p align="center"><strong>9 testing stages</strong></p></td>
<td><p align="center"><strong>42%</strong></p></td>
<td><p align="center"><strong>92%</strong></p></td>
</tr>
</table>
<p>* Six test stages, 33% cost and 85% removal efficiency are U.S. averages.</p>
</blockquote>
<ol>
<li value="9"><strong>Increase domain knowledge of testers.</strong></li>
</ol>
<blockquote>
<p>Testers need domain knowledge in order to know where they should focus their testing efforts. The testing footprint mentioned earlier can be an effective tool for focusing the testing effort – but it requires that testers really understand what real users do. How can testers acquire domain knowledge? They need to spend time observing what real customers do when they use your software. They also need to spend time in Customer Support listening to the kinds of problems customers encounter on a day-to-day basis.</p>
</blockquote>
<ol>
<li value="10"><strong>Measure progress using customer satisfaction surveys and defect removal efficiency.</strong></li>
</ol>
<blockquote>
<p>Measuring customer satisfaction on a regular basis and responding to feedback is a sure sign of a best-in-class company. When customer problems arise, best-in-class companies use the opportunity to win back customer trust by going above and beyond what customers would typically expect. For example, instead of simply pushing out hot fixes, find a way to delight customers by doing things like extending their support contract or providing free support for a limited time.</p>
<p>Another sign of a best-in-class company is measuring and using the defect removal efficiency metric to continually improve. Customer satisfaction and defect removal efficiency are good examples of measurements that can be tied to overall corporate quality improvement goals.</p>
</blockquote></td>
</tr>
<tr>
<td align="left" valign="top"><img width="145" height="96" src="/newsletter/vol9/no1/vol9no1_clip_image002.jpg"></td>
<td> </td>
<td align="left" valign="top" class="BodyText"><p><strong>How to win over unhappy customers – or not</strong></p>
<p>Okay, so you own one of the Ford cars with the defective MyFord Touch software. What happens now? </p>
<blockquote>
<p>Ford recently announced they are sending updated software on thumb drives directly to about 300,000 customers so they can install new software themselves. Alternatively, they can bring their care in to a dealer who will do the installation for them.</p>
<p>If you want to do the upgrade yourself, it will take about an hour of your time and the <strong>car must be running the whole time</strong>. This will waste about a gallon of expensive gas. Let’s see, I need to spend an hour of my time and waste a gallon of my gas to fix a defective product that I paid a lot of money for… hmmm…</p>
</blockquote>
<p>Am I missing something here or are all those unhappy MyFord Touch customers still unhappy?</p>
<p>‘till next time…</p></td>
</tr>
</table>
<br>
<br>
<a name="morsel"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="145" align="left" valign="top"><img src="/images/monthly_morsels.gif" alt="Monthly Morsels" width="145" height="35"></td>
<td width="15"> </td>
<td width="440" align="left" valign="top"><p> Every month in this space, you’ll find additional information related to this month’s topic.</p>
<ol>
<li> Priddle, A., “Ford sending fixes for MyFord Touch systems”, <em>Detroit Free Press</em>, Mar 6, 2012.<br>
<br>
</li>
<li>Jones, C., “Software Defect Removal Efficiency”, <em>IEEE Computer,</em> April 1996.<br>
<br>
</li>
<li>Jones, C., “SOFTWARE DEFECT REMOVAL: THE STATE OF THE ART IN 2010”.<br>
<br>
</li>
<li>Charette, R., “This Car Runs on Code”, <em>IEEE Spectrum</em>, February 2009.<br>
<br>
</li>
<li>Bunkley, N., “After Ratings Drop, Ford Reworks Touch Screens”, <em>NY Times</em>, March 5, 2012. </li>
</ol>
</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="145" align="left" valign="top"><img src="/images/about_SQC.gif" alt="About SQC" width="145" height="35"></td>
<td width="15"> </td>
<td width="440" align="left" valign="top">Software Quality Consulting provides a full-range of software engineering services for safety-critical industries and mission-critical projects. Our goal is to help create safety-critical and mission-critical software that meets our client’s needs, complies with all applicable standards and regulations, with the highest level of quality possible, and in the most cost-effective and timely manner possible.
<p>To learn more about how we can help your organization, <a href="/index.html?AboutSQC" target="_blank">visit our web site</a> or <a href="mailto:[email protected]">send us an email</a>.</p></td></tr>
</table>
<br>
<div align="center" class="Reference">Food for Thought, Predictable Software Development, Act Like a Customer,<br>
and ALAC are trademarks of Software Quality Consulting, Inc.<br>
Copyright 2012. Software Quality Consulting, Inc. All rights reserved.<br>
Graphic design by <a href="http://www.sarahcoledesign.com/" target="_blank">Sarah Cole Design</a>.</div>
<a name="bottom"> </a></body>
</html>