|
Server : Apache/2.4.62 System : FreeBSD fbsdweb2.web.rcn.net 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64 User : www ( 80) PHP Version : 8.3.8 Disable Function : NONE Directory : /domains/srakitin/OLD/newsletter/vol5/no7/ |
Upload File : |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Food for Thought: Are More Software Catastrophes Inevitable?</title>
<link href="/newsletter/StyleSheet.css" rel="stylesheet" type="text/css">
</head>
<OpenTracking/>
<!-- Do NOT delete previous line if you want to get statistics on the number of opened emails -->
<body>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0">
<tr align="center" valign="top">
<td colspan="2"><img src="/newsletter/images/FoodForThoughtLogo.gif" alt="Food for Thought" width="600" height="105"></td>
</tr>
<tr class="Reference">
<td align="left" valign="top"><p>An e-newsletter published by<br>
Software Quality Consulting, Inc. </p>
</td>
<td align="right" valign="top"><p>November 2008, Vol. 5 No. 7<br>
[<a href="/newsletter/vol5/no7/vol5no7.txt" target="_blank">Text-only Version</a>]</p>
</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top">
<p>Welcome to <em><strong>Food for Thought™</strong></em>, an e-newsletter from <strong><a href="/index.html?Intro" target="_blank">Software Quality Consulting</a></strong>. I've created free subscriptions for my valued business contacts. If you find this newsletter informative, I encourage you to continue reading. Feel free to pass this newsletter along to colleagues by clicking this <strong><a href="http://ui.constantcontact.com/roving/sa/fp.jsp?plat=i&p=f&m=sctz69n6">Forward Email</a></strong> link. If you’ve received this newsletter from a colleague and would like to subscribe, please click this <strong><a href="/newsletter/Subscribe.htm?Newsletter" target="_blank">Enter New Subscription</a></strong> link. If you don't wish to receive this newsletter, click the <strong><a href="#bottom">SafeUnSubscribe</a></strong>™ link at the bottom of this newsletter, and you won’t be bothered again.</p>
<p>Your continued feedback on this newsletter is most welcome. Please send your comments and suggestions to <strong><a href="mailto:[email protected]">[email protected]</a></strong>.</p></td>
</tr>
</table>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/InThisIssue.gif" alt="In This Issue" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p>In <a href="#article"><strong>This Months’ Topic</strong></a>, I discuss the increasing dependence on software and the risks associated with this dependence... <br>
<br>
Regular features to look for each month are:</p>
<ul>
<li> <a href="#morsel"><strong>Monthly Morsels</strong></a><br>
Hints, tips, techniques and reference info related to this month’s topic</li>
</ul>
<ul>
<li> <a href="#calendar"><strong>Calendar</strong></a><br>
Conferences, workshops, and meetings of interest to software engineers, QA engineers and anyone interested in software development</li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<a name="article"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/ThisMonthsTopic.gif" alt="This Month's Topic" width="114" height="37"></td>
<td width="15"> </td>
<td width="471" align="left" valign="top" class="BodyText"><p align="center" class="Headline">Are More Software Catastrophes Inevitable?
<br>
<br>
<em>The Challenge of Creating Dependable Software</em></p>
<p> We have come to depend on software for many aspects of daily life. Every day we rely on millions of lines of code often without realizing it...</p>
<blockquote>
<p> Your programmed digital alarm clock rings and switches to soothing nature sounds that gently wake you from a night’s sleep. As you hit the shower, the programmable thermostat has already raised the temperature setting so that the kitchen will be nice and warm when you come down for breakfast. The programmable coffee maker starts brewing your favorite caffeinated beverage at just the right time. After breakfast, you head out to work. Your car’s on-board computer monitors everything, including your anti-lock brakes, engine, emissions, airbags, seat belts, tire pressure, audio system, and of course, where you are. On the way to work, you stop at an ATM kiosk for some cash. You get a call on your cell phone and respond by sending a text message. Back on the highway, you pay tolls using a transponder that bills your credit card. Your GPS navigation system receives updates on traffic conditions and recommends alternate routes so you arrive at your office in the shortest possible time…</p>
</blockquote>
<p> By the time you have arrived for work in the morning, you relied on millions of lines of code without ever giving it a second thought. Every day people all over the world depend on software to work correctly and safely - even though we know that <strong><a href="/newsletter/vol2/no11/vol2no11.html" target="_blank">all software is defective.</a></strong> The reality is that:</p>
<blockquote>
<p>“We rely on software and sometimes it fails us.” [1]</p>
</blockquote> <p> Consider some areas where software is currently being used:</p>
<ul>
<li> Nuclear Power Plants </li>
<li> Chemical/Petro-chemical Processing Plants </li>
<li> Railway Signaling Systems </li>
<li> Controlling National Electricity Grid </li>
<li> Medical Devices </li>
<li> Avionics and Air Traffic Management </li>
</ul>
<p>While many system failures are harmless, there have been dozens of cases where software has led to severe economic loss as well as loss of life. For examples of software failures, see [1], [2], and [3] </p>
<p>Software is routinely used in many highly complex, interconnected systems. With each new use, there are new risks. Risk assessments used to identify and mitigate safety risks are commonly required in industries such as medical devices, avionics, nuclear power, etc. However, a risk assessment does not guarantee that software will not harm people - at best, it merely lowers risk to an “acceptable level”. </p>
<p>If we are to continue to depend on software, some critical issues must be addressed:</p>
<ul>
<li> Can software-based systems be made <strong>dependable</strong> in a cost-effective manner? </li>
</ul>
<ul>
<li> Can we provide assurance that a quantifiable level of <strong>dependability</strong> has been achieved? </li>
</ul>
<p>As software-based systems continue to increase in complexity and risk, the onus is on system developers to ensure these systems are <strong>dependable</strong> and most importantly, <strong>safe</strong>. </p>
<p>As a society, we entrust trained and licensed professional engineers to design buildings and bridges that are not likely to collapse. Even with the best engineers, the best inspections, the best materials, and the best construction techniques, <strong><a href="http://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collapse" target="_blank">buildings</a></strong> and <strong><a href="http://www.dot.state.mn.us/i35wbridge/index.html" target="_blank">bridges</a></strong> do fail on occasion. If we compare how software-based systems are designed and developed to how buildings and bridges are designed and developed, we find that:</p>
<table width="471" border="1" cellpadding="5" cellspacing="0" bordercolor="#000000" class="BodyText">
<tr align="center" valign="middle">
<td width="92"><p> </p></td>
<td width="182"><p><strong> Buildings and Bridges </strong></p></td>
<td width="198"><p><strong> Software-based Systems </strong></p></td>
</tr>
<tr>
<td width="92" valign="top"><p><strong> Design </strong></p></td>
<td width="182" valign="top"><p> Can <strong>only</strong> be performed by licensed and registered <strong><a href="http://www.nspe.org/index.html" target="_blank">Professional Engineers</a></strong>. </p></td>
<td width="198" valign="top"><p> Can be performed by software and/or systems engineers with no requirements for demonstrating competency. </p></td>
</tr>
<tr>
<td width="92" valign="top"><p><strong> Construction </strong></p></td>
<td width="182" valign="top"><p> Performed by skilled craftsman required to undergo years of training and apprenticeship. </p></td>
<td width="198" valign="top"><p> Anyone with a CS degree can be a software engineer and design and build safety-critical systems. There is no apprenticeship period and often no mentoring. </p></td>
</tr>
<tr>
<td width="92" valign="top"><p><strong> Inspections </strong></p></td>
<td width="182" valign="top"><p> Required by local, state and national laws and building codes. </p></td>
<td width="198" valign="top"><p> More often than not, inspections are not done, especially when projects are behind schedule. </p></td>
</tr>
<tr>
<td width="92" valign="top"><p><strong> Liability </strong></p></td>
<td width="182" valign="top"><p> Construction firms are often liable if people are harmed as a result of <strong><a href="http://en.wikipedia.org/wiki/Big_Dig_ceiling_collapse" target="_blank">poor workmanship.</a></strong></p></td>
<td width="198" valign="top"><p> Software developers often have little or <strong><a href="http://cse.stanford.edu/class/cs201/projects-00-01/ucita/controversy.html" target="_blank">no liability for poor workmanship</a></strong> unless explicitly required by contract. </p></td>
</tr>
</table>
<p><strong>Some Terms and Concepts </strong></p>
<p>Before we continue this discussion, we need to define some terms and concepts: </p>
<blockquote>
<p><strong> Dependable</strong></p>
<p> A system is dependable when it can be depended on to produce the consequences for which it was designed with no adverse affects.</p>
<p><strong> Dependability</strong></p>
<p> Dependability as applied to a computer system is defined by the <strong><a href="http://www.dependability.org/" target="_blank">IFIP 10.4 Working Group on Dependable Computing and Fault Tolerance</a></strong> as:</p>
<p> " [..] the trustworthiness of a computing system which allows reliance to be justifiably placed on the service it delivers [..]" </p>
<p><strong> Risk</strong></p>
<p> Risk is often expressed as the combination of the likelihood of occurrence of harm and severity of that harm should it occur. As such, risk is something that needs to be actively managed.</p>
<p> <img width="386" height="258" src="/newsletter/vol5/no7/vol5no7_clip_image002_0001.jpg"> </p>
<p> This picture (from IEC Standard 60601-1-4) illustrates that risk is the combination of likelihood and severity. The location of line defining the <strong>Intolerable Region</strong> is often determined on a case-by-case basis.</p>
<p><strong> Learn more...</strong><strong><a href="/training/fda/risk1.html" target="_blank">Risk Management Training for Medical Device Manufacturers</a></strong></p>
</blockquote> <p> Now that we have some context, let’s look at this problem in a bit more detail.</p>
<p><strong> How big is this problem?</strong></p>
<p> We know that the best, most highly skilled software developers inject on average one defect for every 8 lines of code they write. [3] We also have anecdotal information suggesting that through peer reviews and testing, we typically find about 95% of the injected defects. The result is on average, released software has a defect density in the range of 5-6 defects per thousand lines of code (KLOC).<br>
<br>
</p>
</td>
</tr>
<tr>
<td align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><p style="font-family:Times New Roman;font-size:12.0pt;"><img width="110" height="68" src="/newsletter/vol5/no7/vol5no7_clip_image001.jpg"></p>
<p align="center" class="Reference">Most late model cars have about<br>
10 million lines of code (MLOC).<br>
2010 model year cars are expected to have upwards<br>
of 100 MLOC.</p></td>
<td> </td>
<td align="left" valign="top" class="BodyText"><p>So if we look at a typically software-based system that has one million lines of code (MLOC), here’s what we’d expect:</p>
<blockquote>
<p><strong> Defects injected</strong> using 1 defect /8 lines of code = ~120,000 defects </p>
<p><strong> Defect removed</strong> assuming 95% found = 114,000 defects </p>
<p><strong> Defects remaining</strong> (defects injected - defects removed) = <strong> 6,000 defects</strong></p>
</blockquote>
<p> To put this in perspective, your average late model car has about 10 MLOC. Given the above, there could be as many as <strong> 60,000</strong> defects that remain in the software that controls your car. One of those defects led to a <strong><a href="http://dso.com/indepth/showArticle.jhtml?articleID=175007283" target="_blank">significant recall</a></strong>.</p>
<p><strong> What are the issues?</strong></p>
<p> Recently, a prestigious group of researchers from the <strong><a href="http://sites.nationalacademies.org/nrc/index.htm" target="_blank">National Research Council</a> </strong>published a book on the issues related to developing software for dependable systems. Their sobering assessment of the situation:</p>
<blockquote>
<p> “Society is increasingly dependent on software. Software failures can cause or contribute to serious accidents that result in death, injury, significant environmental damage, or major financial loss. Such accidents have already occurred and without intervention, the increasingly pervasive use of software - especially in arenas such as transportation, heath care, and the broader infrastructure - may make them more frequent and more serious.” [2]</p>
</blockquote>
<p> The problem is exacerbated by a pervasive lack of evidence about both the incidence and severity of software failures. This lack of evidence has led the researchers to the following conclusions: [2]</p>
<ul>
<li> Better evidence is needed, so that approaches aimed at improving the dependability of software can be objectively assessed. </li>
</ul>
<ul>
<li> For now, the pursuit of dependability in software systems should focus on the construction and evaluation of evidence. </li>
</ul>
<p>What we must recognize is that software development best practices are <strong>necessary but not sufficient</strong> to ensure that software systems are dependable and safe. We need to adopt and incorporate <strong>software safety</strong> practices, such as risk assessment, hazard analysis, threat models, safe states, use of static analysis tools, etc., to improve software dependability.</p>
<p><strong> Certifiably Dependable Software</strong></p>
<p> The researchers found that software may not be declared “dependable” based solely on the software development process used to build it. There needs to be a notion of “certifiably dependable” that is - dependability claims must be supported by evidence.</p>
<p> A variety of certification schemes exist today. However, none are robust enough to be recommended as examples to follow. Current certification schemes are focused on specific industries - such as the scheme used by the US Federal Aviation Administration (FAA) to certify software-based systems used in avionics and air traffic management. Software security is another area where a certification scheme is used - the <strong><a href="http://www.niap-ccevs.org/" target="_blank">National Information Assurance Partnership </a></strong>(NIAP) licenses 3 rd party labs to certify security software products based on the <strong><a href="http://www.niap-ccevs.org/cc-scheme/" target="_blank">Common Criteria</a></strong> for software security as defined in<strong><a href="http://www.niap-ccevs.org/cc-scheme/cc_docs/" target="_blank"> ISO/IEC Standard 15408</a></strong>.</p>
<p><strong> What can be done?</strong></p>
<p> The researchers’ proposed approach for improving software dependability is based on:</p>
<ul>
<li><strong> Explicit Claims</strong></li>
</ul>
<blockquote>
<p>“No system can be ‘dependable’ in all aspects and under all conditions. So to be useful, a claim of dependability must be explicit. It must articulate precisely the properties the system is expected to exhibit and the assumptions about the system’s environment upon which the claim is contingent. The claim should also indicate explicitly the level of dependability claimed, preferably in quantitative terms.” [2]</p>
</blockquote>
<ul>
<li><strong> Evidence</strong></li>
</ul>
<blockquote>
<p>“For a system to be regarded as dependable, concrete evidence must be presented that substantiates the dependability claim. Because testing alone is insufficient to establish properties, the [dependability] case will typically combine evidence from testing with evidence from analysis.” [2]</p>
</blockquote>
<ul>
<li><strong> Expertise</strong></li>
</ul>
<blockquote>
<p>“Expertise - in software development, in the domain under consideration, and in the broader systems context, among other things - is necessary to achieve dependable systems.” [2]</p>
</blockquote>
<p> The researchers then identified the following <strong>recommendations</strong>: [2]</p>
<ul>
<li> Make the most of effective software development technologies and formal methods. </li>
</ul>
<ul>
<li> Follow proven principles of software development - take a systems perspective and exploit simplicity. </li>
</ul>
<ul>
<li> Make a dependability case for a given system and context: evidence, explicitness, and expertise. </li>
</ul>
<ul>
<li> Demand more transparency, so that customers and users can make informed judgments about dependability. </li>
</ul>
<ul>
<li> Make use of but do not rely solely on process and testing. </li>
</ul>
<ul>
<li> Base certification on inspection and analysis of the dependability claim and the evidence offered in its support.</li>
</ul>
</td>
</tr>
<tr>
<td align="left" valign="top" background="/newsletter/images/RedSpacer.gif"><img width="110" height="171" src="/newsletter/vol5/no7/vol5no7_clip_image002_0000.jpg"> </td>
<td> </td>
<td align="left" valign="top" class="BodyText"><p>Current software development practices are not sufficient to develop software that is both dependable and safe. We need to integrate new methods and new techniques into the software development process that are focused on reducing risk and improving dependability. The <strong>IEEE Software Engineering Standards Committee</strong> (SESC) has already begun taking steps in this direction.</p>
<p> The <strong><a href="http://74.125.45.104/search?q=cache:7eiWeod2UXIJ:standards.computer.org/sesc/s2esc_excom/minutes/2008-07/Schultz-Moore-Zaman-mtg159.ppt+IEEE+Standard+12207-2008&hl=en&ct=clnk&cd=7&gl=us" target="_blank">IEEE Software Engineering Standards</a></strong> are being revised to incorporate a systems perspective into software development. In addition, <strong>IEEE Standard 12207:2008 Systems and Software Engineering - Software Life Cycle Processes</strong> [4] now includes requirements for risk and risk assessment. The forthcoming revision of <strong>IEEE Standard 1012 Software Verification and Validation</strong> will likely also addresses the issue of risk by requiring that software components be assessed based on risk.</p>
<p><strong> Summary</strong></p>
<p> One of the key findings of the <strong>National Research Council’s</strong> book on software dependability is:</p>
<blockquote>
<p> “Avoidable software failures have already been responsible for loss of life and for large economic losses. The quality of software produced by the industry is extremely variable, and there is inadequate oversight in some critical areas. Unless improvements are made, more pervasive deployment of software in the civic infrastructure may lead to catastrophic failures. Software has the potential to bring benefits to society, but it will not be possible to realize these benefits - especially in critical applications - unless software becomes more dependable.” [2]</p>
</blockquote>
‘Til next time...</td>
</tr>
</table>
<br>
<br>
<a name="morsel"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/MonthlyMorsels.gif" alt="Monthly Morsels" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p> Every month in this space, you’ll find additional information related to this month’s topic.</p>
<ul>
<li><strong> References <br>
<br>
</strong>
<ol>
<li> Wiener, L., <em>Digital Woes - Why We Should Not Depend on Software</em>, Addison-Wesley, 1993. <br>
<br>
</li>
<li> Jackson, D., et. al., <em>Software for Dependable Systems - Sufficient Evidence?</em>, National Research Council, National Academies Press, 2007.<br>
<br>
</li>
<li>Leveson, N., <em>Safeware – System Safety and Computers</em>, Addison-Wesley, 1995.<br>
<br>
</li>
<li><em>ISO/IEC/IEEE Std 12207:2008 System and Software Engineering - Software Life Cycle Processes</em>, IEEE, Piscataway, NJ.<br>
<br>
</li>
</ol>
</li>
<li><strong>Additional Resources</strong>
<br>
<br>
<ol>
<li><strong><a href="http://www.dependability.org/" target="_blank">IFIP Working Group 10.4 on Dependable Computing and Fault Tolerance</a></strong>.<strong><br>
<br>
</strong></li>
<li><strong><a href="https://store.thedacs.com/" target="_blank">Software Safety System Handbook</a></strong> – A Technical and Managerial Team Approach, DoD Joint Services Computer Resources Management Group, 1999 included in DACS Software Reliability Sourcebook.<strong></strong><strong> </strong> <br>
<br>
</li>
<li> Humphrey, W., “<strong><a href="http://www.sei.cmu.edu/news-at-sei/columns/watts_new/2004/3/watts-new-2004-3.pdf" target="_blank">The Quality Attitude</a></strong>”,<em> news@sei newsletter</em>, Number 3, 2004. <br>
<br>
</li>
<li>Humphrey, W., “<strong><a href="http://www.sei.cmu.edu/news-at-sei/columns/watts_new/2004/1/watts-new-2004-1.htm" target="_blank">Defective Software Works</a></strong>”, <em>news@sei newsletter</em>, Number 1, 2004.</li>
</ol>
</li>
</ul>
</td>
</tr>
</table>
<br>
<br>
<a name="calendar"></a>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/Calendar.gif" alt="Calendar" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p> Every month you’ll find news here about local and national events that are of interest to the software community…</p>
<ul>
<li><strong> Software Quality Calendar</strong></li>
</ul>
<blockquote>
<p>There are many organizations that sponsor monthly meetings, workshops, and conferences of interest to software professionals. <strong><a href="/links/upcoming.html" target="_blank">Find out what’s happening…</a></strong></p>
</blockquote>
<ul>
<li><strong> Workshops Offered by Software Quality Consulting</strong></li>
</ul>
<blockquote>
<p>Software Quality Consulting offers workshops in many topics related to software process improvement. <strong><a href="/seminars/courses.html" target="_blank">Get more info…</a></strong></p>
</blockquote></td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td width="114" align="right" valign="top" background="/newsletter/images/RedSpacer.gif"><img src="/newsletter/images/AboutSQC.gif" alt="About SQC" width="114" height="37"></td>
<td width="15"> </td>
<td align="left" valign="top"><p> Software Quality Consulting provides consulting, training, and auditing services tailored to meet the specific needs of clients. We help clients fine-tune their software development processes and improve the quality of their software products. The overall goal is to help clients achieve Predictable Software Development™ – so that organizations can consistently deliver quality software with promised features in the promised timeframe. </p>
To learn more about how we can help your organization, <strong><a href="/index.html?AboutSQC" target="_blank">visit our web site</a></strong> or <strong><a href="mailto:[email protected]">send us an email</a></strong>.</td>
</tr>
</table>
<br>
<br>
<table width="600" border="0" align="center" cellpadding="0" cellspacing="0" class="BodyText">
<tr>
<td align="left" valign="top"><p> I hope this newsletter has been informative and helpful. Your comments and feedback are most welcome. <strong><a href="mailto:[email protected]">Send me your feedback…</a></strong></p>
<p>Thanks,</p>
<p> <img src="/newsletter/images/BusinessCard.gif" width="270" height="121" align="right"><img src="/newsletter/images/Signature.gif" width="90" height="68"><br>
Steve Rakitin<br>
<br>
<strong><a href="mailto:[email protected]">[email protected]</a></strong></p></td>
</tr>
</table>
<div align="center"><br>
<FONT class="Reference">Food for Thought, Predictable Software Development, Act Like a Customer,<br>
and ALAC are trademarks of Software Quality Consulting, Inc.<br>
Copyright 2008. Software Quality Consulting, Inc. All rights reserved.<br>
Graphic design by <B><U><FONT color=blue><a href="mailto:[email protected] ">Sage Studio</a></FONT></U></B></FONT></div>
<a name="bottom"> </a></body>
</html>