Food for Thought: An e-newsletter published by Software Quality Consulting, Inc.
March 2006, Vol. 3 No. 3
Design for Testability 

To view a web version of this newsletter, click on the following link:
http://www.swqual.com/newsletter/vol3/no3/vol3no3.html.

--------------------------------------------------------------------------------

Welcome to Food for Thought(TM), an e-newsletter from Software Quality 
Consulting (http://www.swqual.com/index.html?Intro). I've created free
subscriptions for my valued business contacts. If you find this newsletter
informative, I encourage you to continue reading. Feel free to pass this
newsletter along to colleagues by clicking this Forward Email link
(http://ui.constantcontact.com/roving/sa/fp.jsp?plat=i&p=f&m=sctz69n6). If
you�ve received this newsletter from a colleague and would like to subscribe,
please click this Enter New Subscription link
(http://www.swqual.com/newsletter/Subscribe.htm?Newsletter). If you don't wish
to receive this newsletter, click the SafeUnSubscribe(TM) link at the bottom of
this newsletter, and you won�t be bothered again.

Your continued feedback on this newsletter is most welcome. Please send 
your comments and suggestions to [email protected].

--------------------------------------------------------------------------------

***In This Month�s Issue***

In This Months� Topic, I discuss applying design for testability 
principles to software testing... 

Regular features to look for each month are:

- Monthly Morsels
  Hints, tips, techniques and reference info related to this month�s topic 

- Calendar
  Conferences, workshops, and meetings of interest to software engineers, 
  QA engineers and anyone interested in software development 

--------------------------------------------------------------------------------

***This Month�s Topic***

DESIGN FOR TESTABILITY

Many software quality problems are the result of poorly written 
requirements. We know that 40-60 percent of all software defects are 
caused by errors in requirements. [1] What this means is that on a typical 
product development cycle that has say, 500 defects - as many as 300 are 
caused by poorly written requirements.

While most companies recognize the importance of having requirements, many 
haven�t recognized the importance of having good requirements - 
requirements that are well-written, unambiguous, and testable. Since so 
many defects are caused by poorly written requirements, companies should 
be motivated to improve the quality of their requirements...

WHAT CAN BE DONE TO IMPROVE REQUIREMENTS?

There are a couple of ways to do this:

- First, provide training (http://www.swqual.com/training/requirements.html)
  for those people who typically write   requirements. We can�t expect people to
  do a good job of writing   requirements if they haven�t been trained in what
  that means.

  Read more about writing good requirements...
  (http://www.jiludwig.com/Testability.html)

  Check the Monthly Morsels section for references on requirements... 

- Second, conduct effective peer reviews of requirements to increase the 
  likelihood of finding requirements problems early. Some training may be 
  required here as well since most people have not been taught how to 
  perform effective peer reviews
  (http://www.swqual.com/training/peer_reviews.html) on requirements documents.
  For example, when reviewing requirements, one question that should be asked is
  �Is every requirement testable?�

  Read more about peer reviews...
  (http://www.processimpact.com/pr_goodies.shtml)

  Check the Monthly Morsels section for additional references on peer 
  reviews... 
  
- And third, introduce Design for Testability concepts into the 
  organization... 

WHAT IS DESIGN FOR TESTABILITY?

Design for Testability (DFT) is a concept developed by electrical 
engineers to improve the designs of circuit boards so that these boards 
could be tested using conventional automated testing equipment (ATE). By 
bringing certain test points out to the edge, the test engineer writing 
the ATE program is able to test more of the functions the board is 
intended to perform, thus lowering the failure rate in the field and 
improving overall quality.

Wikipedia defines Design for Testability as:

  Tests are applied at several steps in the hardware manufacturing flow 
  and, for certain products, may also be used for hardware maintenance in 
  the customer�s environment. The tests generally are driven by test 
  programs that execute in Automatic Test Equipment (ATE) or, in the case 
  of system maintenance, inside the assembled system itself. In addition 
  to finding and indicating the presence of defects (i.e., the test 
  fails), tests may be able to log diagnostic information about the nature 
  of the encountered test fails. The diagnostic information can be used to 
  locate the source of the failure.

DFT principles long used by hardware engineers can be applied to both 
software design and software testing... 

WHAT IS SOFTWARE TESTABILITY?

James Bach [4] and David Gelperin [5] define testability as anything that 
makes software easier to test by making it easier to design and execute 
tests.

Bach [4] describes software testability in terms of: 

- Control. The better we can control it, the more the testing can be 
  automated and optimized. 

- Visibility. What we see is what we test. 

- Operability. The better it works, the more efficiently it can be 
  tested. 

- Simplicity. The less there is to test, the more quickly we can test it. 

- Understandability. The more information we have, the smarter we test. 

- Suitability. The more we know about the intended use of the software, 
  the better we can organize our testing to find important bugs. 

- Stability. The fewer the changes, the fewer the disruptions to testing. 

Testability, as described by Bret Pettichord, has more of a design 
component to it:

  �Testability is a design issue and needs to be addressed with the design 
  of the rest of the system. Projects that defer testing to the later 
  stages of a project will find their programmers unwilling to consider 
  testability changes by the time testers actually roll onto the project 
  and start making suggestions. Testability takes cooperation, 
  appreciation and a team commitment to reliability.� [2]

Improving software testability is clearly an important objective in order 
to reduce the number of defects that result both from poorly written 
requirements as well as poorly designed software. Let�s look at how we can 
apply the principles of DFT to software testing...

APPLYING DFT PRINCIPLES TO SOFTWARE TESTING 

Design for Testability principles can be adapted for software testing in 
at least two ways. First, we can use DFT principles to help clarify 
requirements that may be poorly written so that they can be more easily 
tested. Second, we can use DFT principles to improve the testability when 
using automated testing tools. 
Let�s look at what this means...

DFT FOR MANUAL TESTING

Poorly written requirements create problems for developers and testers. 
Not only can such requirements be difficult to understand, they can be 
interpreted in several ways. Rewriting requirements to remove ambiguity 
often isn�t done for a variety of reasons � there�s no time, the author 
doesn�t see the value, etc. So the tester is left to guess - How did the 
developer interpret this? How should I interpret this? Getting 
clarification from the author of the requirements document often is an 
exercise in frustration. Asking the developer how they interpreted the 
requirement can often raise more questions than it resolves. There�s a 
better way...

Testers can rewrite ambiguous requirements � using alternative techniques 
- so that the requirements become clear and more importantly, testable. 
After rewriting an ambiguous requirement, the tester can tactfully present 
the rewritten requirement to both the author - like this...

  �I had difficulty understanding this section of the requirements the way 
  it was written so I rewrote it like this. Is this what you meant?�

Once the author agrees to the rewritten requirement, the tester can share 
this with the developer.

Some techniques for rewriting requirements to improve testability include 
using:

- Work Flow Diagrams 
- Flowcharts 
- Structured English 
- Truth Tables 

FLOWCHARTS

Work flow diagrams and flowcharts are excellent tools for expressing 
requirements in a way that leads to better understanding.

Expressing portions of the requirements using a flowchart enables testers 
to see all of the paths that need to be tested. This technique also makes 
it easy to see if all paths were included in the English text. Frequently, 
rewriting requirements using a flowchart helps identify missing 
information. Here�s a simple example (see flow chart in the HTML version - http://www.swqual.com/newsletter/vol3/no3/vol3no3.html).
 
As a tester, having the requirements expressed in a flowchart makes it 
much easier to visualize tests that are required. Using a flowchart in 
this example also makes it easier to ensure that all paths are defined, 
including error handling conditions � something that is easy to overlook 
in English.

Testers can also use colored highlighters to mark the paths covered by 
each test, ensuring that all paths are in fact covered.

For an example of rewriting requirements using Structured English, see the 
September 2004 newsletter...
(http://www.swqual.com/newsletter/vol1/no1/vol1no1.html)

TRUTH TABLES

Here is an interesting example of comparing the testability of 
requirements written in English and the same requirements expressed in a 
truth table. Truth tables are useful when dealing with several discrete 
variables that are related.

Which is easier to understand, implement, and test?

Requirements expressed in English
(OP = Old Password NP = New Password)

User enters NP. Application determines if password meets these 
rules. 

- If OP is correct, NP and Confirm NP are same, NP passes 
  configuration edit, and has not been used during prior two 
  password changes, confirm change with a successfully changed 
  password message. Display Message = Password successfully changed. 

- If OP is correct and NP and Confirm NP match but do not conform to 
  configuration settings, a message describing error is displayed. 
  OP, NP and Confirm NP are blank after user selects �OK� on error 
  message. �OK� button brings user to Change Password screen. Error 
  Message = Password you entered does not conform to format 
  specified by your system administrator. Enter a valid password. 

- If OP is valid and NP and Confirm NP do not match, a message 
  describing error is displayed. OP, NP and Confirm NP are blank 
  after user selects �OK� on error message. �OK� button brings user 
  to Change Password screen. Error Message = NP and Confirm NP 
  entries do not match. Try again. 

- If OP is correct and NP and Confirm NP match and pass 
  configuration but has been used prior by user during previous two 
  changes a message is displayed. OP, NP and Confirm NP are blank 
  after user selects �OK� on error message. �Ok� button brings user 
  to Change Password screen. Error Message � Password used too 
  recently. Try again.� 

- If NP and Confirm NP match and pass configuration but OP is 
  invalid, a message is displayed. OP, NP and Confirm NP are blank 
  after user selects �OK� on error message. �Ok� button brings user 
  to Change Password screen. Error Message � OP entered is invalid. 
  Try again. 

Now here�s the same requirements rewritten using a truth table.

(see flow chart in the HTML version �
http://www.swqual.com/newsletter/vol3/no3/vol3no3.html)

  Messages: 

  1.  �Password successfully changed� 
  2.  �The password entered does not conform to the format specified by your sys
       admin. Enter a valid password.� 
  3.  �New Password and Confirm New Password entries do not match. Try 
      again.� 
  4.  �The password you entered has been used recently. Try again.� 
  5.  �The Old Password you entered is invalid. Try again.� 
  
  Not only is the truth table easier to understand than the text, each row 
  of the truth table represents a set of conditions that need to be 
  tested. 

DFT FOR AUTOMATED TESTING 

The principle of rewriting requirements to remove ambiguity applies 
regardless of whether you�re doing manual or automated testing.

Applying DFT principles is a bit different for automated testing. Brian 
LeSuer [3] says that applications need to be designed for automated 
testing. Specifically, he recommends that application developers follow 
these rules to help improve the testability of the application: 

- Expose application data 
- Choose automation-friendly third-party controls 
- Uniquely name application pages and objects 
- Add �hidden� controls 
- Use standard objects 
- Modify custom controls 
- Externalize functions in an API 
- Use proper naming conventions 
- Use only standard widgets 
- Expose �hidden� controls 
- Use unique page names 

Bret Pettichord [2] says that testability is a critical aspect of 
successful test automation efforts:

- Automation requires testability. Whether testing via an API or a GUI, 
  some accommodations to testability are necessary for successful 
  automation. If you�re going to have to do it, why not plan for it up 
  front? 

- Successful automation requires a focus on testability. Again, regardless 
  of approach, the greater the emphasis on testability within the 
  development team, the more likely automated testing will successful. 

- Test automation always affects test veracity to some degree . Automating 
  tests always makes them somewhat different from actual customer usage. 
  Wise testers will understand these deviations and develop compensating 
  testing strategies, rather than act as though it can be eliminated. 

BOTTOM LINE...

Improving testability affects requirements and design. Rewriting ambiguous 
requirements using alternative techniques leads to requirements that are 
less ambiguous, more understandable and as a result, more testable. When 
the requirements are more testable, your customers will find fewer bugs.

For test automation to be successful, testers and developers need a common 
understanding of what is required in order to make most effective use of 
the test automation tools. 

�Till next time...

--------------------------------------------------------------------------------

***Monthly Morsels***

Every month in this space you�ll find additional information related to 
this month�s topic.

- References:

  [1] Leffingwell, Dean, �Calculating the Return on Investment from More 
  Effective Requirements Management�, American Programmer, April 1997, p. 
  13-16.

  [2] Pettichord, Bret, �Design for Testability�, presented at Pacific 
  Northwest Software Quality Conference, October 2002.

  [3] Lesuer, Brian, �Everything You Always Wanted to Know About Test 
  Automation...�, presented at Software Quality Group of New England, Nov 9, 
  2005.

  [4] Bach, James, �Heuristics of Software Testability�, 1999.

  [5] Gelperin, David, �Assessing Support for Test,� Software Quality 
  Engineering Report, January 1999. 

- Books:

  Requirements 

  - Goldsmith, Robin, Discovering the REAL Business Requirements for 
    Software Project Success, Artech House, 2004. 

  - Wiegers, Karl, Software Requirements, Microsoft Press, 1999. 

  - Leffingwell, D. and Widrig, D., Managing Software Requirements: A Use 
    Case Approach, 2 nd Ed, Addison-Wesley, 2003 
  
  Peer Reviews and Inspections 

  - Wiegers, Karl, Peer Reviews in Software, Addison-Wesley, 2002.

  - Rakitin, Steven, Software Verification and Validation for 
    Practitioners and Managers, 2 nd ed., Artech House, 2001. 

--------------------------------------------------------------------------------

***Calendar***

Every month you�ll find news here about local and national events that 
are of interest to the software community ...

- A panel discussion on Offshore Outsourcing sponsored by the Boston SPIN 
  and the Software Quality Group of New England (SQGNE) will be held on 
  Tuesday May 16th. Find out more... 
  (http://www.swqual.com/links/upcoming.html)

- Software Quality Calendar

  There are many organizations that sponsor monthly meetings, workshops, 
  and conferences of interest to software professionals. Find out what�s 
  happening... 
  (http://www.swqual.com/links/upcoming.html)

- Workshops Offered by Software Quality Consulting

  Software Quality Consulting offers workshops in many topics related to 
  software process improvement. Get more info... 
  (http://www.swqual.com/seminars/courses.html)

--------------------------------------------------------------------------------

***About SQC***

Software Quality Consulting provides consulting, training, and auditing 
services tailored to meet the specific needs of clients. We help clients 
fine-tune their software development processes and improve the quality of 
their software products. The overall goal is to help clients achieve 
Predictable Software Development(TM) � so that organizations can consistently 
deliver quality software with promised features in the promised timeframe. 

To learn more about how we can help your organization, visit our web site
(http://www.swqual.com/index.html?AboutSQC) or send us an email
[email protected]).

--------------------------------------------------------------------------------

I hope this newsletter has been informative and helpful. Your comments and 
feedback are most welcome. Send me your feedback... ([email protected])

Thanks,

Steve Rakitin
[email protected]

Food for Thought, Predictable Software Development, Act Like a Customer,
and ALAC are trademarks of Software Quality Consulting, Inc.
Copyright 2006. Software Quality Consulting, Inc. All rights reserved.
Graphic design by Sage Studio