Food for Thought: Best Practices - What's Best for Your Organization?
An e-newsletter published by Software Quality Consulting, Inc.
November 2004, Vol. 1 No. 3 

To view a web version of this newsletter, click on the following link:
http://www.swqual.com/newsletter/vol1/no3/vol1no3.html.

-------------------------------------------------------------------------

Welcome to Food for Thought(TM), an e-newsletter from Software Quality 
Consulting (http://www.swqual.com/). I've created free subscriptions for
my valued business contacts. If you find this newsletter informative,
I encourage you to continue reading. Feel free to pass this newsletter
along to colleagues by clicking the Forward Email link at the bottom of
this page. If you've received this newsletter from a colleague and would
like to subscribe, please click this New Subscription link
(http://www.swqual.com/newsletter/Subscribe.htm). If you don't
wish to receive this newsletter, click the SafeUnSubscribe(TM) at the bottom
of this page, and you won't be bothered again.
 
Feedback from last month's edition was very positive and encouraging. Your 
feedback is most welcome. Please send your comments and suggestions to 
[email protected].

---------------------------------------------------------------------------

***In This Issue***

This month we discuss the issue of Best Practices and how can you 
determine if something purported to be a "best practice" is really going 
to make a difference for your organization.

Regular features to look for each month are: 

- Monthly Morsel: Hints, tips, techniques and reference info related to 
  this month's topic 

- Calendar: Conferences, workshops, and meetings of interest to software 
  engineers, QA engineers and anyone interested in software development. 

-------------------------------------------------------------------------

***This Month's Topic***

Best Practices 

What's best for your organization?

There's been lots of talk about "best practices" over the years. What's 
usually been lacking in all of this talk is: 

- A definition for what constitutes something that is called a "best 
  practice"

- A process by which a candidate practice is considered for the 
  designation "best" 

- Evidence that something called a "best practice" really works

What we need is the equivalent of the Good Housekeeping Seal for software 
development practices. 

Further, while there is a generally high level of awareness regarding best 
practices, the adoption of so-called "best practices" tends to be very 
low. 

The term "best practice" has clearly been overused and in many cases, 
misused:

  "While there might be a few universally good practices (such as peer 
  reviews), and there are certainly required practices (such as 
  configuration and requirements management), most practices have hidden 
  assumptions and conditions for use, and there is little empirical data 
  to support evaluation and selection." 

  Turner, R., "Seven Pitfalls To Avoid in the Hunt for Best Practices", 
  IEEE Software, Jan-Feb 2003, p. 67-69. 

Let's try to correct this situation by starting with some definitions of 
"best practice": 

  The American Society for Quality defines "best practice" as a superior 
  method or innovative practice that contributes to the improved 
  performance of an organization, usually recognized as "best" by other 
  peer organizations. 

Capers Jones definition is 

  "...to be deemed a 'best practice', it would be reasonable to require that 
  the practice must have been observed in a minimum of 10 companies and 50 
  software projects. Furthermore, the practice must be considered by those 
  who used it to have been beneficial in all or most of the projects." 

  Jones, C., Software Assessments, Benchmarks, and Best Practices, 
  Addison-Wesley, 2000 

A presentation at the 3rd OSD Conference on the Acquisition of Software 
Intensive Systems in January2004 offered this definition: An activity or 
procedure that has produced outstanding results in another situation and 
could be adapted to improve effectiveness, efficiency, ecology, and/or 
innovativeness in another situation. 

The Interoperability Clearinghouse Glossary of Terms defines best practice 
as an activity or procedure that has produced outstanding results in 
another situation and could be adapted to improve effectiveness, 
efficiency, ecology, and/or innovativeness in another situation. 

And there are dozens more definitions floating around. If we can't seem to 
agree on a definition for best practices, how we say there are such 
things? 

The truth of the matter is that the first part of the first definition 
given above is probably the most reasonable definition: 
a superior method or innovative practice that contributes to the improved 
performance of an organization 

It's the second part of that definition (usually recognized as "best" by 
other peer organizations) that causes problems. For example: 

- Who are these peer organizations? 

- How do they recognize a practice as being "best"? 

- Are there some hurdles that a candidate practice must meet before it 
  earns the designation "best"? 

- What evidence is there that something really is "best"? 

More on these questions in a bit. First, let's look at several groups that 
have identified so-called "best practices". 

Airlie Council 

One of the most widely quoted sources of "best practices" is the Airlie 
Council. 

The Airlie Council 
1. Norm Brown 
2. Vic Basili 
3. Frank McGrath 
4. Ed Yourdon 
5. Lou Mazzucche
6. Capers Jones
7. Tom DeMarco
8. Larry Putnam 
9. John Manzo
10. Mike Evans
11. Tim Lister
l2. Tom McCabe (not pictured)
13. Grady Booch (not pictured)
14. Roger Pressman (not pictured) 

The Airlie Council was convened in the late 1990's by an order from 
Congress to identify software best practices that could significantly 
improve software acquisition activities on behalf of the Department of 
Defense. Through their efforts, they identified what has become known as 
the Nine Principal Best Practices. These include: 

- Formal Risk Management 
- Agreement on Interfaces 
- Formal Inspections 
- Metrics Based Scheduling and Management 
- Binary Quality Gates at the Inch-Pebble Level 
- Project-Wide Visibility of Progress to Plan 
- Defect Tracking Against Quality Targets 
- Configuration Management 
- People-Aware Management Accountability 
- Nine Best Practices - The Software Management Framework, prepared by the 
- Niwot Ridge Consulting Group, June 1999

Software Productivity Research (SPR) 

Capers Jones' company was very active in collecting data on practices used 
by many companies to develop software. Through his surveys, SPR was able 
to identify those companies considered "best in class". From these 
companies, he identified software development practices that these "best 
in class" companies had in common. The idea here is that these practices 
in some way were responsible for a company being considered "best in 
class". These practices include: 

- Quality Measurements 
- Defect Prevention 
- Defect and quality estimation automation 
- Defect tracking automation 
- Complexity analysis tools 
- Test coverage analysis tools 
- Formal inspections 
- Formal testing by testing specialists 
- Formal quality assurance groups 
- Executive and managerial understanding of quality 
  
  Jones, C., Software Quality: Analysis and Guidelines for Success, 
  Boston, MA: International Thomson Computer Press, 1997. 

Software Program Manager's Network (SPMN) 

The SPMN was established in 1992 by the Assistant Secretary of the Navy to 
identify proven industry and government software best practices and convey 
these practices to managers of large-scale DoD system acquisition 
programs.  The SPMN 16 Critical Software Practices(TM) specifically address 
underlying cost and schedule drivers that have caused many software 
intensive systems to be delivered over budget, behind schedule and with 
significant performance shortfalls.  The SPMN 16 Critical Software 
Practices(TM) are organized in three groups:
  
  PROJECT INTEGRITY 
  1. Adopt Continuous Program Risk Management 
  2. Estimate Cost and Schedule Empirically
  3. Use Metrics to Manage
  4. Track Earned Value
  5. Track Defects against Quality Targets
  6. Treat People as the most important resource
 
  CONSTRUCTION INTEGRITY 
  7. Adopt Life Cycle Configuration Management
  8. Manage and Trace Requirements
  9. Use System-based software design 
  10. Ensure data and database interoperability
  11. Define and control interfaces
  12. Design twice, code once
  13. Assess reuse risks and costs
 
  PRODUCT STABILITY AND INTEGRITY 
  14. Inspect requirements and design 
  15. Manage testing as a continuous process
  16. Compile and smoke test frequently 

Software Engineering Institute (SEI) Capability Maturity ModelSM (CMM) 
Many organizations believe that the CMMSM is useful as a resource for 
identifying and implementing so-called "best practices". However, a quick 
visit to the SEI website (http://www.sei.cmu.edu/cmmi/adoption/sunset-faq.html#SUN11)
reveals the following statement from the SEI:
 
  "The Software CMM is twelve years old and no longer represents current 
  best practices in software engineering."
 
How can you determine if these so-called "best practices" will work for 
your organization?
 
Now that we have presented some definitions of what might constitute a 
"best practice", we need to revisit the questions identified earlier:
 
  Who are the peer organizations who identify and recognize "best 
  practices"?

  Currently, there are many organizations (as illustrated by the several 
  sources above) who have claimed to identify so-called "best practices". 
  Currently, there is no one single organization dedicated solely to 
  identifying and recognizing potential "best practices" for software 
  development... 

  How do these organizations recognize a practice as being "best"?

  Each of the organizations identified above use very different criteria 
  for recognizing a practice as being "best"... 

  Are there some hurdles that a candidate practice must meet before it 
  earns the designation "best"?

  Again, since each organization has different criteria, the hurdles 
  associated with achieving the designation "best practice" are also very 
  different... 

  What evidence is there that something really is "best"?

  Evidence (quantitative or qualitative) is sorely lacking. Capers Jones 
  has been the primary source of what little data has been published (Get 
  more info...). The IEEE Computer Society has published quantitative data 
  on the effectiveness of Software Inspections.
 
Sorting it All Out
 
The bottom line is that you need to apply common sense in determining if a 
so-called "best practice" might work well for your organization. Here are 
some tips that should help: 

  Look at those practices that have been identified by more than one 
  organization as a so-called "best practice". This may help increase 
  confidence that a practice will result in a positive improvement.

  For example, if we look at these sources of potential "best practices" 
  identified above, we can see that there is some overlap:
 
  See Table in HTML file: http://www.swqual.com/newsletter/vol1/no3/vol1no3.html 

  Note: The numbers in the Airlie Council, Best in Class Companies, and 
  SPMN columns refer to the numbers in the respective lists above. The 
  numbers in the SEI CMM column refer to the CMM level where the practice 
  is required. 

  Establish your organization's current performance baseline. This is a 
  critical and often overlooked step in helping to identify where software 
  development improvement is needed.

  Identify a small number of key performance measures that you can use to 
  baseline your organization's performance. For example, one simple 
  measure that can be indicative of an organization's performance is the 
  number of unplanned bug fix releases between major releases of software.

  Perform Root Cause Analysis on those performance measures in order 
  identify areas where software development improvement is required. Then, 
  identify practices that are likely to help improve performance in those 
  areas.

  For example, for many organizations, decreasing time to market is a 
  common desire. First, precisely define what is meant by this measure. 
  What does it include, what does it not include. How is time to market 
  measured? And, what is the current performance level?

  Once you've defined the measure, let's say your objective is to decrease 
  time to market by 10% over the current level. Now, you are in a position 
  to review potential practices and select those that can have a positive 
  impact on decreasing time to market. 

  Determine if a change to your software development process had the 
  desired affect.

  After instituting a change to your development process, say 
  incorporating peer reviews, assess your baseline again and determine if 
  the change achieved the desired improvement. 

The Bottom Line
 
You can't expect the performance of your organization to improve if the 
continue to follow the same process.

- "Best practices" are only "best" if they work for you. 

- Involve your staff in identifying opportunities for improvement. 

In the coming months, we'll discuss some of the so-called "best practices" 
identified in this article in more depth...

----------------------------------------------------------------------------

***Monthly Morsels***

Every month in this space you'll find additional information related to 
this month's topic.
 
Best Practice Information 

- Get more info on the Nine Best Practices from the Airlie Council
  (http://www.niwotridge.com/PDFs/NineBestPractices.pdf)

- Get more info on the 16 Critical Software Practices(TM) from the SPMN 
  (http://www.spmn.com/16CSP.html)

- Wheeler, D., et. al, editors, Software Inspection: An Industry Best 
  Practice, IEEE Computer Society Press, 1996, Order Number BP07340

Books and Articles 

-   Jones, C., Software Quality: Analysis and Guidelines for Success, 
    Boston, MA: International Thomson Computer Press, 1997. 

-   Jones, C., Software Assessments, Benchmarks, and Best Practices, 
    Addison-Wesley, 2000 

-   Nine Best Practices - The Software Management Framework, prepared by 
    the Niwot Ridge Consulting Group, June 1999 

-   Harrison, W., "Best Practices: Who Says So?", IEEE Software, Jan-Feb 
    2004, p. 8-9. 

-   Turner, R., "Seven Pitfalls To Avoid in the Hunt for Best Practices", 
    IEEE Software, Jan-Feb 2003, p. 67-69. 

--------------------------------------------------------------------------

***Calendar***

Every month, you'll find news here about local and national events that 
are of interest to the software community ... 

Software Quality Calendar

- There are many organizations that sponsor monthly meetings, workshops, 
  and conferences of interest to software professionals. Find out what's 
  happening... 
  (http://www.swqual.com/links/upcoming.html)

Workshops Offered by Software Quality Consulting 

- Software Quality Consulting offers workshops in many topics related to 
  software process improvement. Get more info... 
  (http://www.swqual.com/seminars/courses.html)

--------------------------------------------------------------------------

***About SQC***

Software Quality Consulting provides consulting, training, and auditing 
services tailored to meet the specific needs of clients. We help clients 
fine-tune their software development processes and improve the quality of 
their software products. The overall goal is to help clients achieve 
Predictable Software Development(TM) - so that organizations can consistently 
deliver quality software with promised features in the promised timeframe.

To learn more about us and how we can help your organization, visit our 
web site (http://www.swqual.com/) or send us an email ([email protected]).

--------------------------------------------------------------------------- 

I hope this newsletter has been informative and helpful. Your comments and 
feedback are most welcome. Send me your feedback...
Thanks,

Steve Rakitin
[email protected]

Food for Thought and Predictable Software Development are trademarks of Software 
Quality Consulting, Inc.
 
Copyright (c) 2004. Software Quality Consulting, Inc. All rights reserved.

Graphic design by Sage Studio ([email protected])