Food for Thought: Project Retrospectives
An e-newsletter published by Software Quality Consulting, Inc.
June 2005, Vol. 2 No. 6 

To view a web version of this newsletter, click on the following link:
http://www.swqual.com/newsletter/vol2/no6/vol2no6.html.

--------------------------------------------------------------------------------

Welcome to Food for Thought(TM), an e-newsletter from Software Quality 
Consulting (http://www.swqual.com/?Intro). I've created free subscriptions for
my valued business contacts. If you find this newsletter informative, I
encourage you to continue reading. Feel free to pass this newsletter along to
colleagues by clicking this Forward Email link. If you�ve received this
newsletter from a colleague and would like to subscribe, please click this Enter
New Subscription link (http://www.swqual.com/newsletter/Subscribe.htm). If you
don't wish to receive this newsletter, click the SafeUnSubscribe(TM) link at the
bottom of this newsletter, and you won�t be bothered again.

Your continued feedback on this newsletter is most welcome. Please send 
your comments and suggestions to [email protected]. 

--------------------------------------------------------------------------------

In This Months� Topic, I discuss using Project Retrospectives as an 
alternative to traditional Post-mortems... 

Regular features to look for each month are:

- Monthly Morsels
  Hints, tips, techniques and reference info related to this month�s topic 

- Calendar
  Conferences, workshops, and meetings of interest to software engineers, 
  QA engineers and anyone interested in software development 

--------------------------------------------------------------------------------

***This Month�s Topic***

Project Retrospectives - A kinder, gentler, more productive way to learn from past mistakes
 
Most every organization, at one time or another, has experienced the pain 
of a Project Post-Mortem. The story goes something like this: 

  From the beginning, the Project Team knew that not only was the schedule 
  unrealistic, but the requirements were ill-defined, the resources were 
  far less than what was needed, and commitments were being made to key 
  customers without consulting R&D. 

  Everyone worked as hard as they could on the project, often spending 
  60-80 hours per week. In the end, the product was delivered several 
  months later than planned, had fewer features than were promised, and 
  had too many bugs. 

  Customers were not happy with what they received since it was months 
  late, didn�t have all the promised features, and crashed or locked up 
  frequently. 

  Management asked for a Post-mortem to review what had happened. The 
  Project Manager scheduled a meeting about a month after the software was 
  released. People were given the usual short notice that this was 
  happening. 

  At the Post-mortem, the Project Manager tried to control the discussion 
  by asking for examples of things that worked well and things that didn�t 
  work well. After about a half-hour of tense discussion, the blaming and 
  finger pointing started. QA blamed Development for being 6 weeks late 
  and for delivering code that was very buggy. Development blamed project 
  management for not spending enough time clarifying requirements up front 
  and for allowing requirements to change right up to the last weeks of 
  the project. By the end of the meeting, everyone was angry and nothing 
  of value was recorded. 

  On the next project, the same mistakes were made again. 

This �story� unfortunately happens far too often. Often, when post-mortems 
are held, people tend to either say very little or not show up. As a 
result, nothing is learned and nothing changes. 

The term �post-mortem� has some negative connotations to it. People have 
come to expect that �post-mortems� are nothing more than forums for 
exacting retribution and venting frustration. In terms of return on 
investment (ROI), �post-mortems� usually rank low since little of value is 
learned. While Management may be satisfied that a post-mortem was held, 
the organization continues to make the same mistakes on the next project 
because nothing was learned and nothing was changed. 

Here then is the first thing to learn about project post-mortems: 

  If you always do what you�ve always done,
  you�ll always get what you�ve always gotten.

Thankfully, there is a better way. 

ENTER PROJECT RETROSPECTIVES... 

Norm Kerth  The better way is called Project Retrospectives and was 
developed and refined by Norman Kerth [1]. Norm has developed a much more 
effective way to glean �wisdom� from projects. 

Post-mortems are meetings that typically occur with no planning, no 
preparation, are led by a project team member who may or may not have good 
facilitation skills, and may take only a few hours at most. 

Project Retrospectives on the other hand, are �events�. Here are some key 
attributes:

-  They are planned 
-  People come prepared � click here to learn how to prepare...
   (http://www.swqual.com/newsletter/Retrospective.pdf)
-  An experienced facilitator plans the event and leads the discussion 
-  Retrospectives typically take 2 -3 days (yes, days) 

By making an investment in learning from the past, organizations can 
significantly reduce the likelihood that they will repeat the same 
mistakes on the next project. 

Planning a retrospective is critical to maximize the ROI. The facilitator 
meets with the project manager and selected project team members to get a 
sense of what happened, to gather relevant project data, to understand the 
organization (roles, responsibilities, and reporting relationships), get a 
sense of the culture, and to identify any hot button issues. The more 
familiar the facilitator is with these issues the better able he or she 
will be to handle them when they surface. 

SEARCHING FOR WISDOM 

Wisdom is defined as:

- accumulated knowledge or erudition or enlightenment 
- trait of utilizing knowledge and experience with common sense and 
  insight 
- ability to apply knowledge or experience or understanding with common 
  sense and insight 

When project teams work together, they learn stuff. Some stuff is not that 
important and some stuff is very important. The important stuff has the 
potential to become �wisdom�.

How do our experiences on projects become wisdom? Well, when we 
collectively can discuss our experiences in a safe and open environment, 
those discussions can often lead to �light bulb moments� � when people 
realize how their actions, their procedures, their behavior, etc., affects 
others and, as a result, affects the product.

SAFETY AND TRUST ARE ESSENTIAL

One of the key problems associated with your typical post-mortem is that 
many people may not feel that they can speak openly for fear of 
retribution or because they don�t know how to tactfully say what�s on 
their mind. Kerth addresses this key issue by introducing the notion that 
safety and trust are essential. He says:

  �Regardless of what we discover, we must understand and truly believe 
  that everyone did the best job he or she could, given what was known at 
  the time, his or her skills and abilities, the resources available, and 
  the situation at hand.� [1]

Kerth�s Project Retrospectives are based on this notion and everyone 
participating must buy into this. Respect for individuals is also a very 
important aspect of Project Retrospectives. Kerth [1] defines some ground 
rules that are set out at the beginning: 

- When someone is speaking, we will try to not interrupt them 
- We will accept everyone�s opinion without judgment 
- We will talk from our own perspective, and not speak for anyone else 
- There will be no jokes made at the expense of anyone in the room 
- When someone is holding the designated object, then only that person 
  may speak 
- While everyone is encouraged to contribute, your participation in 
  discussions and exercises is optional 

The designated object should be something silly that helps defuse any 
tension that may be present. As a facilitator for a recent Project 
Retrospective, I used a Yoda Pez Dispenser as the designated object. At 
the end of the Retrospective, everyone received their very own Pez 
Dispenser as a reminder of the event.

Once the ground rules are established, the team can now work on creating 
an environment of safety and trust. Kerth recommends doing this with some 
exercises.

THE �CREATE SAFETY� EXERCISE [1] 

The objective of this critical exercise is to help create an atmosphere 
where everyone feels comfortable talking openly about important issues. 
Without a safe environment, we can�t get to the real issues that need to 
be discussed.

Kerth [1] suggests measuring safety by using a secret ballot and a comfort 
scale...

Comfort Level... 

5 Hey, no problem, I feel comfortable saying anything. 
4 I�ll say most things but a few things might be hard to say. 
3 I�ll share some things but keep some things to myself. 
2 I�m not going to say much. I�ll let others bring up issues. 
1 I won�t let managers know what I really think. 

Based on the results of the vote, the group determines if:

- Everyone feels reasonably safe talking openly 
- Or, additional steps are needed to increase the level of safety... 

The secret ballot is held and the facilitator records the results. If they 
are mostly 4�s and 5�s, there�s no problem and the team can move on. If 
the results are mostly 2�s and 3�s, there is a problem and the facilitator 
needs to address it.

  To address the issue of safety, the group is asked to stand and form 
  natural affinity groups. This is done by having the team members move 
  near people with whom they have worked closely with on this project and 
  away from people with whom they worked very little. Team members may 
  only move themselves and may not talk in this exercise. Once the 
  affinity groups are formed, each group finds a corner of the room and 
  they brainstorm ideas for how to increase the level of safety. Each 
  group brings a list of items that are then discussed with the whole 
  group. Once the whole group reaches consensus on these ideas, a second 
  secret ballot is taken and hopefully, the level of safety is now 4�s and 
  5�s...

The bottom line is the Retrospective can�t begin until everyone feels 
safe talking openly and honestly about their experiences. Once everyone 
feels safe, we can move on to the next exercise...

THE �DEFINE SUCCESS� EXERCISE [1]

Kerth [1] says that purpose of this exercise is to �establish the idea 
that the project�s degree of success is not a relevant measure to use 
while people are trying to learn from their experiences.� Every project, 
even acknowledged failures, have aspects about which the project team 
should be proud of � in addition to things that can be improved upon.

This is a good exercise to use immediately after the Create Safety 
exercise because it helps get people talking and listening. An experienced 
facilitator can tell if people really do feel safe by the nature of this 
discussion.

We begin the Define Success Exercise by asking the project team members 
for their definition of success. This activity results in the first of 
several lists that are created during the Retrospective. Each response is 
listed.

We then discuss this definition:

  A successful project is one on which everyone says �I wish we could do 
  that over again � the very same way.�

Based on this definition, the project team is asked: Was this project a 
�success�?

The facilitator leads the discussion on this question � ensuring that 
everyone has the opportunity to talk � coaxing those that seem quiet � to 
help build the feeling of safety and trust.

We then look at some (albeit dated) industry data [2].

- 31% of projects are canceled before they ever get completed 
- 53% of projects cost 189% of their original estimates 
- On average, only 16% of software projects are completed on-time and 
  on-budget 
- For small software companies, 78% of their software projects are 
  deployed with at least 74% of their original features and functions 

We now compare what the project team accomplished against this backdrop. 
Again, the goal is to get people to start talking about the project in a 
way to help people see that there were successes as well as failures � all 
is not bad.

INDIANA JONES MEETS CSI

The real work in a Project Retrospective is a lot like a cross between 
Indiana Jones (archeology) and the TV show Crime Scene Investigators 
(forensic science). Each person is asked to bring to the Retrospective 
specific artifacts that they felt were important to the project from their 
perspective. Artifacts can be documents, e-mails, coffee mugs, photos, or 
anything that has specific meaning relative to the project. We make sure 
to bring project schedules (planned and actual) to discuss. We go around 
the room and each person discusses the artifact they brought, it�s 
significance to them and to the project.

Once this discussion is completed, the team is ready to start building the 
Project Timeline...

THE �DEVELOP TIME LINE� EXERCISE [1]

Part of the problem with post-mortems is that most people tend to remember 
only what occurred during the last part the project. The end of most 
projects is often the most contentious and stressful.

We need to uncover and discuss evidence of what actually occurred 
throughout the entire project � not just the end. The way to do this is to 
discuss the artifacts and then build the project timeline based on factual 
information.

The facilitator asks the affinity groups to move to their corner of the 
room. The work is inclusive not consensual which means everyone�s opinions 
are valid. Each group identifies significant events that occurred during 
the project as follows:

- Write each event on a post-it note along with approximate date... 
- Anyone who thinks an event was important creates a post-it note for it... 
- Use the artifacts present in the room to stimulate your memory... 

While the affinity groups are working, the facilitator hangs flip chart 
paper on a large wall in the meeting room and months are noted along the 
top.

Once the affinity groups are finished, they start placing their events on 
the timeline. As a result, much discussion ensues.

Eventually, all the post-it notes are placed and everyone is general 
agreement with the result. Now, stand back and see what actually happened 
from start to finish!

THE �MINING FOR GOLD� EXERCISE [1]

Once the timeline is created and its significance absorbed, it�s time for 
the team to perform the last and most important exercise. For this 
exercise, the facilitator creates five lists on five flip chart easels.

- What worked well that we don�t want to forget

  This list is for those things that most agree worked well and that we 
  want to continue doing on the next project... 

- What did we learn

  Here is where we document cause and effects that have lead to learning... 

- What should we do differently on the next project

  This list is for issues that need to be done better next time 

- What still puzzles us

  This list is for issues that the project team doesn�t have enough 
  information or skill to resolve. The issues on this list often require 
  training or more thorough investigation... 

- What do we need to discuss in more detail

  Issues on this list are things that need further discussion, 
  clarification, information... 

Usually, the discussion will take off on it�s own. If necessary, the 
facilitator can help jump start the process by having the team look at the 
timeline and asking questions such as:

- What event is most significant? 
- What event surprised you? 
- Are there any patterns that you can see? 
- What events don�t you understand? 

As the discussion progresses, the facilitator (or a volunteer) adds items 
to the five lists as agreed by the team. This is the most important part 
of the process! By spending time to build a safe and trusting environment, 
you�ll get the most valuable results. This just can�t happen using the old 
post-mortem approach.

BRINGING CLOSURE TO THE PROCESS

The Project Team determines when the Retrospective is done. When that 
happens, the facilitator can capture the information from the five lists 
along with other important information from the event. This information is 
reviewed with Management. Specific action items are identified along with 
a responsible person and a date for each action item.

This is the part of the process that leads to change. If Management 
resists change, I would suggest reminding them of my favorite mantra:

   If you always do what you�ve always done,
   you�ll always get what you�ve always gotten.

Till next time... 

On a sad note...
Norm Kerth is a highly respected consultant who suffered a disabling brain 
injury in an automobile accident. As a result, he cannot work and lives on 
a limited income. You can help him by sending a donation. Checks can be 
made payable to Norm Kerth Benefit Fund and sent to Norm Kerth Benefit 
Fund c/o Process Impact, 11491 SE 119th Drive, Clackamas, OR 97015-8778. 
You can also visit Karl Wieger�s website (Process Impact �
http://www.processimpact.com/goodies.shtml) for more details 
about contributing to the fund. Thanks.

--------------------------------------------------------------------------------

***Monthly Morsel***

Every month in this space you�ll find additional information related to 
this month�s topic.

- References:

  [1] Kerth, N. L., Project Retrospectives � A Handbook for Team Reviews, 
  Dorset House, 2001.

  [2] The Chaos Report, prepared by the Standish Group, 1994. 

- On-line Resources:

  - Project Retrospectives Website � Norm Kerth�s site about his process...
    (http://www.retrospectives.com/index.html)

  - A Powerpoint Presentation that covers the Retrospective Process
    (http://www.cs.unca.edu/%7Emanns/RetrospectivesPresentationatOOPSLA03.ppt)

--------------------------------------------------------------------------------

***Calendar***

Every month, you�ll find news here about local and national events that 
are of interest to the software community ...

- Software Quality Calendar

  There are many organizations that sponsor monthly meetings, workshops, 
  and conferences of interest to software professionals. Find out what�s 
  happening... (http://www.swqual.com/links/upcoming.html)

- Workshops Offered by Software Quality Consulting

  Software Quality Consulting offers workshops in many topics related to 
  software process improvement. Get more info...
(http://www.swqual.com/seminars/courses.html)

--------------------------------------------------------------------------------

***About SQC***

Software Quality Consulting provides consulting, training, and auditing 
services tailored to meet the specific needs of clients. We help clients 
fine-tune their software development processes and improve the quality of 
their software products. The overall goal is to help clients achieve 
Predictable Software Development(TM) � so that organizations can consistently 
deliver quality software with promised features in the promised timeframe.
To learn more about how we can help your organization, visit our web site
(http://www.swqual.com/) or send us an email ([email protected]).

--------------------------------------------------------------------------------

I hope this newsletter has been informative and helpful. Your comments and 
feedback are most welcome. Send me your feedback... ([email protected])

Thanks,

Steve Rakitin
[email protected]

Food for Thought and Predictable Software Development are trademarks of Software 
Quality Consulting, Inc. 
Copyright � 2005. Software Quality Consulting, Inc. All rights reserved. Graphic 
design by Sage Studio