The 3 statistical tools of project troubleshooting – root cause analysis, Pareto charts and weighted decision tables – are best illustrated using a practical example.
A web portal for Travel Comforts, an online travel company, is being developed by SoftTech Services Ltd, a software services company. Dave was identified as the co-coordinator from Travel Comforts company and is the customer contact for SoftTech.
The project starts off well with Steve and Julia from the SoftTech project team eliciting the requirements from Dave, the customer contact, and a few other stakeholders that Dave put in touch with the elicitation team. Steve, the project manager of the SoftTech team, was happy with the way elicitation was carried out and got the requirements to document signed off by Dave.
The project was originally planned to be of 8-month duration and currently, the project timeline is in its 6th month. In the 6th month, the project is still in its design phase. The design phase has overshot its deadline by 2 months and usual project tracking procedures have not been able to pull back the schedule; the projection is that the final delivery will be delayed by 2 months. A worried Steve while sipping coffee in the cafeteria pours out his heart to his colleague and the conversation is as follows:
Steve: Looks like this Travel Comforts project is doomed.
Colleague: Why? What happened?
Steve: Design is not getting over at all, man! And all my efforts to close it so far have gone fruitless.
Colleague: Is the design so complex?
Steve: No, that’s not the problem. Even after we got a sign-off on the requirements document, new requirements keep coming in and existing requirements keep changing. And because of these fluctuating requirements, we are unable to freeze the design. That’s the problem and it’s frustrating.
Colleague: Why are the requirements changing?
Steve (with a surprised look): What do you mean? The customer is changing the requirements and that is why requirements are changing.
Colleague: I mean, are you not imposing the change control procedure?
Steve: Well, if there are one or two changes, then we can do a detailed impact analysis and follow the change control procedure thoroughly. But with the flood gates open and requirements changing day in and day out, I have run out of my bandwidth to follow the change control procedures for every single requirement.
Colleague: Have you informed your customer contact about the consequences? I mean the possible delays because of the changes?
Steve: Dave is the customer contact. He knows the consequences of these changes, but he says he is helpless. His stakeholders, these people who work with the Travel Comforts portal to earn business for the Travel Comforts company, are giving him new requirements and changing the requirements.
Colleague: Why are the stakeholders changing the requirements so often?
Steve: How am I supposed to know that?
Colleague: Steve, I’ll tell you what – your project is now a troubled project. Your usual effort to correct the situation won’t be of any use. You need firefighters. If you take my advice, escalate the matter to your delivery head and get some super project manager to come and troubleshoot your project and revive it.
Steve: Hmmm, I see your point. Let me give it a shot…
The coffee meeting concludes and Steve takes his colleague’s advice seriously. He escalates the matter to the delivery head and gets Harry, an expert project manager in SoftTech Services, to look into the project and put it back on track.
Harry duly comes forward, studies the project environment, and after suggesting a couple of minor corrective actions, gets into a RCA of requirement changes, identifies the root cause, suggests the most effective corrective actions and leaves the project.
After a couple of weeks, Steve bumps into his colleague in the cafeteria once again.
Colleague: You look quite relieved, Steve. What happened?
Steve: I took your inputs seriously and Harry, the super PM, did the troubleshooting for us and the project is now back on track.
Colleague: Can you explain in detail?
Steve: Sure. Harry asked us a series of questions grouped systematically as part of RCA. At some places, he probed further based on the answers. We hit the jackpot when he asked us whether all the stakeholders are giving new requirements and changing requirements or only a few. It was easy to dig out this information and we figured out that most changes were coming from two stakeholders namely Bob and Roger. We then probed further to see why they were doing so and found out that neither Dave (the customer contact) nor our elicitation team had elicited requirements from them in the first place.
Colleague: Oh my god! So what did you do then?
Steve: Simple! We simply put the project on hold, elicited requirements from Bob and Roger, and reworked the plan, Dave was cooperative all through and we got his sign-off on the new plan. We are back on track now with the new plan.
Colleague: So, the requirements are not changing now?
Steve: Well, there is that one odd change that comes now and then which is manageable. I have now tightened the Change Control Procedure and requirements are under control now. The flood has been arrested for sure!
Colleague: Why couldn’t your customer contact figure out the problem? He is the one who was receiving requirement changes and could have figured out why Bob and Roger were changing requirements.
Steve: Well, you see, certain things that you make out easily when you go through formal training, don’t occur automatically when you don’t have that training. Harry had gone through several formal pieces of training, including one on Requirements Management and this is what made the difference. Dave was operating on a slightly wrong paradigm. He thinks that being the software guy, he is mainly responsible for requirements and he put together a bunch of requirements based on earlier documentation and similar products. He arranged for our elicitation team to meet a few stakeholders, just to cover gaps if any. And it seems Bob and Roger were on vacation during this period and Dave just arranged meetings with whichever stakeholder was available at that time. Since Dave had accounted for requirements for the package module also, he thought that he had everything covered. But actually, Bob and Roger were the stakeholders for the package module. And when they started changing requirements, Dave simply thought that these two people were troublesome stakeholders and he was just living with the situation.
Colleague: And how did his impression change then?
Steve: Harry convinced Dave that the primary responsibility and ownership of requirements rested with the business stakeholders and he also provided references from various authorities on requirements engineering. Harry also convinced Dave that when Bob and Roger returned from the vacation, they should have been consulted proactively to elicit requirements from them, and leaving them out was a mistake. This is how Dave was convinced that it was his omission. He agreed to arrange for elicitation from Bob and Roger and also signed off on the revised plan.
Colleague: Wow! That’s a significant turnaround, Steve. Have you documented all this?
Steve: Well, the most important questions and the analysis of them have been documented in an RCA table, a Pareto chart, and a decision table; not the entire set of details in an exhaustive manner.
Colleague: Can you please forward them to me? I would like to sit with you and go through an exhaustive coverage also sometimes.
Steve: Sure, and let’s do that during a lean period.
Colleague: Well then, who pays the cafeteria bill now?
Steve: It’s on me! Not only because I am relaxed today, but also need to give you a treat for suggesting getting external help for revival.
Colleague: Thanks. Cheers.
Documentation of the Project Troubleshooting Strategy
Following are the artifacts that Steve forwards to his colleague.
The output of the RCA carried out is shown below and then detailed in a tabular format for easy readability:
Factors considered under each category are as follows:
Men: There are multiple, distinct roles involved in requirements and they are the stakeholders who provide the requirements, the customer contact who facilitates the stakeholders providing the requirements, the project manager, the analyst team who elicits requirements, and the developers. Each of these roles was probed to identify the root cause.
Materials: Physical raw materials make sense in a manufacturing environment and this category had to be adapted to a software environment. It was decided to replace this category with software tools used in the development life cycle.
Machine: Machine makes sense in a manufacturing environment and this was replaced by the platform and the software modules being developed.
Method: The life cycle model and the project management processes came under this category and were probed.
Management and environment: These were probed separately initially and hence were not included in the RCA.
Following were the probing questions and findings in this RCA:
| Sl No. | Category | Questions | Findings | Sub questions | Findings |
| 1. | Men | Have requirements been elicited from all stakeholders? | No, only from customer contact and a few others | ||
| Are the requirement changes coming from all the stakeholders or a few specific ones? | Majority of changes coming from Bob and Roger | ||||
| Is the project manager familiar with the change control procedures? | Yes | Is he able to enforce it? | No; infeasible because of a huge number of changes | ||
| Have analysts trained in requirements elicitation? | They have gone through a self-study | ||||
| Is the customer contact aware of the consequences of change? | Yes, but he can’t stop the stakeholders | ||||
| Is there a problem with the dev team understanding the requirements? | No, the analysts and dev team are completely in sync with the requirements | ||||
| 2. | S/W tools | Is any requirement management s/w being used? | No, requirements are documented in MS Word. | ||
| 3. | Methods | Have the analysts used the appropriate elicitation techniques? | Not necessarily. But the same techniques are used by everybody. | Is inadequate expertise in elicitation techniques the reason for requirements change? | Doesn’t seem to be; Requirements haven’t been elicited from Bob and Roger. |
| 4. | Software modules | Are the requirement changes coming in all modules or a few specific ones? | Most requirement changes are in the package deal module for which Bob and Roger are the stakeholders. |
After identifying the root causes, the frequency of changes due to each reason was determined and plotted using a Pareto chart as follows:
The next step was to identify the corrective actions, evaluate them and choose the most effective one for implementation. The potential corrective actions were brainstormed and their rankings were tabulated in the weighted decision table below. All scores are out of 10 with 1 being the lowest score and 10 the highest. As ease of implementation and effectiveness both were equally important factors, the weight assigned is 1 for both factors.
| Sl # | Corrective action | Ease of implementation | Weight | Effectiveness | Weight | Cum score |
| 1 | Keep the travel package module out of scope for this release | 1 | 1 | 9 | 1 | 9 |
| 2 | Tighten the Change Request procedure | 5 | 1 | 7 | 1 | 35 |
| 3 | Pause the project, elicit requirements from Bob and Roger, re-plan and restart the project | 5 | 1 | 10 | 1 | 50 |
| 4 | Pause the project, restart the entire requirements phase, re-plan and restart the project | 3 | 1 | 5 | 1 | 15 |
| 5 | Acquire a requirement management tool | 5 | 1 | 5 | 1 | 25 |
| 6 | Train the analysts in elicitation techniques | 9 | 1 | 3 | 1 | 27 |
Keeping the package module for the next release would have brought down the requirement changes drastically, and in that sense, would have been highly effective. This decision was near impossible to implement as from a business perspective, the package module had to be a part of this release.
Restarting the whole requirements phase did not make sense, as requirement changes were low in other modules, and hence it would be neither easy nor effective.
Options 5 and 6 were good long-term measures but would have very limited effectiveness in containing the requirement changes coming from Bob and Roger.
Therefore options 2 and 3 were decided to be implemented.
In summation, this example illustrates that the fishbone diagram helped in digging deeper in a systematic manner and identifying all possible causes. The Pareto chart then helped in narrowing down the most important causes that needed to be addressed. The weighted decision table then helped to choose the most appropriate corrective actions. The use of all 3 statistical tools enabled a troubled project to be brought back on track, and knowledge of these (explained in A Guide to the Project Management Body of Knowledge, 5th edition, Project Management Institute Inc.) will similarly help project managers to attain successful results.
