The 95 percent syndrome in Software Development

Being a Project Manager, Architect, Analyst, Developer or anybody involved in the development of software, you have probably suffered the 95 percent syndrome.Imagine a software development project where things are going reasonably well: things are on schedule, Quality Assurance says things delivered are OK, the customer feel the progress, people are happy because they don’t have to work overtime and they architect and developers can apply the Best Practises. You are very close to the end of the development cycle (if you are working with an iterative and incremental methodology) and confident about the result. It’s time to review the overall status of the project, and then you check with the development team what is done and what is pending.
Project Manager: ‘Requirement X is implemented’
Architect: ‘Yes, it has been developed’
Quality Assurance: ‘We have tested it and it has some bugs’
PM: ‘How many bugs?’
QA: ‘About 50’.
PM: ’50???’
Architect: ‘Yes, but they are trivial.’
PM: ‘OK, great. I need an estimation for these bugs’.
Architect: ‘OK. We have first to finish the pending tasks.’

And iterating over each requirement, we find out that most of the requirements are not fully implemented. Your feelings of confidence on the schedule are starting to fade away…
PM: ‘What about the status of the Integration Environment?’
Architect: ‘Well, we are having some trouble putting together all the stuff developed by the team’.
PM: ‘What can of problems?’
Architect: ‘Stupid things like different naming conventions in the URLs and connection strings, corrupt data in the database, etc…’
PM: ‘Who is in charge of fixing this things?’

PM: ‘I know, everybody is busy finishing their tasks… But somebody must work in the Integration environment’.
Architect: ‘I can do it…’
PM: ‘No, I need you to work with the developers to fix the bugs.’
Architect: ‘Me fixing bugs?’
PM: ‘Yes, what’s wrong?’
Architect: ‘Nothing… I will work in the bugs’.
PM: ‘I know nobody wants to fix bugs. OK. Can developer X work on the Integration stuff?
Architect: ‘He has pending tasks’.
PM: ‘I know… We can assign his pending tasks to developer Y.’
Architect: ‘But the context change can kill his productivity. Can we contract another developer?’
PM: ‘No, budget is very tight and the a contractor will need to be in context anyway?’.

So now your feelings of confidence have fully faded away. Now it’s time to calculate the delays and reschedule the project. Things are not going so well, so you decided to meet more often to track the status of the project. After a couple of Status Meetings, you realise that the bugs are being fixed, but new bugs arise. The integration environment does not work properly yet, and now the Business Analysts come with requirement changes from the client. Meanwhile, the client is getting nervous because he cannot see the progress like he saw in the beginning of the project. Developers are now working overtime to fix bugs and complete pending tasks and some of them are starting to complain about your poor management skills in the coffee breaks.

So what have happened here?

I think this is part of the Software Development Game. The 95 percent syndrome comes as result of our education as Software Engineers. At the university they told us that everything has a solution, that there is only a way to build software. Wrong. They did not tell us that the biggest problem of software development is uncertain. Software Engineering has a very big problem as a Science: It has been created by Mathematicians and not by Engineers. For a Mathematician, uncertain is Hell. So they avoided uncertainty. I still can remember how Software Development was seen as a mathematical function,

Where x where the requirements, and Y was the software build. Terrible mistake. The waterfall model was defeated and iterative-incremental models came. And things were better. Now we had the following polynomial representation of software development.

Where each monomial can be an iteration. The good thing of this approach is that it assumes is not possible to build software as an image of the requirements, but it’s possible to build software that can get closer and closer to the requirements. Why? let’s put both formula together:

<p>
  </b></span>Going back with the mere mortals, it&#8217;s not strange to have a beautifully crafted Project Plan and a Gantt chart with a workload assigned for each tasks. We can create easily a chart like this:
</p>

<div style="padding: 1em 0pt; text-align: center;">
  <img style="width: 354px; height: 255px;" src="http://docs.google.com/File?id=dcvnhchp_66zprq8bd9" />
</div>

<p>
  <img src="http://www.blogger.com/post-edit.g?blogID=11583932&postID=3490945643825334407" alt="" /><br />The Project Manager thinks that the tangent of the function can vary, and with the changes he can control the delays from the schedule, and recalculate the schedule. But he is wrong. <span style="font-weight: bold;">He is wrong because he is assuming a function like the one above, a function that assumes that the workload to build a requirement is the same at the beginning of the project and at the end.</p> 
  
  <p>
    </span>The truth is that the function that describes the progress of a development is like this:<br /><span style="font-weight: bold;"><br /></span> 
    
    <div style="padding: 1em 0pt; text-align: center;">
      <img style="width: 336px; height: 241px;" src="http://docs.google.com/File?id=dcvnhchp_67g8qwpchn" />
    </div>
    
    <p>
      And this chart says a lot of things:
    </p>
    
    <ol>
      <li>
        The workload to build a requirement is dependant of when is being developed. There are a lot of factors (integratio<br /> n tasks, QA team performance, requirement changes&#8230;).
      </li>
      <li>
        Requirements being developed at the end of the cycle are more &#8220;expensive&#8221; than requirements at the beginning. The requirement can be simple in itself, but now it&#8217;s part of a very complex situation.
      </li>
      <li>
        It&#8217;s almost unavoidable to leave requirements out of the iteration. If you want to keep the schedule, you will have to move requirements to the next iteration.
      </li>
      <li>
        And 5% of the project can cost like 95% of the project. <span style="font-weight: bold;">This is the 95 percent syndrome.</span>
      </li>
    </ol>
    
    <p>
      What can we do to manage the 95 percent syndrome? I&#8217;m afraid this is part of the Software Development game and we can&#8217;t avoid it. So we will have to live with it. How? My personal recipes:
    </p>
    
    <ol>
      <li>
        Ask the client to classify the requirements by priority before starting the iteration.
      </li>
      <li>
        Ask the architect to classify the requirements by complexity before starting the iteration.
      </li>
      <li>
        Build highest priority and complexity first.
      </li>
      <li>
        Schedule several &#8216;bug fixing days&#8217; in between the iteration. Stop the development progress and work on the bugs these days.
      </li>
      <li>
        Keep in mind that Integration tasks depends directly on the amount of elements to integrate and it&#8217;s not a linear function. Probably it can a be logarithmic.
      </li>
      <li>
        Keep a buffer in the Project Plan, and do not say a word to anybody.
      </li>
      <li>
        Do not accept changes in the current iteration. If you don&#8217;t have any choice, then reschedule the project as if you were punishing the client.
      </li>
      <li>
        Explain to the client on every meeting that most of the times some requirements are moved to the next iteration. Since most important requirements are built first in the iteration, only low priority requirements are moved.
      </li>
      <li>
        Do not reschedule the iteration eternally. Try to close it, even with less of the 95% of the requirements.
      </li>
    </ol>
    
    <p>
      These are my thoughts about this special syndrome, but I would like to hear from you. Please post your tips and recipes.</silence>
    </p>

Related Articles