Estimating Projects and Meeting the Estimates

OVERVIEW

Accurate estimating is one of the biggest challenges faced by project teams.  Projects commonly miss their estimates by orders of magnitude.  Standish reports “52.7% of projects will overrun their initial cost estimates by 189%”.   Why is it so difficult to estimate projects?  Is it an estimation problem or a management problem? 

The Project Management Institute’s Book of Knowledge (PMBOK) provides very little guidance for estimating projects.  It describes the various factors that should be considered such as scope, resources, and risks and lists the updated artefacts but very little guidance is provided. 

ANALYSIS

How do we create a repeatable estimating process when so many projects are creating a unique product?  This lack of repeatability results in a heavy dependence on the experience of the person producing the estimate. Experience-based estimates may explain variations of 50% but the Standish reports show variations close to 200%.  There must be other factors. 

Significant variations between the estimated costs and actual costs can be the result of the can also be the result of management issues.  When a project team creates an estimate, that estimate should represent a commitment.  If the project team does not view the estimate as a commitment, then there is very little chance the estimate will be met. 

SOLUTIONS

This section discusses options for addressing the variability resulting from the unique nature of each project and management considerations that will result in more predictable outcomes. Meeting project schedules and estimates requires a combination of repeatable estimating and consistent management practices, and an acceptance of the commitment to meet the dates and estimates. 

1.  Define Standards and Models

While the results from individual projects are unique and rarely repeatable, the definition of standards and models allows for the definition of repeatable activities that can be combined to create unique results. 

2.  Manage Creativity

Creative activities are the most difficult to estimate.  How do we know when we are done being creative?   While many people who participate in projects want to be creative, few people have the skills and the discipline required.  Creative activities must be defined and assigned to individuals with time limits for completion.     

3.  Adjust for Available Skills Levels

There are very few times when the desired skill levels are available.  Repeatable estimates have to be defined based on an assumed skill set but when the desired skill levels are not available, a project manager has to re-assess the estimated efforts and durations and ensure the available staff skills are sufficient for meeting the commitment.

4.  Manage the Learning Curve

Technology is changing rapidly.  Many projects involve the deployment of technology and few people on the project have significant experience with the new technology.  As a result, the team learns the technology by experimentation as they complete their assignments.  This type of on-the-job training is very difficult to estimate.  A project that deploys new technology should include an evaluation period by a small experienced group who will make decisions regarding best practices and recommended usage and then train the rest of the team.

5.  Manage Requirements and Scope

While this is an obvious recommendation, a common reason for project cost over-runs is the failure to manage requirements and scope.  Many people treat consider requirements and scope as synonymous.  They are not and they must be managed separately. 

Requirements management should begin by defining the expected business outcomes.  Functional requirements must be defined that support the desired outcomes.  Finally, technical requirements and design specifications should describe how the functional requirements will be delivered.  All of these types of requirements must be identified and documented and managed. 

The scope of the project identifies which of the requirements will be estimated and delivered.  Some requirements may not be approved and others may not be deferred.  Project teams must document and obtain formal approval for all requirements that are included in the project scope.  If the decision is made to change the project scope to add or remove requirements, then the estimate and schedule should also be updated.    

6.  Treat Estimates as Commitments

When the project team is established, the estimates should be reviewed by lead members of the team and any issues should be identified and resolved.  Once the issues are resolved, the team leads should accept the estimate as a commitment.   When assignments are made to individual participants, they should also review their assigned estimate and raise issues or commit to the estimate and date. 

7.  Manage Issues

Issues include anything that may impede the ability of the team to complete the assigned tasks.  All issues should be logged and assessed to determine their impact to the estimate and schedule.  Ownership for resolving the issues should also be assigned.    

SUMMARY

Effective estimating cannot be achieved without commitment and effective management.

Software Defects – Symptom or Problem

All software has defects and perfection would be difficult and expensive to achieve.  Some defects are recurring and some are undetected for many years.  It is important to understand that software defects are symptoms of other problems such as process deficiencies or inadequate management or oversight.   Defects may also be the acceptable result of risk mitigation and cost management decisions.  In other words, management made a choice to limit testing and allow for the possibility of defects. 

Identifying and fixing defects is difficult, disruptive, and expensive.  Preventing defects is much more cost effective.  In order to prevent defects, we need to anticipate them and implement processes and management techniques to prevent them.  How do we do this?

1. Obtain agreement on the definition and types of defects.  Examples: If code is confusing and inefficient but it provides the desired capability, is it defective?  ITIL refers to “Fitness for Purpose” and “Fitness for Use”.  The ISO 9126 software quality standards provide additional criteria. 
2. For each defect type, determine the likelihood of defects and the expected impacts.
3. Identify preventative mechanisms for each type of defect which include code review and testing
4. Track defects and the reason the defect occurred and communicate the information to participants so they can learn to anticipate and prevent defects
5. Define rules for developing applications that are specific to each type of technology.  This will improve standardization and avoid high-risk development techniques.

Should the PMO be owned by IT or the “Business”?

Neither! We describe the “business” as if it is a single entity. The “business” is a collection of departments whose priorities and objectives frequently conflict. The functions of a PMO are best assigned to a corporate entity which spans the business departments and IT and is closely aligned with the overall business strategy.

The PMO should have oversight responsibilities for the entire life cycle of all projects (including non-IT). This will provide the best chance of addressing conflicting priorities, standarding across the enterprise, and aligning with the “business” objectives.