Estimating Projects and Meeting the Estimates

OVERVIEW

Accurate estimating is one of the biggest challenges faced by project teams.  Projects commonly miss their estimates by orders of magnitude.  Standish reports “52.7% of projects will overrun their initial cost estimates by 189%”.   Why is it so difficult to estimate projects?  Is it an estimation problem or a management problem? 

The Project Management Institute’s Book of Knowledge (PMBOK) provides very little guidance for estimating projects.  It describes the various factors that should be considered such as scope, resources, and risks and lists the updated artefacts but very little guidance is provided. 

ANALYSIS

How do we create a repeatable estimating process when so many projects are creating a unique product?  This lack of repeatability results in a heavy dependence on the experience of the person producing the estimate. Experience-based estimates may explain variations of 50% but the Standish reports show variations close to 200%.  There must be other factors. 

Significant variations between the estimated costs and actual costs can be the result of the can also be the result of management issues.  When a project team creates an estimate, that estimate should represent a commitment.  If the project team does not view the estimate as a commitment, then there is very little chance the estimate will be met. 

SOLUTIONS

This section discusses options for addressing the variability resulting from the unique nature of each project and management considerations that will result in more predictable outcomes. Meeting project schedules and estimates requires a combination of repeatable estimating and consistent management practices, and an acceptance of the commitment to meet the dates and estimates. 

1.  Define Standards and Models

While the results from individual projects are unique and rarely repeatable, the definition of standards and models allows for the definition of repeatable activities that can be combined to create unique results. 

2.  Manage Creativity

Creative activities are the most difficult to estimate.  How do we know when we are done being creative?   While many people who participate in projects want to be creative, few people have the skills and the discipline required.  Creative activities must be defined and assigned to individuals with time limits for completion.     

3.  Adjust for Available Skills Levels

There are very few times when the desired skill levels are available.  Repeatable estimates have to be defined based on an assumed skill set but when the desired skill levels are not available, a project manager has to re-assess the estimated efforts and durations and ensure the available staff skills are sufficient for meeting the commitment.

4.  Manage the Learning Curve

Technology is changing rapidly.  Many projects involve the deployment of technology and few people on the project have significant experience with the new technology.  As a result, the team learns the technology by experimentation as they complete their assignments.  This type of on-the-job training is very difficult to estimate.  A project that deploys new technology should include an evaluation period by a small experienced group who will make decisions regarding best practices and recommended usage and then train the rest of the team.

5.  Manage Requirements and Scope

While this is an obvious recommendation, a common reason for project cost over-runs is the failure to manage requirements and scope.  Many people treat consider requirements and scope as synonymous.  They are not and they must be managed separately. 

Requirements management should begin by defining the expected business outcomes.  Functional requirements must be defined that support the desired outcomes.  Finally, technical requirements and design specifications should describe how the functional requirements will be delivered.  All of these types of requirements must be identified and documented and managed. 

The scope of the project identifies which of the requirements will be estimated and delivered.  Some requirements may not be approved and others may not be deferred.  Project teams must document and obtain formal approval for all requirements that are included in the project scope.  If the decision is made to change the project scope to add or remove requirements, then the estimate and schedule should also be updated.    

6.  Treat Estimates as Commitments

When the project team is established, the estimates should be reviewed by lead members of the team and any issues should be identified and resolved.  Once the issues are resolved, the team leads should accept the estimate as a commitment.   When assignments are made to individual participants, they should also review their assigned estimate and raise issues or commit to the estimate and date. 

7.  Manage Issues

Issues include anything that may impede the ability of the team to complete the assigned tasks.  All issues should be logged and assessed to determine their impact to the estimate and schedule.  Ownership for resolving the issues should also be assigned.    

SUMMARY

Effective estimating cannot be achieved without commitment and effective management.

Software Defects – Symptom or Problem

All software has defects and perfection would be difficult and expensive to achieve.  Some defects are recurring and some are undetected for many years.  It is important to understand that software defects are symptoms of other problems such as process deficiencies or inadequate management or oversight.   Defects may also be the acceptable result of risk mitigation and cost management decisions.  In other words, management made a choice to limit testing and allow for the possibility of defects. 

Identifying and fixing defects is difficult, disruptive, and expensive.  Preventing defects is much more cost effective.  In order to prevent defects, we need to anticipate them and implement processes and management techniques to prevent them.  How do we do this?

1. Obtain agreement on the definition and types of defects.  Examples: If code is confusing and inefficient but it provides the desired capability, is it defective?  ITIL refers to “Fitness for Purpose” and “Fitness for Use”.  The ISO 9126 software quality standards provide additional criteria. 
2. For each defect type, determine the likelihood of defects and the expected impacts.
3. Identify preventative mechanisms for each type of defect which include code review and testing
4. Track defects and the reason the defect occurred and communicate the information to participants so they can learn to anticipate and prevent defects
5. Define rules for developing applications that are specific to each type of technology.  This will improve standardization and avoid high-risk development techniques.

What should be the role of a PMO?

OVERVIEW

The role of a Project Management Office (PMO) varies significantly from one organization to the next.  Some PMO’s simply provide a pool of IT project managers to manage projects.  At the other end of the spectrum, PMOs may be the part of the business with ultimate responsibility to define, authorize and monitor projects.  The responsibility to define and enforce project management processes is also common to PMO’s. 

PMO Role

In order to obtain a variety of opinions and observations, the question “What is a PMO?” was posted to a public blog which generated a wide variety of responses.  This section summarizes these responses.

• Get the work in (Proposals), provide Project Managers to lead through completion.
• Act as an interface between Program/Project Boards and the Project delivery teams.
• Screen projects for business value
• Provide the first level of escalation for project teams and customers
• Ensure the management approach matches the project type
• Establish and enforce processes that leverage industry best practices such as PMBOK and Prince2
• Oversee delivery to ensure compliance with policies and procedures, incident management, change management, strategic reviews, operation reviews, on time delivery of the projects in quantifiable manner.
• Drive project execution performance and visibility through the introduction of project management standards
• Projects and IT services have a life-cycle from Inception through Implementation and Benefits Realisation.  The PMO is in an ideal position to coordinate across the life cycle
• PMO should be structured around optimizing people
• Evaluate the technology, complexity, novelty and pace required for the project

One of the respondents cited Crawford, J.K., et al. (2008). Seven Steps to Strategy Execution which identifies the following PMO responsibilities:

• Incorporate lessons learned into Processes, Standards and Methodologies
• Develop and deploy professional project managers to manage projects
• Take the lead role for training and development of project managers and team members in the project management processes, culture, and roles
• Provide Project Support to assist with estimating, budgeting, planning, scheduling, status reporting and issue tracking.
• Establish and maintain project-related Software Tools by identifying, integrating, maintaining, and monitoring performance of the tools.
• Mentoring and Coaching departments who which to manage projects themselves.
• Assume Portfolio Management responsibility to act as a clearinghouse for project information and to make sure decision makers have the best information. 

Issues

The following issues were identified during the discussion:

• “The cobbler’s children have no shoes”. IT spends most of their time building management solutions for the business.  They neglect to implement processes and tools for effectively managing IT. PMO’s must combine information from various disconnected sources spend long hours trying to reconcile and evaluate progress.
• A project begins with Inception where potential projects/ideas are identified, prioritized, and authorized. Most PMO’s are part of the IT organization and IT’s focus is the delivery of projects.  As a result the PMO doesn’t get involved until a formal project has been authorized.  PMO involvement typically ends when the project is completed and no one is responsible for validating the desired benefits were realized. 

OBSERVATIONS

An analysis of the responses yielded the following observations: 

• There is a wide variety of opinions regarding the role of a PMO and a lack of consensus.  Most of the responses described an advisory, training, staffing, or a supporting role for the PMO with limited authority. 
• The suggestions were primarily IT-centric with minimal discussion of business issues and only one comment about the role of a PMO with respect to Portfolio Management.
• Many of the suggestions implied a common theme which is the identification and mitigation of risks but this was never directly stated.
• There was no discussion regarding the role of the business sponsors and stakeholders and the PMO’s involvement in defining and managing these roles.
• There was no discussion of the role of a PMO with respect to Application Support.

RECOMMENDATIONS

Based on this analysis, it appears the benefits to the business could be greatly increased by integrating the following responsibilities into the role of a PMO: 

• Provide coordination and direction throughout all cycles of a project from Inception to Benefits Realization.  This includes the identification and verification of the value proposition.   
• Assume the responsibilities for Portfolio Management to classify, organize, and prioritize projects.  This should also include a lead role in aligning requirements and resolving conflicts across organizational boundaries.
• Take a lead role in project risk assessments, risk mitigation, and ongoing project performance monitoring.  Project teams can become so focussed on solving tactical project issues they lose sight of the big picture and the overall objectives.  This type of oversight can help a PMO identify issues early and help prevent problems. 
• Include support oversight in the scope of the PMO since it is directly impacted by projects
• Implement monitoring and assessment capabilities to support these recommendations.