SOFTWARE ACQUISITION GOLD PRACTICE

Track Earned Value

FOCUS AREA:    COST – Project Management

Definition and Summary:  An integrated system of project management and control that enables a contractor and their customer to monitor the progress of a project in terms of integrated cost, schedule, and technical performance measures.

 

Earned value management, which is used to track earned value, is an integrated system of project management and control that enables a Contractor and their customer to monitor the progress of a project in terms of integrated cost, schedule, and technical performance measures.  The Contractor/developer owns the process but the Acquirer/customer has full and timely visibility of the information contained within it.  Traditional project management practice tends to compare actual costs with planned expenditures, and confuses actual costs with actual progress.  EVM provides a third reference point that is an objective view of the status of the effort, i.e., the value to the end goal of the work completed to date..

Successful implementation of Earned Value Management principles can result in:

 

• Better Visibility into Program Performance
• Reduced Cycle Time to Deliver a Product
• Increased Accountability
• Reduced Risk
  

 

DESCRIPTION OF THE PRACTICE:

 

SUMMARY DESCRIPTION

Earned value project management involves planning work to a manageable level of detail such that it is feasible to allocate a portion of the budget to each planned work unit (work package), and then tracking progress by the accumulated “value” of completed work units.  As work is performed, it is “earned” on the same basis as it was planned, in dollars or other quantifiable units.  As the work units are completed, the project earns the budgeted value associated with those work units.  This method associates a dollar value with work completed so that it can be compared with the actual spending (to determine cost variance – potential cost overruns), and the planned spending (to determine schedule variance – potential schedule slippage).  In this manner, planned and actual spending are integrated with actual work performed.  The integration provides greater visibility into the real project status for all stakeholders and thus creates a scenario for better management of risks, for early determination of whether a project is in trouble, and for estimating what will be needed to complete it.

Although tracking earned value occurs during project execution, it cannot be accomplished if appropriate project planning and budget allocation has not occurred up front.  The figure below depicts, at a high level, the activities that are necessary in order to effectively implement the principles of earned value management. 

 

DETAILED DESCRIPTION

Earned value (EV) management is an integrated system of project management and control that enables a Contractor and their customer to monitor the progress of a project in terms of integrated cost, schedule, and technical performance measures.  The Contractor/developer owns the process but the Acquirer/customer has full and timely visibility of the information contained within it.  This represents a recent migration in focus from acquirers using EV for inspection and oversight to developers using EV for project management.  Traditional project management practice tends to compare actual costs with planned expenditures, and confuses actual costs with actual progress.  EVM provides a third reference point that is an objective view of the status of the effort, i.e., the value to the end goal of the work completed to date.

Although tracking earned value occurs during project execution, it cannot be accomplished if appropriate project planning and budget allocation has not occurred up front.  The figure below depicts, at a high level, the activities that are necessary in order to effectively implement the principles of earned value management. 

 

DEFINE THE WORK:  The Project Manager must decompose the project into distinct discrete manageable tasks or groups of tasks (work packages) with decisive outputs and specific measurable entry and exit criteria.  Each work package has a short duration, or can be divided into a series of milestones whose status can be objectively measured.  Each work package can be assigned a start and finish date, a budget value, and can be integrated with higher-level schedules.  This activity is often referred to as developing the Work Breakdown Structure (WBS).  It is important to balance the level of detail in the WBS with the needs of the project, with the ultimate goal being the ability to realistically estimate the cost of accomplishing each task (earned value).  Providing too much detail creates an overload of data, creating a tracking nightmare, and stifling the creativity of developers; lack of detail may mask vital information.  Guidelines on EVM from the UK Ministry of Defense suggest that 3-4 levels in the WBS are sufficient for most projects, but complex projects (such as building a ship) may require five or six levels.

SCHEDULE AND BUDGET:  Once the effort is identified through the WBS, the project manager must prepare a budget and schedule for accomplishing the work.  This is in contrast to “backing into a schedule” based on an arbitrary fixed dollar amount.  Details of budgeting and scheduling are beyond the scope of this document, but essentially involve identifying what resources are needed and how much effort will be required in what time frame to complete each of the tasks in the WBS.  What is critical to being able to track earned value is that a portion of the budget is allocated for each work package that comprises the WBS and that the WBS adequately defines all work necessary to meet the agreed-upon requirements for the project.

MEASURE PERFORMANCE:  This activity focuses on performance, not just planned vs. actual spending.  It involves tracking a number of measures starting very early in the project, and analyzing the data to determine real project status.  Important measures are:

Primary Measures

·         Budget Cost of Work Scheduled (BCWS) – the spending plan; the dollars (or hours) planned for the effort.  The cumulative planned expenditures would equal the total dollars budgeted for the effort for the specified time period.  With EVM, the spending plan serves as a performance baseline for making predictions about cost and schedule variance and estimates of completion.

·         Actual Cost of Work Performed (ACWP) – actual spending; the cumulative actual expenditures on the effort viewed at regular intervals within the project duration.

·         Budgeted Cost of Work Performed (BCWP) – earned value, the measure of technical accomplishment; the cumulative budgeted value (dollars or hours) of work actually completed.  It may be calculated as the sum of the values budgeted for the work packages actually completed, or calculated as the percent work complete multiplied by the planned cost of the project.

Derived/Calculated Measures

From the three primary measures it is possible to derive measures that can be used to accurately assess the status of the project and predict its future state.

·         Cost Variance (CV) – The numerical difference between the earned value (BCWP) and the actual cost (ACWP).  CV = BCWP – ACWP.  (Another way of thinking of this is the difference between the planned and actual costs of work completed.)

·         Schedule Variance (SV) - An indicator of how much a program is ahead of or behind schedule.  SV = BCWP – BCWS.  (Another way of thinking of this: earned value – planned budget, or the difference between the value of work accomplished for a given period and the value of the work planned).  Schedule variance is presented well in chart format.

·         Cost Performance Index (CPI) – The cost efficiency factor representing the relationship between the actual cost expended and the earned value.  CPI = BCWP/ACWP.  A CPI ≥ 1 suggests a relatively efficient cost factor, while a CPI <1 may be cause for concern.

·         Schedule Performance Index (SPI) – The planned schedule efficiency factor representing the relationship between the earned value and the initial planned schedule.  SPI = BCWP/BCWS.  A SPI ≥ 1 is good. SPI < 1 suggests actual work is falling behind the planned schedule.

·         Budget at Completion (BAC) – sum total of the time-phased budget.  Synonymous with “Performance Measurement Baseline”.

·         Estimate to Complete  (ETC) – A calculated value, in dollars or hours, that represents the cost of work required to complete remaining project tasks.  ETC = BAC – BCWP.

·         Estimate at Complete (EAC) – A calculated value, in dollars or hours, that represents the projected total final costs of work when completed.  EAC = ACWP + ETC.

 

In looking at the list of important measures earned value (BCWP) is one of the three basic measures from which the other measures are derived.  Without it, the other measures are not possible.

Earned value credit should be binary, with 0 percent being given before task completion and 100 percent given when completion of each work unit is validated.  Establishing specific measurable exit criteria for each task makes it easier to track task completion, and thus credit the earned value of the task to the project so that the earned value of the project at any given point in time is obtained by “simple math” rather than by subjective assessment.

COMMUNICATE PERFORMANCE STATUS:  Tracking earned value is of little value if the estimating and analysis capability that it provides is not used to manage the project.  Although originally required for reporting project status to the acquirer, in recent years there has been a migration of focus.  EVM is now viewed as a project management technique, as well.  Its usefulness is broader than simply reporting project status up the management chain.  There are some important reasons to communicate the project status (represented in terms of earned value) to all stakeholders.

·         Promote Accountability - When developers understand how their individual work (or lack thereof) influences the project, they tend to be more focused on their specific work goals.  They also better understand the significance of estimating the amount of work needed to complete specific tasks.  There exists a mindset among some project managers that they should “protect” their developers from the distraction of project metrics.  In reality, communicating project status to the development staff tends to establish a sense of accountability for their assigned pieces of the project and often results in more realistic estimates for completion of future tasks.

·         Reporting real project status, including earned value, at regular intervals provides an opportunity to address potential problems early in the project when it is still possible to resolve problems and avoid cost overruns and schedule slippage.  The project team takes a proactive approach to prevent problems from occurring.  Management uses the information to resolve issues that are beyond the control of the project team.  The time interval should be at least monthly, regardless of the size and duration of a project, and more frequent for some projects.  Many practitioners experienced with earned value management indicate that the project team should review project earned value weekly, because it can alert the team to specific problem areas before they develop into major problems.

Illustrative Example:

Here are some examples of how tracking earned value improves the accuracy of communicating real project status.

Assuming a project budget of $300K (with $100K allocated for the first four months), planned to be implemented over 12 reporting periods (1^st 4 periods shown here), a sufficiently detailed WBS, and an accurate method of crediting earned value, Table 1 provides the primary project level data that would be captured under principles of EVM to communicate project status.  Data typically captured (and measures reported) under a traditional approach are shaded blue in both tables.  Rows shaded gold represent additional data collected and measures reported using an EVM approach.

Table 1:  Primary Measures of Earned Value Management

 

Under the traditional project-reporting paradigm (Planned vs. Actual spending), at the end of 4 periods this data shows that the project is underrun by $8K.  However, when “tracking earned value” is added as an additional reference point, the data shows that the project is overrun by $12K (see Cost Variance in Table 2).  This is the effect of integrating EV into the project monitoring process.  It focuses on reporting progress from the perspective of work completed, and that focus brings a whole new dimension to the status of a project.  In this case, the difference in project status ($20K), as reported by the two approaches, amounts to 20% of the funding ($100K).  Table 2 illustrates other metrics derived from earned value data that can be used, not only to better assess the current status of the project, but also to make more realistic predictions about where the project is headed.

Table 2:  Derived Measures of Project Status

Communicating earned value does not make problems go away automatically, but it can provide objective consistent measures that are useful in focusing attention on the “real” status of the project, as follows:

·         Cost and schedule efficiency factors <1 are cause for concern

·         Downward trend in CPI is consistent with government findings in monitoring projects since the 1970s (see below)

·         Even with fluctuation, data reported early in the project is a good predictor of what will happen later

·         Frequent reporting of EV data supports trend analysis that can better communicate the direction of the project

The DoD has been able to make several determinations about projects in general by using data collected from projects where tracking earned value was required (over 400 projects since 1977).

·         Without exception, the cumulative CPI does not significantly improve between the period of 15% and 85% of contract performance period.  In fact, it tends to decline.  For example, a CPI of 0.75, derived from data at the 15% mark suggests that if the budget and schedule remain constant, only 75% of the work will be completed at project end.  This clearly indicates early in the project a need to adjust budget, schedule, and/or the scope of work.

·         Studies show that EACs based on both the CPI and the SPI tend to be significantly higher and are generally more accurate (Christensen, 1996)

Typical Project Status Chart Using EV

 

CHARACTERISTICS OF IMPLEMENTATION:

 

SUMMARY CHARACTERISTICS

 

NO DATA CURRENTLY AVAILABLE

 

ANTICIPATED BENEFITS OF IMPLEMENTATION:

 

Successful implementation of the Earned Value Management principles can result in:

 

• Better Visibility into Program Performance

The combination of advance planning, baseline maintenance, and earned value analysis yields earlier and better visibility into program performance than is provided by non-integrated methods of planning and control.

• Reduced Cycle Time to Deliver a Product

Earned value management is premised on careful detailed planning – task decomposition, scheduling, and budgeting.  This planning often addresses/prevents problems from surfacing later in the effort that result in rework.  Thus, as rework is prevented cycle time may, in fact, be reduced.

 

• Fosters Accountability

When the developer, at the personal level, understands how their pieces fit into the overall project effort they tend to focus on delivery of a quality product.  Additionally, over time they are better able to estimate the work required to complete a task, thereby improving the overall accuracy of the budget/estimating process for future efforts.

 

• Reduced Risk

Because earned value measures enable realistic estimates of completion (for both cost and schedule) to be derived early in the project, it is possible to make adjustments and take corrective action to mitigate the risk of cost overruns and schedule slippage.

 

DETAILED CHARACTERISTICS

 

Key Characteristics of the Track Earned Value Gold Practice

RELATIONSHIPS TO OTHER PRACTICES:

The Figure below represents a high-level process architecture for the subject practice, depicting relationships among this practice and the nature of the influences on the practice (describing how other practices might relate to this practice).  These relationship statements are based on definitions of specific “best practices” found in the literature and the notion that the successful implementation of practices may “influence” (or are influenced by) the ability to successfully implement other practices.  A brief description of these influences is included in the table below.

 

Process Architecture for the "Track Earned Value" Gold Practice

 

Summary of Relationship Factors for the Track Earned Value Practice

RESOURCES:        

 

§        Websites

·         Management-Technologies, Inc. has developed an Earned Value Maturity Model

http://www.mgmt-technologies.com/evmtech.html

·         Earned Value Management (EVM) website (sponsored by OSD)

http://www.acq.osd.mil/pm/

·         NASA Earned Value Management (EVM) Website:  The mission of this site is to provide a primary on-line reference point for EVM theory, application, and use as an integrated project management tool within NASA.

http://evm.nasa.gov/

·         Defense Material Organization – Australia:  Provides insight into how EVM is being implemented within the Australian Defense community.

http://www.defence.gov.au/dmo/esd/evm/index.cfm

·         Ministry of Defense, United Kingdom – Acquisition Management Website

http://www.ams.mod.uk/ams/content/docs/evm2/evmacq2.htm

§        Tools and Methods

State-of-the-art methods and tools that may be useful in implementing and improving the effectiveness of earned value management include:

 

• Team Software Process
• Personal Software Process
• Earned Value Management Maturity Model
• Project Management Tools that aid in establishing and tracking EV

 

This site, part of DAU's Acquisition community Connection (ACC), provides links to material on software tools for earned value management.

http://pmcop.dau.mil/simplify/ev.php?ID=52966_201&ID2=DO_TOPIC

Dekker, Ltd., founded in 1984, provides a complete product line of management solution tools including Trakker Earned Value Management System.

http://www.dtrakker.com/products/dt_evm.htm

§        Experts/Contact Points

Paul Solomon, Northrop Grumman Corp. (paul.solomon@pb-ev.com)

Quentin W. Fleming, Primavera Systems, Inc. (QuentinF@Primavera.com)

 

§        Training Opportunities:

·         DAU Acquisition Community Connection (ACC) EVM Training Center has several on-line references and training resources

http://pmcop.dau.mil/simplify/ev.php?ID=52967_201&ID2=DO_TOPIC

·         Tecolote Research, Inc. is a privately held firm that is nationally known for providing integrated financial management services, including earned value training, for clients within the U.S. Government

http://www.tecolote.com/Services/EarnedValue.htm

·         Humphreys & Associates, Inc. is a management consulting firm specializing in Earned Value Project Management since 1978 with both on-line and conventional courses customized for both public and private sector organizations

http://www.humphreys-assoc.com/on-line/courses/courses.html

http://www.humphreys-assoc.com/products/materials/materials.html

http://www.humphreys-assoc.com/workshops/workshops.html

 

·         Management Technologies provides two courses in EVM

http://www.mgmt-technologies.com/seminars.html

·         VitalThought is a Project Management company that specializes in the Earned Value technique of project management as defined by the DoD Cost Schedule Control System Criteria

http://www.vitalthought.com/Training/training.html

·         Quentin W. Fleming Project Management Consultancy provides a one-day seminar covering all fundamentals necessary to implement projects that employ the earned value project management concept

http://www.quentinf.com/ - Project%20Management

·         Program Management Associates, Inc. (PMA) is an SBA-qualified small business, incorporated in 1988, dedicated to project management support and training

http://www.pma-inc.com/earned.htm

 

 

§        Bibliography:

 

 

APPENDICES

DEFINITIONS:

 

Earned Value Management (EVM) is an integrated system of project management and control that enables a contractor and their customer to monitor the progress of a project in terms of integrated cost, schedule, and technical performance measures.

-    [Acquisition Management Website, Ministry of Defense, UK]

Earned value is a management technique that relates resource planning to schedules and to technical cost and schedule requirements.  All work is planned, budgeted, and scheduled in time-phased ''planned value'' increments constituting a cost and schedule measurement baseline.  There are two major objectives of an earned value system: to encourage contractors to use effective internal cost and schedule management control systems; and to permit the customer to be able to rely on timely data produced by those systems for determining product-oriented contract status.

-    Earned Value Management Website

SOURCES (Origins of the Practice):

 

The concept of earned value originated with industrial engineers in factory settings in the early 1900s.  They assessed their cost performance by comparing the physical factory work output (earned value) against the actual costs incurred, and then to the physical work planned.  They defined cost variance as the difference between the actual costs and the earned value.  This definition is critical to implementing the earned value concept. 

§         DoD Financial Management Orders, since 1967

Although expressed with different terminology the basic principles of earned value management have been recommended for use in the DoD since the early 1960s when it issued a directive that imposed 35 Cost/Schedule Control Systems Criteria (C/SCSC) on all private and industrial firms that wished to participate in future major systems procurements.  That directive focused on oversight and inspection and caused EV to be viewed only as a reporting tool.

In 1996, and as a result of participation by private industry, the DoD endorsed a replacement of the C/SCSC, which became known as the industry standard “Earned Value Management System” (EVMS) and reduced the number of formal criteria to 32.  The standard, identified as EIA-748, has since been updated and is now available (for a fee) from the ANSI Standards body.

RECOMMENDING SOURCES:

 

§         USD (A&T), August 1999

In August of 1999, Dr. Jack Gansler, USD (A&T), signed a memorandum announcing that the DoD had adopted the ANSI Earned Value Management System (EVMS) Standard for use on defense acquisitions.  The standard incorporates best business practices that have proven to provide strong benefits for program and enterprise planning and control.  It is available electronically for a fee (Doc # EIA-748) from Global Engineering Documents.

§         DoD 5000.2-R, “Mandatory Procedures for Major Defense Acquisition Programs (MDAPS) and Major Automated Information System (MAIS) Acquisition Programs”, 5 April 2002, (canceled 30 October 2002)

§         Interim Defense Acquisition Guidebook, 30 October 2002 [INTERIM 2002] [This document now supercedes DoD 5000.2-R.]

 

C2.9.3.4  integrated contract performance management.  The four paragraphs within section C2.9.3.4 provide detail on the contract situations in which the ANSI EVMS guidelines and the Cost/Schedule Status Report (C/SSR) DID DI-MGMT-81467 (DoD 5010.12-L) must be followed.

C2.9.3.4.1  The PM shall obtain integrated cost and schedule performance data to monitor program execution.  The PM shall require contractors to use internal management control systems that produce data that a) indicate work progress; b) properly relate cost, schedule, and technical accomplishment; c) are valid, timely and able to be audited; and d) provide DoD PMs with information at a practical level of summarization.  Unless waived by the Milestone Decision Authority (MDA), the PM shall require that contractors’ management information systems used in planning and controlling contract performance meet the EVMS Guidelines set forth in the ANSI/EIA-748-98 document.

Ap4  appendix 4  EVMS guidelines, mandatory procedures, & Reporting.  This appendix reproduces the guidelines contained in the ANSI/EIA-748-98 document.

 

§         Capability Maturity Model Integration (CMMI), Version 1.1 Staged, Software Engineering Institute, CMU/SEI-2002-TR-012, TR-012, March 2002

 

Three key process areas (KPAs) specifically address tracking work performance and comparing it to scheduled or planned activity, although they do not specifically require the use of the earned value measure.

Pg 97  Project Planning KPA. SP1.1.  Establish a top-level work breakdown structure (WBS) to estimate the scope of the project.  In the list of sub-practices the following text appears:

 

The top-level WBS is intended to help in gauging the project work effort in terms of tasks and organizational roles and responsibilities.  The amount of detail in the WBS at this more detailed level helps in developing realistic schedules, thereby minimizing the need for management reserve.

 

Pg 101  SP 1.4  Estimate the project effort and cost for the work products and tasks based on estimation rationale.

 

Pg 104  SP 2.1.  Establish and maintain the project’s budget and schedule.

 

PP 124 – 125  Project Monitoring and Control KPA. SG 1 Monitor Project Against Plan.  Actual performance and progress of the project are monitored against the project plan.  The sub-practices include the following:

Monitor progress against the schedule.  Progress monitoring typically includes:

·          Periodically measuring the actual completion of activities and milestones

·         Comparing actual completion of activities and milestones against the schedule documented in the project plan

·         Identifying significant deviations from the schedule estimates in the project plan

Monitor the project's cost and expended effort.  Effort and cost monitoring typically includes:

·         Periodically measuring the actual effort and cost expended and staff assigned

·         Comparing actual effort, costs, staffing, and training to the estimates and budgets documented in the project plan

·         Identifying significant deviations from the budgets in the project plan.

 

Pg 154  Measurement & Analysis KPA.  The purpose of M&A is to develop and sustain a measurement capability that is used to support management information needs.

The integration of measurement and analysis activities into the processes of the project supports the following:

·         Objective planning and estimating

·         Tracking actual performance against established plans and objectives

·         Identifying and resolving process-related issues

·         Providing a basis for incorporating measurement into additional processes in the future

 

PG 159  SP 1.2 Specify Measures.  Measurement objectives are refined into precise quantifiable measures. … Earned Value is identified as an example of a commonly used derived measure.

 

GLOSSARY

 

 

CASE STUDIES FROM THE LITERATURE:

 

Practical Software Measurement:  Performance-Based Earned Value

Successful software project management can be achieved by focusing on requirements, selecting the most effective software metrics, and using Earned Value Management.  Best practices and lessons learned by the Northrop Grumman team in developing weapons system software for the B-2 Stealth Bomber are discussed.

http://www.testablerequirements.com/Articles/solomon.htm

EVM and Software Project Management – Our Story

The Software Division at Tinker Air Force Base in Oklahoma has used earned value management (EVM) methods for more than 15 years.  These management methods have had significant influence in the improvement of software development and maintenance practices of the organization.  This article, in a story-telling manner, describes the use of EVM for managing software, and how its system of management facilitated a natural evolution that lead to recognition, awards, and more importantly, on-time, at-cost, quality software.  http://stsc.hill.af.mil/crosstalk/2002/11/lipke.html

 

All the Right Behavior

Software projects using the Team Software ProcessSM (TSP^SM) have an unusually high rate of on-time completion.  One of several key factors contributing to this accomplishment is the effective way TSP teams make use of earned value techniques to iteratively refine their plan as they work it.  Because earned value is reviewed weekly, and because no value is earned either at the personal or the team levels until a task is fully completed, software engineers are highly motivated to perform good earned value practices.  This article expounds upon this principle and examines how TSP teams succeed with earned value.   http://stsc.hill.af.mil/crosstalk/2002/09/webb.html