SOFTWARE ACQUISITION GOLD PRACTICE

BINARY QUALITY GATES AT THE INCH-PEBBLE LEVEL

FOCUS AREA:   PROJECT MANAGEMENT - Measurement 

Definition and Summary:  The practice of defining completion criteria for each task in the lowest-level activity network as gates that assess either the quality of the products produced, or the adequacy and completeness of the finished process, and then tracking completion at the binary level (done or not done).

 

The practice of implementing binary quality gates at the inch-pebble level involves two stages of activities, (1) planning, and (2) monitoring, which are summarized in the following figure.

The general premise of this practice is that by pushing the project participants to plan at the fine-grained level, specific development activities can be far more effectively identified, planned and tracked.

Successful implementation of the practice of establishing “Binary Quality Gates at the Inch-pebble Level” can result in:

 

·         Improved estimates for cost and schedule over time

·         Improved quality

·         Improved accuracy in project tracking

·         Reduced risk

 

 

DESCRIPTION OF THE PRACTICE:

 

SUMMARY DESCRIPTION

The practice of implementing binary quality gates at the inch-pebble level involves two stages of activities, (1) planning, and (2) monitoring, which are summarized in the figure below.

The general premise of this practice is that by pushing the project participants to plan at the fine-grained level, specific development activities can be far more effectively identified, planned and tracked [Niwot, 1999].

The key elements of the “Binary Quality Gates at the Inch-Pebble Level” practice are:

·         Decompose tasks to the inch-pebble level

·         Verify that tasks produce a tangible product

·         Establish binary quality gates

·         Map tasks to quality goals and/or other requirements

·         Verify inch-pebble definitions for tasks on the critical path

·         Map quality gates to the Work Breakdown Structure

·         Use inch-pebble task definition for resource and schedule planning

 

DETAILED DESCRIPTION

This practice addresses a problem that has been prevalent in software projects for decades.  The significance of the problem is proportional to the size and complexity of the project.  The problem is often referred to as the 90% syndrome (sometimes called the 80% syndrome).  Figure 1 illustrates the problem in terms of a graph typically used to monitor progress on a project at a high level.  Resources are allocated and a schedule derived based on high-level project plans.  Development progresses and is reported in terms of actual expenditure of resources against planned expenditure, along with an overlay of the estimated % completion of the effort.  A crisis arises when the project nears completion in terms of cost and schedule, but actual completion of the software seems to be stalled at 90%.  Stakeholders at this point begin to question the validity of reporting progress as “% complete” – but it is often too late in the program to compensate for the inaccuracy.  Developers often resort to unrealistic schedules (and sometimes compromise quality) in order to complete the project in the designated time frame, or, cost and schedule overruns result.  Additionally, the project data itself (that could be used for improving estimates on future projects) is of little value for estimating future efforts.  The remaining “10%” of the project often is closer to 50% in reality.  This is traumatic for a project because most of the resources have already been used up and the project is close to its deadline for completion.  It results in a “lose/lose” situation for both the developer and the acquirer.

Figure 1:  Graphical Illustration of the “90% Syndrome”

 

The apparent “leveling” of effort at (or near) the 90% stage occurs:

(1)     Because the original estimates of % complete were inaccurate

(2)     Initial task planning did not include enough granularity to allow adequate estimating of the effort and schedule required to complete the task

(3)     Both the planning estimates and the monitoring estimates are based on subjective assessment of the developer with little or no quantitative basis

Discussions on project status are “illusionary”.

The practice of implementing binary quality gates at the inch-pebble level involves two stages of activities, (1) planning, and (2) monitoring, which are summarized in Figure 2.

The general premise of this practice is that by pushing the project participants to plan at the fine-grained level, specific development activities can be far more effectively identified, planned and tracked [Niwot, 1999].

Project Planning Activities

Decompose tasks to the inch-pebble level: The concept of “inch-pebbles” has been presented in a variety of definitions in the literature.  Some authors purport that an inch-pebble is a task (or sub-task) that can be completed in a day.  Others describe an inch-pebble in more relative terms, indicating that as a general rule it should be not more than 5% of the project duration and that at least 95% of the task can be performed by a single “lowest level” organization.  For a 50 week project, an inch-pebble is, therefore, a task that can be completed in a 2 to 3 week period by a single entity.

The essence of this step in the practice is best captured in the phrase “no longer than an inch, no heavier than a pebble”.  Decompose tasks into a level of granularity that removes uncertainty and ambiguity, and reduces complexity of the overall task to something manageable, thus accommodating specification of a realistic schedule for completion.  Breaking large tasks into multiple small tasks helps one estimate them more accurately, and often reveals work activities one might not have thought of otherwise.  It also facilitates project status tracking.

Verify that inch-pebble tasks produce a tangible product that is necessary for a required deliverable: Keep the focus on what has to be built/delivered, the functionality and quality of the software itself, not project fringe activity.  Not every task on a project needs to be decomposed to the inch-pebble level.  The purpose of this step of the practice is to ensure accurate planning and reporting in order to reduce risk.  Focus this practice on areas that have risk.

Establish Binary Quality Gates: Establish objective acceptance criteria and tests for determining that the output of a scheduled task is acceptable and the task is, therefore, complete.  This ensures that all participants are clear on the success factors and also promotes accountability for completion.  Although multiple criteria may exist for a task, the task is not considered complete unless all of the criteria are met.  Thus, task completion is a binary state, complete or not complete.  The concept of partial completion is not acceptable in this framework.  It is extremely important to define and communicate what the quality gates are before the start of the task (inch-pebble) so that developers (participants) know what they will be held accountable for.

Verify that tasks address quality goals: Compare tasks against quality goals and other project requirements to ensure adequate coverage of all the requirements that are important to the customer.  Have all the goals been addressed?  Forgetting about an important goal might mean that tasks will need to be added to the project later (upon discovery) and this would impact both the reporting of project status and the schedule itself.  A good rule of thumb is that all tasks that are on the critical path should be decomposed to the inch-pebble level.  This reduces the risk of the project completion being delayed due to “unforeseen” tasks needing to be completed.

Verify that quality gates are compatible with the WBS: Quality gates specify the criteria for stating that a task is complete.  They must be aligned with the WBS so that participants understand what they are accountable for.  Quality gates serve as project mini-milestones.  A high level task is an aggregate of smaller tasks.  While upper management (or the customer) is primarily interested in project status from the high-level task perspective, the development team needs to focus on the specific tasks, not the aggregate.  The inch-pebble tasks must roll up to the higher level tasks and must be in synch with the milestones of the high-level tasks.

Use inch-pebble task definition to develop estimates of resources and schedule:  The primary reason for decomposing a task to the inch-pebble level is to get a better understanding of the task itself, to clarify what really has to be done, in order to better predict how long it will really take.  In some programs, cost and schedule have already been determined based on high-level task statements.  Using inch-pebbles to develop more realistic estimates will reveal where the problems are in the proposed schedule.  It is then important for management to communicate with the stakeholders regarding the cases where the proposed schedule is unrealistic.  Not addressing an unrealistic schedule is a significant project risk.  Inch-pebbles provide the managers with the information needed to negotiate actions to mitigate that risk.

Project Monitoring and Reporting Activities

Apply quality gates WITHOUT EXCEPTION to determine task completion:  Applying binary quality gates brings objectivity to the assessment process.  Quality gates are only valuable if enforced.  Adhering to the use of quality gates for tracking task completion facilitates project tracking overall and establishes greater credibility to the status reporting process.  Application focuses attention on schedule slippage and the source before it becomes a major problem.  Creating inch-pebbles and establishing quality gates is a lot of work.  It is wasted if the quality gates are not applied.

Track inch-pebble task completion:  Typically, there are many criteria that must be met in order to pass through the quality gate.  The use of checklists at this level helps to ensure that problems do not “slip through the cracks”.  What is deemed important is on the checklist.  Typically, individuals have been assigned responsibility for the various items and are held accountable for them.  Recording the assessment of each of the criteria, and accountability, work together to ensure built-in quality.  In contrast, when the criteria status is not documented, it is neither manageable, nor credible.  It is too easy to say, “I think that was done last week, or, I think Joe did that”.

Use binary quality gates as the basis for reporting status on larger aggregate tasks:  If a high-level task is broken out into ten inch-pebble tasks, and eight of the ten tasks are complete, then it is meaningful to report that the high level task is 80% complete, because there is some reasonable certainty that the remaining two tasks do, in fact, represent 20% of the high level task, and will be completed in a fixed predictable amount of time (because they are inch-pebbles).  Task monitoring at the inch-pebble level through binary quality gates provides the objectivity that allows for meaningful aggregation of task status at the higher level.  Depending on the project, an organization may also elect to apply the binary concept to the higher level tasks as well.  The task is reported not done until all inch-pebbles are completed.  Project milestones are essentially binary quality gates placed on a schedule.

When should the practice be implemented?

There is a school of thought that asserts that software development is a discovery process and that forcing planning at too detailed a level is counterproductive to the success of the project.  The controversy seems to center around when it is appropriate to implement this practice.

Royce [Royce, 1998] comments that this practice may be appropriate for the production stage of development, but not for the engineering stage.  He notes that taking this approach too early in the life-cycle, before the requirements and architecture have stabilized, uses valuable time and resources and typically requires that planning be re-baselined as major changes occur.

The extreme application of this practice could result in a micromanagement nightmare where excessive specification and tracking of inch-pebbles causes excessive meetings and endless man-hours tracking tasks.  When inch-pebbles become the deliverables, tracking almost overshadows the project itself.

Rothman [Rothman, 1999] comments that certain kinds of projects do not lend themselves immediately and totally to inch-pebbles:

§     Really big projects (50-100 people for 5 to 10 years):  In this scenario requirements inevitably change.  It makes sense to use inch-pebbles only for the current phase of the project to achieve major milestones and help manage customers’ and technical staffs’ expectations.

§     A research project:  Because you don’t always know where you are going it is difficult to create a specific plan for getting there.  Inch-pebbles are created when the research is transitioned out of the lab into a development/production environment.

§     A new product:  Enabling technology has not been defined or created.  In this environment, inch-pebble planning is used in the production phase of development, where the product is being finalized for delivery.

In all of the above scenarios the projects have the following attributes:

§     Project manager cannot completely define the projects in the beginning or even in the middle

§     Project manager may not know what the final deliverable will be

§     Completion status of the project is not critical because the schedule is not a major risk

The acquisition community can best support implementation of this practice by specifying that this practice be employed at the judgment of both the developer and the acquirer and applied to specific tasks, as warranted, to mitigate risk.  In contrast, a contract stating that the practice must be used unilaterally throughout the project, or requiring that inch-pebbles be defined as milestones (deliverables), can, in fact, have an adverse impact on the success of the project.

An Example of Quality Gates:

The quality gate approach can be seen as a set of predefined quality criteria that a software development project must meet in order to proceed from one stage of its lifecycle to the next.  Figure 3 shows a typical schema of quality gates.  [Note:  In the description of Gate 1, “Ops” refers to Operations; in the Gate 4 description, “SLA” refers to Service Level Agreements.]

Figure 3:  Typical Schema of Quality Gates [Charvat, 2003]

 

The subject practice is about developing such a schema for the tasks of a project at a much finer granularity.  Each gate essentially consists of a checklist of activities.  It always starts with the question “Did we get through the previous gate yet?”  The items in the checklist are defined as having a binary status.

Although this schema shows the gates as sequential, some gates may be concurrent, each depending on a previous gate.  Beyond having a beginning and an ending gate, a strict sequential order is not required.  The analysis that occurs in mapping the quality gates contributes to improved project design.

 

CHARACTERISTICS OF IMPLEMENTATION:

 

SUMMARY CHARACTERISTICS

 

NO DATA CURRENTLY AVAILABLE

 

ANTICIPATED BENEFITS OF IMPLEMENTATION:

 

Successful implementation of the practice of establishing “Binary Quality Gates at the Inch-pebble Level” can result in:

 

·         Improved estimates for cost and schedule over time

Defining inch-pebbles allows one to better understand how long a task or phase will take, and what the exit criteria will be.  This, in turn, provides the information that allows more accurate estimating of the resources and schedule needed to complete the effort, and provides more accurate information for use in estimates of future projects.

·         Improved quality

Project managers continuously communicate the process and build quality directly into the project.  It increases the focus on a well-designed product.

·         Improved accuracy in project tracking

Because one defines the tasks in small increments, and a task is either complete or not, there is no need for continual status and task checking, or for micro-management of the project.  Project status is obvious at any time in the project. 

·         Reduced risk

Use of inch-pebbles and quality gates (with its inherent phase-by-phase checklist approach) reduces the risk of missing a project schedule due to missing, misunderstood, or incomplete tasks.  Often it reduces development time by getting it done right the first time.

 

DETAILED CHARACTERISTICS

 

Key Characteristics of the “Binary Quality Gates at the Inch-pebble Level” Gold Practice

Characteristic	Comments
Applies to the sub task or project level	Several hundred inch-pebbles may be needed, even for a small project.  This level of granularity is needed only to clarify specific development tasks, and to provide a binary metric for representing the status of task completion in an objective way that facilitates aggregation to represent the status of the project as a whole.
Aids in developing estimates	The technical staff defines inch-pebbles to clarify what is necessary to complete each task.  They tend to be accurate at this level of granularity.  Estimates are derived from aggregate counts of inch-pebbles. Estimating is a matter of calculating (# of inch-pebbles defined multiplied by the inch-pebble duration) Over time, technical staff who plan at the inch-pebble level become good estimators of their effort
Objective assessment of completion status	Use of binary quality gates at the inch-pebble level minimizes the subjectivity in determining completion status Task is complete only when all of the established criteria are met Project status can be objectively measured by counting the number of inch-pebbles completed and remaining
Must be built into the project	Completion criteria (for quality gates) must be established during planning, prior to development Best suited for enterprises that have the desire to instill a quality approach to the way they manage projects Needs to be driven from the top down for all IT projects Quality gates need to be integrated with both the development and deployment processes of your IT project
Need tracking tools for implementation	Quantity of inch-pebbles necessitates having some tools to facilitate monitoring and tracking; otherwise, the tracking becomes a quagmire.
Uses formal checklists throughout the project	Formal checklists are used throughout the life of a project Checklists can be revised as needed, but the process is documented Using checklists reduces the variability that can occur with quality assurance activities
Formal acceptance and sign-off at each gate	Formal sign-off and acceptance occurs at each gate Responsible stakeholders are identified when the gate is established.  They know up front what is expected.
Quality and integrity  assessment always occurs	Assessment of the quality and integrity of the product takes place If the gate is established and applied without exception, then the assessment occurs.  It will be as good as the plan.
Ensures good communication	Information is communicated to the correct stakeholders (i.e., deployment hands off to operations, etc.) because both the type of information and the recipients are identified in the checklists All phase transitions are good candidates for quality gates

 

 

RELATIONSHIPS TO OTHER PRACTICES:

 

The Figure below represents a high-level process architecture for the subject practice, depicting relationships among this practice and the nature of the influences on the practice (describing how other practices might relate to this practice).  These relationship statements are based on definitions of specific “best practices” found in the literature and the notion that the successful implementation of practices may “influence” (or are influenced by) the ability to successfully implement other practices.  A brief description of these influences is included in the table below.

 

Process Architecture for the "Binary Quality Gates at the Inch-Pebble Level" Gold Practice

 

Summary of Relationship Factors

INPUTS TO THE PRACTICE
Identify and plan for needed quality gates	Establishing binary quality gates at the inch-pebble level is a strategy to achieve various quality goals/requirements of the project.  Performance-Based Specifications, Goal-Question-Metric Approach, and Manage Requirements practices address the clarification of requirements that drive project planning, and aid the planner in deciding where quality gates are needed, and what criteria should be established.   Establish Clear Goals and Decision Points is a practice recommended by the Defense Science Board (DSB) to be applied at the program level in order to align program plans with strategic defense planning.  It addresses establishing cost, performance, and schedule goals, and major decision points (e.g. milestones, decision reviews, and interim progress reviews).  Establishing binary quality gates accomplishes the same thing, but is operative at the micro level rather than the macro level.  The practices share the essential concepts.  Binary quality gates might be applied within a project to ensure meeting a milestone that has been identified for a program.
Establish a structure for capturing project artifacts	Establishing binary quality gates “at the inch-pebble level” means that extensive quantities of task descriptions need to be organized, and then tracked through completion.  Configuration Management (CM) practices provide the planning and organization necessary to manage the development and use of the volume of project artifacts generated as a result of implementing the quality gate/inch-pebble approach.  In some cases the CM strategy can automate some aspects of reporting task completion based on pre-defined quality gates.
Prepare cost and schedule estimates	Metrics-Based Scheduling and Management supports binary quality gates and defining inch-pebbles because it is concerned with maintaining statistical quality control of costs and schedules.  This requires early calculation of size metrics, projection of costs and schedules from empirical patterns, and tracking of project status through the use of captured result metrics.  This practice is fact/data oriented and as such creates the need for accurate estimates of time and resources for the tasks that comprise a project.  It establishes a culture in which decomposing tasks to the inch-pebble level is viewed as valuable if it results in more accurate objective estimates that can be aggregated to provide better cost and schedule estimates.
OUTPUTS FROM THE PRACTICE
Apply quality gate exit criteria	Practices such as “Demonstration-Based Reviews”, Model-Based Testing” and “Statistical Process Control” may be used to meet the exit criteria established for a quality gate.  For example, a quality gate may be established relative to the work product achieving a particular level of performance.  Part of the criteria might be that “X” number of test cases generated from the test model all pass.  A quality gate relating to the integration process might specify that a smoke test be performed daily during the integration phase.  A quality gate relating to releasing code for a specific revision of a software product might require that a formal inspection be conducted.  In all cases, the criteria are inch-pebble tasks that can be given a binary status (such as pass/fail, or done/not done), bringing objectivity to the assessment of the quality gate.
Monitor and track progress	Defining binary quality gates and tracking at the inch-pebble level provides objectivity in monitoring and tracking progress.  They simplify determining when to credit a task’s earned value.  In some cases, the earned value can be credited automatically based on the inch-pebble tracking because of the binary status and the defined/documented relationship of inch-pebble tasks to their parent task.  Adherence to tracking task status at the binary level both clarifies and facilitates the automation of defect tracking.   By breaking the work down into inch-pebbles we create a time metric that can easily be used for more accurate reporting of project status.  For example, if task A (within a project) breaks out into 40 inch-pebbles, and we have consistently defined our inch-pebble duration to be two weeks, then we can reasonably estimate the task will require about 80 person weeks. If task A has 4 inch-pebbles and task B has 16, and we have competed task A, we know we are not 50% complete, we are only 20% complete.  The real value comes as developers improve their skills at defining inch-pebbles consistently.

RESOURCES:        

§        Websites

·         Software Program Managers Network web site for 16 Critical Software Practices for Performance-based Management.  Note: The activities of this practice have been folded into practices in the “Project Integrity” grouping on the SPMN web site.  The specific practice, as defined in this document, is no longer identified by name on the SPMN site.  There does not seem to be a web site that specifically identifies this practice.  http://www.spmn.com/16CSP.html

§        Tools and Methods

State-of-the-art methods and tools that may be useful in implementing and improving the effectiveness of “Binary Quality Gates at the Inch-pebble Level” include:

 

·         Integrated project management tools

Such tools would allow the creation of a sophisticated Work Breakdown Structure (WBS) that can be used to align tasks with requirements, document and track tasks at the inch-pebble level, and then generate reports on aggregate task status.  Such tools should also link the tasks to resources and schedule.  One example of such a tool is Microsoft Project.

·         Mind mapping tools

These tools assist the developer in organizing their thinking with visual maps of their thoughts.  The tools typically function from a star tree model allowing the user to create branches and then move the branches around to reflect solidifying their ideas.  The tools are open-ended, allowing the user to use as many levels as it takes to get to the inch-pebble granularity.  Two such tools currently on the market are Mind Manager, and Star Tree Studio. Some of these tools have the capability to export their data to project management tools and presentation tools.

“The Little Yellow Book of Software Management Questions”, SPMN, 1997, contains a list of questions relating to this practice that can assist the user in assessing the degree to which the practice is being implemented effectively on their project. You must register with the SPMN in order to download this book. From this product page click on GUIDEBOOKS and follow the directions for registering and accessing this and other product offerings.

http://www.spmn.com/products.html

Microsoft Project is a tool that can be used to develop the task decomposition (also known as WBS) to inch-pebble level granularity. The tool also provides for tracking these tasks and automating the update of quality gate status. MS Project is one of several tools developed by Microsoft that is functionally integrated with MS Office, but is packaged as a standalone tool.

§        Experts/Contact Points

§        Jason P. Charvat is an accomplished project management consultant with extensive international experience in information technology projects.  Jay is a senior manager for RCG Information Technology, a national IT professional services company based in Edison N.J., specializing in IT design, development, integration, and project management.  He is the author of Project Management Nation and Project Management Methodologies.  He can be reached at jcharvat@rcgit.com.

§        Norman Brown, past Director, Software Program Managers Network (SPMN), now with Quadrant-1.  POC information is as follows:

Email: norm@quadrant-1.com

Mailing address:           

Quadrant-1

2510 N. Nelson St.

Arlington, VA 22207

 

·         Johanna Rothman, President, Rothman Consulting Group, Inc., has written articles about defining and using inch-pebbles, and provides consulting services as well.  POC information is as follows:

Johanna Rothman

781-641-4046

jr@jrothman.com

http://www.jrothman.com/

 

Mailing address:

38 Bonad Rd.

Arlington, MA 02476

 

§        Training Opportunities:

·         Software Quality Consulting, Inc. offers a variety of courses that address project planning and tracking activity although currently there is no course specifically dedicated to this practice.

http://www.swqual.com/seminars/courses.html

·         Rothman Consulting, Inc. provides consulting services and customized training on inch-pebbles and other project planning and management skills.  http://www.jrothman.com/

 

§        Bibliography:

[Charvat, 2003]	Charvat, Jason P., “How to Use Quality Gates to Guide IT Projects”, Builder.com, 03 Feb. 2003 http://www.builderau.com.au/manage/project/print.htm?TYPE=story&AT=20271712-39024668t-20000988c
[Christensen, 2001]	Christensen, Mark J., Thayer, Richard H., “The Project Manager’s Guide to Software Engineering Best Practices”, IEEE Computer Society, 2001.ISBN 0-7695-1199-6
[CMU/SEI-2002-TR-012]	“Capability Maturity Model Integration (CMMISM) for Systems Engineering, Software Engineering, Integrated Product and Process Development, and Supplier Sourcing, (CMMI-SE/SW/IPPD/SS, V1.1), Staged Representation”, Software Engineering Institute, March 2002 http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr012.pdf
[DOD 5000.2, 2003]	DoD Instruction 5000.2, “ Operation of the Defense Acquisition System”, May 2003 http://dod5000.dau.mil/
[DODD 5000.2, 2002]	DoD 5000.2-R, “Mandatory Procedures for Major Defense Acquisition Programs (MDAPS) and Major Automated Information System (MAIS) Acquisition Programs”, 5 April 2002 http://www.acq.osd.mil/ar/doc/020405.Regulation.pdf
[INTERIM, 2002]	“Interim Defense Acquisition Guidebook”, 30 October 2002 (Replaces DoD 5000.2-R, canceled 30 October 2002)
[McGrath, 1998]	McGrath, Frank; “More Best Practices – But Not a Checklist”, STC98 – SPMN Track, 1998 http://www.stc-online.org/cd-rom/1998/slides/t1fmcgra.PDF
[Niwot, 1999]	“Nine Best Practices: The Software Management Framework”, Niwot Ridge Consulting, June 1999 http://www.niwotridge.com/PDFs/NineBestPractices.pdf
[Rothman, 1999]	Rothman, Johanna, “How to Use Inch-Pebbles When You Think You Can’t”, web site, Rothman consulting Group, Inc. http://www.jrothman.com/Papers/Howinch-pebbles.html
[Rothman, 2002]	Rothman, Johanna, “Release Criteria: Is This Software Done?”, STQE Magazine, March/April 2002 http://www.jrothman.com/Papers/releasecriteria.html
[Royce, 1998]	Royce, Walker, “Software Project Management: A Unified Framework”, Addison-Wesley, 1998
[Royce, 2001]	“Improving Software Development Economics”,  The Rational Edge, May 2001
[SPMN, 1997]	“The Little Yellow Book of Software Management Questions”, Software Program Managers Network, April 1997
[SPMN, 1998]	“The Program Managers Guide to Software Acquisition Best Practices”, Software Program Managers Network, April 1998 http://www.spmn.com/products.html Registration required in order to download this book. From this product page click on GUIDEBOOKS and follow the directions for registering and accessing this and other product offerings.
[Turner, 2002]	Turner, R.G., “Implementation of Best Practices in U.S. Department of Defense Software-Intensive System Acquisitions”, Ph.D. Dissertation, George Washington University, 31 January 2002

 

 

APPENDICES

DEFINITIONS:

 

Using binary quality gates at the inch-pebble level is the practice of establishing acceptance/completion criteria for the lowest level tasks (detailed tasks of short duration) and then tracking task completion at the binary level.  The task is considered complete only when all of the established criteria have been met.  The binary indication is meeting the predefined criteria (or not).  No partial credit is given.  Criteria may take the form of gates that assess either the quality of the products produced, or the adequacy and completeness of the finished process.  Some samples of gates are: technical reviews, completion of a specific set of tests that integrate or qualify software components, demonstrations, or project audits.  Quality gates can be applied at any time during the project, including solicitation.

[SPMN, 1998]

Monitoring the status of a project by tracking binary completion of relatively small tasks.  Activities are either incomplete or complete.  This is to prevent the “80% complete” syndrome where the estimated completion figure is reported without particular rigor.

-          [Turner, 2002]

SOURCES (Origins of the Practice):

 

·         Software Program Managers Network (SPMN), Airlie Software Council, 1995

This practice is one of the original nine principal best practices identified by the Airlie Software Council in 1995, which was comprised of software experts and industry leaders.  The council claimed that, if implemented, these practices would improve software development and maintenance productivity and quality, reduce cost, and improve user satisfaction.

RECOMMENDING SOURCES:

·         Defense Acquisition Deskbook, Feb. 2002, Frontline Wisdom & Advice

§     Air Force Material Command (afmc) - Binary Quality Gates (Entry/Exit Criteria)

Scope: Successful programs establish means to unambiguously assess the quality of products produced or the adequacy and completeness of the finished process.

Description: Completion of each task in the lowest-level tasking or activity network or work breakdown structure needs to be defined by an objective binary indication.  These completion events should be in the form of gates that assess either the quality of the products produced, or the adequacy and completeness of the finished process.  When planning and project monitoring is not based on sufficient detail, any picture of where the program is and how it is progressing may be judgmental and thus inaccurate.  The use of binary quality gates, applied to a predefined plan and criteria, allow the value earned in terms of task completion to be unambiguously determined.  That is, either a task is fully complete or no credit is taken.  To apply this approach without injecting significant lags in program status reporting, lower level tasks must be of short duration, expend only a small percentage of total program budget, and have objective evidence of successful completion.  To the extent possible, the "effort-expended" and "results-achieved" indicators should be the same data used to implement company cost and schedule control.

·         Interim Defense Acquisition Guidebook, 30 October 2002 [INTERIM 2002]

C5.2.3.5.6  Software Management.  The Program Manager (PM) shall manage and engineer software intensive systems using best processes and practices known to reduce costs, schedule, and performance risks.

·         Capability Maturity Model Integration (CMMI), Version 1.1 Staged, Software Engineering Institute, CMU/SEI-2002-TR-012, March 2002

While not specifically addressing the “Binary Quality Gates at the Inch-pebble Level” by name, the CMMI documents the necessary activity under several of its key process areas (KPA).

Pages 97–98  “Project Planning” KPA.  Relative to estimating the scope of a project, the CMMI directs the planner to “Establish a top-level work breakdown structure (WBS).  The WBS evolves with the project.  Initially a top-level WBS can serve to structure the initial estimating.  The development of a WBS divides the overall project into an interconnected set of manageable components.”  Specific sub practices include:

 

1.       Develop a WBS based on the product architecture

2.       Identify the work packages in sufficient detail to specify estimates of project tasks, responsibilities, and schedule.  The top-level WBS is intended to help in gauging the project work effort in terms of tasks and organizational roles and responsibilities.  The amount of detail in the WBS at this more detailed level helps in developing realistic schedules, thereby minimizing the need for management reserve.

Pages 119-120  Generic Practice (GP) 2.8.  Monitor and control the project planning process against the plan for performing the process and take appropriate corrective action.

GP 2.9.  Objectively evaluate adherence of the project planning process against its process description, standards, and procedures, and address noncompliance.

Pages 178–181  “Process and Product Quality Assurance” KPA.  This area addresses objectively evaluating the designated performed processes, work products and services against the applicable process descriptions, standards, and procedures, communicating quality issues and ensuring resolution of noncompliance issues with the staff and managers.

Sub practices include:

1.       Promote an environment (created as part of project management) that encourages employee participation in identifying and reporting quality issues

 

2.       Establish and maintain clearly stated criteria for the evaluations

 

3.       Use the stated criteria to evaluate work products and performed processes for adherence to process descriptions, standards, and procedures

 

4.       Identify each noncompliance found during the evaluation

 

5.       Identify lessons learned that could improve processes for future products and services

 

GLOSSARY

 

 

CASE STUDIES FROM THE LITERATURE:

 

To date, the DACS has not found any case studies that specifically address the practice of using quality gates at the inch-pebble level.  Typically, the practice is addressed as part of a broader Earned Value Management (EVM) approach, or in the context of general project planning strategy.  Case studies that focus on the effectiveness of EVM have not singled out the use of binary quality gates or planning at the inch-pebble level.  They are simply viewed as characteristics of EVM.

Rothman [Rothman, 1999] describes a scenario in which the implementer (planner) estimated the time to develop a module as 14 days, and then went back and identified the inch-pebble tasks associated with the module development.  The total duration of inch-pebble tasks was 28 days, twice as long as the original estimate.  He had simply forgotten some tasks in his original planning.  Imagine the impact of this estimate variance for a large project with hundreds of modules, and the misleading picture it might present.