FORMAL RISK MANAGEMENT
Definition and
Summary: Software Risk
Management is a
proactive approach for minimizing the uncertainty and potential loss associated
with a project. A risk is an event or condition that, if it occurs, has a
positive or negative effect on a project’s objectives. The three common
characteristics of risk are (1) it represents a future event, (2) it has a
probability of occurring of greater than 0%, but less than 100%, and (3) the
consequence of the risk must be unexpected or unplanned for. Future events can
be categorized as opportunity-focused (positive risk) if their consequences are
favorable, or as threat-focused (negative risk) if their consequences are unfavorable.
“If you can’t afford to mitigate the risk now, be absolutely
sure that you can afford to resolve the problem later when it happens.” [Universal, 2002]
Providing insights to
support informed decision making is the primary objective of Risk Management. In
practice, Risk Management concentrates on performing bottom-up, detailed,
continuous assessment of risk and opportunity. It focuses on addressing the
day-to-day operational risks that a program faces. Risk Management follows a
two-stage, repeatable and iterative process of assessment (i.e., the
identification, estimation and evaluation of the risks confronting a program)
and management (i.e., the planning for, monitoring of, and controlling of the
means to eliminate or reduce the likelihood or consequences of the risks
discovered). It is performed continually over the life of a program, from
initiation to retirement.
Effective implementation of formal risk management is based
on the following set of benefits resulting from the process:
§
Appropriately
tailored risk management strategies are defined and implemented
§
Potential
problems (risks) that could impact project success are identified
§
The likelihood
and consequences of these risks are understood
§
A priority order
in which risks should be addressed is established
§
Mitigation
alternatives appropriate for each potential problem are carefully considered
based on project circumstances
§
Optimized
mitigation techniques for all risks above their thresholds are selected
§
Contingency
plans in case the risk mitigates are developed proactively, rather than as a
result of fire-fighting
§
Information to
improve risk management policies is captured, analyzed and acted upon
§
Risk management
processes/procedures are systematically and periodically reviewed and improved
to further reduce risk
SUMMARY
DESCRIPTION
There are a
variety of risks that confront the global software industry, as illustrated in
the figure below. The characteristics of the legal, social, economic and
competitive environments impose constraints and opportunities that help to
define the nature of the risks (and their exposure levels) for suppliers,
buyers, and other stakeholders in the software acquisition and development
process.
Rather than
meeting risks head on, however, and despite increased efforts to promote sound
risk management processes, the software industry, as a whole, tends to be in
denial about whether risks exist on their programs and, if they do, how they
should go about addressing them. The figure below, adapted from the Software
Program Manager’s Network (SPMN) “The Little Book of Bad Excuses” [SPMN, 1998] highlights
some of the most common traps that all too many programs have fallen into over
the years, and continue to fall into today. As a result, symptoms that
organizations are not effectively practicing risk management continue to
flourish, including a continual state of project instability, constant
fire-fighting, multiple schedule slippages because of recurring surprise
factors, and constantly operating in a high-stress, crisis management
environment.
(click to enlarge) |
A formal
risk management process is a continuous process for systematically addressing
risk throughout the product/project life-cycle. Risks can be introduced (or
latently reside) at the very earliest stages of the software life-cycle. The
ability to identify risks earlier translates into earlier risk removal, at less
cost, which promotes higher project success probability.
DETAILED
DESCRIPTION
INTRODUCTION
Providing insights to
support informed decision making is the primary objective of Risk Management. In
practice, Risk Management concentrates on performing bottom-up, detailed,
continuous assessment of risk and opportunity. It focuses on addressing the
day-to-day operational risks that a program faces. Risk Management follows a
two-stage, repeatable and iterative process of assessment (i.e., the
identification, estimation and evaluation of the risks confronting a program)
and management (i.e., the planning for, monitoring of, and controlling of the
means to eliminate or reduce the likelihood or consequences of the risks
discovered). It is performed continually over the life of a program, from initiation
to retirement.
There are a variety of risks that confront the global software industry, as
illustrated in Figure 1 [McManus,
2004], which will be discussed in more detail. The characteristics
of the legal, social, economic and competitive environments impose constraints and
opportunities that help to define the nature of the risks (and their exposure
levels) for suppliers, buyers, and other stakeholders in the software
acquisition and development process.
Figure 1: Risk and the Global Software Industry [McManus, 2004]
Rather than
meeting them head on, however, and despite increased efforts to promote sound
risk management processes, the software industry, as a whole, tends to be in
denial about whether risks exist on their programs and, if they do, how they
should go about addressing them. Figure 2, adapted from the Software Program
Manager’s Network (SPMN) “The Little Book of Bad Excuses” [SPMN, 1998] highlights
some of the most common traps that all too many programs have fallen into over
the years, and continue to fall into today. As a result, symptoms that
organizations are not effectively practicing risk management continue to flourish,
including a continual state of project instability, constant fire-fighting,
multiple schedule slippages because of recurring surprise factors, and
constantly operating in a high-stress, crisis management environment.
Figure
2 (click to enlarge):
The Stubborn Face of Risk Denial (adapted from The Little Book of Bad
Excuses) [SPMN, 1998]
There are a
number of good resources that helped to initially define and add rigor to
software risk management approaches. Capers Jones provided some historical
perspective on the evolution of software risk assessment processes through the
evolution of Software Productivity Research (SPR) activities into key process
areas of the Software Engineering Institute (SEI) Capability Maturity Model
(CMM). Additional rigor was added to the software risk management approach
through the contributions of Barry Boehm (“Software Risk Management” [Boehm, 1989]) and
Robert N. Charette (“Software Engineering Risk Analysis and Management” [Charette, 1989] and
“Application Strategies for Risk Analysis” [Charette, 1991]).
There are
numerous reasons as to why formal risk management processes have not been
implemented more widely, however, or enjoyed more success. Some of the more
interesting suggestions have been made by Kontio [Kontio, 2002], who suggests that the success
of the business market (before the dot.com bubble burst) coupled with the
expected rigor of a formal risk management process was felt to be, simply, not
worth the investment in time or money (see Table 1).
Table 1: Reasons for Lack of Progress in Software Risk
Management [Kontio, 2002]
|
Reasons for Lack of Progress
|
Proposed New Agenda
|
|
The .com boom was a false perception of success:
Companies were not concerned about risks; only
opportunities. Investors were uninterested in conservative, low-risk
investments.
|
Business attitudes in software companies need to take
both risk and opportunities into account
|
|
The wrong risk management techniques were being used:
Simple techniques (checklists, scoring tables,
complex calculation tools) deployed in 1990’s were not necessarily based on
sound theories, good assumptions, or accurate mathematics. Industry
practitioners may have felt that spending time on risk management was not
cost-effective.
|
Biased and unsound software risk management methods
should be replaced by better methods
|
|
Risk management was too bureaucratic:
Numerous report templates and standards were
defined, and many lengthy reports produced, but not necessarily read or acted
upon. Therefore, risk management created high overhead cost while producing
little of value.
|
Software risk management needs to be made cost
effective and easy, while still based on sound principles
|
|
Risk management was too standard:
Many risk management approaches/tools aimed to
make their use easier by standardizing on some aspects of their models.
Risks often valued only through money, calendar or effort impacts. Similar
checklists are applied to large defense contracts and small start-up
companies. Realities are much more complex than this.
|
Software risk management needs to be application or
situation specific. There are no “silver bullets” or “one size fits all”
solutions.
|
|
Risk management was not transparent:
Many
risk management methods/tools use complex calculations and forms to
document/analyze risk information. Little attention was paid to make sure
all decision-makers understood the process and results. Since they didn’t
understand them, they didn’t have confidence in them and could not
communicate the results well.
|
Risk management methods must emphasize communication
and understandability of results
|
|
Researchers were too theoretical and consultants too
naïve:
Researchers
ignored practical implications or constraints. Lack of interdisciplinary
research (risk management methods not leveraged from psychologists,
economists, civil engineers, etc.). Consultants have used/reused the same
old simple methods without true concern about their serious limitations.
|
Researchers need to provide more practice results.
Consultants need to upgrade their knowledge and gain a deeper understanding
of methods/tools.
|
The
literature has, however, reported some success in risk management’s effect on
projects. In one study conducted by the Project Management Institute (PMI) [Voetsch, 2004], the
underlying rationale was to gain an appreciation of the role of risk in
relation to other project management success factors. In order to accomplish
this, the study used a population with a demonstrated sensitivity to risk,
namely PMI’s Risk Special Interest Group. Based on the 187 responses received
during the winter of 2002-2003, the research showed that risk management
success followed when adequate support and resources were applied to implement
it. Specifically, conclusions were drawn that:
·
Senior management’s sensitivity to risk increases the frequency
of risk management practices
·
Availability of adequate resources increases the use and
effectiveness of risk management tools
·
More frequent use of formal risk planning practices increases the
use and effectiveness of risk monitoring
·
Senior management support for up-front risk planning increases
the probability of project success
The
referenced report contains some relatively positive quantitative findings regarding
the use of risk management practices, but on the negative side:
·
Less than 5% of respondents reported “almost always” receiving
encouragement for risk-taking vs. 18% who “rarely” received such encouragement
·
Only 12% of respondents reported “almost always” receiving
adequate resources for risk management, while only 7% reported that project team
members “almost always” receive training in risk planning and impact analysis
·
Only 20% of respondents reported that they “almost always” used
formal risk reviews, and only 8% said that they used risk audits during their
projects
·
Only 21% of respondents reported that their projects are
completed according to their specifications or the original Statement of Work
(SOW)
·
Only 30% of respondents reported “almost always” completing
their projects on time, and only 26% reported completing them within budget
·
Only 40% of respondents reported that their projects “almost
always” are received with customer satisfaction upon delivery
Clearly,
there is significant progress that needs to be made in order to use risk
management techniques effectively.
The
Concept of Positive Risk
Although
there has been considerable debate regarding the proper definition of software
risk, the traditional consensus has been that risk always involves two
characteristics: uncertainty and loss. While uncertainty certainly still holds
true, there is a growing perception that risk can also involve opportunity as
well as loss.
Positive
risk refers to risk that we initiate ourselves because we see a potential
opportunity along with a potential for failure (the negative risk associated
with “loss” of the opportunity). There are several kinds of opportunities that
can be leveraged in projects if responses to them are well-timed and prompt
action is initiated [Kahkonen,
2001]. These include:
·
Business opportunities, e.g., product development, customer care
during the project life cycle, and focused attention on high profit margin
activities
·
Operational opportunities, e.g., value-added, do what is
important, minimize rework
·
Systemic opportunities, which typically mean long-term savings resulting
from improved safety, insurance, etc.
Two
examples of positive risk follow [Mochal, 2002]. Example 1 is a software development
project that is scheduled to take 90 days. The customer would prefer earlier
delivery, and would get more value from an earlier delivery, but understands and
concurs with the 90-day development period. Utilizing a new software-testing
tool might allow delivery in 60 days, but this is the first time that your
organization has used the tool (lack of expertise; learning curve required).
If the tool doesn’t work out, delivery might take 110 days, rather than 90.
In our
second example, it is estimated to take six months to complete a medium-sized software
development project. You may be able to complete the project sooner using
Extreme Programming techniques, delivering the first iteration in three months,
then monthly updates after that. There is a risk, however, that new techniques
will not work well, or that staff may resist the changes, causing a delay in
the project.
Although
most of the remaining discussion will focus on the traditional, negative
aspects of risk, it would be beneficial to think about how the techniques of
formal risk management presented here could be tailored to successfully
leverage positive risk.
As we get
further into our discussion on formal risk management processes and techniques,
keep in mind that there are always trade-offs that need to be considered in
terms of how much to invest in risk management versus the expected payback over
the total software project life-cycle. Figure 3 helps to illustrate this
point.
Figure
3: The Role of Risk in Software Affordability
This figure
illustrates the concept that a relative optimum point of investment exists for
deciding how much money and other resources should be applied to a formal risk
management program. Proceeding to the right on the x-axis represents
decreasing risk, and CTLCC represents the total software life-cycle
cost, CRR is the cost associated with investment in risk reduction
activities, and COM are the total operating and maintenance costs
associated with the ability to successfully reduce risk. Starting at the
origin, increasing investment in formal risk management has the impact of
decreasing the project risk while simultaneously decreasing the total
life-cycle cost of the software. Beyond our optimal point, however, continuing
to invest in risk reduction activities will still continue to reduce risk, but
the cost of doing so will exceed the degree of improvement experienced in
operating and maintenance costs. Even though we’re reducing risk, the total
life-cycle cost of the software will increase. These are the types of
decisions that must be made in determining the scope of a formal risk
management program, and they influence our choices in how we choose to handle
risk. Issues regarding risk avoidance, mitigation, transferal, etc., will be
discussed later, but there is no silver bullet that can guide the
decision-making process. Each project (and the needs of the business at
specific points in time and under certain market conditions) will most likely
require specifically tailored risk management solutions.
The Most Common/Serious
Software Risks
There are
numerous reasons as to why formal risk management is difficult to implement
effectively. These include the sheer number of risk factors that have been
identified in the literature. For example, Capers Jones assessed several
hundred organizations and observed over 100 risk factors (of which 60 he
discusses in detail in [Jones,
1994]). He observed, however, that few projects have more than 15
active risk factors at any one time, but many projects have approximately six
simultaneous risk factors.
Another reason
for the relatively low implementation of formal risk management methods in
practice are, according to [Kontio,
1998], the fact that risk is an abstract or fuzzy concept for which
users lack the necessary tools to more accurately define risk for a deeper
analysis. In addition, many risk management methods may be based on risk
quantification. Users may not have the ability to provide accurate estimates
for probability and loss/opportunity projections required for a reliable risk
analysis. Table-based approaches can sometimes be too biased or too coarse for
proper risk prioritization. Risks may also have different implications for
different stakeholders (or, conversely, be perceived differently by different
stakeholders). Existing risk management methods may not provide support for
dealing with these differences. Risks may also affect a project in more than
one way. For example, most risk management approaches focus on cost, schedule
or quality risks, but there may be combinations of risks or other
characteristics such as future required maintenance, company reputation, or
potential liability/litigation that should be considered important in
influencing the decision-making process. Finally, many current risk management
techniques may be perceived as too costly or too complex to use. Simple,
straightforward risk management techniques that require an acceptable amount of
time to produce results might be the answer.
So, what
kinds of risks are likely to be encountered in the acquisition and development
of software?
There are a
number of sources in the literature that expound upon the different types of
software risks and problems. References and, where available, Internet links
are included in the bibliography
section of this document. These references include [Jones, 1994], [Boehm, 1991], [McManus, 2004], [GAO, 2004a], [GSAM, 2000], [DAU, 2003], [STSC, 2003], [Buttigieg], [Kontio, 2002], and [Weigers, 1998].
Since there
is a great deal of overlap between them (although some have some slightly
different perspectives on potential software project risks) only a few of them
will be addressed in the body of this document, primarily because the
references provide some additional insight beyond just identifying the
potential software program risks. Readers should use these references as a
trigger for identifying risks, causes and potential solutions that should be
considered in support of their own projects.
Table 2 is
of particular interest because of the discussions of each risk in the context
of other potential related problems on the project, or in the larger
organization. It provides an interesting summary perspective on the concept of
compound risks. Jones book goes into much more detail regarding these risks.
An expanded version of this table has been prepared and made available by the
DACS at http://www.goldpractices.com/practices/frm/SoftwareRisks-Jones1994.doc.
Table 2: The Most Serious Software Risks [Jones, 1994]
|
Software
Risk
|
Risk
Characteristics
|
Possible Compounding Risks
|
|
Canceled Projects
|
A.
Software projects that are
terminated prior to delivery to their intended customers
|
·
Almost always associated with Friction
with Users and Friction with Senior Management
·
Strong association with Low
Employee Morale, Low Managerial Morale and High
Staff Attrition
·
Strongly associated with Lost
Business Opportunities and Low Market Shares
·
Canceled projects can be caused
by Inadequate Planning, Inadequate Cost Estimating,
Missed Schedules, Long Schedules, Excessive
Time to Market, Cost Overruns, Inexperienced
Management, Management Malpractice, Inexperienced
Clients, Creeping User Requirements, Inadequate
Development Processes, Low Productivity, and Low
Quality
|
|
Creeping User Requirements
|
A.
New requirements or significant
modifications to existing requirements that are made after a basic set of
requirements have been mutually agreed to
B.
Widespread failure to anticipate
changing requirements, with no plans for how to deal with them
|
·
Associated with Friction
with Users, and sometimes with Friction with Senior Management
·
Causative factors of Missed
Schedules, Excessive Time to Market, and Cost
Overruns
·
Contributing factor to Excessive
Schedule Pressure and Low Staff Morale
·
Creeping requirements can be
caused by Inexperienced Users, Inexperienced Management,
Inadequate Methodologies, Inadequate Cost Estimating
and Inadequate Measurement
|
|
Excessive Schedule Pressure
|
A.
Coercive demands by software
clients to deliver software applications in a shorter time period than is
technically achievable
B.
Coercive demands by software
executives or project managers to deliver code, specifications or
documentation in a shorter time period than is technically possible
|
·
Significant contributor to Low
Staff Morale and High Staff Turnover
·
Observed on a majority of Cancelled
Projects and those with Cost Overruns
·
Main contributor to Low
Quality
·
Excessive schedule pressure can
be caused by Creeping User Requirements, Inadequate
Estimating, Inadequate Measurement, Inadequate
Planning and Management Malpractice
|
|
Inaccurate Cost Estimating
|
A.
Automated estimating methods that
vary by >30% when calibrated against fully-measured control projects
B.
Automated partial estimating
methods that omit, ignore or fail to include >30% of the activities and
tasks associated with software projects unless specifically limited to
selected activities
C.
All forms of manual estimating
over 1000 Function Points
|
·
Associated with Cost
Overruns, Long Schedules, Missed Schedules,
Excessive Time to Market, Lost Business, Friction
with Senior Management, Friction with Users, Inaccurate
Quality Estimates, Inaccurate Reliability Estimates, Low
Quality and Cancelled Projects
·
Inaccurate cost estimates can be
caused by Corporate Politics, Creeping Requirements,
Inadequate Measurement, Inadequate Software Management
Curricula, Inadequate Software Engineering Curricula, Inexperienced
Management, Lag Time in Technology Adoption, Inaccurate
Metrics, Poor Support Structures and Poor
Technology Transfer
|
|
Inaccurate Metrics
|
A.
Common software metrics that
violate standard economic assumptions
B.
Common software metrics that
behave in paradoxical, counterintuitive or unpredictable ways
C.
“SLOC” and “KLOC” metrics when
used as general-purpose software quality or productivity metrics
D.
“Cost per defect” when used as an
indicator of quality costs
E.
“Software Science” metrics when
used as general productivity or quality indicators
F.
Ratios and percentages when used
as general productivity indicators
|
·
Associated with Poor
Technology Investment, False Productivity Claims, Inadequate
Cost Estimating and Inadequate Measurements
·
Causative factors to Missed
Schedules, Long Schedules, Low Productivity
and Cancelled Projects
·
Contributing factors to Friction
with Executives, Friction with Users, Friction
with Contractors and Low Management Morale
·
Characteristic (C) is associated
with Poor Technology Investments, False Productivity
Claims, Low Productivity, Inadequate Cost
Estimating and Inadequate Measurements
·
Characteristic (D) is associated
with Poor Technology Investments, Low Quality and
Inadequate Defect Removal
·
Inaccurate metrics can be caused
by Academic Malpractice, Management Malpractice, Inadequate
Management Curricula and Inadequate Standards
|
|
Inadequate Measurement
|
A.
Failure to record the basic and
special factors that influence software projects, coupled with either
inaccurate or missing quantitative data on size, staffing, schedules,
resources, costs and quality, also coupled with the use of paradoxical or
unsound metrics
|
·
Root cause of Low
Productivity, False Productivity Claims, Low
Quality, Long Schedules, Missed Schedules,
Excessive Time to Market, Inadequate Estimating, Friction
with Senior Management, Friction with Users, Low
User Satisfaction, Canceled Projects, Inadequate
Methodologies, Inadequate Tools, Poor
Organization Structures, Inadequate Specialization, Inadequate
Capital Investment and Inadequate Office Environments
·
Inadequate measurements can be
caused by Inadequate Software Management Curricula, Inadequate
Software Engineering Curricula, Inexperienced Management,
Lag Time in Technology Adoption, Inaccurate Metrics,
Poor Organization Structures, Poor Technology Transfer
and Poor Corporate Culture
|
|
Low Productivity
|
A.
Software projects that are
significantly below U.S. industry averages for their class, type and size
B.
Any project that averages less
than five Function Points per person month in net project productivity (the
“current” U.S. average – Note that this is the U.S average as of the 1994 publication date
of Caper Jones’ book)
C.
Any project perceived as being of
low productivity by its clients or by senior management
|
·
Basic contributor to Friction
with Users and Friction with Senior Management
·
Causative factor to False
Productivity Claims
·
May sometimes be a factor leading
to Canceled Projects and Excessive Time to Market
·
Low productivity can be caused by
Low Levels of Reusability, Inexperienced Management,
Inexperienced Staff, Inadequate Capital Investment,
Inadequate Office Environments, Inadequate Methodologies,
Inadequate Tools, Inadequate Measurements, Inadequate
Metrics and Short-Range Improvement Planning
|
|
Low Quality
|
A.
Software where significant
numbers of users are dissatisfied with quality and/or software receiving more
than 0.25 user defect reports per Function Point per year
|
·
Key contributor to Low User
Satisfaction and Friction with Users
·
Contributing factor to Low
Market Shares and Excessive Time to Market
·
Causative factor to Friction
with Senior Management, Low Staff Morale, Error-Prone
Modules and High Maintenance Costs
·
Often associated with Low
Productivity and Long Schedules
·
Low quality can be caused by Inexperienced
Management, Inexperienced Staff, Creeping User
Requirements, Inadequate Estimating, Inadequate
Planning, Inadequate Defect Removal, Inadequate
Measurements and Excessive Schedule Pressure
|
|
Malpractice (Project Management)
|
A.
Performing managerial functions
so poorly that harm/damage occurs to projects which are under the manager’s
scope of authority
B.
Failure to exert proper authority
to prevent harm/damage to projects which are under the manager’s scope of
authority
C.
Repeated/significant errors (more
than 35%) in performance of basic software management tasks of schedule planning
and cost estimating
D.
Repeated delivery of software
packages with excessive defect levels
E.
Failure to keep adequate
measurement records of projects produced under their authority
|
·
Significant contributor to High
Staff Attrition Rate and Low Staff Morale
·
Strongly associated with Canceled
Projects, Corporate Politics, Cost Overruns,
Excessive Schedule Pressure, Excessive Time to Market,
Friction with Users, Long Schedules, Low
Quality, Missed Schedules, Technical Malpractice
and Silver Bullet Syndrome
·
Management malpractice can be
caused by Inadequate Management Training, Inadequate
Career Planning, Excessive Schedule Pressure, Executive
Malpractice, Resistance to Change and Creeping
User Requirements
·
May be caused by Inadequate
Compensation Plans
|
|
Silver Bullet Syndrome
|
A.
The concept that a single tool or
methodology will be the “silver bullet” that cures software productivity or
quality problems
|
·
Significant contributor to Friction
with Users and Friction with Senior Management
·
Causative factor for Canceled
Projects, Excessive Time to Market, Long
Schedules and Cost Overruns
·
Contributes to High Staff
Attrition, Low Staff Morale, Low Productivity,
Low Quality and Short-Range Improvement Planning
|
Table 3 is
derived from the Guidelines for Successful
Acquisition and Management of Software-Intensive Systems (GSAM)”, Version 3.0 [GSAM, 2000]. It is included here due to its more direct focus on
the software acquisition process, and is noteworthy for its focus on the
technical aspects of a project, rather than being limited to cost, schedule and
organizational risks.
Table 3: Significant Acquisition Risks to Address in the
Acquisition Strategy [GSAM,
2000]
|
Risk Area
|
Significant Acquisition Program Risks
|
|
Threat
|
·
Uncertainty in threat accuracy
and stability
·
Sensitivity of design &
technology to threat
·
Vulnerability of system to threat
countermeasures
·
Vulnerability of program to
intelligence penetration
|
|
Requirements
|
·
Operational requirements not properly
established or vaguely stated for program phase
·
Requirements are not stable
·
Required operating environment
not described
·
Requirements do not address
logistics and suitability
·
Requirements are too constrictive
– identify specific solutions that force high cost
|
|
Design
|
·
Design implications not
sufficiently considered in concept exploration
·
System will not satisfy user
requirements
·
Mismatch of user manpower or
skill profiles with system design solution or human-machine interface
problems
·
Increased user skills or more
training requirements identified late in the acquisition process
·
Design not cost effective
·
Design relies on immature
technologies or excessive use of COTS to achieve performance objectives
|
|
Test & Evaluation
|
·
Test planning not initiated early
in program
·
Testing does not address the
ultimate operating environment
·
Test procedures do not address
all major performance and suitability specifications
·
Test facilities not available to
accomplish specific tests, especially system-level tests
·
Insufficient time to test
thoroughly
|
|
Modeling & Simulation
|
·
Same risks as those identified
for Test & Evaluation (T&E)
·
Modeling & Simulation (M&S)
not verified, validated or accredited for the intended purpose
|
|
Technology
|
·
Program depends on unproven
technology for success, or there are no alternatives
·
Program success depends on achieving
advances in state-of-the-art technology
·
Potential advances in technology
will result in less-than-optimal cost-effective system, or make system
components obsolete
·
Technology has not been
demonstrated in the required operating environment
·
Technology relies on complex
hardware, software or integration design
·
Program lacks proper tools and
M&S capability to assess alternatives
|
|
Logistics
|
·
Inadequate supportability late in
development or after fielding, resulting in need for engineering changes,
increased costs, and/or schedule delays
·
Life-cycle costs not accurate
because of poor logistics supportability analysis (LSA)
·
LSA results not included in
cost-performance trade-offs
·
Design trade studies do not
include supportability considerations
|
|
Development
|
·
Development implications not
considered during concept exploration
·
Development not sufficiently
considered during design
·
Inadequate planning for long-lead
items and vendor support
·
Development processes not proven
·
Prime contractors do not have
adequate plans for controlling subcontractors
·
Sufficient development tools not
readily available for cost-effective production
·
Contract offers no incentive to
upgrade tools, improve processes, or reduce costs
|
|
Concurrency
|
·
Immature or unproven technologies
will not be adequately developed prior to system production
·
Development funding will be
available too early, i.e., before the development effort has sufficiently
matured
·
Concurrency established without
clear understanding of risks
|
|
Developer Capability
|
·
Developer has limited experience
in specific type of development
·
Contractor has poor track record
relative to costs and schedule
·
Contractor experiences loss of
key personnel
·
Prime contractor relies heavily
on subcontractors or Commercial Off-The-Shelf (COTS) items for major development
efforts
·
Contractor will require
significant capitalization to meet program requirements
|
|
Cost/Funding
|
·
Realistic cost objectives not
established early
·
Marginal performance capabilities
incorporated at excessive costs; satisfactory cost-performance tradeoffs not
done
·
Excessive life-cycle costs due to
inadequate treatment of support requirements
·
Significant reliance on software
·
Funding profile does not match
acquisition strategy
·
Funding profile not stable from
budget cycle to budget cycle
|
|
Schedule
|
·
Schedule not considered in
tradeoff studies
·
Schedule does not reflect
realistic acquisition planning
·
Acquisition Program Baseline
(APB) schedule objectives not realistic and obtainable
·
Resources not available to meet
schedule
|
|
Management
|
·
Acquisition strategy does not
give adequate consideration to various essential elements, e.g., mission
need, test and evaluation, technology, etc.
·
Subordinate strategies and plans
are not developed in a timely manner or based on the acquisition strategy
·
Proper mix (experience, skills,
stability) of people not assigned to Program Management Office (PMO) or to
Contractor team
·
Effective risk assessments not
performed or results not understood and acted upon
|
Table 4,
although it reverts back to the more “traditional” high-level risk concerns of
cost, schedule and organization, is useful because it includes information
related to symptoms, potential root causes of the risk, quantitative leading
indicators that the risk has manifested, and generic mitigation techniques to
address the risk.
Table 4: Standard Risks for Project Failure [STSC, 2003]
