SOFTWARE ACQUISITION GOLD PRACTICE

ENSURE INTEROPERABILITY

 

FOCUS AREA:   TECHNICAL PERFORMANCE – Systems Engineering

 

Definition and Summary:  The ability of systems, units, or forces to provide services to (and accept services from) other systems, units, or forces and to use the services so exchanged to enable them to operate effectively together.

 

The definition of interoperability encompasses both a technical and an operational capability.  The technical capability (ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces) addresses issues of connectivity among systems, data and file exchange, networking, and other communication related scenarios.  The operational capability (ability of systems, units, or forces to use the services so exchanged to enable them to operate effectively together) addresses the degree to which value is derived from that technical capability.  Identifying technical requirements for interoperability is challenging but straightforward; ensuring “effectiveness” of the technical solution is much more complex because the operational environment in which effectiveness is assessed is a moving target.

Successful implementation of the Ensure Interoperability Gold Practice can result in:

§         Increased likelihood of actually achieving specific interoperability objectives

§         Significant savings across the interoperability domain

§         A migration plan for making legacy systems “interoperable”

 

 

DESCRIPTION OF THE PRACTICE:

 

SUMMARY DESCRIPTION

The definition of interoperability encompasses both a technical and an operational capability.  The technical capability (ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces) addresses issues of connectivity among systems, data and file exchange, networking, and other communication related scenarios.  The operational capability (ability of systems, units, or forces to use the services so exchanged to enable them to operate effectively together) addresses the degree to which value is derived from that technical capability.  Identifying technical requirements for interoperability is challenging but straightforward; ensuring “effectiveness” of the technical solution is much more complex because the operational environment in which effectiveness is assessed is a moving target.

The figure to the right presents the high-level activities that comprise the practice of ensuring interoperability.  The graphic model is used to symbolize the “un-ending” nature of the practice.  These high-level activities reflect the principles of a modern systems engineering approach to development – but the outcome is not a program, or system, or product.  The outcome is progress of a particular program, system, or project (or progress in collaboration among developers) in contributing to achieving or sustaining a defined state of interoperability identified as meeting a need in a specified environment at a particular point in time.   

 

The key concepts/steps of the “Ensure Interoperability” practice are:

 

·         Recognize the challenge

·         Understand the scope

·         Specify interoperability requirements

·         Plan a strategy for achieving interoperability

·         Implement the plan

·         Evaluate/assess progress toward achieving interoperability

 

DETAILED DESCRIPTION

The definition of interoperability encompasses both a technical and an operational capability.  The technical capability (ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces) addresses issues of connectivity among systems, data and file exchange, networking, and other communication related scenarios.  The operational capability (ability of systems, units, or forces to use the services so exchanged to enable them to operate effectively together) addresses the degree to which value is derived from that technical capability.  Identifying technical requirements for interoperability is challenging but straightforward; ensuring “effectiveness” of the technical solution is much more complex because the operational environment in which effectiveness is assessed is a moving target.

The need to address interoperability is stated everywhere in the literature.  Everyone says “Make it happen” but few are saying exactly what activities are necessary to “make it happen”.  This is due in part to the “vastness” and complexity of the issue and the fact that many different factions (people and organizations) have a role to play.  If you are a program manager responsible for a national security system your activities for ensuring interoperability are vastly different from the activities of a commercial developer making network components.  If you are an acquisition specialist responsible for contracting language your activities are different than those of the requirements analyst.  The practice of ensuring interoperability therefore involves recognizing the complexity and understanding the scope of the interoperability issue with respect to the particular system and your particular level of involvement before taking action.  This awareness establishes the framework for identifying all of the stakeholders and drives requirements definition, planning, and decision making for the remainder of the effort.  Because of the ever-changing operational environment over time, interoperability is never “done”.  There must be a process for assessing/evaluating the degree of interoperability among “systems, units, and forces” over time and making adjustments as the technology and operational needs change.  Although this practice is primarily aimed at military functions there is the notion that there are things that can be done in a general sense to ensure the ability of a system to interoperate with other (yet to be defined) systems in the future.  Particularly on the commercial side of the house, ensuring interoperability is about building products (components, applications, or systems) that have the greatest likelihood of being interoperable with other/future vendor products at some future point in time.  Ensuring interoperability in this scenario has a slightly different focus and corresponding activities.

Figure 1 presents the high-level activities that comprise the practice of ensuring interoperability.  The graphic model is used to symbolize the “un-ending” nature of the practice.  These high-level activities reflect the principles of a modern systems engineering approach to development – but the outcome is not a program, or system, or product.  The outcome is progress of a particular program, system, or project (or progress in collaboration among developers) in contributing to achieving or sustaining a defined state of interoperability identified as meeting a need in a specified environment at a particular point in time.   

 

Recognize the challenge!  Joint Vision 2010 and Joint Vision 2020 define a challenge and provide a roadmap for defense in the 21^st century.  Joint and international interoperability are crucial elements in realizing these visions. [DSP, 2001].  In a recent presentation at the Software Engineering Institute (SEI) Conference on Acquisition of Software Intensive Systems Mark Kasunic, from the SEI, described the challenge as follows [Kasunic, 2003]:

Interoperability is the number one problem in joint force & combined operations.  It is the CINC’s top issue^*.

The problem may be getting worse

·    Real-world operations, evaluations and exercises continue to highlight joint/combined warfighting capability shortfalls

·    As new coalition partners develop, complex systems are acquired, and “fixes” to past problems are applied in stove-piped fashion

·    Joint Vision 2010 and 2020 call for increasingly network-centric warfare, dependent upon fully interoperable systems

^*     As stated by Ms. Robin Quinlan, Deputy Director, Systems Interoperability, Office of the Secretary of Defense [Quinlan, 2000].

 

 

·         By definition, interoperability extends beyond the boundaries of any singular system, project, or program.  To command the developer to make a system “interoperable” without providing further scope is meaningless.  The challenge lies in identifying all the stakeholders and “communicating” across programs, and across organizational boundaries in order to clearly define the level of interoperability that is required and the participating systems.

·         Interoperability is not a binary state.  There are different degrees (or levels) of interoperability and different kinds of interoperability.  It is impossible to say “system A is interoperable but system B is not”.  Someone has to emphatically state how much interoperability (what level? what functionality?) is necessary and what systems constitute a particular interoperability domain.  Whose job is that?  Who truly understands which of the thousands of systems that exist truly need to interoperate, and at what level?  How do developers acquire this “big picture”?  What organizational structures and activities are necessary to support this type of communication?  How is this activity funded?

·         Interoperability is a volatile attribute of any given system because the requirements of any given system in the domain may change, as well as the impact of technology insertion.  Both factors may alter the interoperability state across the domain and necessitate re-assessment and corrective actions to “sustain” a desired state of interoperability.

·         How is interoperability funded?  Solving the interoperability issue often involves collaboration among many organizations.  How does each organization support that essential activity?  Testing and evaluation is significantly different for interoperability than it is for other program attributes.  Who should perform the testing?  How is testing funded?

·         How is interoperability communicated?  What practices must be put in place to ensure that interoperability assessments (evaluations) are consistent across the domain?  What measures (and criteria) should be used to communicate the status of interoperability among the participants in the domain?

·         Demanding interoperability does not guarantee it will be realized.  In many situations the technology is not sufficiently mature enough to meet the demand.  Acquirers need to find a balance between what is desired (or needed) and what is technically possible.  Someone needs to be looking ahead at the various research efforts now going on that are addressing interoperability challenges.  The Air Force Research Laboratory has a research program called “Joint Battlespace Infosphere” (JBI) that seeks to address the interoperability challenge.  There are others.

 

Although the primary motivation for this Gold Practice is to support military capability, the same challenges exist in the civilian/business world where data integrity, timely access to accurate information, and high level decision making are best supported by having interoperable systems.  Yesterday’s competitors are now today’s partners.  Big Defense contractors are acquiring other companies at a rapid pace and consequently acquiring a multitude of systems – many with no interoperability attributes.  What is an effective process for bringing those systems under control and sustaining productivity?  Can they be modified to be interoperable?  Or, should they be thrown away in favor of establishing an enterprise system?  In both the military and commercial environments life-cycle costs to support mission goals are a significant consideration in specifying and prioritizing interoperability requirements.

 

Understand the scope:  Individuals (and organizations), interacting with each other, effect the achievement of any desired level of interoperability.  Planning for interoperability is directly impacted by perceptions of the scope of the issue.  If the acquirer/developer of system A is not aware that system A must interoperate with systems D and E in a variety of ways, as well as with systems B and C, it is likely that there will be shortfalls in planning and budgeting that will impact the degree of interoperability achieved.  A program manager responsible for an MDAP program has a different focus than the project manager of a developer organization building a network component for general sale – although both are concerned about interoperability.

Understanding scope addresses:

·         What specific interoperability capabilities are needed

·         Who (organizations, programs, systems, etc.) is (needs to be) involved and in what capacity?  Who is the appropriate authority to identify needs?  Who will test or certify

·         What conventions, regulations, guides, policies standards, etc., apply

·         How will this understanding be communicated throughout the subject domain (across a program and to all members of the development team)

·         What (if any) is the required sequence of events that must occur

·         Who should assess the interoperability status among the subject systems, and how

·         What is my role (responsibility) versus the role of others

Specify interoperability requirements:  This activity bridges the gap between the general “mandate/demand/expectation” for interoperability and the reality of making it happen.  Customers/warfighters must provide/share their “operational” view of interoperability, identifying their functional and performance needs and the participating systems or organizations that satisfy the needs.  Standards Bodies/Developers bring a technical view of interoperability to the table, identifying policies, conventions and standards that are useful in attaining the various kinds and levels of interoperability.  Program managers/project managers provide knowledge of their current and planned programs/systems capabilities and costs.  Through collaboration these “stakeholders” analyze, clarify and identify the specific (detailed) interoperability requirements that each system must meet.  Requirements trade-offs negotiation, (another DACS Gold Practice), is a significant part of this process; interoperability requirements are analyzed and prioritized relative to other needs such as supportability, and affordability over the life of the system.  Some requirements may not be attainable in the reality of current technology and so are placed on a “wish” list.  The essential element of this activity is involving the right stakeholders.

Absent the regulation and guidance of the military environment, how does the individual project manager determine when interoperability is important?  If any one of these statements is true, then there are probably some interoperability requirements that need to be identified.

The system or component:

·          Generates data that is used by another system

·          Processes or consumes data that is generated by another system

·          Relies on another system for delivery of data

·          Is software that operates on the same platform as another system

 

Plan a strategy for achieving interoperability:  Planning to ensure interoperability is the same as planning for any other desired capability.  Follow good systems engineering /project planning practices (perhaps what you are already doing) to address interoperability as you would do for any other requirement.  Planning may take a different direction depending on the scope of the issue, who you are, and what your program/system is.  You may need to plan and budget for meetings that cross organizational boundaries.  The time frame required to elicit interoperability requirements may be longer due to the need to interact with multiple organizations/programs.  It may involve planning a migration over a period of two or three years to take a legacy system to a desired state of interoperability (or to replace it).  You may need to allocate special resources to investigate/analyze available technology and standards, or you may need to plan how to effect a cultural change in your staff that will result in adopting standards already defined as the solution for a specific interoperability capability.  A significant part of planning involves determining how attainment of interoperability will be evaluated or assessed for each system, and across the interoperability domain.  In some environments (such as programs that are part of the NSS) most of the high level interoperability requirements have already been defined, and are common to all systems in the domain.  Additionally the Joint Interoperability Test Command (JITC) has been established to evaluate and certify systems (with respect to interoperability) prior to releasing them for use in the field. JITC works with the development team throughout the life cycle.  Planning in these environments focuses on coordination with such external organizations as much as on internal development.  Many programs are required to implement the Joint Technical Architecture (JTA) as a step toward achieving interoperability.  This may make sense for your program regardless of whether or not it is required.

An important step in planning for interoperability is in planning for training and other activity that communicates/educates the staff (collaborators) about the scope of the effort, the stakeholders, the plan for attainment, and the evaluation strategy.

Although interoperability is vitally important on many programs, it is still one of several performance requirements.  Planning for interoperability should be blended into the overall project or program plan, not treated in isolation.

 

 

Implement the plan: Follow the plan! Walk the Talk! Adjust as necessary.

 

Evaluate/assess progress toward achieving interoperability :  As is the case with most performance requirements, the techniques for  evaluating interoperability depend on the operational environment of the system, and the scope of the requirements.  The important elements of this step are:

·         An evaluation process is established.  This may include a certification process by an external authority, independent assessments, and developer tests and evaluations.

·         The evaluation process addresses the interoperability requirements

·         Test results adequately communicate the status of the system with respect to those requirements

·         Interoperability status is shared among the stakeholders within the interoperability domain through a common representation

 

DoD Directive (DoDD) 4630.5 and DoD Instruction (DoDI) 4630.8 mandates joint and combined interoperability certification testing for "all Command, Control, Communications, and Intelligence (C3I) systems developed for use by US forces.”  This certification must be obtained prior to fielding a new system.  DoDI 4630.8 directs DISA to "develop and conduct a C3I systems interoperability testing and certification program."  Further amplification is contained in the Chairman, Joint Chiefs of Staff Instruction 6212.01B.  The principal goal is to establish increased interoperability through a quality, cost effective testing program administered throughout the life cycle of a system.  This life cycle starts with the requirements identified during the acquisition phase and continues through the retirement of the system.  Certification is based on JITC review and analysis of system requirements, test plans, and test results.  Testing may be conducted by JITC or another certified test agency.  JITC approval of other agency test plans is required to ensure they meet certification criteria requirements prior to the start of testing.

 

Figure 2 is extracted from a JITC Common Operational Picture Lab Briefing dated 1/7/02 and used with permission of JITC author, Michael Keoster.  It is an excellent illustration of the complexity and scope of the interoperability issue.  The blue circle in the middle represents an operational need (high level requirement) to be addressed – in this case, Situational Awareness COP.  The various systems involved are represented by the rectangles and ovals around the outer edge.  They are grouped by their affiliation/origin, (Army, Navy, Air Force/Marine, Joint Services).  The various arrows represent the types of interfaces needed and the directional flow of data/information.  Systems are color-coded Grey – future interface, Green – JITC IOP
Certified, and Red – In Use but Not Certified.

 

This diagram identifies the interfaces that need to be evaluated for one operational scenario.  The fact that only a small percentage of the systems in use are certified is testimony to the immensity of the issue, and the relevance of interoperability to other operational (functional) needs.  Certifying systems prior to fielding them is an admirable goal that is very difficult to attain.  Meanwhile, do systems performing vital functions get shelved while waiting for certification?  This diagram shows the realistic approach to that scenario.

The diagram does show that there may be opportunity for collaboration in developing interfaces that perform the same essential functions.  When different systems are built using common standards and conventions, the testing of those systems is less complicated because the variance in what must be evaluated is reduced.

Types of Testing

There are three major types of testing that are significant for evaluating the “interoperability” of a system or system-of-systems.

·         Conformance testing is the process of assessing the compliance of a product to the defining specification or standard.  Specialized test tools are used to exercise a product to determine if the proper actions and reactions are produced.  The test tool is normally the only device the product being evaluated is connected to.  Successful completion of a conformance test will enhance the probability of interoperability with other products that have been successfully conformance tested.

·         Interoperability testing is the process of assessing the ability of a system to exchange usable electronic information with systems of other services or nations as specified in its requirements documents.  Specialized test tools are used to monitor performance of products to determine if the proper actions and reactions are produced.  A system is certified as interoperable at the completion of successful interoperability testing.

·         Operational testing is the process of evaluating the performance of products in an operational environment.  Due to the resource requirements needed to generate realistic environments, operational testing is usually conducted in conjunction with military exercises.

Essential Measurement Scenarios

Here are some scenarios for presenting test results in a manner that communicates the status to the stakeholders and calls attention to those systems or programs that warrant further attention.  This material is extracted from Kasunic’s presentation at the Software Acquisition Conference in Jan 2003 [Kasunic, 2003].

Figure 3 at right shows the results of conformance testing.  The entries rate as pass/marginal/fail (green, yellow, or red) the compliance of systems S₁, S₂ … S_n with the relevant standards and guidance.  This can be established on a standard-by-standard basis, or a collection of standards.  What is important is that the thresholds for each level of compliance are applied consistently across all the systems.  Accomplishing this consistency requires the cooperation /collaboration across the program boundaries, or perhaps necessitates an “oversight” committee or group to establish the thresholds.

 

 

Figure 4 addresses direct interoperability testing (technical interoperability) results among the systems of a particular operational domain.  The pairwise interoperability of the systems indicated in the row and column headings is rated as pass/marginal/fail (green, yellow, or red).  This method of tracking interoperability status accommodates the variances in the levels of interoperability required among systems and still focuses attention on the overall interoperability picture.  For example S1 and S2 may only require a simple level of data interoperability that is easily achieved, while S5 and S3 require high degrees of interoperability necessary for decision-making in a combat scenario.

 

 

The value of this chart is dependent upon having clear precise interoperability requirements defined for each system, and having pre-defined thresholds for representing the interoperability status of the pairs.  It can be a very powerful tool for tracking interoperability but it also hints at the immensity of the task when viewed from the perspective of an individual program manager.  The complexity of the task increases (almost exponentially) with the number of systems, and system life cycles.  Whenever requirements changes to any one system have a potential impact on interoperability all systems in the interoperability domain need to be re-assessed with respect to that impact (one row in the Figure 3 matrix).

Figure 5 addresses presentation of operational testing results.  Such results may be obtained from military exercises or modeling and simulation exercises that approximate the operational view.  This type of testing focuses on the “effectiveness” of the technical capability (the 2^nd part of the interoperability definition) in enabling the desired levels of communication and interaction among participants (systems and ultimately people) in the prescribed operational setting.

 

Each mission “slice” may require a slightly different collection of systems with some systems having a common role (information flow) in most missions.  The ability of systems to perform may be impacted by the characteristics of the operational environment resulting in different behavior in the various missions.  The interoperability status of S1 and S6 may be different for slice #1 than for slice #2 (even though it had an acceptable status in pair-wise systems testing).  Additionally, operational effectiveness often depends on a “sequence” of interoperating systems rather than a single pair.  As illustrated in mission slice #1 there may be multiple paths of information flow that can be used to accomplish the communication/interaction (S1 to S9).  Even though a red flag appears between S1 and S6 operational effectiveness is not compromised because an alternative path exists for getting from S1 to S9.  Operational testing may reveal the need for modifications or additions to the interoperability requirements for specific systems over time.  Graphing operational scenarios, as shown in Figure 4, and overlaying the results of pair-wise system testing on the graph can be useful in determining the priorities for corrective action.  For example, if the S1/S6 pair is always on the critical path for mission effectiveness, with few or no alternatives it probably would be elevated to a higher priority than it would have under the mission slice #1 scenario.

 

These charts and graphs are not a silver bullet for ensuring interoperability, but they do focus attention on the problem areas.  They also illustrate the need for collaboration and coordination among programs in both defining and evaluating interoperability.

 

The LISI Model (see available tools section) now under development by MITRE provides a means to measure, understand and track the interoperability of systems.  It provides a reference model for interoperability and then extends it into an implementation in the form of a capability maturity model for interoperability.  It defines and assesses systems against increasing levels of sophistication for system-to-system interaction.  It provides an audit trail for linking systems interoperability metrics to specific mission operations and may therefore be invaluable as a basis for evolutionary planning. [LISI, 2002]

CHARACTERISTICS OF IMPLEMENTATION:

 

SUMMARY CHARACTERISTICS

 

NO DATA CURRENTLY AVAILABLE

 

ANTICIPATED BENEFITS OF IMPLEMENTATION:

 

Successful implementation of the Ensure Interoperability Gold Practice can result in:

 

§        Increased Likelihood of Actually Achieving Specific Interoperability Objectives.  The high level demand for interoperability is broken down into meaningful, realistic, detailed prioritized interoperability requirements that are attainable with current technology.

§        Significant Savings Across the Interoperability Domain.  Both cost and schedule “savings” can be obtained through coordination and collaboration that provide oversight for identifying interoperability requirements, evaluating the interoperability of systems and sharing the results within the interoperability domain.  Although this requires a cultural change, it avoids duplicating expensive evaluation processes for each system and can spread the costs across the domain.

§        A Migration Plan for Making Legacy Systems “Interoperable”.  The unending process focus of this practice may support the development of sensible strategies for migrating existing systems (that were not initially designed for interoperability) to function in new interoperable environments.

 

 

DETAILED CHARACTERISTICS

 

Key Characteristics of the” Ensure Interoperability” Gold Practice

Characteristic	Comments
Relies On Standards-Based Architectures	·          Achieving various degrees of interoperability is dependent on distinct systems complying with standards, conventions, and protocols specified for the entire domain ·          Conforming to a domain-specific joint architecture addresses many of the interoperability requirements
Requires Collaboration & Coordination Among Programs	·          By definition, interoperability is concerned with how multiple systems must interact ·          Representatives of those systems become key stakeholders in eliciting and evaluating interoperability requirements ·          May require cultural changes to effect successful collaboration
Requirements Management & Negotiation	·          Must balance what is desired with what is realistically achievable (technology may not yet exist for some scenarios) ·          Stakeholders must prioritize interoperability requirements relative to functional and other performance requirements, while considering total cost of ownership
Many Kinds of Interoperability	·          Data interoperability is different from connectivity, and so on ·          The two broadest views are operational and technical interoperability ·          Technical interoperability typically enables operational interoperability
Several Levels	·          Interoperability is typically not a binary state among systems; there are degrees (levels); systems may be interoperable in some ways but not others ·          The desired levels of interoperability must be delineated as requirements statements ·          Required interoperability (capabilities) is not necessarily the same for all systems in the domain
Complex Costing	·          Complexity of costing interoperability due to volatility and size of the interoperability domain ·          Difficult to estimate the cost of testing because laboratory settings cannot fully address operational scenarios of a combat environment; dependent on military exercises ·          Systems belonging to other countries, and outside of the military (weather, civilian emergency systems) scope are often part of an interoperability domain
Complex Measurement Strategy	·          3 major dimensions for measuring interoperability; technical conformance to standards, technical interoperability of system pairs, and operational effectiveness of interoperability ·          Ensuring consistency across programs in measuring and establishing and communicating thresholds is very complex ·          Collaboration within the domain (or an oversight body) is necessary to ensure a meaningful measurement strategy
Certification Process Required	·          MDAC programs (and all the systems they interoperate with) must be certified by JITC process (or equivalent) prior to release for use in the field ·          Monitoring of this process is an immense task ·          How are all the “other” systems (including those of coalition forces) identified and brought under the certification process ·          What is the reality of the situation?  Are the services using “yet to be certified” systems?

RELATIONSHIPS TO OTHER PRACTICES:

 

The Figure below represents a high-level process architecture for the subject practice, depicting relationships among this practice and the nature of the influences on the practice (describing how other practices might relate to this practice).  These relationship statements are based on definitions of specific “best practices” found in the literature and the notion that the successful implementation of practices may “influence” (or are influenced by) the ability to successfully implement other practices.  A brief description of these influences is included in the table below.

Process Architecture for the "Ensure Interoperability" Gold Practice

 

 

Summary of Relationship Factors

INPUTS TO THE PRACTICE
Identify Specific Interoperability Issues	These practices contribute in various ways to identifying and prioritizing interoperability requirements for a system or system-of-systems.  Although their implementation is “not required to ensure interoperability”, use of one or more of these practices (in any combination) may facilitate achieving your goal.  Goal-Question Metric Approach supports getting from the broadly defined desire to be “interoperable” to metrics that specify how you will know what you have actually achieved.  Formal Risk Management helps in prioritizing interoperability requirements relative to other needs, and relative to acceptable levels of risk.  Manage Requirements and Requirements Trade-Off/Negotiations address the complexity of the interoperability issue, accepting that requirements change over time along with the operational environment and that programs must plan to manage that change.  Performance-based Specifications focus on the “what” (outcome) rather than the “how”.  This is important because it gives the developer room to innovate in a rapidly changing “cutting edge” technical area in order to meet challenging interoperability requirements. Agreement on Interfaces addresses participation of the stakeholders (warfighters) early on in defining their operational needs and explaining what level of communication and interaction they require among the systems of a particular operational environment (interoperability domain).  This establishes the scope of the interoperability issue in any given domain and provides guidance for the specification of actual performance requirements.
Plan for Interoperability	The complexity and technical focus of the interoperability issue necessitates a focus on process, and significant planning for coordination and collaboration in addition to technical development over the life of the system.  Establish Clear Goals and Decision Points helps to address the challenge and scope of the interoperability issue and allocate roles and responsibility across the domain.  The Business Case (operational need) drives requirements and maintenance activity.  As the environment changes it impacts the priorities established for interoperability.  Technology Insertion plays a vital role in sustaining interoperability across a collection of systems in a volatile environment.  Use of the Open Systems Approach in development facilitates technology insertion and thus increases the likelihood of sustaining interoperability over the life of the systems.  Planning for and demonstrating interoperability at the Architecture Level tends to minimize cost and increase flexibility over the life cycle.
Resolve Specific Interoperability Issues	These practices address the actual design and implementation activities that support ensuring interoperability.  Structured Development Methods (specifically the iterative process model) may be necessary to deliver increasing levels of interoperability over time, and also to validate the defined requirements.  Decisions regarding Use of COTS/NDI, and about Reuse of “Custom” Components are major factors, not only in achieving the requirements but improving affordability of the product over the life cycle.  The complexity of the interoperability warrants taking a process approach to software development and life cycle maintenance.  Independent Expert Reviews and Software Capability Evaluations (SCEs) help identify strengths and weaknesses in the processes that are in place and help the organization target interoperability improvement initiatives.
Demonstrate That Interoperability Is Achieved	Measuring and communicating the degree of interoperability among systems in a specific operational domain is very complex.  Model-Based Testing is the “next best thing” to the operational environment itself.  Creating the actual operational environment for testing is seldom feasible because of cost and the volatility of that environment.  Formal Inspections may prevent development from “going astray”.  They are typically conducted with respect to specific criteria (such as designing for interoperability, or conformance to a particular standard) and so serve as preventive action to uncover early problems which might otherwise go un-detected until final testing when they are more costly to fix.  Demonstration-Based Reviews “show” rather than talk about interoperability.  This provides an opportunity for stakeholders to both verify and validate the interoperability requirements while there is still time to take corrective action if necessary.  Project artifacts are the primary tool for communicating about the requirements, design, and progress of a project.  Using Formal Model-Based Notation is essentially agreeing on some standards for describing the project and using those standards to share information over the life cycle of the project.  Typical applications of this practice are architectural notation, requirements notation (UML), and testing scorecards.  Agreement on how interoperability status of systems shall be communicated across the domain is critical to ensuring interoperability.
OUTPUTS FROM THE PRACTICE
Make Sure Interoperability Issues Are Addressed	Because interoperability is a significant goal that typically involves multiple programs, many processes and practices influence the overall practice of ensuring interoperability but the reverse is not necessarily true.  Implementing Ensuring Interoperability does not directly impact many other practices.  The exceptions are Best Value Awards and Acquisition Process Improvement.  In seeking best value, the acquirer may look for evidence of a process-oriented approach to the interoperability issue.  Organizations that have implemented and documented this practice would likely score higher in a “best value” evaluation assuming other technical competencies are equal.  The activities related to requirements development and planning for interoperability which result in a high degree of success in achieving interoperability may trigger Acquisition Process Improvement which, in turn, increases the likelihood of ensuring interoperability over time.

 

RESOURCES:        

 

§        Websites

o        DISA Joint Interoperability Test Command

http://jitc.fhu.disa.mil/

 

o        DoD Defense Standardization Program – Interoperability Web Site

http://www.dsp.dla.mil/

 

o        DoD Joint Technical Architecture – JTA On-Line Web Site

http://jtaonline.disa.mil/

 

o        DoD Joint Technical Architecture –Web Site

http://jta.disa.mil/

 

o        Systems Engineering Data Representation and Exchange Standardization (SEDRES) - European-initiated Web site with International Participation

http://www.sedres.com/

 

§        Tools and Methods

State-of-the-art methods and tools that may be useful in implementing and improving the effectiveness of Ensure Interoperability include:

§        Modeling & Simulation Tools

M & S tools provide a discipline for developing a broad level of understanding of how the parts of a system (or system-of-systems) interact which is seldom achievable any other way.  Computer simulation can be used to optimize performance of systems through trade-off analysis, verify the correctness of design, and develop “virtual environments” for training, war gaming, and maintainability studies.

 

§        Levels of Information System Interoperability (LISI). 

Rather than a single measure, LISI is actually a collection of related models, a tool for use in applying these models, a set of metrics and techniques for applying the models, and an initiative (or process) aimed at using these models to address a wide set of interoperability objectives.  At its core, LISI is based around classifying levels of interoperability by the “richness” of the communication that a particular system or group of systems allows (C4ISR Architecture Working Group, 1998).  LISI is currently under development by MITRE. For further information see  http://deskbook.dau.mil/askaprof-akss/normal/qdetail2.asp?cgiSubjectAreaID=3&cgiQuestionID=9196 and http://deskbook.dau.mil/askaprof-akss/normal/qdetail2.asp?cgiSubjectAreaID=9&cgiQuestionID=6759

 

·        Acquisition Streamlining and Standardization Information System (ASSIST On-line).

Note that the ASSIST database is the official source of all documents listed in the DoD Index of Specifications and Standards (DODISS) and all Data Item Descriptions (DIDs).

http://assist1.daps.dla.mil/quicksearch/

 

·         Joint Interoperability Tool,

maintained by JITC, provides high speed access to key interoperability information.  The heart of the system is an extensive data repository featuring the JITC Lessons Learned reports, JITC Test Reports, the NATO Interface Guide, Joint Interoperability Certification letters, and other interoperability documents and references; as well as a high speed search engine to quickly access data.  This tool gives a quick and easy on-line capability, which identifies system/equipment characteristics, tested configurations and practical “how-to” information to facilitate interoperability.  [Potential users must register and be validated prior to access.]

https://jit.fhu.disa.mil/

 

§        Experts/Contact Points

Mark Kasunic, Senior Member of the Technical Staff, Software Engineering Institute, Carnegie Mellon, is developing a “State of the Practice” report on Interoperability.  As such, he is well informed on efforts to measure interoperability. mailto:mkasunic@sei.cmu.edu?subject=Measuring Interoperability

 

Col. Thomas Andrew, U.S. Air Force, Commander of DISA’s Joint Interoperability Test Command, is responsible as the DoD’s sole certifier of joint interoperability for systems.

 

§        Training Opportunities:

o        Defense Acquisition University

§         PQM 103 Defense Specification Management

§         PQM 104 Specification Selection and Application

§         PMT 202 Multinational Program Management

 

http://www.dau.mil/catalog/cat2003/Chapter4.pdf

 

§        Bibliography:

 

 

APPENDICES

DEFINITIONS:

 

Interoperability:  The ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces and to use the services so exchanged to enable them to operate effectively together.

-    Joint Publication 1-02

 

Interoperability:  The ability of people, procedures, and equipment to operate together effectively and efficiently under all conditions of battle.

- [JITC, 2001]

SOURCES (Origins of the Practice):

NONE INDICATED

RECOMMENDING SOURCES:

 

§         DoD 5000.2-R, “Mandatory Procedures for Major Defense Acquisition Programs (MDAPS) and Major Automated Information System (MAIS) Acquisition Programs”, 5 April 2002 [CANCELLED 30 October 2002]

§         Interim Defense Acquisition Guidebook, 30 October 2002 [INTERIM 2002]

 

C1.4 Acquisition Program Baseline (APB) C1.4.1  Every acquisition program shall establish an APB beginning at program initiation.  The PM shall base the APB on users' performance requirements.  Performance shall include interoperability, supportability and, as applicable, environmental requirements.  C1.4.3.1.1  The total number of performance parameters shall be the minimum number needed to characterize the major drivers of operational performance, supportability, and interoperability.

 

C2.2 REQUIREMENTS C2.2.1  The acquisition strategy shall provide a summary description of the requirement the acquisition is intended to satisfy.  The summary shall highlight aspects of the requirement that are driven by family-of-systems or mission area requirements for interoperability.

 

C2.6 PROGRAM MANAGEMENT C2.6.7  Planning for Simulation-Based Acquisition (SBA) and Modeling and Simulation (M&S).  SBA is the robust and interactive use of M&S throughout the product life cycle.  C.2.6.7.4  SBA/M&S shall support efficient test planning; pre-test results prediction; validation of system interoperability.

 

C.2.7 Design Considerations Affecting the Acquisition Strategy.  

 

C2.7.2  Interoperability.All acquired systems shall be interoperable with other U.S. and allied defense systems, as defined in the requirements and interoperability documents.  The Program Manager (PM) shall describe the treatment of interoperability requirements.  This description shall identify enabling system engineering efforts such as network analysis, interface control efforts, open systems, data management, and standardization.  It shall also identify related requirements or constraints (e.g., treaties or international standardization agreements) that impact interoperability requirements (e.g., standards required by the DoD Joint Technical Architecture (JTA) or the systems, forces, units, etc., for which interoperability is, or could be an issue), and any waivers or deviations that have been obtained or are anticipated being sought.  The acquisition strategy shall reflect full compliance with the interoperability policies in subparagraph and for IT, including National Security Systems (NSS), section 0.  The Milestone Decision Authority (MDA) shall adjudicate interoperability issues.  C2.7.2.1  Information Interoperability.  The PM shall identify and assess the technical, schedule, cost, and funding critical path issues (i.e., issues that could impact the PM's ability to execute the acquisition strategy) related to interoperability for the PM’s acquisition program.  The PM shall identify the critical path issues in other program(s) (i.e., system(s)) that will exchange information with the PM’s delivered system, and assess the potential impact of these issues on the PM’s program.

 

C2.8 SUPPORT STRATEGY C2.8.5.3  Training.  The acquisition strategy shall include a description of interoperability requirements necessary to support unit and joint training architectures.

 

C2.9.2.2  International INTEROPERABILITY.  The growing requirement for effective international coalitions requires a heightened degree of international interoperability.  Reciprocal trade and international cooperative programs with allies and friendly nations serves this end.  Programs shall strive to achieve deployment and sustainability of interoperable systems with our potential coalition partners.  C2.3.1.3   Provide an assessment of the advantages and disadvantages, with regard to program timing, life-cycle costs, technology sharing, standardization, and interoperability, of a cooperative program with one or more major allies or North Atlantic Treaty Organizations (NATO) organizations

 

C3.1 TESTS & EVALUATION.  Provide an assessment of the advantages and disadvantages, with regard to program timing, life-cycle costs, technology sharing, standardization, and interoperability, of a cooperative program with one or more major allies or NATO organizations.

 

C3.2.3.2  T & E Guidelines  C3.2.3.2.2.12  All DoD MDAPs, programs on the Office of Secretary of Defense (OSD)  Test and Evaluation (T&E) Oversight list, post-acquisition (legacy) systems, and all programs and systems that must interoperate with them, are subject to interoperability evaluations throughout their life cycles to validate their ability to support mission accomplishment.  At the discretion of the Under Secretary of Defense for Acquisition Technology and Logistics (USD(AT&L)), Assistant Secretary of Defense for Command, Control, Communications, and Intelligence (ASD(C3I)), Director of Operational Test and Evaluation (DOT&E), United States Joint Forces Command (USJFCOM), and the Joint Staff, they shall place programs and systems deemed to have significant interoperability deficiencies on the Interoperability Watch List.  PMs for a program on the Watch List will be required to undertake corrective actions to address interoperability deficiencies in order to be removed from the Interoperability Watch List.

 

C3.5 Certification of Readiness for Operational Test & Evaluation (OT&E)  C3.5.4  The developing agencies shall review all available interoperability assessments (e.g., Operational Assessments (OA), Joint Interoperability Test Command (JITC) interoperability assessments, and standards conformance reports) during Operational Test Readiness Reviews (OTRR) to highlight potentially critical interoperability problems for assessment during OT&E.

 

C3.6 Operational Test & Evaluation (OT&E).  C3.6.1.11  For systems with joint interoperability requirements, all available interoperability assessments (e.g., OAs, JITC interoperability assessments, standards conformance reports) should be reviewed during the OTRR before conducting Initial Operational Test & Evaluation (IOT&E).  Potentially critical interoperability problems shall be highlighted for assessment during OT&E.

 

C5.2 SYSTEMS ENGINEERING. The Systems Engineering (SE) process shall … C5.2.2.2 ensure the interoperability and integration of all operational, functional, and physical interfaces.

 

C5.3 REQUIREMENTS ANALYSIS. The PM shall work with the user to establish and refine operational and design requirements.  Together, they shall determine appropriate operational performance objectives, within affordability constraints.  These analyses shall allocate and balance interoperability requirements among systems that must interoperate successfully to satisfy all appropriate Capstone Requirements Documents (CRD) the proposed system falls under.  C5.3.2.3.5.11  SE  Activities …Interoperability.  All acquisition programs shall satisfactorily address interoperability and integration.  Users shall specify, and the appropriate authority shall validate, thresholds and objectives during the requirements generation process.  The Joint Staff shall certify interoperability requirements.

 

C6.3 Information Interoperability  C6.3.2  The Operational  Requirements Document (ORD) sponsor shall characterize information interoperability, as applicable, within a family of systems, a mission area, and a mission, for all IT systems, including NSS.  In developing the ORD, the ORD sponsor shall consider using the products described in the Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) Architecture Framework (renamed the DoD Architecture Framework in versions 2.1 and later) and universal resources such as the JTA.  The ORD sponsor shall apply the following guidance to information interoperability.  C6.3.2.1  Manage, verify and maintain information interoperability throughout the system life cycle.  C6.3.2.2  Participate in interoperability and supportability M&S assessments that are performed by the Military Departments or Lead Executive Component to determine the level of interoperability between systems and identify incompatibilities.

 

§         Capability Maturity Model Integration (CMMI), Version 1.1, Software Engineering Institute, CMU/SEI-2002-TR-011, TR-012, March 2002

 

The CMMI does not specifically mention the practice of “ensuring interoperability” but there are several key process areas (KPAs) that address the activities that are necessary to realize a desired interoperability capability.  Many of the activities discussed under these KPAs are in fact Gold Practices that may be very useful in any software intensive initiative.  The most relevant KPAs are summarized in the following paragraphs.  What is significant is that they represent a process focus with process improvement “built-in”.  The complexity of the interoperability issue warrants a process oriented approach.

Pages 206-227  “Requirements Development” Key Process Area (KPA).  Under CMMI the organization establishes a process for developing requirements:

 

Develop Customer Requirements	·         Addresses defining a set of customer requirements to use in the development of product requirements
Develop Product Requirements	·         Addresses defining a set of product or product-component requirements to use in the design of products and product components
Analyze and Validate Requirements	·         Addresses the necessary analysis of customer, product, and product-component requirements to define, derive, and understand the requirements

 

PP 228-258 “Technical Solutions” KPA.  The purpose of Technical Solution is to design, develop, and implement solutions to requirements.  Solutions, designs, and implementations encompass products, product components, and product-related lifecycle processes either singly or in combinations as appropriate.  The process area focuses on the following:

o        Evaluating and selecting solutions (sometimes referred to as “design approaches,” “design concepts,” or “preliminary designs”) that potentially satisfy an appropriate set of allocated requirements

o        Developing detailed designs for the selected solutions (detailed in the context of containing all the information needed to manufacture, code, or otherwise implement the design as a product or product component)

o        Implementing the designs as a product or product component

 

PP 281 – 298 “Verification”  KPA.  Its purpose is to ensure that selected work products

meet their specified requirements.

 

PP 299-311 “Validation“ KPA.  Its purpose is to demonstrate that a product or product component fulfills its intended use when placed in its intended environment.

 

Verification ensures that “you built it right;” whereas, validation ensures that “you built the right thing.”

 

PP 446-460 “DECISION ANALYSIS AND RESOLUTION” KPA.  The purpose of this process area is to analyze possible decisions using a formal evaluation process that evaluates identified alternatives against established criteria.  A formal evaluation process is a structured approach to evaluating alternative solutions against established criteria to determine a recommended solution to address an issue.  A formal evaluation process involves the following actions:

 

·          Establishing the criteria for evaluating alternatives

·          Identifying alternative solutions

·          Selecting methods for evaluating alternatives

·          Evaluating the alternative solutions using the established criteria and methods

·          Selecting recommended solutions from the alternatives based on the evaluation criteria

 

GLOSSARY

 

 

CASE STUDIES FROM THE LITERATURE:

 

NO REFERENCED CASE STUDIES AT THIS TIME