![[NASA Logo]](../Images/nasaball.gif)
| |
NASA Procedures and Guidelines |
This Document is Obsolete and Is No Longer Used.
|
|
P.1 Purpose
P.2 Applicability
P.3 Authority
P.4 Applicable Documents and Forms
P.5 Measurement/Verification
P.6 Cancellation
1.1 Background
1.2 Framework for Systems Engineering Procedural Requirements
1.3 Guiding Principles of Technical Excellence
1.4 Framework for Systems Engineering Capability
1.5 Document Organization
2.1 Roles and Responsibilities Relative to System Engineering Practices
2.2 Tailoring and Customizing
3.1 Introduction
3.2 Common Technical Processes Requirements
4.1 Introduction
4.2 Prior to Contract Award
4.3 During Contract Performance
4.4 Contract Completion
5.1 Life-Cycle
5.2 Life-Cycle and Technical Review Requirement
6.1 Systems Engineering Management Plan Function
6.2 Technical Team Responsibilities
Figure 1-1 - Hierarchy of Related Documents
Figure 1-2 - Documentation Relationships
Figure 1-3 - Technical Excellence - Pillars and Foundation
Figure 1 4 - SE Framework
Figure 3 1 - Systems Engineering (SE) Engine
Figure 3-2 - Application of SE Engine Common Technical Processes Within System Structure
Figure 3-3 - Sequencing of the Common Technical Processes
Figure 3-4 - SE Engine Implemented for a Simple Single-Pass Waterfall-Type Life Cycle
Figure 5 1 - NASA Uncoupled and Loosely Coupled Program Life Cycle
Figure 5-2 - NASA Tightly Coupled Program Life Cycle
Figure 5-3 - NASA Single-Project Program Life Cycle
Figure 5 4 - The NASA Project Life Cycle
Figure A-1 - Enabling Product Relationship to End Products
Table 5-1 - SE Work Product Maturity
Table G-1 - SRR Entrance and Success Criteria for Programs
Table G-2 - SDR Entrance and Success Criteria for Programs
Table G-3 - MCR Entrance and Success Criteria
Table G-4 - SRR Entrance and Success Criteria
Table G-5 - MDR/SDR Entrance and Success Criteria (Projects and Single-Project Program)
Table G-6 - PDR Entrance and Success Criteria
Table G-7 - CDR Entrance and Success Criteria
Table G-8 - PRR Entrance and Success Criteria
Table G-9 - SIR Entrance and Success Criteria
Table G-10 - TRR Entrance and Success Criteria
Table G-11 - SAR Entrance and Success Criteria
Table G-12 - ORR Entrance and Success Criteria
Table G-13 - MRR/FRR Entrance and Success Criteria
Table G-14 - PLAR Entrance and Success Criteria
Table G-15 - CERR Entrance and Success Criteria
Table G-16 - PFAR Entrance and Success Criteria
Table G-17 - DR Entrance and Success Criteria
Table G-18 - Disposal Readiness Review Entrance and Success Criteria
Table G-19 - Peer Review Entrance and Success Criteria
Table G-20 - PIR/PSR Entrance and Success Criteria
Table G-21 - DCR Entrance and Success Criteria
Table J-1 - Deleted Requirements and Justification
|
|
|
|
|
1 |
01/19/2021 |
Updated with admin changes: Section 3.1.5.9 Editorial fix; Section 5.2.2.4 Reference fix; Appendix A "Will" to "can" in Engineering Unit definition; Appendix G Changes to Table G-9; Success Criteria 7 and 8, Table G-12; Entrance Criteria 9.d, and Table G-13, Entrance criteria 7.e |
|
2 |
02/22/2022 |
Updated with admin changes: Appendix E, TRL, delete example b. under TRL 2 |
This document establishes the NASA processes and requirements for implementation of Systems Engineering (SE) by programs/projects. NASA SE is a logical systems approach performed by multidisciplinary teams to engineer and integrate NASA's systems to ensure NASA products meet the customer's needs. Implementation of this systems approach will enhance NASA's core engineering capabilities while improving safety, mission success, and affordability. This systems approach is applied to all elements of a system (i.e., hardware, software, and human) and all hierarchical levels of a system over the complete program/project life cycle.
a. This NASA Procedural Requirement (NPR) applies to NASA Headquarters and NASA Centers, including component facilities and technical and service support centers. This NPR applies to NASA employees and NASA support contractors that use NASA processes to augment and support NASA technical work. This NPR applies to the Jet Propulsion Laboratory (JPL), a Federally Funded Research and Development Center, other contractors, grant recipients, or parties to agreements only to the extent specified or referenced in the appropriate contracts, grants, or agreements. (See Chapter 4.)
b. This NPR applies to air and space flight, research and technology, information technology (IT), and institutional programs and projects. Tailoring the requirements in this NPR and customizing practices, based on criteria such as system/product size, complexity, criticality, acceptable risk posture, and architectural level, is necessary and expected. See Section 2.2 for tailoring and customizing descriptions. For IT programs and projects, see NPR 7120.7 for applicable SE tailoring.
c. In this document, projects are viewed as a specific investment with defined goals, objectives, and requirements, with the majority containing a life-cycle cost, a beginning, and an end. Projects normally yield new or revised products or services that directly address NASA strategic needs. They are performed through a variety of means, such as wholly in-house, by Government, industry, international or academic partnerships, or through contracts with private industry.
d. The requirements enumerated in this document are applicable to all new programs and projects, as well as to all programs and projects currently in the Formulation Phase, as of the effective date of this document. (See NPR 7120.5, NASA Space Flight Program and Project Management Requirements; NPR 7120.7, NASA Information Technology and Institutional Infrastructure Program and Project Management Requirements; or NPR 7120.8, NASA Research and Technology Program and Project Management Requirements; for definitions of program phases.) This NPR also applies to programs and projects in their Implementation Phase as of the effective date of this document. For existing programs/projects regardless of their current phase, waivers or deviations allowing continuation of current practices that do not comply with one or more requirements of this NPR, may be granted using the Center's Engineering Technical Authority (ETA) Process.
e. Many other discipline areas perform functions during the program/project life cycle and influence or are influenced by the engineering functions performed and, therefore, need to be fully integrated into the SE processes. These discipline areas include but are not limited to health and medical, safety, reliability, maintainability, quality assurance, IT, cybersecurity, logistics, operations, training, human system integration, planetary protection, and environmental protection. The description of these disciplines and their relationship to the overall program/project management life-cycle are defined in other NASA directives; for example, the safety, reliability, maintainability, and quality assurance requirements and standards are defined in the Office of Safety Mission Assurance (OSMA) directives and standards, and health and medical requirements are defined in the Office of the Chief Health and Medical Officer (OCHMO) directives and standards. For example, see NASA-STD-3001, NASA Space Flight Human System Standard Volume 1 and Volume 2, and NPR 8705.2, Human-Rating Requirements for Space Systems.
f. In this NPR, all mandatory actions (i.e., requirements) are denoted by statements containing the term "shall." The requirements are explicitly shown as [SE-XX] for clarity and tracking purposes as indicated in Appendix H. The terms "may" or "can" denote discretionary privilege or permission, "should" denotes a good practice and is recommended but not required, "will" denotes expected outcome, and "are/is" denotes descriptive material.
g. In this NPR, all document citations are assumed to be the latest version, unless otherwise noted.
a. National Aeronautics and Space Act, 51 U.S.C. § 20113(a).
b. NPD 1000.0, NASA Governance and Strategic Management Handbook.
c. NPD 1000.3, The NASA Organization.
d. NPD 1001.0, NASA Strategic Plan.
e. Government Contract Quality Assurance, 48 CFR, subpart 1846.4.
f. NPD 2570.5, NASA Electromagnetic Spectrum Management.
g. NPD 7120.4, NASA Engineering and Program/Project Management Policy.
h. NPR 1441.1, NASA Records Management Program Requirements.
i. NPR 2570.1, NASA Radio Frequency (RF) Spectrum Management Manual.
j. NPR 7120.5, NASA Space Flight Program and Project Management Requirements.
k. NPR 7120.7, NASA Information Technology and Institutional Infrastructure Program and Project Management Requirements.
l. NPR 7120.8, NASA Research and Technology Program and Project Management Requirements.
m. NPR 7150.2, NASA Software Engineering Requirements.
n. NPR 8000.4, Agency Risk Management Procedural Requirements.
o. NPR 8590.1, Environmental Compliance and Restoration Program.
p. NPR 8705.2, Human-Rating Requirements for Space Systems.
q. NPR 8705.5, Technical Probabilistic Risk Assessment (PRA) Procedures for Safety and Mission Success for NASA Programs and Projects.
r. NPR 8820.2, Facility Project Requirements (FPR).
s. NASA-HDBK-2203, NASA Software Engineering Handbook.
t. NASA-STD-3001, NASA Space Flight Human System Standard.
u. NASA/SP-2010-576, NASA Risk-Informed Decision Making Handbook.
v. NASA/SP-2011-3422, NASA Risk Management Handbook.
w. NASA/SP-2015-3709, Human Systems Integration (HSI) Practitioner's Guide.
x. NASA/SP-2016-6105, NASA Systems Engineering Handbook.
y. NASA/SP-2016-6105-SUPPL, Expanded Guidance for NASA Systems Engineering.
a. Compliance with this document is verified by the Office of the Chief Engineer by surveys, audits, reviews, and/or reporting requirements.
b. Compliance, including tailoring, for programs and projects is documented by appending a completed Compliance Matrix for Programs/Projects (see Appendix H) to the Systems Engineering Management Plan (SEMP) or other equivalent program/project documentation and by submitting the review products and plans identified in this document to the responsible NASA officials at the life-cycle and technical reviews. Programs and projects may substitute a matrix that documents compliance with their particular Center implementation of this NPR, if applicable.
P.6 Cancellation
NPR 7123.1B, NASA Systems Engineering Processes and Requirements, dated April 18, 2013.
1.1.1 Systems engineering at NASA requires the application of a systematic, disciplined engineering approach that is quantifiable, recursive, iterative, and repeatable for the development, operation, maintenance, and disposal of systems integrated into a whole throughout the life cycle of a project or program. The emphasis of SE is on safely achieving stakeholder functional, physical, operational, and performance (including human performance) requirements in the intended use environments over the system's planned life within cost and schedule constraints.
1.1.2 This NPR complements the NASA policy requirements for the administration, management, and review of all programs and projects, as specified in:
a. NPR 7120.5.
b. NPR 7120.7.
c. NPR 7120.8.
d. NPR 7150.2, NASA Software Engineering Requirements.
e. NPR 8590.1, Environmental Compliance and Restoration Program.
f. NPR 8820.2, Facility Project Requirements (FPR).
1.1.3 The processes described in this document build upon and apply best practices and lessons learned from NASA, other governmental agencies, and industry to clearly delineate a successful model to complete comprehensive technical work, reduce program and technical risk, and increase the likelihood of mission success. The requirements established in this NPR should be tailored and customized for criteria such as system/product size, complexity, criticality, acceptable risk posture, architectural level, development plans, and schedule following the guidance of Section 2.2.
1.1.4 Precedence
The order of precedence in case of conflict between requirements is 51 U.S.C. § 20113(a)(1), National Aeronautics and Space Act; NPD 1000.0, NASA Governance and Strategic Management Handbook; NPD 1000.3, The NASA Organization; NPD 7120.4, NASA Engineering and Program/Project Management Policy; and NPR 7123.1, NASA Systems Engineering Processes and Requirements.
1.1.5 Figures
1.1.5.1 Figures within this NPR are informational.
1.2.1 Institutional requirements are the responsibility of the institutional authorities. They focus on how NASA does business and are independent of any particular program or project. These requirements are issued by NASA Headquarters and by Center organizations and are normally documented in NASA Policy Directives (NPDs), NASA Procedural Requirements (NPRs), NASA Standards, Center Policy Directives (CPDs), Center Procedural Requirements (CPRs), and Mission Directorate (MD) requirements. Figure 1-1 shows the flow down from NPD 1000.0 through Program and Project Plans.
Figure 1-1 - Hierarchy of Related Documents
1.2.2 This NPR focuses on SE processes and requirements. It is one of several related Engineering and Program/Project NPRs that flow down from NPD 7120.4, as shown in Figure 1-2.
Figure 1-2 - Documentation Relationships
1.3.1 The Office of the Chief Engineer (OCE) provides leadership for technical excellence at NASA. As depicted in Figure 1-3, there are four pillars to achieving technical excellence and strengthening the SE capability. These pillars are intended to ensure that every NASA program and project meets the highest possible technical excellence.
Figure 1-3 - Technical Excellence - Pillars and Foundation
a. Clearly Documented Requirements, Policies, and Procedures. Given the complexity and uniqueness of the systems that NASA develops and deploys, clear policies and procedures are essential to mission success. All NASA technical policies and procedures flow directly from NPD 1000.0. Policies and procedures are only as effective as their implementation, facilitated by personal and organizational accountability and effective training. OCE ensures policies and procedures are consistent with and reinforce NASA's organizational beliefs and values. OCE puts in place effective, clearly documented policies and procedures, supplemented by guidance in handbooks and standards to facilitate optimal performance, rigor, and efficiency among NASA's technical workforce.
b. Effective Training and Development. NASA is fortunate that the importance of its mission allows it to attract and retain the most capable technical workforce in the world. OCE bears responsibility for providing this workforce with the technical training and development necessary to carry out the Agency's missions. At the Agency level, NASA's Academy of Program/Project and Engineering Leadership (APPEL) provides for the development of engineering leaders and teams within NASA. APPEL is augmented by technical leadership development at many Centers. Training consists of more than just transferring a set of skills. In addition to ensuring that NASA's technical workforce is knowledgeable about standards, specifications, processes, and procedures, the training available through APPEL and other curriculums is rooted in an engineering philosophy that grounds NASA's approach to technical work and decision making. These offerings give historical and philosophical perspectives that teach and reinforce NASA's organizational values and beliefs. OCE provides full support for training and development activities that will allow NASA to maximize the abilities of its technical workforce.
c. Balancing Risk. Risk is an inherent factor in any spacecraft, aircraft, or technology development. Proper risk management entails striking a balance between the tensions of program/project management and engineering independence. Engineering rigor cannot be sacrificed for schedules and budgets, and likewise programmatic concerns cannot be overlooked in the development of the technical approach to a given program or project; technical risk will be consciously and deliberately traded against budget and schedule. The Engineering Technical Authority (ETA) is responsible for ensuring risks are considered and good engineering practices are followed in technical development and implementation. OCE oversees all activities related to the exercise of ETA across the Agency. Section 2.1.6 of this document contains additional information on the ETA responsibilities.
d. Continuous Communications. Communication lies at the heart of all leadership and management challenges. Most major failures in NASA's history have stemmed in part from poor communication. Among the Agency's technical workforce, communication takes a myriad of forms: continuous risk management (CRM)/risk-informed decision making (RIDM), data sharing, knowledge management, knowledge sharing, dissemination of best practices and lessons learned, and continuous learning to name but a few. The complexity of NASA's programs and projects demands a rigorous culture of continuous and open communication that flourishes within the context of policies and procedures and knowledge transfer, while empowering individuals at all levels to raise concerns without fear of adverse consequences. OCE promotes a culture of continuous communications.
1.3.2 Personal and organizational accountability and responsibility lay the foundation for technical excellence.
a. Personal Accountability. Personal accountability means that each individual understands that he or she is responsible for the success of the mission. Each person, regardless of position or area of responsibility, contributes to success. What NASA does is so complex and interdependent that every component needs to work for the Agency to be successful. All of those who constitute NASA's technical community need to possess the knowledge and confidence to speak up when something is amiss in their or anyone else's area of responsibility to ensure mission success.
b. Organizational Responsibility. NASA's technical organizations have a responsibility to provide the proper training, tools, and environment for technical excellence. Providing the proper environment for technical excellence means establishing regular and open communication so that individuals feel comfortable exercising their personal responsibility. It also requires ensuring that those who prefer to remain in the technical field (instead of management) have a satisfying and rewarding career track (e.g., NASA Technical Fellows, ST/SL or GS-15 technical leads).
1.3.3 A central component of the environment for technical excellence is strengthening the SE capability.
1.4.1 The framework for SE capability consists of three elements—the common technical processes, tools and methods, and training for a skilled workforce. The relationship of the three elements is illustrated in Figure 1-4. The integrated implementation of the three elements of the SE framework is intended to strengthen and improve the overall capability required for the efficient and effective engineering of NASA systems. Each element is described below.
Figure 1 4 - SE Framework
a. The common technical processes of this NPR provide what has to be done to engineer quality system products and achieve mission success. These processes are applied to the integration of hardware, software, and human systems as one integrated whole. This NPR describes the common SE processes as well as standard concepts and terminology for consistent application and communication of these processes across the Agency. This NPR, supplemented by NASA/SP-2016-6105, NASA Systems Engineering Handbook, and endorsed SE standards, also describes a structure for applying the common technical processes.
b. Tools and methods range from the facilities and resources necessary to perform the technical work to the clearly documented policies, processes, and procedures that allow personnel to work safely and efficiently. Tools and methods enable the efficient and effective completion of the activities and tasks of the common technical processes. The SE capability is strengthened through the infusion of advanced methods and tools into the common technical processes to achieve greater efficiency, collaboration, and communication among distributed teams. The NASA Systems Engineering Handbook is a resource for methods and tools to support the Centers' implementation of the required technical processes in their program/projects.
c. A well-trained, knowledgeable, and experienced technical workforce is essential for improving SE capability. The workforce will be able to apply NASA and Center tools and methods for the completion of the required SE processes within the context of the program or project to which they are assigned. In addition, they will be able to effectively communicate requirements and solutions to customers, other engineers, and management to work efficiently and effectively on a team. Issues of recruitment, retention, and training are aspects included in this element. The OCE will facilitate training the NASA workforce on the application of this and associated NPRs.
1.4.2 Improvements to SE capability can be measured through assessing and updating the implementation of the common technical processes, use of adopted methods and tools, and workforce engineering training.
1.5.1 This SE NPR is organized into the following chapters:
a. The Preface describes items such as the purpose, applicability, authority, and applicable documents of this NPR.
b. Chapter 1 describes the SE framework and document organization.
c. Chapter 2 describes the institutional and programmatic requirements, including roles and responsibilities. Tailoring of SE requirements and customizing SE practices are also addressed.
d. Chapter 3 describes the core set of common Agency-level technical processes and requirements for engineering NASA system products throughout the product life-cycle.
e. Chapter 4 describes the activities and requirements to be accomplished by assigned NASA technical teams or individuals (NASA employees and NASA support contractors) when performing technical oversight of a prime or other external contractor.
f. Chapter 5 describes the life-cycle and technical review requirements throughout the program and project life-cycles. Appendix G contains entrance/success criteria guidance for each of the reviews.
g. Chapter 6 describes the Systems Engineering Management Plan (SEMP), including the SEMP role, functions, and content. Appendix J of NASA/SP-2016-6105 provides details of a generic SEMP annotated outline.
2.1.1 General
The roles and responsibilities of senior management are defined in part in NPD 1000.0 and NPD 7120.4. The roles and responsibilities of program and project managers are defined in NPR 7120.5, NPR 7120.7, NPR 7120.8, NPR 8820.2, and other NASA directives. This NPR establishes SE processes and responsibilities.
2.1.1.1 For programs and projects involving more than one Center, the governing Mission Directorate or mission support office determines whether a Center executes a program/project in a lead role or in a supporting role. For Centers in supporting roles, compliance to this NPR should be jointly negotiated and documented in the lead Center's program/project SEMP or other equivalent program/project documentation along with approval through the lead Center's ETA process.
2.1.1.2 The roles and responsibilities associated with program and project management and Technical Authority (TA) are defined in the Program and Project Management NPRs (for example, NPR 7120.5 for space flight projects). Specific roles and responsibilities of the program/project manager and the ETA related to the SEMP are defined in Sections 2.1.6 and 6.2 of this NPR.
2.1.2 Office of the Chief Engineer (OCE)
2.1.2.1 The NASA Chief Engineer is responsible for policy, oversight, and assessment of the NASA engineering and program/project management process; implements the ETA process; and serves as principal advisor to the Administrator and other senior officials on matters pertaining to the Agency's technical capability and readiness to execute NASA programs and projects.
2.1.2.2 The NASA Chief Engineer provides overall leadership for the ETA process for programs and projects, including Agency engineering policy direction, requirements, and standards. The NASA Chief Engineer hears appeals of engineering decisions when they cannot be resolved at lower levels.
2.1.3 Mission Directorate or Headquarters Program Offices
2.1.3.1 The Mission Directorate Associate Administrator (MDAA) is responsible for establishing, developing, and maintaining the Programmatic Authority (i.e., policy and procedures, programs, projects, budgets, and schedules) in managing programs and projects within their Mission Directorate.
2.1.3.2 When programs and projects are managed at Headquarters or within Mission Directorates, that program office is responsible for the requirements in this NPR. Technical teams residing at Headquarters will follow the requirements of this NPR unless tailored by the governing organization and responsible ETA. The technical teams residing at Centers will follow Center-level process requirement documents.
2.1.3.3 The Office of the Chief Information Officer provides leadership, planning, policy direction, and oversight for the management of NASA information and NASA information technology (IT).
2.1.4 Center Directors
2.1.4.1 The Center Director is responsible for establishing, developing, and maintaining the Institutional Authority (e.g., processes and procedures, human capital, facilities, and infrastructure) required to execute programs and projects assigned to their Center. This includes:
a. Ensuring the Center is capable of accomplishing the programs, projects, and other activities assigned to it in accordance with Agency policy and the Center's best practices and institutional policies by establishing, developing, and maintaining institutional capabilities (processes and procedures, human capital—including trained/certified program/project personnel, facilities, and infrastructure) required for the execution of programs and projects.
b. Performing periodic program and project reviews to assess technical and programmatic progress to ensure performance in accordance with their Center's and the Agency requirements, procedures, processes, and other documentation.
c. Working with the Mission Directorate and the program and project managers, once assigned, to assemble the program/project team(s) and to provide needed Center resources.
d. Providing support and guidance to programs and projects in resolving technical and programmatic issues and risks.
2.1.4.2 The Center Director is responsible for developing the Center's ETA policies and practices consistent with Agency policies and standards. The Center Director is the Center ETA responsible for Center engineering design processes, specifications, rules, best practices, and other activities necessary to fulfill mission performance requirements for programs, projects, and/or major systems implemented by the Center. The Center Director delegates the Center ETA implementation responsibility to an individual in the Center's engineering leadership. The Center ETA supports processing changes to, and waivers or deviations from, requirements that are the responsibility of the ETA. This includes all applicable Agency and Center engineering directives, requirements, procedures, and standards.
Note: Centers may employ and tailor relevant government or industry standards that meet the intent of the requirements established in this NPR to augment or serve as the basis for their processes. A listing of endorsed technical standards is maintained on the NASA Technical Standards System under "Endorsed Standards" https://standards.nasa.gov/endorsed_standards.
2.1.4.3 [SE-01] through [SE-05] deleted.
Note: Rather than resequence the remaining requirements, the original requirement numbering was left intact in case Centers or other organizations refer to these requirement numbers in their flow-down requirement documents. Appendix J is provided to account for the deleted requirements. For each requirement that was deleted, the justification for its deletion is noted.
2.1.5 Technical Teams
2.1.5.1 Systems engineering is implemented by the technical team in accordance with the program/project SEMP or other equivalent program/project documentation. The makeup and organization of each technical team is the responsibility of each Center or program and includes all the personnel required to implement the technical aspects of the program/project.
2.1.5.2 The technical team, in conjunction with the Center's ETA, is responsible for completing the compliance matrix in Appendix H, capturing any tailoring, and including it in the SEMP or other equivalent program/project documentation.
2.1.5.3 For systems that contain software, the technical team ensures that software developed within NASA, or acquired from other entities, complies with NPR 7150.2.
a. NPR 7150.2 elaborates on the requirements in NPR 7123.1 and determines the applicability of requirements based on the Agency's software classification.
b. NPD 7120.4 contains additional Agency principles for the acquisition, development, maintenance, and management of software.
2.1.5.4 The technical team ensures that human systems integration activities, products, planning, and execution align with NASA/SP-2015-3709, Human Systems Integration (HSI) Practitioner's Guide.
2.1.6 Engineering Technical Authority
2.1.6.1 The ETA establishes and is responsible for the engineering design processes, specifications, rules, best practices, and other activities necessary to fulfill programmatic mission performance requirements. Centers delegate ETA to the level appropriate for the scope and size of the program/project, which may be Center engineering leadership or individuals. When ETA is used in this document, it refers generically to different levels of ETA.
2.1.6.2 ETAs or their delegates at the program or project level:
a. Serve as members of program or project control boards, change boards, and internal review boards.
b. Work with the Center management and other TA personnel to ensure that the quality and integrity of program or project processes, products, and standards of performance related to engineering, SMA, and health and medical reflect the level of excellence expected by the Center and the TA community.
c. Ensure that requests for waivers or deviations from ETA requirements are submitted to, and acted on, by the appropriate level of ETA.
d. Assist the program or project in making risk-informed decisions that properly balance technical merit, cost, schedule, and safety across the system.
e. Provide the program or project with the ETA view of matters based on their knowledge and experience and raise needed dissenting opinions on decisions or actions. (See Dissenting Opinion Sections of NPR 7120.5, NPR 7120.8, and NPR 7120.7.)
f. Serve as an effective part of NASA's overall system of checks and balances.
2.1.6.3 The ETA for the program or project leads and manages the system engineering activities. (Note that these responsibilities can be delegated by the ETA to Chief Engineer or other personnel as needed). A Center may have more than one engineering organization and delegates ETA to different areas as needed. The ETA may be delegated as appropriate to the size, complexity, and type of program/project. For example, ETA may be delegated to a line manager that is independent of the project for smaller projects or to the CIO for purely IT projects.
2.1.6.4 To support the program/project and maintain ETA independence and an effective check and balance system, the ETA:
a. Will seek concurrence by the program/project manager when a program/project-level ETA is appointed.
b. Cannot approve a request for a waiver or deviation from a non-technical derived requirement established by a Programmatic Authority.
c. May approve a request for a waiver or deviation from a technical derived requirement if he/she ensures that the appropriate independent Institutional Authority subject matter expert who is the steward for the involved technology, has concurred in the decision to approve the requirement waiver.
2.1.6.5 Although a limited number of individuals make up the ETA, their work is enabled by the contributions of the program's or project's working-level engineers and other supporting personnel (e.g., contracting officers). The working-level engineers do not have formally delegated Technical Authority and consequently may not serve in an ETA capacity. These engineers perform the detailed engineering and analysis for the program/project with guidance from their Center management and/or lead discipline engineers and support from the Center engineering infrastructure. They deliver the program/project products (e.g., hardware, software, designs, analysis, and technical alternatives) that conform to applicable programmatic, Agency, and Center requirements. They are responsible for raising issues to the program/project manager, Center engineering management, and/or the program/project ETA and are a key resource for resolving these issues.
2.1.6.6 Requirement [SE-06] concerning SEMP approval was moved to Section 6.1.8.
Tailoring can be differentiated from customizing as described in NASA/SP-2016-6105. Tailoring is removing requirements by use of waiver or deviation. Customizing is meeting the intent of the requirement through alternative approaches and does not require waivers or deviations.
2.2.1 Tailoring SE Requirements
2.2.1.1 SE requirements tailoring is the process used to seek relief from SE NPR requirements when that relief is consistent with program or project objectives, acceptable risk, and constraints.
2.2.1.2 The tailoring process (which can occur at any time in the program or project life cycle) results in deviations or waivers to requirements depending on the timing of the request (see Appendix A for definition of deviation and waiver).
2.2.1.3 The results of the program/project technical team's tailoring SE requirements from either this NPR, or a particular Center's implementation of this NPR, will be documented in the SEMP or other equivalent project documentation, along with supporting rationale that includes the risk evaluation, and documented approvals through the Center's ETA process.
2.2.2 Customizing SE Practices
2.2.2.1 Customizing is the adaptation of SE practices that are used to accomplish the SE requirements as appropriate to the size, complexity, and acceptable risk of the program/project.
2.2.2.2 Technical teams under the guidance of the project ETA are encouraged to customize these recommended SE practices so that the intent of the SE practice is being met in the most effective and efficient manner. The results of this customization do not require waivers or deviations but should be documented in the program/project SEMP or other equivalent program/project documentation.
2.2.3 Considerations for Tailoring or Customizing
Refer to NASA, SP-2016-6105 for examples of tailoring and customizing.
2.2.3.1 Considerations for tailoring or customizing should include but are not limited to:
a. Scope and visibility (e.g., organizations and partnerships involved, international agreements, amount of effort required).
b. Risk tolerance and failure consequences.
c. System size, functionality, and complexity (e.g., human space flight/flagship science vs. subscale technology demonstration).
d. Human involvement (e.g., human interfaces, critical crew (flight, ground) functions, interaction with, and control/oversight of (semi-) autonomous systems).
e. Impact on Agency IT security and national security.
f. Impact on other systems.
g. Longevity.
h. Serviceability (both ground and in-flight).
i. Constraints (including cost, schedule, degree of insight/oversight permitted with partnerships or international agreements).
j. Safety, quality, and mission assurance.
k. Current level of technology available.
l. Availability of industrial capacity.
3.1.1 This chapter establishes the core set of common technical processes and requirements to be used by NASA programs or projects in engineering system products during all life-cycle phases to meet phase success criteria and program/project objectives. The 17 common technical processes are enumerated according to their description in this chapter and their interactions shown in Figure 3-1. This SE common technical processes model illustrates the use of:
a. System design processes for "top-down" design of each product in the system structure.
b. Product realization processes for "bottom-up" realization of each product in the system structure.
c. Cross-cutting technical management processes for planning, assessing, and controlling the implementation of the system design and product realization processes and to guide technical decision making (decision analysis).
3.1.2 The SE common technical processes model is referred to as an "SE engine" in this NPR to stress that these common technical processes are used to drive the development of the system products and associated work products required by management to satisfy the applicable product life-cycle phase success criteria while meeting stakeholder expectations within cost, schedule, and risk constraints.
3.1.3 This chapter identifies the following for each of the 17 common technical processes:
a. The specific requirement for Program/Project Managers to identify and implement (as defined in Section 3.2.1) the ETA-approved process.
b. A brief description of how the process is used as an element of the Systems Engineering Engine.
3.1.4 Typical practices for each process are identified in NASA/SP-2016-6105, where each process is described in terms of purpose, inputs, outputs, and activities. It should be emphasized that the practices documented in the handbook do not represent additional requirements that need to be executed by the technical team but provide best practices associated with the 17 common technical processes. As the technical team develops a tailored and customized approach for the application of these processes, sources of SE guidance and technical standards, such as NASA/SP-2016-6105 and endorsed industry standards, should be considered. Appendix I provides a list of NASA and endorsed military and industry standards applicable to Systems Engineering and available on the NASA Technical Standards System, found at https://standards.nasa.gov/endorsed_standards, and should be applied as appropriate for each program or project. For additional guidance on mapping HSI into the SE Engine, refer to NASA/SP-2015-3709, Section 3.0.
Figure 3 1 - Systems Engineering (SE) Engine
3.1.5 The context in which the common technical processes are used is provided below: (Refer to "The Common Technical Processes and the SE Engine" in NASA/SP-2016-6105 for further information.)
3.1.5.1 The common technical processes are applied to each product layer to concurrently develop the products that will satisfy the operational or mission functions of the system (end products) and that will satisfy the life-cycle support functions of the system (enabling products). In this document, a product layer is a horizontal slice of the product breakdown hierarchy and includes both the end product and its associated enabling products. The enabling products facilitate the activities of system design, product realization, operations and mission support, sustainment, and end-of-product-life disposal or recycling by having the products and services available when needed.
3.1.5.2 The common technical processes are applied to design a system solution definition for each product layer down and across each level of the system structure and to realize the product layer end products up and across the system structure. Figure 3-2 illustrates how the three major sets of processes of the Systems Engineering (SE) Engine (system design processes, product realization processes, and technical management processes) are applied to each product layer within a system structure.
Figure 3-2 - Application of SE Engine Common Technical Processes
Within System Structure
3.1.5.3 The common technical processes are used to define the product layers of the system structure in each applicable phase of the relevant life-cycle to generate work products and system products needed to satisfy the success criteria of the applicable phase. Figure 3-3 depicts the sequencing of the processes.
Figure 3-3 - Sequencing of the Common Technical Processes
3.1.5.4 There are four system design processes applied to each product-based product layer from the top to the bottom of the system structure:
a. Stakeholder Expectation Definition.
b. Technical Requirements Definition.
c. Logical Decomposition.
d. Design Solution Definition. (See Figure 3-1 and Figure 3-2.)
3.1.5.5 During the application of these four processes to a product layer, it is expected that there will be a need to apply activities from other processes yet to be completed and to repeat process activities already performed to arrive at an acceptable set of requirements and solutions. There also will be a need to interact with the technical management processes to aid in identifying and resolving issues and making decisions between alternatives. For software products, the technical team ensures that the process executions comply with NPR 7150.2, software design requirements. The technical team also ensures that human capabilities and limitations are understood and how those human capabilities or limitations impact the hardware and software of any given system in terms of design. Refer to NASA/SP-2015-3709.
3.1.5.6 There are five product realization processes. Four of the product realization processes are applied to each end product of a product layer from the bottom to the top of the system structure:
a. Either Product Implementation for the lowest level or Product Integration for subsequent levels.
b. Product Verification.
c. Product Validation.
d. Product Transition. (See Figure 3-1 and Figure 3-2.)
3.1.5.7 The form of the end product realized will depend on the applicable product life-cycle phase, location within the system structure of the product layer containing the end product, and the success criteria of the phase. Typical early phase products are reports, models, simulations, mockups, prototypes, or demonstrators. Typical later phase products may take the form of qualification units, final mission products, and fully assembled payloads and instruments.
3.1.5.8 There are eight technical management processes—Technical Planning, Technical Requirements Management, Interface Management, Technical Risk Management, Configuration Management, Technical Data Management, Technical Assessment, and Decision Analysis. (See Figure 3-1 and Figure 3-2.) These technical management processes supplement the program and project management directives (e.g., NPR 7120.5), which specify the technical activities for which program and project managers are responsible.
3.1.5.9 Note that during the design and realization phases of a project, all 17 processes are used. After the end product is developed and placed into operations the Technical Management processes in the center chamber of the SE Engine will continue to be employed. For more information on the use of the SE Engine during the operational phase, refer to NASA/SP-2016-6105.
3.1.5.10 The common technical processes are applied by assigned technical teams and individuals trained in the requirements of this NPR.
3.1.5.11 The assigned technical teams and individuals use the appropriate and available sets of tools and methods to accomplish required common technical process activities. This includes the use of modeling and simulation as applicable to the product phase, location of the product layer in the system structure, and the applicable phase success criteria.
3.1.6 Relationship of the SE Engine to the SE Vee.
The NASA SE Engine is a highly versatile representation of the core SE processes necessary to properly engineer a system. It can be used for any type of life-cycle including waterfall, spiral, and agile. It allows for use in very simple to highly complex systems. The NASA SE Engine had its heritage in a classic SE Vee, and if being used for a simple one-pass waterfall-type life-cycle, the right and left chambers of the engine can be represented as shown in Figure 3-4. For a more detailed description of how the SE Engine evolved from the SE Vee, refer to the NASA Systems Engineering Handbook.
Figure 3-4 - SE Engine Implemented for a Simple Single-Pass Waterfall-Type Life Cycle
3.2.1 For Section 3.2, "identify" means to either use an approved process or a customized process that is approved by the ETA or their delegate. "Implement" includes documenting and communicating the approved process, providing resources to execute the process, providing training on the process, and monitoring and controlling the process. The technical team is responsible for the execution of these 17 required processes per Section 2.1.5.
3.2.2 Stakeholder Expectations Definition Process
3.2.2.1 Program/Project Managers shall identify and implement an ETA-approved Stakeholder Expectations Definition process to include activities, requirements, guidelines, and documentation, as tailored and customized for the definition of stakeholder expectations for the applicable product layer [SE-07].
3.2.2.2 The Stakeholder Expectations Definition process is used to elicit and define use cases, scenarios, concept of operations, and stakeholder expectations for the applicable product life-cycle phases and product layer. This includes expectations such as:
a. Operational end products and life-cycle-enabling products of the product layer.
b. Affordability.
c. Operator or user interfaces.
d. Expected skills and capabilities of operators or users.
e. Expected number of simultaneous users.
f. System and human performance criteria.
g. Technical authority, standards, regulations, and laws.
h. Factors such as health and medical, safety, planetary protection, orbital debris, quality, cybersecurity, context of use by humans, reliability, availability, maintainability, electromagnetic compatibility, interoperability, testability, transportability, supportability, usability, and disposability.
i. For crewed missions, crew health and performance capabilities and limitations, risk posture, crew survivability, and system habitability.
j. Local management constraints on how work will be done (e.g., operating procedures).
3.2.2.3 The baselined stakeholder expectations are used for validation of the product layer end product during product realization. At this point, Measures of Effectiveness (MOEs) are defined. For more information of MOEs refer to NASA/SP-2016-6105, NASA Systems Engineering Handbook.
3.2.3 Technical Requirements Definition Process
3.2.3.1 Program/Project Managers shall identify and implement an ETA-approved Technical Requirements Definition process to include activities, requirements, guidelines, and documentation, as tailored and customized for the definition of technical requirements from the set of agreed upon stakeholder expectations for the applicable product layer [SE-08].
3.2.3.2 The technical requirements definition process is used to transform the baselined stakeholder expectations into unique, quantitative, and measurable technical requirements expressed as "shall" statements that can be used for defining a design solution for the product layer end product and related enabling products. This process also includes validation of the requirements to ensure that the requirements are well-formed (clear and unambiguous), complete (agrees with customer and stakeholder needs and expectations), consistent (conflict free), and individually verifiable and traceable to a higher level requirement or goal. As part of this process, Measures of Performance (MOPs) and Technical Performance Measures (TPMs) are defined. For more information of MOPs and TPMs, refer to NASA/SP-2016-6105, NASA Systems Engineering Handbook.
3.2.4 Logical Decomposition Process
3.2.4.1 Program/Project Managers shall identify and implement an ETA-approved Logical Decomposition process to include activities, requirements, guidelines, and documentation, as tailored and customized for logical decomposition of the validated technical requirements of the applicable product layer [SE-09].
3.2.4.2 The logical decomposition process is used to improve understanding of the defined technical requirements and the relationships among the requirements (e.g., functional, behavioral, performance, and temporal) and to transform the defined set of technical requirements into a set of logical decomposition models and their associated set of derived technical requirements for lower levels of the system and for input to the design solution definition process.
3.2.5 Design Solution Definition Process
3.2.5.1 Program/Project Managers shall identify and implement an ETA-approved Design Solution Definition process to include activities, requirements, guidelines, and documentation, as tailored and customized for designing product solution definitions within the applicable product layer that satisfy the derived technical requirements [SE-10].
3.2.5.2 The Design Solution Definition process is used to translate the outputs of the logical decomposition process into a design solution definition that is in a form consistent with the product life-cycle phase and product layer location in the system structure and that will satisfy phase success criteria. This includes transforming the defined logical decomposition models and their associated sets of derived technical requirements into alternative solutions, then analyzing each alternative to be able to select a preferred alternative and fully defining that alternative into a final design solution definition that will satisfy the requirements.
3.2.5.3 These design solution definitions will be used for generating end products, either by using the product implementation process or product integration process, as a function of the position of the product layer in the system structure and whether there are additional subsystems of the end product that need to be defined. The output definitions from the design solution (end product specifications) will be used for conducting product verification.
3.2.6 Product Implementation Process
3.2.6.1 Program/Project Managers shall identify and implement an ETA-approved Product Implementation process to include activities, requirements, guidelines, and documentation, as tailored and customized for implementation of a design solution definition by making, buying, or reusing an end product of the applicable product layer [SE-11].
3.2.6.2 The Product Implementation Process is used to generate a specified product of a product layer through buying, making, or reusing in a form consistent with the product life-cycle phase success criteria and that satisfies the design solution definition (e.g., drawings, specifications).
3.2.7 Product Integration Process
3.2.7.1 Program/Project Managers shall identify and implement an ETA-approved Product Integration process to include activities, requirements, guidelines, and documentation, as tailored and customized for the integration of lower level products into an end product of the applicable product layer in accordance with its design solution definition [SE-12].
3.2.7.2 The Product Integration Process is used to transform lower level, verified and validated end products into the desired end product of the higher level product layer through assembly and integration.
3.2.8 Product Verification Process
3.2.8.1 Program/Project Managers shall identify and implement an ETA-approved Product Verification process to include activities, requirements/specifications, guidelines, and documentation, as tailored and customized for verification of end products generated by the product implementation process or product integration process against their design solution definitions [SE-13].
3.2.8.2 The Product Verification process is used to demonstrate that an end product generated from product implementation or product integration conforms to its requirements as a function of the product life-cycle phase and the location of the product layer end product in the system structure. Special attention is given to demonstrating satisfaction of the MOPs defined for each MOE during performance of the technical requirements definition process.
3.2.9 Product Validation Process
3.2.9.1 Program/Project Managers shall identify and implement an ETA-approved Product Validation process to include activities, requirements, guidelines, and documentation, as tailored and customized for validation of end products generated by the product implementation process or product integration process against their stakeholder expectations [SE-14].
3.2.9.2 The Product Validation process is used to confirm that a verified end product generated by product implementation or product integration fulfills (satisfies) its intended use when placed in its intended environment and to ensure that any anomalies discovered during validation are appropriately resolved prior to delivery of the product (if validation is done by the supplier of the product) or prior to integration with other products into a higher level assembled product (if validation is done by the receiver of the product). The validation is done against the set of baselined stakeholder expectations. Special attention should be given to demonstrating satisfaction of the MOEs identified during performance of the stakeholder expectations definition process. The type of product validation is a function of the form of the product and product life-cycle phase and in accordance with an applicable customer agreement.
3.2.10 Product Transition Process
3.2.10.1 Program/Project Managers shall identify and implement an ETA-approved Product Transition process to include activities, requirements, guidelines, and documentation, as tailored and customized for transitioning end products to the next higher level product layer customer or user [SE-15].
3.2.10.2 The Product Transition process is used to transition a verified and validated end product that has been generated by product implementation or product integration to the customer at the next level in the system structure for integration into an end product or, for the top-level end product, transitioned to the intended end user. The form of the product transitioned will be a function of the product life-cycle phase and the location within the system structure of the product layer in which the end product exists.
3.2.11 Technical Planning Process
3.2.11.1 Program/Project Managers shall identify and implement an ETA-approved Technical Planning process to include activities, requirements, guidelines, and documentation, as tailored and customized for planning the technical effort [SE-16].
3.2.11.2 The Technical Planning process is used to plan for the application and management of each common technical process, including tailoring of organizational requirements and requirements specified in this NPR. It is also used to identify, define, and plan the technical effort applicable to the product life-cycle phase for product layer location within the system structure and to meet program/project objectives and product life-cycle phase success criteria. A key document generated by this process is the SEMP (See Chapter 6).
3.2.12 Requirements Management Process
3.2.12.1 Program/Project Managers shall identify and implement an ETA-approved Requirements Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of requirements throughout the system life-cycle [SE-17].
3.2.12.2 The Requirements Management process is used to:
a. Manage the product requirements identified, baselined, and used in the definition of the product layer products during system design.
b. Provide bidirectional traceability back to the top product layer requirements.
c. Manage the changes to established requirement baselines over the life-cycle of the system products.
3.2.13 Interface Management Process
3.2.13.1 Program/Project Managers shall identify and implement an ETA-approved Interface Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of the interfaces defined and generated during the application of the system design processes [SE-18].
3.2.13.2 The Interface Management process is used to:
d. Establish and use formal interface management to assist in controlling system product development efforts when the efforts are divided between Government programs, contractors, and/or geographically diverse technical teams within the same program or project.
e. Maintain interface definition and compliance among the end products and enabling products that compose the system, as well as with other systems with which the end products and enabling products will interoperate.
3.2.14 Technical Risk Management Process
3.2.14.1 Program/Project Managers shall identify and implement an ETA-approved Technical Risk Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of the risk identified during the technical effort [SE-19].
3.2.14.2 The Technical Risk Management process is used to make risk-informed decisions and examine, on a continuing basis, the potential for deviations from the program/project plan and the consequences that could result should they occur. This enables risk-handling activities to be planned and invoked as needed across the life of the program or project to mitigate impacts on achieving product life-cycle phase success criteria and meeting technical objectives. The technical team supports the development of potential health and medical, safety, cost, and schedule impacts for identified technical risks and any associated mitigation strategies. NPR 8000.4, Agency Risk Management Procedural Requirements, is to be used as a source document for defining this process and NPR 8705.5, Technical Probabilistic Risk Assessment (PRA) Procedures for Safety and Mission Success for NASA Programs and Projects, provides one means of identifying and assessing technical risk. While the focus of this process is the management of technical risk, the highly interdependent nature of health and medical, safety, technical, cost, and schedule risks require the broader program/project team to consistently address risk management with an integrated approach. NASA/SP-2011-3422, NASA Risk Management Handbook, provides guidance for managing risk in an integrated fashion.
3.2.15 Configuration Management Process
3.2.15.1 Program/Project Managers shall identify and implement an ETA-approved Configuration Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for configuration management [SE-20].
3.2.15.2 The Configuration Management process for end products, enabling products, and other work products placed under configuration control is used to:
a. Identify the items to be placed under configuration control.
b. Identify the configuration of the product or work product at various points in time.
c. Systematically control changes to the configuration of the product or work product.
d. Maintain the integrity and traceability of the configuration of the product or work product throughout its life.
e. Preserve the records of the product or end product configuration throughout its life-cycle, dispositioning them in accordance with NPR 1441.1, NASA Records Management Program Requirements.
3.2.16 Technical Data Management Process
3.2.16.1 Program/Project Managers shall identify and implement an ETA-approved Technical Data Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of the technical data generated and used in the technical effort [SE-21].
3.2.16.2 The Technical Data Management Process is used to plan for, acquire, access, manage, protect, and use data of a technical nature to support the total life-cycle of a system. This process is used to capture trade studies, cost estimates, technical analyses, reports, and other important information.
3.2.17 Technical Assessment Process
3.2.17.1 Program/Project Managers shall identify and implement an ETA-approved Technical Assessment process to include activities, requirements, guidelines, and documentation, as tailored and customized for making assessments of the progress of planned technical effort and progress toward requirements satisfaction [SE-22].
3.2.17.2 The Technical Assessment process is used to help monitor progress of the technical effort and provide status information for support of the system design, product realization, and technical management processes. A key aspect of the technical assessment process is the conduct of life-cycle and technical reviews throughout the system life-cycle in accordance with Chapter 5.
3.2.18 Decision Analysis Process
3.2.18.1 Program/Project Managers shall identify and implement an ETA-approved Decision Analysis process to include activities, requirements, guidelines, and documentation, as tailored and customized for making technical decisions [SE-23].
3.2.18.2 The Decision Analysis process, including processes for identification of decision criteria, identification of alternatives, analysis of alternatives, and alternative selection, is applied to technical issues to support their resolution. It considers relevant data (e.g., engineering performance, quality, and reliability) and associated uncertainties. Decision analysis is used throughout the system life-cycle to formulate candidate decision alternatives and evaluate their impacts on health and medical, safety, technical, cost, and schedule performance. NASA/SP-2010-576, NASA Risk-Informed Decision Making Handbook, provides guidance for analyzing decision alternatives in a risk-informed fashion.
4.1.1 Work contracted in support of programs and projects is critical to mission success. Inputs or requirements in support of a solicitation (such as Requests for Proposals (RFP)) typically include a Statement of Work, product requirements, Independent Government Estimate, Data Requirements List, Deliverables List, and Surveillance Plan. These should be developed considering the risk posture of the program/project and fit within the cost and schedule constraints. In addition to developing the product requirements, a critical aspect of the solicitation is for the technical team to define the insight and oversight requirements. "Insight" is a monitoring activity, whereas "oversight" is an exercise of authority by the Government. The Federal Acquisition Regulation and the NASA Supplement to the Federal Acquisition Regulation govern the acquisition planning, contract formation, and contract administration process. Authority to interface with the contractor can be delegated only by the contracting officer. The activities listed in Section 4.2 will be coordinated with the cognizant contracting officer. Detailed definitions for insight and oversight are provided in 48 CFR, sbpt. 1846.4. As stated in Section 1.1.3, the requirements should be appropriately tailored and customized for system/product size, complexity, criticality, acceptable risk posture, and architectural level.
4.1.2 This chapter defines a minimum set of technical activities and requirements for a NASA program/project technical team to perform before contract award, during contract performance, and upon completion of the contract on program/projects. These activities and requirements are intended to supplement the common technical process activities and requirements of Chapter 3 and thus enhance the outcome of the contracted effort and ensure the required integration between work performed by the contractor and the program or project.
4.2.1 The NASA technical team shall define the engineering activities for the periods before contract award, during contract performance, and upon contract completion in the SEMP or other equivalent program/project documentation [SE-24].
4.2.2 The content of Appendix J of NASA/SP-2016-6105 should be used as a guide in the development of the SEMP or other equivalent program/project documentation.
4.2.3 The NASA technical team shall establish the technical inputs to the solicitation appropriate for the product(s) to be developed, including product requirements and Statement of Work tasks [SE-25].
4.2.3.1 The technical team uses knowledge of the 17 common technical processes to identify products and desired practices to include in the solicitation.
4.2.4 The NASA technical team shall determine the technical work products to be delivered by the offeror or contractor, to include contractor documentation that specifies the contractor's SE approach to the scope of activities described by the 17 common technical processes [SE-26].
4.2.5 The NASA technical team shall provide the requirements for technical insight and oversight activities planned in the NASA SEMP or other equivalent program/project documentation to the contracting officer for inclusion in the solicitation [SE-27].
4.2.6 Care should be taken that no requirements or solicitation information is divulged prior to the release of the solicitation.
4.2.7 The NASA technical team shall participate in the evaluation of offeror proposals in accordance with applicable NASA and Center source selection procedures [SE-28].
4.2.7.1 This requirement ensures that the proposal addresses the requirements, products, and processes specified in the solicitation.
4.3.1 The NASA technical team, under the authority of the contracting officer, shall perform the technical insight and oversight activities established in the contract including modifications to the original contract [SE-29].
4.3.2 The requirements levied on the technical team in Section 4.2 for establishing the contract applies to any modifications or additions to the original contract.
4.4.1 The NASA technical team shall participate in the review(s) to finalize Government acceptance of the deliverables [SE-30].
4.4.2 The NASA technical team shall participate in product transition as defined in the NASA SEMP or other equivalent program/project documentation [SE-31].
5.1.1 NPR 7120.5 defines four types of programs that may contain projects:
a. Uncoupled programs.
b. Loosely coupled programs.
c. Tightly coupled programs.
d. Single-project programs.
5.1.1.1 Which life-cycle a program/project uses will be dependent on what type of program/project it is and whether the program/project is producing products for space flight, advanced technology development, information technology, infrastructure, or other applications.
5.1.1.2 A specific life-cycle may be required by associated project management NPRs. For example, NPR 7120.5 defines the life-cycles for space flight programs and projects, and NPR 7120.7 defines life-cycles for IT. For Announcement of Opportunity (AO) driven projects, refer to NPR 7120.5, Section 2.2.7.1. For purposes of illustration, life-cycles from NPR 7120.5 are repeated here in Figures 5-1 through 5-4.
5.1.2 The application of the common technical processes within each life-cycle phase produces technical results and work products that provide inputs to life-cycle and technical reviews and support informed management decisions for progressing to the next life-cycle phase.
5.1.3 Each program and project will perform the life-cycle reviews as required by or tailored in accordance with their governing program/project management NPR, applicable Center policies and procedures, and the requirements of this document. These reviews provide a periodic assessment of a program or project's technical and programmatic status and health at key points in the life-cycle. The technical team provides the technical inputs to be incorporated into the overall program/project review package. Appendix G provides guidelines for the entrance and success criteria for each of these reviews with a focus on the technical products. Additional programmatic work products may also be required by the governing program/project NPR. Programs/projects are expected to tailor the reviews and customize the entrance/success criteria as appropriate to the size/complexity and unique needs of their activities. Approved tailoring is captured in the SEMP or other equivalent program/project documents.
5.1.4 The progress between life-cycle phases is marked by key decision points (KDPs). At each KDP, management examines the maturity of the technical aspects of the program/project. For example, management evaluates the adequacy of the resources (staffing and funding) allocated to the planned technical effort, the technical maturity of the product, the management of technical and nontechnical internal issues and risks, and the responsiveness to any changes in stakeholder expectations. If the technical and management aspects of the program/project are satisfactory, including the implementation of corrective actions, then the program/project can be approved by the designated Decision Authority to proceed to the next phase. Program and project management NPRs (NPR 7120.5, NPR 7120.7, and NPR 7120.8) contain further details relating to life-cycle progress.
Note: For example only. Refer to Figure 2-2 in NPR 7120.5 for the official life cycle. Table 2-3 reference in Footnote 5 above is in NPR 7120.5.
Figure 5 1 - NASA Uncoupled and Loosely Coupled Program Life-Cycle
Note: For example only. Refer to Figure 2-3 in NPR 7120.5 for the official life cycle. Table 2-4 reference in Footnote 5 above is in NPR 7120.5.
Figure 5-2 - NASA Tightly Coupled Program Life-Cycle
Note: For example only. Refer to Figure 2-4 in NPR 7120.5 for the official life cycle. Table 2-5 reference in Footnote 5 above is in NPR 7120.5.
Figure 5-3 - NASA Single-Project Program Life-Cycle
Note: For example only. Refer to Figure 2-5 in NPR 7120.5 for the official life cycle. Table 2-5 reference in Footnote 2 above is in NPR 7120.5.
Figure 5 4 - The NASA Project Life-Cycle
5.1.5 Life-cycle reviews are event based and occur when the entrance criteria for the applicable review are satisfied. (Appendix G provides guidance.) They occur based on the maturity of the relevant technical baseline as opposed to calendar milestones (e.g., the quarterly progress review, the yearly summary).
5.1.6 Accurate assessment of technology maturity is critical to technology advancement and its subsequent incorporation into operational products. The program/project ensures that Technology Readiness Levels (TRLs) and/or other measures of technology maturity are used to assess maturity throughout the life-cycle of the program/project. When other measures of technology maturity are used, they should be mapped back to TRLs. The definition of the TRLs for hardware and software are defined in Appendix E. Moving to higher levels of technology maturity requires an assessment of a range of capabilities for design, analysis, manufacture, and test. Measures for assessing technology maturity are described in NASA/SP-2016-6105. The initial technology maturity assessment is done in the Formulation phase and updated at program/project status reviews. The program/project approach for maturing and assessing technology is typically captured in a Technology Development Plan, the SEMP, or other equivalent program/project documentation.
5.2.1 Planning
5.2.1.1 The technical team shall develop and document plans for life-cycle and technical reviews for use in the program/project planning process [SE-32].
5.2.1.2 The life-cycle and technical review schedule, as documented in the SEMP or other equivalent program/project documentation, will be reflected in the overall program/project plan. The results of each life-cycle and technical review will be used to update the technical review plan as part of the SEMP (or other equivalent program/project documentation) update process. The review plans, data, and results should be maintained and dispositioned as Federal Records.
5.2.1.3 The technical team ensures that system aspects interfacing with crew or human operators (e.g., users, maintainers, assemblers, and ground support personnel) are included in all life-cycle and technical reviews and that HSI requirements are implemented. Additional HSI guidance is provided in NASA/SP-2015-3709 and NASA/SP-2016-6105/SUPPL Expanded Guidance for NASA Systems Engineering Volumes 1 and 2.
5.2.1.4 The technical team ensures that system aspects represented or implemented in software are included in all life-cycle and technical reviews and that all software review requirements are implemented. Software review requirements are provided in NPR 7150.2, with guidance provided in NASA-HDBK-2203, NASA Software Engineering Handbook.
5.2.1.5 The technical team shall participate in the life-cycle and technical reviews as indicated in the governing program/project management NPR [SE-33]. Additional description of technical reviews is provided in NASA/SP-2016-6105, NASA Systems Engineering Handbook and in NASA/SP-2014-3705, NASA Spaceflight Program & Project Management Handbook. (For requirements on program and project life cycles and management reviews, see the appropriate NPR, e.g., NPR 7120.5.)
5.2.2 Conduct
5.2.2.1 The technical team shall participate in the development of entrance and success criteria for each of the respective reviews [SE-34]. The technical team should utilize the guidance defined in Appendix G as well as Center best practices for defining entrance and success criteria.
5.2.2.2 The technical team shall provide the following minimum products at the associated life-cycle review, at the indicated maturity level. If the associated life-cycle review is not held, the technical team will need to seek a waiver or deviation to tailor these requirements. If the associated life-cycle review is held but combined with other life-cycle reviews or resequenced, this is considered customization and therefore no waiver is required (but approach should still be documented in the SEMP or Review Plan for clarity).
a. Mission Concept Review (MCR):
(1) Baselined stakeholder identification and expectation definitions [SE-35].
(2) Baselined concept definition [SE-36].
(3) Approved MOE definition [SE-37].
b. System Requirements Review (SRR):
(1) Baselined SEMP (or other equivalent program/project documentation) for projects, single-project programs, and one-step AO programs [SE-38].
(2) Baselined requirements [SE-39].
c. Mission Definition Review/System Definition Review (MDR/SDR):
(1) Approved TPM definitions [SE-40].
(2) Baselined architecture definition [SE-41].
(3) Baselined allocation of requirements to next lower level [SE-42].
(4) Initial trend of required leading indicators [SE-43].
(5) Baseline SEMP (or other equivalent program/project documentation) for uncoupled, loosely coupled, tightly coupled, and two-step AO programs [SE-44].
d. Preliminary Design Review (PDR):
(1) Preliminary design solution definition [SE-45].
e. Critical Design Review (CDR):
(1) Baseline detailed design [SE-46].
f. System Integration Review (SIR):
(1) Updated integration plan [SE-47].
(2) Preliminary Verification and Validation (V&V) results [SE-48].
g. Operational Readiness Review (ORR):
(1) [SE-49] deleted.
(2) [SE-50] deleted.
(3) Preliminary decommissioning plans [SE-51].
h. Flight Readiness Review (FRR):
(1) Baseline disposal plans [SE-52].
(2) Baseline V&V results [SE-53].
(3) Final certification for flight/use [SE-54].
i. Decommissioning Review (DR):
(1) Baseline decommissioning plans [SE-55].
j. Disposal Readiness Review (DRR):
(1) Updated disposal plans [SE-56].
5.2.2.3 Table 5-1 shows the maturity of primary SE work products at the associated life-cycle reviews for all types and sizes of programs/projects. The required SE products identified above are notated with "**" in the table. For further description of the primary SE work products, refer to Appendix G. For additional guidance on software product maturity for program/project life-cycle reviews, refer to NASA-HDBK-2203. Additional programmatic work products are required by the governing program/project management NPRs, but not listed herein.
5.2.2.4 The expectation for work products identified as "baselined" in Section 5.2.2.2 and Table 5-1 is that they will be at least final drafts going into the designated life-cycle review. Subsequent to the review, the final draft will be updated in accordance with approved review comments, Review Item Discrepancies (RID), or Requests for Action (RFA) and formally baselined.
5.2.2.5 Terms for maturity levels of technical work products identified in this section are addressed in detail in Appendix F.
5.2.2.6 The technical team ensures that each program or project hosting equipment, experiments, or payloads with radio frequency (RF) requirements include success criteria in all life-cycle and technical reviews to receive approval from the responsible Center spectrum manager that program or project spectrum goals and progress are being achieved and satisfy all spectrum regulatory requirements. Spectrum certification requirements are provided in NPD 2570.5 and NPR 2570.1, NASA Radio Frequency (RF) Spectrum Management Manual. NPR 2570.1 takes precedence over this document regarding spectrum related procedures and processes.
Table 5-1 - SE Work Product Maturity
**Item is a required product for that review.
1 For projects, single-project programs, and one-step AO programs.
2 For uncoupled, tightly coupled, loosely coupled programs, and two-step AO programs.
5.2.2.7 Technical teams shall monitor technical effort through periodic technical reviews [SE-57].
5.2.2.8 For each type of program/project, technical efforts are monitored throughout the life- cycle to ensure that the technical goals of the program/project are being achieved and that the technical direction of the program/project is appropriate.
5.2.2.9 A technical review is an evaluation of the program/project, or element thereof, by the technical team and other knowledgeable participants for the purposes of:
a. Assessing the status of and progress toward accomplishing the planned activities.
b. Validating the technical tradeoffs explored and design solutions proposed.
c. Identifying technical weaknesses or marginal design and potential problems (risks) and recommending improvements and corrective actions.
d. Making judgments on the activity's readiness for the follow-on events, including additional future evaluation milestones to improve the likelihood of a successful outcome.
e. Making assessments and recommendations to the program/project team, Center, and Agency management.
f. Providing a historical record of decisions that were made during these formal reviews which can be referenced at a later date.
g. Assessing the technical risk status and current risk profile.
5.2.3 Completion
5.2.3.1 Life-cycle reviews are considered complete when the following are accomplished:
a. Agreement (including with the appropriate TA) exists for the disposition of all RIDs and RFAs.
b. The review board report and minutes are complete and distributed.
c. Agreement (including with the appropriate TA) exists on a plan to address the issues and concerns of insufficient program/project performance with respect to the LCR success criteria in the review board's report.
d. Agreement (including with the appropriate TA) exists on a plan for addressing the actions identified out of the review.
e. Liens against the review results are closed, or an adequate and timely plan exists for their closure.
f. Differences of opinion between the program/project under review and the review board(s) have been resolved, or a timely plan exists to resolve the issues.
g. A report is given by the review board chairperson to the appropriate management and governing Program Management Committees (PMCs) charged with oversight of the program/project.
h. Appropriate procedures and controls are instituted to ensure that all actions from reviews are followed and verified through implementation to closure.
i. The Program/Project Decision Authority signs a decision memo (e.g., memorandum or other appropriate format) documenting successful completion of the review.
6.1.1 A Systems Engineering Management Plan (SEMP) is used to establish the technical content of the engineering work early in the Formulation phase for each program/project and updated as needed throughout the program/project life-cycle. The resulting technical plan represents the agreed to and approved tailoring of the requirements of this NPR and the customizing of SE practices to satisfy program/project technical requirements.
6.1.1.1 The SEMP provides the specifics of the technical effort and describes what common technical processes will be used, how the processes will be applied using appropriate activities, how the program/project will be organized to accomplish the activities, and the technical resources required (including cost, schedule, and personnel) for accomplishing the activities. The process activities are driven by the critical events during any phase of a life-cycle (including operations) that set the objectives and work product outputs of the processes and how the processes are integrated. (See Appendix J of NASA/SP-2016-6105 for a suggested annotated outline for the SEMP.)
6.1.1.2 The SEMP provides the communication bridge between the program/project management team and the executing technical team. It also facilitates effective communication within the technical team.
6.1.1.3 The SEMP provides the framework to realize the appropriate work products of the applicable program/project life-cycle phases to provide management with necessary information for assessing technical progress.
6.1.1.4 The SEMP may be a stand-alone document or may be included as sections within other documentation such as the program or project plan.
6.1.1.5 The SEMP provides the basis for implementing the technical effort and communicating what will be done and by whom, when, where, how, and why it is being done including any applicable constraints on the implementation. In addition, the SEMP identifies the roles and responsibility interfaces of the technical effort and how those interfaces will be managed.
6.1.1.6 The SEMP is the vehicle that documents and communicates the technical approach, including the application of the common technical processes; resources to be used; and key technical tasks, activities, and events along with their metrics and success criteria. The SEMP communicates the technical effort that will be performed by the assigned technical team to the team itself, managers, customers, and other stakeholders.
6.1.1.7 The SEMP is a living document that captures a program/project's current and evolving SE strategy and its relationship with the overall program/project management effort throughout the life cycle of the system. Whereas the primary focus is on the current and upcoming phase in which the technical effort will be done, the planning extends to a summary of the technical efforts that are planned for future phases. The SEMP's purpose is to guide all technical aspects of the program/project.
6.1.2 The SEMP is consistent with higher level SEMPs and the Program/Project Plan, allowing for tailoring and customization. For example, a Project level SEMP would be consistent with the Program level SEMP and the Project Plan.
6.1.3 The content of a SEMP for an in-house technical effort may differ from an external technical effort. For an external technical effort, the NASA SEMP should include details on developing requirements for source selection, monitoring performance, and transferring and integrating externally produced products to NASA. (See Appendix J of NASA/SP-2016-6105 for further details.)
6.1.4 The NASA SEMP also provides the basis for determining the required contractor's documentation specifying their SE approach to the scope of activities described by the 17 common technical processes (See Section 4.2.3).
6.1.5 The ETA shall approve the SEMP, waiver or deviation authorizations, and other key technical documents to ensure independent assessment of technical content [SE-06].
6.2.1 Working with the Program/Project Manager, the technical team under the guidance of the ETA determines the appropriate level within the system structure at which SEMPs are to be developed, taking into account factors such as number and complexity of interfaces, operating environments, and risk factors.
6.2.2 The technical team establishes the initial SEMP early in the Formulation phase and updates it as necessary to reflect changes in scope or improved technical development. The technical team will have their approaches approved through the Center's ETA process 7. As changes occur, the SEMP will be updated by the technical team, reviewed and reapproved by both the Center's ETA and the program/project manager, and presented at subsequent life-cycle reviews or their equivalent. The SEMP is updated at major life-cycle reviews through the SIR.
6.2.3 The technical teams shall define in the program/project SEMP how the required 17 common technical processes, as tailored, will be recursively applied to the various levels of program/project product layer system structure during each applicable life-cycle phase [SE-58].
6.2.4 The technical team baselines the SEMP per the Center's procedures and the governing PM policy. (For example, for spaceflight projects under NPR 7120.5, it is baselined at SRR for projects and single-project programs and System Definition Review (SDR) for loosely coupled programs, tightly coupled programs, and uncoupled programs). The content of Appendix J of NASA/SP-2016-6105 should be used as a guide for producing the work product. For small projects, the SEMP material can be incorporated in the Project Plan provided the ETA approves the SEMP material.
6.2.5 The technical team shall ensure that any technical plans and discipline plans are consistent with the SEMP (or equivalent program/project documentation) and are accomplished as fully integrated parts of the technical effort [SE-59].
6.2.6 The technical team shall establish TPMs for the program/project that track/describe the current state versus plan [SE-60]. These measures are typically described in the SEMP per Appendix J of NASA/SP-2016-6105 guide.
6.2.7 The technical team shall report the TPMs to the Program/Project Manager on an agreed-to reporting interval [SE-61].
6.2.8 A technical leading indicator is a subset of the TPMs that provides insight into the potential future states. The technical team shall ensure that the set of TPMs include the following leading indicators:
a. Mass margins for projects involving hardware [SE-62].
b. Power margins for projects that are powered [SE-63].
6.2.9 The technical team shall ensure that a set of review trends is created and maintained that includes closure of review action documentation (RIDs, RFAs, and/or Action Items as established by the project [SE-64].
Acceptable Risk: The risk that is understood and agreed to by the program/project, governing PMC, Mission Directorate, and other customers such that no further specific mitigating action is required. (Some mitigating actions might have already occurred.)
Activity: A set of tasks that describe the technical effort to accomplish a process and help generate expected outcomes.
Affordability: The practice of balancing system performance and risk with cost and schedule constraints over the system life, satisfying system operational needs in concert with strategic investment and evolving stakeholder value.
Approve (with respect to Technology Maturation Products from Appendix F): Used for a product, such as Concept Documentation, that is not expected to be put under classic configuration control but still requires that changes from the “approved” version are documented at each subsequent “update.”
Baseline: An agreed-to set of requirements, designs, or documents that will have changes controlled through a formal approval and monitoring process.
Baseline (with respect to Technology Maturation Products from Appendix F): Indicates putting the product under configuration control so that changes can be tracked, approved, and communicated to the team and any relevant stakeholders. The expectation on products labeled “baseline” is that they will be at least final drafts going into the designated review and baselined coming out of the review. Baselining a product does not necessarily imply that it is fully mature at that point in the life-cycle. Updates to baselined documents require the same formal approval process as the original baseline.
Bidirectional Traceability: The ability to trace any given requirement/expectation to its parent requirement/expectation and to its allocated children requirements/expectations.
Brassboard: A medium fidelity functional unit that typically tries to make use of as much of the final product as possible and begins to address scaling issues associated with the operational system. It does not have the engineering pedigree in all aspects but is structured to be able to operate in simulated operational environments in order to assess performance of critical functions
Breadboard: A low fidelity unit that demonstrates function only, without respect to form or fit. It often uses commercial and/or ad hoc components and is not intended to provide definitive information regarding operational performance.
Certification Package: The body of evidence that results from the verification activities and other activities such as reports, special forms, models, waivers, or other supporting documentation that is evaluated to indicate the design is certified for flight/use.
Component Facilities: Complexes that are geographically separated from the NASA Center or institution to which they are assigned but are still part of the Agency.
Concept of Operations (ConOps): Developed early in Pre-Phase A, describes the overall high-level concept of how the system will be used to meet stakeholder expectations, usually in a time sequenced manner. It describes the system from an operational perspective and helps facilitate an understanding of the system goals. It stimulates the development of the requirements and architecture related to the user elements of the system. It serves as the basis for subsequent definition documents and provides the foundation for the long-range operational planning activities (for nominal and contingency operations). It provides the criteria for the validation of the system. In cases where an Operations Concept (OpsCon) is developed, the ConOps feeds into the OpsCon and they evolve together. The ConOps becomes part of the Concept Documentation.
Construction of Facilities: A NASA corporate program that funds planning for future facility needs, design of facilities projects, revitalization projects (repair, rehabilitation, and modification of existing facilities), construction of new facilities, and acquisition of collateral equipment.
Contractor: For the purposes of this NPR, an individual, partnership, company, corporation, association, or other service having a contract with the Agency for the design, development, manufacture, maintenance, modification, operation, or supply of items or services under the terms of a contract to a program or project within the scope of this NPR. Research grantees, research contractors, and research subcontractors are excluded from this definition.
Corrective Action: Action taken on a product to correct and preclude recurrence of a failure or anomaly, e.g., design change, procedure change, personnel training.
Critical Event: An event in the operations phase of the mission that is time sensitive and is required to be accomplished successfully in order to achieve mission success. These events will be considered early in the life-cycle as drivers for system design.
Customer: The organization or individual that has requested a product and will receive the product to be delivered. The customer may be an end user of the product, the acquiring agent for the end user, or the requestor of the work products from a technical effort. Each product within the system hierarchy has a customer.
Customizing: The modification of recommended SE practices that are used to accomplish the SE requirements. Examples of these practices are in NASA/SP-2016-6105.
Decision Authority: The individual authorized by the Agency to make important decisions for programs and projects under their authority.
Derived Requirements: Requirements arising from constraints, consideration of issues implied but not explicitly stated in the high-level direction provided by Agency and Center institutional requirements or factors introduced by the selected architecture and design.
Deviation: A documented authorization releasing a program or project from meeting a requirement before the requirement is put under configuration control at the level the requirement will be implemented.
Documentation: Captured information and its support medium that is suitable to be placed under configuration control. Note that the medium may be paper, photograph, electronic storage (digital documents and models), or a combination thereof.
Enabling Products: The life-cycle support products and services (e.g., production, test, deployment, training, maintenance, and disposal) that facilitate the progression and use of the operational end product through its life-cycle. Since the end product and its enabling products are interdependent, they are viewed as a system. Program/project responsibility thus extends to responsibility for acquiring services from the relevant enabling products in each life-cycle phase. When a suitable enabling product does not already exist, the program/project that is responsible for the end product can also be responsible for creating and using the enabling product. An example is below in Figure A-1.
Figure A-1 – Enabling Product Relationship to End Products
Engineering Technical Authority: One of the three identified lines of technical authority (i.e., Engineering, Safety and Mission Assurance, and Health and Medical). ETA includes individuals who have been formally delegated Technical Authority that flows from the Administrator to the NASA Chief Engineer and to the Center Directors for further delegation to Center engineering leadership and individuals. These individuals are funded independent from a program or project and are a key part of NASA’s system of checks and balances that provides independent oversight of programs and projects in support of safety and mission success. The ETA establishes and is responsible for the engineering processes, specifications, rules, best practices, and other activities throughout the life-cycle, necessary to fulfill programmatic mission performance requirements. The ETA for the program or project leads and manages the engineering activities, including systems engineering, design, development, sustaining engineering, and operations.
Engineering Unit: A high fidelity unit that demonstrates critical aspects of the engineering processes involved in the development of the operational unit. Engineering test units are intended to closely resemble the final product (hardware/software) to the maximum extent possible and are built and tested so as to establish confidence that the design will function in the expected environments. In some cases, the engineering unit can become the final product, assuming proper traceability has been exercised over the components and hardware handling.
Entrance Criteria: Guidance for minimum accomplishments the program or project fulfills prior to a life-cycle review.
Expectation: A statement of needs, desires, capabilities, and wants that are not expressed as a requirement (not expressed as a “shall” statement). Once the set of expectations from applicable stakeholders is collected, analyzed, and converted into a “shall” statement, the “expectation” becomes a “requirement.” Expectations can be stated in either qualitative (non-measurable) or quantitative (measurable) terms. Expectations can be stated in terms of functions, behaviors, or constraints with respect to the product being engineered or the process used to engineer the product.
Federal Records: All books, papers, maps, photographs, machine-readable materials, digital models, or other documentary materials, regardless of physical form or characteristics, made or received by an agency of the U.S. Government under Federal law or in connection with the transaction of public business and preserved or appropriate for preservation by that agency or its legitimate successor as evidence of the organization, functions, policies, decisions, procedures, operations, or other activities of the Government or because of the informational value of the data in them.
Final (with respect to Technology Maturation Products from Appendix F): Applied to products that are expected to exist in a specified form (e.g., minutes and final reports).
Formulation Phase: The first part of the NASA management life cycle defined in NPR 7120.5, where system requirements are baselined, feasible concepts are determined, a system definition is baselined for the selected concept(s), and preparation is made for progressing to the Implementation phase.
Human Systems Integration (HSI): An interdisciplinary and comprehensive management and technical process that focuses on the integration of human considerations into the system acquisition and development processes to enhance human system design, reduce life-cycle ownership cost, and optimize total system performance. Human system domain design activities associated with operations, training, human factors engineering, safety, quality, maintainability and supportability, habitability, and survivability are considered concurrently and integrated with all other SE design activities.
Identify (with respect to identification of processes in Chapter 3): To either use an approved process or a customized process that is approved by the ETA or their delegate.
Implement (with respect to Implementation of processes in Chapter 3): To document and communicate the approved process, provide resources to execute the process, provide training on the process, and monitor and control the process.
Implementation Phase: The part of the NASA management life-cycle defined in NPR 7120.5, where the detailed design of system products is completed and the products to be deployed are fabricated, assembled, integrated, and tested and the products are deployed to their customers or users for their assigned use or mission.
Information Technology Plan: A plan that provides the Information System Description, which encompasses the complete set of interconnected IT systems, their subsystems and components, and the system dataset and log data. This plan includes the IT system configuration management, network diagram, the system interconnections, the data flow, the data type, and the data categorization/data tagging/metadata. This plan is a foundational element for the IT System Security Plan and facilitates correct reporting for the Federal Information Technology Acquisition Reform Act (FITARA). This plan is required for all programs and projects. It would include corporate IT, industrial control systems, and mission IT (including all computing systems, avionics buses, and other related components). For a space system the network diagram would include all IT nodes such as, but not limited to, the Launch Control Center, mission control center, data processing center(s), science operations center, and on-board system IT.
Information Technology System Security Plan: The formal document prepared by the information system owner (or common security controls owner for inherited controls) that provides an overview of the security requirements for the system and describes the security controls in place or planned for meeting those requirements. The plan can also contain as supporting appendices or as references, other key security-related documents such as a risk assessment, privacy impact assessment, system interconnection agreements, contingency plan, security configurations, configuration management plan, and incident response plan.
Initial (with respect to Technology Maturation Products from Appendix F): Applied to products that are continually developed and updated as the program or project matures.
Insight: An element of Government surveillance that monitors contractor compliance using Government-identified metrics and contracted milestones. Insight is a continuum that can range from low intensity, such as reviewing quarterly reports, to high intensity, such as performing surveys and reviews.
Institutional Authority: Institutional Authority encompasses all those organizations and authorities not in the Programmatic Authority. This includes Engineering, Safety and Mission Assurance, and Health and Medical organizations; Mission Support organizations; and Center Directors.
Iterative: Application of a process to the same product or set of products to correct a discovered discrepancy or other variation from requirements. (See Recursive and Repeatable.)
Joint Confidence Level: A process and product that helps inform management of the likelihood of a project’s programmatic success. The probability that cost will be equal to or less than the targeted cost and that schedule will be equal to or less than the targeted schedule date.
Key Decision Point (KDP): The event at which the Decision Authority determines the readiness of a program/project to progress to the next phase of the life cycle (or to the next KDP).
Key Performance Parameters (KPP): Those capabilities or characteristics (typically engineering-based or related to health and medical, safety, or operational performance) considered most essential for successful mission accomplishment. Failure to meet a KPP threshold can be cause for the program, project, system, or advanced technology development to be reevaluated or terminated or for the system concept or the contributions of the individual systems to be reassessed. A program/project’s KPPs are identified and quantified in the program/project baseline. (See Technical Performance Parameter.)
Laboratory Environment: An environment that does not address in any manner the environment to be encountered by the system, subsystem, or component (hardware or software) during its intended operation. Tests in a laboratory environment are solely for the purpose of demonstrating the underlying principles of technical performance (functions) without respect to the impact of environment.
Leading Indicator: A measure for evaluating the effectiveness of how a specific activity is applied on a program or project in a manner that provides information about impacts likely to affect the system performance objectives. A leading indicator may be an individual measure or collection of measures predictive of future system (and project) performance before the performance is realized. The goal of the indicators is to provide insight into potential future states to allow management to take action before problems are realized. A technical leading indicator is a subset of the Technical Performance Measures (TPMs) that provides insight into the potential future states.
Logical Decomposition: The decomposition of the defined technical requirements by functions, time, and behaviors to determine the appropriate set of logical and data architecture models and related derived technical requirements. Models may include functional flow block diagrams, timelines, data control flow, states and modes, behavior diagrams, operator tasks, system data, metadata, data standards, taxonomy, and functional failure modes.
Loosely Coupled Programs: Programs that address specific objectives through multiple space flight projects of varied scope. While each individual project has an assigned set of mission objectives, architectural and technological synergies and strategies that benefit the program as a whole are explored during the Formulation process. For instance, Mars orbiters designed for more than one Mars year in orbit are required to carry a communication system to support present and future landers.
Measure of Effectiveness (MOE): A measure by which a stakeholder’s expectations will be judged in assessing satisfaction with products or systems produced and delivered in accordance with the associated technical effort. An MOE is deemed to be critical to not only the acceptability of the product by the stakeholder but also critical to operational/mission usage. An MOE is typically qualitative in nature or not able to be used directly as a “design-to” requirement.
Measure of Performance (MOP): A quantitative measure that, when met by the design solution, will help ensure that an MOE for a product or system will be satisfied. MOPs are given special attention during design to ensure that the MOEs with which they are associated are met. There are generally two or more measures of performance for each MOE.
Operational Environment: The environment in which the final product will be operated. In the case of space flight hardware/software, it is space. In the case of ground-based or airborne systems that are not directed toward space flight, it will be the environments defined by the scope of operations. For software, the environment will be defined by the operational platform.
Operations Concept (OpsCon): Developed later in the life-cycle and baselined at PDR, a more detailed description of how the flight system and the ground system are used together to ensure that the concept of operation is reasonable. This might include how mission data of interest, such as engineering data, scientific data, and data standards/metadata are captured, returned to Earth, processed, made searchable, accessible, and available to users, and archived for future reference. The OpsCon should describe how the flight system and ground system work together across mission phases for planning, training, launch, cruise, critical activities, science observations, and end of mission to achieve the mission. This product should be informed by the ConOps and they should evolve together. They may exist as a single product or separate products.
Other Interested Parties: Groups or individuals that are not customers of a planned technical effort but may be affected by the resulting product, the manner in which the product is realized or used, or who have a responsibility for providing life-cycle support services. A subset of “stakeholders.” (See Stakeholder.)
Oversight: An element of Government surveillance that occurs in line with the contractor’s processes in which the Government retains and exercises the right to concur or non-concur with the contractor’s decisions.
Peer Review: See Peer Review in Appendix G, Table G-19.
Preliminary (with respect to Technology Maturation Products from Appendix F): The documentation of information as it stabilizes but before it goes under configuration control. It is the initial development leading to a baseline. Some products will remain in a preliminary state for multiple reviews. The initial preliminary version is likely to be updated at a subsequent review but remains preliminary until baselined.
Process: A set of activities used to convert inputs into desired outputs to generate expected outcomes and satisfy a purpose.
Product: A part of a system consisting of end products that perform operational functions and enabling products that perform life-cycle services related to the end product or a result of the technical efforts in the form of a work product (e.g., plan, baseline, or test result).
Product Layer: The end product is decomposed into a hierarchy of smaller and smaller products. The product layer is defined as a horizontal slice of this product breakdown hierarchy and includes both the end product and associated enabling products.
Product Realization: The act of making, buying, or reusing a product or the assembly and integration of lower level realized products into a new product, as well as the verification and validation that the product satisfies its appropriate set of requirements and the transition of the product to its customer.
Program: A strategic investment by a Mission Directorate (or mission support office) that has defined goals, objectives, architecture, funding level, and a management structure that supports one or more projects.
Program Commitment Agreement: The contract between the Administrator and the cognizant Mission Directorate Associate Administrator (MDAA) or Mission Support Office Directorate (MSOD) Associate Administrator for implementation of a program.
Project: A specific investment having defined goals, objectives, requirements, life-cycle cost, a beginning, and an end. A project yields new or revised products or services that directly address NASA’s strategic needs. They may be performed wholly in-house; by Government, industry, or academia partnerships; or through contracts with private industry.
Prototype Unit: The prototype unit demonstrates form, fit, and function at a scale deemed to be representative of the final product operating in its operational environment. A subscale test article provides fidelity sufficient to permit validation of analytical models capable of predicting the behavior of full-scale systems in an operational environment.
Radio Frequency Authorization: Given by the National Telecommunications and Information Administration (NTIA) for the use of radio frequency spectrum for radio transmissions for telecommunications or for other purposes.
Realized Product: The desired output from application of the five Product Realization Processes. The form of this product is dependent on the phase of the product life-cycle and the phase success criteria.
Recursive: Value that is added to the system by the repeated application of processes to design next lower layer system products or to realize next upper layer end products within the system structure. This also applies to repeating application of the same processes to the system structure in the next life-cycle phase to mature the system definition and satisfy phase success criteria.
Relevant Environment: Not all systems, subsystems, and/or components need to be operated in the operational environment in order to satisfactorily address performance margin requirements. Consequently, the relevant environment is the specific subset of the operational environment that is required to demonstrate critical “at risk” aspects of the final product performance in an operational environment. It is an environment that focuses specifically on “stressing” the technology advance in question.
Relevant Stakeholder: A subset of the term “stakeholder” that applies to people or roles that are designated in a plan for stakeholder involvement. Since “stakeholder” may describe a very large number of people, attempting to deal with all of them might require unnecessary time and effort. For this reason, “relevant stakeholder” is used in most practice statements to describe the people identified to contribute to a specific task.
Repeatable: In the context of systems engineering, a repeatable process is a characteristic that can be applied to products at any level of the system structure or within any life-cycle phase.
Request for Action/Review Item Discrepancy (RFA/RID): The most common names for the comment forms that reviewers submit during life-cycle reviews that capture their comments, concerns, and/or issues about the product or documentation. Each Center defines their own RFA/RID disposition process.
Requirement: The agreed upon need, capability, capacity, or demand for personnel, equipment, facilities, or other resources or services by specified quantities for specific periods of time or at a specified time expressed as a “shall” statement. Acceptable form for a requirement statement is individually clear, correct, feasible to obtain, unambiguous in meaning, and can be validated at the level of the system structure at which stated. In pairs of requirement statements or as a set, collectively, they are not redundant, are adequately related with respect to terms used, and are not in conflict with one another.
Review Trends: Metrics that show how the identified life-cycle and technical reviews are progressing such as tracking the closure of action items, RIDs, or RFAs throughout the life-cycle.
Risk: In the context of mission execution, the potential for performance shortfalls, which may be realized in the future, with respect to achieving explicitly established and stated performance requirements. The performance shortfalls may be related to any one or more of the following mission execution domains: (1) safety, (2) technical, (3) cost, and (4) schedule. (See NPR 8000.4.)
Single Point Failure: An independent element of a system (hardware, software, or human), the failure of which would result in loss of objectives, hardware, or crew.
Single-Project Programs: Programs that tend to have long development and/or operational lifetimes, represent a large investment of Agency resources, and have contributions from multiple organizations/agencies. These programs frequently combine program and project management approaches, which they document through tailoring.
Software: In this directive, “software” is defined as (1) computer programs, procedures and possibly associated documentation and data pertaining to the operation of a computer system; (2) all or a part of the programs, procedures, rules, and associated documentation of an information processing system; (3) program or set of programs used to run a computer; (4) all or part of the programs which process or support the processing of digital information; (5) part of a product that is the computer program or the set of computer programs software, and open-source software components. This definition applies to software developed by NASA, software developed for NASA, commercial-off-the-shelf (COTS) software, Government-off-the-shelf (GOTS) software, modified-off-the-shelf (MOTS) software, reused software, auto-generated code, embedded software, the software executed on processors embedded in Programmable Logic Devices (see NASA-HDBK-4008), and open-source software components.
Specification: A document or data that prescribes, in a complete, precise, verifiable manner, the requirements, design, behavior, or characteristics of a system or system component. In this document, specification is treated as a requirement.
Spectrum Certification: A program or project obtains certification by the NTIA (located within the Department of Commerce) that the radio frequency required can be made available before a program or project submits estimates for the development or procurement of major radio spectrum-dependent communication-electronics systems (including all systems employing space satellite techniques).
Spectrum Certification Stage 1, Conceptual: The initial planning effort has been completed, including proposed frequency bands and other available characteristics. Certification of spectrum support for telecommunication systems or subsystems at Stage 1 provides guidance, from the NTIA, on the feasibility of obtaining certification of spectrum support at subsequent stages. The guidance provided will indicate any modifications, including more suitable frequency bands, necessary to assure conformance with the NTIA Manual. (Refer to NPR 2570.1.)
Spectrum Certification Stage 2, Experimental: The preliminary design has been completed and radiation impact assessment, using such things as test equipment or preliminary models may be required. Certification of spectrum support for telecommunication systems or subsystems at Stage 2 is a prerequisite for NTIA authorization of radiation in support of experimentation for systems. It also provides guidance for assuring certification of spectrum support at subsequent stages. (Refer to NPR 2570.1.)
Spectrum Certification Stage 3, Developmental: The major design has been completed, and radiation impact assessment may be required during testing. Certification of spectrum support for telecommunication systems or subsystems at Stage 3 is a prerequisite for NTIA authorization of radiation in support of developmental testing for systems. It also provides guidelines for assuring certification of spectrum support at Stage 4. At this point, the intended frequency band will have been determined and certification at Stage 3 will be required for testing of proposed operational hardware and potential equipment configurations. (Refer to NPR 2570.1.)
Spectrum Certification Stage 4, Operational: Development has been essentially completed, and final operating constraints or restrictions required to assure compatibility need to be identified. Certifying spectrum support for major telecommunication systems or subsystems at Stage 4 is a prerequisite for NTIA authorization to radiate. Tracking, telemetry, and telecommand operations for major satellite networks require NTIA Stage 4 certification of spectrum support before the launch of the spacecraft. Stage 4 certification provides restrictions on the operation of the system or subsystem as may be necessary to prevent harmful interference. (Refer to NPR 2570.1.)
Stakeholder: A group or individual who is affected by or has an interest or stake in a program or project. See “Customer,” “Relevant Stakeholder,” and “Other Interested Parties.”
Success Criteria: Specific accomplishments that need to be satisfactorily demonstrated to meet the objectives of a life-cycle and technical review so that a technical effort can progress further in the life-cycle. Success criteria are documented in the corresponding technical review plan.
System: The combination of elements that function together to produce the capability required to meet a need. The elements include all hardware, software, equipment, facilities, personnel, processes, and procedures needed for this purpose. (Refer to NPR 7120.5.)
Systems Approach: The application of a systematic, disciplined engineering approach that is quantifiable, recursive, iterative, and repeatable for the development, operation, and maintenance of systems integrated into a whole throughout the life-cycle of a project or program.
Systems Engineering Engine: The NASA SE model shown in Figure 3-1 that provides the 17 technical processes and their relationship with each other. The model is called an “SE Engine” in that the appropriate set of processes is applied to the products being engineered to drive the technical effort.
Systems Engineering Management Plan: The SEMP identifies the roles and responsibility interfaces of the technical effort and how those interfaces will be managed. The SEMP is the vehicle that documents and communicates the technical approach, including the application of the common technical processes; resources to be used; and key technical tasks, activities, and events along with their metrics and success criteria.
System of Interest: The system whose characteristics are under consideration regardless of where it lies in the product hierarchy.
System Safety: The application of engineering and management principles, criteria, and techniques to optimize safety within the constraints of operational effectiveness, time, and cost throughout all phases of the system life-cycle.
Tailoring: The process used to seek relief from SE NPR requirements consistent with program or project objectives, allowable risk, and constraints. The tailoring process results in the generation of deviations and waivers depending on the timing of the request.
Technical Authority: Part of NASA’s system of checks and balances that provides independent oversight of programs and projects in support of safety and mission success through the selection of individuals at delegated levels of authority. These individuals are the Technical Authorities. Technical Authority delegations are formal and traceable to the Administrator. Individuals with Technical Authority are funded independently of a program or project. TA originates with the Administrator and is formally delegated to the NASA AA and then to the NASA Chief Engineer for Engineering Technical Authority; the Chief, Safety and Mission Assurance for SMA Technical Authority; the NASA Chief Health and Medical Officer for Health and Medical Technical Authority; and then to the Center Directors.
Technical Performance Measures: The set of performance measures that are monitored by comparing the current actual achievement of the parameters with that anticipated at the current time and on future dates. Used to confirm progress and identify deficiencies that might jeopardize meeting a system requirement. Assessed parameter values that fall outside an expected range around the anticipated values indicate a need for evaluation and corrective action. Technical performance measures are typically selected from the defined set of Measures of Performance (MOPs).
Technical Requirements: The requirements that capture the characteristics, features, functions and performance that the end product will have to meet stakeholder expectations.
Technical Risk: Risk associated with the achievement of a technical goal, criterion, or objective. It applies to undesired consequences related to technical performance, human health and medical, safety, mission assets, or environment.
Technical Team: Members of a multidisciplinary team responsible for defining and implementing the technical aspects of a program or project.
Technology Readiness Level: A scale against which to measure the maturity of a technology. TRLs range from 1 (Basic Technology Research) to 9 (Systems Test, Launch, and Operations).
Tightly Coupled Programs: Programs with multiple projects that execute portions of a mission(s). No single project is capable of implementing a complete mission. Typically, multiple NASA Centers contribute to the program. Individual projects may be managed at different Centers. The program may also include other agency or international partner contributions.
Transition: The act of delivery or moving a product from one location to another location. This act can include packaging, handling, storing, moving, transporting, installing, and sustainment activities.
Uncoupled Programs: Programs implemented under a broad theme and/or a common program implementation concept, such as providing frequent flight opportunities for cost-capped projects selected through AO or NASA Research Announcements. Each such project is independent of the other projects within the program.
Update (with respect to Technology Maturation Products from Appendix F): Applied to products that are expected to evolve as the formulation and implementation processes evolve. Only expected updates are indicated. However, any document may be updated as needed.
Validation (of a product): The process of showing proof that the product accomplishes the intended purpose based on stakeholder expectations and the Concept of Operations. May be determined by a combination of test, analysis, demonstration, and inspection. (Answers the question, “Am I building the right product?”)
Validation (of requirements): The continuous process of ensuring that requirements are well-formed (clear and unambiguous), complete (agrees with customer and stakeholder needs and expectations), consistent (conflict free), and individually verifiable and traceable to a higher level requirement or goal. (Answers the question, “Will I build the right product?”)
Verification (of a product): Proof of compliance with requirements/specifications. Verification may be determined by test, analysis, demonstration, inspection, or a combination thereof. (Answers the question, “Did I build the product right?”)
Waiver: A documented authorization releasing a program or project from meeting a requirement after the requirement is put under configuration control at the level the requirement will be implemented.
| AO | Announcement of Opportunity |
| APPEL | Academy of Program/Project and Engineering Leadership |
| ASM | Acquisition Strategy Meeting |
| CDR | Critical Design Review |
| CERR | Critical Event Readiness Review |
| CMMI | Capability Maturity Model® IntegrationSM |
| ConOps | Concept of Operations |
| COTS | Commercial Off the Shelf |
| CPD | Center Policy Directive |
| CPR | Center Procedural Requirements |
| CPU | Central Processing Unit |
| CRM | Continuous Risk Management |
| DCR | Design Certification Review |
| DR | Decommissioning Review |
| DRR | Disposal Readiness Review |
| EEE | Electrical, Electronic, and Electromechanical |
| EMC | Electromagnetic Compatibility |
| EMI | Electromagnetic Interference |
| ETA | Engineering Technical Authority |
| FA | Formulation Agreement |
| FAD | Formulation Authorization Document |
| FMEA/CIL | Failure Mode and Effects Analysis/Critical Items List |
| FMECA | Failure Mode, Effects, and Criticality Analysis |
| FRR | Flight Readiness Review |
| GIDEP | Government-Industry Data Exchange Program |
| GOTS | Government Off-the-Shelf |
| HSI | Human Systems Integration |
| HSIP | Human Systems Integration Plan |
| ILSP | Integrated Logistics Support Plan |
| IMS | Integrated Master Schedule |
| IP | Institutional Projects |
| IT | Information Technology |
| JCL | Joint Confidence Level |
| JPL | Jet Propulsion Laboratory |
| KDP | Key Decision Point |
| KPP | Key Performance Parameter |
| LRR | Launch Readiness Review |
| MCR | Mission Concept Review |
| MD | Mission Directorate |
| MDAA | Mission Directorate Associate Administrator |
| MDR | Mission Definition Review |
| MOE | Measure of Effectiveness |
| MOP | Measure of Performance |
| MOTS | Modified Off the Shelf |
| MRR | Mission Readiness Review |
| MSD | Mission Support Directorate |
| NODIS | NASA On-Line Directives Information System |
| NPD | NASA Policy Directive |
| NPR | NASA Procedural Requirements |
| NTIA | National Telecommunications and Information Administration |
| OCE | Office of the Chief Engineer |
| OCHMO | Office of the Chief Health and Medical Officer |
| OpsCon | Operations Concept |
| ORR | Operational Readiness Review |
| OSMA | Office of Safety and Mission Assurance |
| PCA | Program Commitment Agreement |
| PDR | Preliminary Design Review |
| PFAR | Post-Flight Assessment Review |
| PIR | Program Implementation Review |
| PLAR | Post-Launch Assessment Review |
| PM | Program or Project Manager |
| PMC | Program Management Committee |
| PRA | Probabilistic Risk Assessment |
| PRR | Production Readiness Review |
| PSR | Program Status Review |
| RF | Radio Frequency |
| RFA | Request for Action |
| RFP | Request for Proposals |
| RID | Review Item Discrepancy |
| RIDM | Risk-Informed Decision Making |
| S&MA | Safety and Mission Assurance |
| SAR | System Acceptance Review |
| SCRM | Supply Chain Risk Management |
| SDR | System Definition Review |
| SE | Systems Engineering |
| SE NPR | Systems Engineering NASA Procedural Requirements |
| SEMP | Systems Engineering Management Plan |
| SIR | System Integration Review |
| SMSR | Safety and Mission Success Review |
| SP | Special Publication |
| SRB | Standing Review Board |
| SRR | System Requirements Review |
| TA | Technical Authority |
| TBD | To Be Determined |
| TBR | To Be Resolved |
| TPM | Technical Performance Measure |
| TRL | Technology Readiness Level |
| TRR | Test Readiness Review |
| U.S.C. | United States Code |
| V&V | Verification and Validation |
Guidance for implementing the core SE processes has been moved to NASA/SP-2016-6105.
The outline for the Systems Engineering Management Plan has moved to Appendix J of NASA/SP-2016-6105.
| TRL | Definition | Hardware Description | Software Description | Success criteria |
|---|---|---|---|---|
| 1 | Basic principles observed and reported. | Scientific knowledge generated underpinning hardware technology concepts/applications. | Scientific knowledge generated underpinning basic properties of software architecture and mathematical formulation. | Peer reviewed documentation of research underlying the proposed concept/application. |
Examples:
|
||||
| 2 | Technology concept and/or application formulated. | Invention begins, practical application is identified but is speculative, no experimental proof or detailed analysis is available to support the conjecture. | Practical application is identified but is speculative; no experimental proof or detailed analysis is available to support the conjecture. Basic properties of algorithms, representations, and concepts defined. Basic principles coded. Experiments performed with synthetic data. | Documented description of the application/concept that addresses feasibility and benefit. |
| Examples: Carbon nanotube composites were created for lightweight, high-strength structural materials for space structures. | ||||
| 3 | Analytical and experimental proof-of-concept of critical function and/or characteristics. | Research and development are initiated, including analytical and laboratory studies to validate predictions regarding the technology. | Development of limited functionality to validate critical properties and predictions using non-integrated software components. | Documented analytical/experimental results validating predictions of key parameters. |
Examples:
|
||||
| 4 | Component and/or breadboard validation in a laboratory environment. | A low fidelity system/component breadboard is built and operated to demonstrate basic functionality in a laboratory environment. | Key, functionality critical software components are integrated and functionally validated to establish interoperability and begin architecture development. Relevant environments defined and performance in the environment predicted. | Documented test performance demonstrating agreement with analytical predictions. Documented definition of potentially relevant environment. |
Examples:
|
||||
| 5 | Component and/or brassboard validated in a relevant environment. | A medium-fidelity component and/or brassboard, with realistic support elements, is built and operated for validation in a relevant environment so as to demonstrate overall performance in critical areas. | End-to-end software elements implemented and interfaced with existing systems/simulations conforming to target environment. End-to-end software system tested in relevant environment, meeting predicted performance. Operational environment performance predicted. Implementations. | Documented test performance demonstrating agreement with analytical predictions. Documented definition of scaling requirements. Performance predictions are made for subsequent development phases. |
Examples:
|
||||
| 6 | System/sub-system model or prototype demonstration in a relevant environment. | A high-fidelity prototype of the system/subsystems that adequately addresses all critical scaling issues is built and tested in a relevant environment to demonstrate performance under critical environmental conditions. | Prototype implementations of the software demonstrated on full-scale, realistic problems. Partially integrated with existing hardware/software systems. Limited documentation available. Engineering feasibility fully demonstrated. | Documented test performance demonstrating agreement with analytical predictions. |
Examples:
|
||||
| 7 | System prototype demonstration in an operational environment. | A high-fidelity prototype or engineering unit that adequately addresses all critical scaling issues is built and functions in the actual operational environment and platform (ground, airborne, or space). | Prototype software exists having all key functionality available for demonstration and test. Well integrated with operational hardware/software systems demonstrating operational feasibility. Most software bugs removed. Limited documentation available. | Documented test performance demonstrating agreement with analytical predictions. |
Examples:
|
||||
| 8 | Actual system completed and "flight qualified" through test and demonstration. | The final product in its final configuration is successfully demonstrated through test and analysis for its intended operational environment and platform (ground, airborne, or space). If necessary*, life testing has been completed. | All software has been thoroughly debugged and fully integrated with all operational hardware and software systems. All user documentation, training documentation, and maintenance documentation completed. All functionality successfully demonstrated in simulated operational scenarios. Verification and Validation completed. | Documented test performance verifying analytical predictions. |
| Note:
*"If necessary" refers to the need to life test either for worn out mechanisms, for temperature stability over time, and for performance over time in extreme environments. An evaluation on a case-by-case basis should be made to determine the system/systems that warrant life testing and the tests begun early in the technology development process to enable completion by TRL 8. It is preferable to have the technology life test initiated and completed at the earliest possible stage in development. Some components may require life testing on or after TRL 5. Examples:
|
||||
| 9 | Actual system flight proven through successful mission operations. | The final product is successfully operated in an actual mission. | All software has been thoroughly debugged and fully integrated with all operational hardware and software systems. All documentation has been completed. Sustaining software support is in place. System has been successfully operated in the operational environment. | Documented mission operational results. |
Examples:
|
||||
Note: In cases of conflict between NASA directives concerning TRL definitions, NPR 7123.1 will take precedence.
F.1 For non-configuration-controlled documents, the following terms and definitions are used in this document:
a. "Initial" is applied to products that are continually developed and updated as the program or project matures.
b. "Final" is applied to products that are expected to exist in this final form, e.g., minutes and final reports.
c. "Update" is applied to products that are expected to evolve as the formulation and implementation processes evolve. Only expected updates are indicated. However, any document may be updated as needed.
F.2 For configuration-controlled documents, the following terms and definitions are used in this document:
a. "Preliminary" is the documentation of information as it stabilizes but before it goes under configuration control. It is the initial development leading to a baseline. Some products will remain in a preliminary state for multiple reviews. The initial preliminary version is likely to be updated at a subsequent review but remains preliminary until baselined.
b. "Baseline" indicates putting the product under configuration control so that changes can be tracked, approved, and communicated to the team and any relevant stakeholders. The expectation on products labeled "baseline" is that they will be at least final drafts going into the designated review and baselined coming out of the review. Baselining a product does not necessarily imply that it is fully mature at that point in the life-cycle. Updates to baselined documents require the same formal approval process as the original baseline.
c. "Approve" is used for a product, such as Concept Documentation, that is not expected to be put under classic configuration control but still requires that changes from the "Approved" version are documented at each subsequent "Update."
d. "Update" is applied to products that are expected to evolve as the formulation and implementation processes evolve. Only expected updates are indicated. However, any document may be updated as needed. Updates to baselined documents require the same formal approval process as the original baseline.
This appendix describes the recommended best practices for entrance and success criteria for the life-cycle and technical reviews required in Chapter 5 regardless of whether the review is accomplished in a one-step or two-step process. The entrance criteria do not provide a complete list of all products and their required maturity levels.1 Terms for maturity levels of technical products defined in the tables of this appendix are addressed in detail in Appendix F. Additional programmatic products may also be required by the appropriate governing NPRs for the project/program.
Tailoring and customizing are expected for projects and programs. The entrance and success criteria and products required for each review will be tailored and customized appropriately for the particular program or project being reviewed. The decision not to tailor and customize life-cycle review criteria should be justified to the ETA.
The recommended criteria in the following tables are focused on demonstrating acceptable program/project technical maturity, adequacy of technical planning and credibility of budget, schedule and risks (as applicable), and readiness to proceed to the next phase. Customized or tailored criteria developed by programs or projects for life-cycle reviews should also be focused on assessing these factors.
Programs and projects use different Appendix G tables for some life-cycle reviews. Programs (except single-project programs) use tables G-1 and G-2 for program-level SRR and SDRs. Projects and single-project programs use the tables starting with G-3.
G.1 System Requirements Review (SRR) for Programs
The SRR for a program is used to ensure that the program’s functional and performance requirements are properly formulated and correlated with the Agency and Mission Directorate strategic objectives.
Table G-1 – SRR Entrance and Success Criteria for Programs
| System Requirements Review for Programs | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.2 System Definition Review for Programs
The SDR for a program evaluates the credibility and responsiveness of the proposed program requirements/architecture to the Mission Directorate requirements, the allocation of program requirements to the projects, and the maturity of the programs mission/system definition. Programs (except single-project programs) should use the entrance and success criteria in Table G-2. For project and single-project programs, refer to Table G-5.
Table G-2 – SDR Entrance and Success Criteria for Programs
| System Definition Review for Programs | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
G.3 Mission Concept Review
The MCR affirms the mission/project need and evaluates the proposed mission’s objectives and the ability of the concept to fulfill those objectives.
Table G-3 – MCR Entrance and Success Criteria
| Mission Concept Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.4 System Requirements Review (SRR) for Projects and Single-Project Programs
The SRR evaluates whether the functional and performance requirements defined for the system of interest are responsive to the program’s requirements and ensures the preliminary project plan and requirements will satisfy the mission. This table is used for projects and single-project programs. For other types of programs, refer to Table G-1.
Table G-4 – SRR Entrance and Success Criteria
| System Requirements Review for Projects and Single-Project Programs | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.5 Mission Definition Review/System Definition Review (MDR/SDR) for Project and Single-Project Programs
The MDR/SDR evaluates whether the proposed mission/system architecture is responsive to the program mission/system functional and performance requirements and whether requirements have been allocated to the next lower product layer and to all functional elements of the mission/system. This table is to be used for projects and single-project programs.
Table G-5 – MDR/SDR Entrance and Success Criteria (Projects
and Single-Project Program)
| Mission Definition Review/ System Definition Review for Projects and Single-Project Programs | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.6 Preliminary Design Review (PDR)
The PDR demonstrates that the preliminary design meets all system of interest requirements with acceptable risk and within the cost and schedule constraints and establishes the basis for proceeding with detailed design.
Table G-6 – PDR Entrance and Success Criteria
| Preliminary Design Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.7 Critical Design Review (CDR)
The CDR demonstrates that the maturity of the design is appropriate to support proceeding with full-scale fabrication, assembly, integration, and test. The CDR determines that the technical effort is on track to complete the system development, meeting functional and performance requirements within the identified cost and schedule constraints at an acceptable risk.
Table G-7 – CDR Entrance and Success Criteria
| Critical Design Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.8 Production Readiness Review (PRR)
For projects developing or acquiring multiple systems/units (typically greater than three or as determined by the project). The PRR determines the readiness of the system developers to efficiently produce the required number of systems. It ensures that the production plans, fabrication, assembly, integration enabling products, operational support, and personnel are in place and ready to begin production.
Table G-8 – PRR Entrance and Success Criteria
| Production Readiness Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.9 System Integration Review (SIR)
An SIR ensures segments, components, and subsystems are on schedule to be integrated into the system of interest, and integration facilities, support personnel, and integration plans and procedures are on schedule to support integration.
Table G-9 – SIR Entrance and Success Criteria
| System Integration Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
G.10 Test Readiness Review (TRR)
A TRR for each planned test or series of tests ensures that the test article (hardware/software), test facility, support personnel, and test procedures are ready for testing and data acquisition, reduction, and control.
Table G-10 – TRR Entrance and Success Criteria
| Test Readiness Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.11 System Acceptance Review (SAR)
The SAR verifies the completeness of the specific end products in relation to their expected maturity level, requirement verification, compliance to stakeholder expectations, and ensures that the system of interest has sufficient technical maturity to authorize its acceptance for operational use or delivery to the launch site or operational environment.
Table G-11 – SAR Entrance and Success Criteria
| System Acceptance Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.12 Operational Readiness Review (ORR)
The ORR ensures that all system and support (flight and ground) hardware, software, personnel, procedures, supporting capabilities, and user documentation accurately reflect the deployed state of the system and are operationally ready.
Table G-12 – ORR Entrance and Success Criteria
| Operational Readiness Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.13 Mission Readiness Review/Flight Readiness Review (MRR/FRR)
The MRR/FRR examines tests, demonstrations, analyses, and audits that determine the system’s readiness for a safe and successful flight or launch and for subsequent flight operations. The MRR/FRR also ensures that all flight and ground hardware, software, personnel, and procedures are operationally ready.
Table G-13 – MRR/FRR Entrance and Success Criteria
| Mission Readiness Review/Flight Readiness Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
***Required per NPD 2570.5.
G.14 Post-Launch Assessment Review (PLAR)
A PLAR evaluates the readiness of the spacecraft systems to proceed with full, routine operations after post-launch deployment. The review also evaluates the status of the project plans and the capability to conduct the mission with emphasis on near-term operations and mission-critical events.
Table G-14 – PLAR Entrance and Success Criteria
| Post-Launch Assessment Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.15 Critical Event Readiness Review (CERR)
A CERR evaluates the readiness of the project and the flight system to execute the critical event during flight operation.
Table G-15 – CERR Entrance and Success Criteria
| Critical Event Readiness Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.16 Post-Flight Assessment Review (PFAR)
The PFAR evaluates how well mission objectives were met during a mission and identifies all flight and ground system anomalies that occurred during the flight and determines the actions necessary to mitigate or resolve the anomalies for future flights of the same spacecraft design.
Table G-16 – PFAR Entrance and Success Criteria
| Post-Flight Assessment Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
G.17 Decommissioning Review (DR)
A DR confirms the decision to terminate or decommission the system and assesses the readiness of the system for the safe decommissioning and disposal of system assets. This review can be applied for the system that was deployed through earlier efforts of this program/project or for a legacy capability that will be replaced by the system being deployed.
Table G-17 – DR Entrance and Success Criteria
| Decommissioning Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.
**Product is required per NPR 7123.1.
G.18 Disposal Readiness Review (DRR)
A DRR confirms the readiness for the final disposal of the system assets. This review can be applied for the system that was deployed through earlier efforts of this program/project or for a legacy capability that will be disposed of and replaced by the system being deployed.
Table G-18 – DRR Entrance and Success Criteria
| Disposal Readiness Review | |
| Entrance Criteria | Success Criteria |
|---|---|
|
|
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence
**Product is required per NPR 7123.1.
G.19 Peer Reviews
Peer reviews provide the technical insight essential to ensure product and process quality. Peer reviews are focused, in-depth technical reviews that support the evolving design and development of a product, including critical documentation or data packages. The participants in a peer review are the technical experts and key stakeholders for the scope of the review. Another purpose of the peer review is to add value and reduce risk through expert knowledge infusion, confirmation of approach, identification of defects, and specific suggestions for product improvements.
Table G-19 – Peer Review Entrance and Success Criteria
| Peer Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.20 Program Implementation Reviews (PIR) and Program Status Reviews (PSR)
PIRs or PSRs are periodically conducted, as required by the Decision Authority and documented in the program plan, during the Implementation phase to evaluate the program’s continuing relevance to the Agency’s Strategic Plan. These reviews assess the program performance with respect to expectations and determine the program’s ability to execute the implementation plan with acceptable risk within cost and schedule constraints.
Table G-20 – PIR/PSR Entrance and Success Criteria
| Program Implementation and Program Status Reviews | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
*Required per NPD 2570.5.
G.21 Design Certification Review (DCR)
This review is not depicted in the standard life-cycle review figures but has proven useful to larger projects such as human space flight. Projects/Centers may choose to add this review to their standard life-cycle if they feel it is useful. The DCR ensures that the design complies with functional and performance requirements, as demonstrated in verification, validation, and qualification evidence. The certified design forms the basis from which system acceptance will be assessed. A DCR should, ideally, be held after a CDR and before a SAR.
Table G-21 – DCR Entrance and Success Criteria
| Design Certification Review | |
|---|---|
| Entrance Criteria | Success Criteria |
|
|
Template Instructions
The Compliance Matrix documents the program/projects compliance or intent to comply with the requirements of this NPR or justification for tailoring. It is attached to the SEMP or other equivalent program/project documentation when submitted for approval. The matrix lists:
Programs/Projects may substitute a matrix that documents their compliance with their particular Center’s implementation of NPR 7123.1.
The “Comply?” column is filled in to identify the program/projects approach to the requirement. An “FC” is inserted for “fully compliant,” “T” for “tailored,” or “NA” for a requirement that is “not applicable.” The column titled “Justification” documents the rationale for tailoring, documents how the requirement will be tailored, or justifies why the requirement is not applicable.
| Req ID | NPR Section | Requirement Statement | Rationale | Comply? | Justification |
|---|---|---|---|---|---|
| SE-01 to 05 | Deleted | See rationale in the Deleted Requirements Table J-1. | |||
| SE-06 | 6.1.8 | The ETA shall approve the SEMP, waiver or deviation authorizations, and other key technical documents to ensure independent assessment of technical content. | This requirement ensures that the ETA has reviewed and approved of key systems engineering documents. | ||
| SE- 07 | 3.2.2.1 | Program/Project Managers shall identify and implement an ETA-approved Stakeholder Expectations Definition process to include activities, requirements, guidelines, and documentation, as tailored and customized for the definition of stakeholder expectations for the applicable product layer. | This requirement ensures that the program/project and the ETA identifies how they will gather and address stakeholder expectations. This ensures that the program/project will gain a thorough understanding of what the customer and other stakeholders expect. | ||
| SE- 08 | 3.2.3.1 | Program/Project Managers shall identify and implement an ETA-approved Technical Requirements Definition process to include activities, requirements, guidelines, and documentation, as tailored and customized for the definition of technical requirements from the set of agreed upon stakeholder expectations for the applicable product layer. | This requirement ensures that the program/project and the ETA identifies how they will select and gain agreement on the technical requirements. | ||
| SE- 09 | 3.2.4.1 | Program/Project Managers shall identify and implement an ETA-approved Logical Decomposition process to include activities, requirements, guidelines, and documentation, as tailored and customized for logical decomposition of the validated technical requirements of the applicable product layer. | This requirement ensures that the program/project and the ETA identifies how they will take the technical requirements for the program/project and glean from them what is needed to accomplish them (e.g., functional block diagrams, timing, architectures). This places the requirements into context and ensures they are understood well enough to begin the design process. | ||
| SE- 10 | 3.2.5.1 | Program/Project Managers shall identify and implement an ETA-approved Design Solution Definition process to include activities, requirements, guidelines, and documentation, as tailored and customized for designing product solution definitions within the applicable product layer that satisfy the derived technical requirements. | This requirement ensures that the program/project and the ETA identifies how they will take the information from the stakeholder expectations, requirements, and logical decomposition and perform the design function. Since all designs are unique, this will describe the general steps that are taken. The specifics for each of the program/projects will be documented in the SEMP or other equivalent program/project documentation. | ||
| SE- 11 | 3.2.6.1 | Program/Project Managers shall identify and implement an ETA-approved Product Implementation process to include activities, requirements, guidelines, and documentation, as tailored and customized for implementation of a design solution definition by making, buying, or reusing an end product of the applicable product layer. | This requirement ensures that the program/project and the ETA identifies how they will execute the designs, whether through buying items off the shelf or contracting to have them built, building/coding them within the Center, or reusing products already developed by another program/project. The specifics for how each program/project will make this determination for the various components/assemblies within the product hierarchy are documented in the SEMP or other equivalent program/project documentation. | ||
| SE- 12 | 3.2.7.1 | Program/Project Managers shall identify and implement an ETA-approved Product Integration process to include activities, requirements, guidelines, and documentation, as tailored and customized for the integration of lower level products into an end product of the applicable product layer in accordance with its design solution definition. | This requirement ensures that the program/project and the ETA identifies how they will approach the integration of products within successive levels of the product hierarchy. This ensures that planning is performed that will enable a smooth integration of products into higher level assemblies. | ||
| SE- 13 | 3.2.8.1 | Program/Project Managers shall identify and implement an ETA-approved Product Verification process to include activities, requirements/specifications, guidelines, and documentation, as tailored and customized for verification of end products generated by the product implementation process or product integration process against their design solution definitions. | This requirement ensures that the program/project and the ETA identifies how they will verify that the end products will comply with each of the technical requirements. | ||
| SE- 14 | 3.2.9.1 | Program/Project Managers shall identify and implement an ETA-approved Product Validation process to include activities, requirements, guidelines, and documentation, as tailored and customized for validation of end products generated by the product implementation process or product integration process against their stakeholder expectations. | This requirement ensures that the program/project and the ETA identifies how they will show that the end products will meet the stakeholder expectations in the intended environment. This is in addition to verifying they meet the stated requirements and ensures the stakeholder is getting what was expected. | ||
| SE- 15 | 3.2.10.1 | Program/Project Managers shall identify and implement an ETA-approved Product Transition process to include activities, requirements, guidelines, and documentation, as tailored and customized for transitioning end products to the next higher level product layer customer or user. | This requirement ensures that the program/project and the ETA identifies how they will handle the end products as they move from one location to another. This includes shipping, handling, transportation criteria, physical security, cybersecurity, and receiving facility storage needs. It ensures that receiving facilities are ready to accept the product and that no damage occurs to the product during handling and transportation. | ||
| SE- 16 | 3.2.11.1 | Program/Project Managers shall identify and implement an ETA-approved Technical Planning process to include activities, requirements, guidelines, and documentation, as tailored and customized for planning the technical effort. | This requirement ensures that the program/project and the ETA identifies how they will perform and document all the technical planning for the program/project. This includes all plans developed for the technical effort —Systems Engineering Management Plans, risk plans, integration plans, and V&V plans. This ensures that the program/project teams are thinking ahead for the work to be performed and capturing that information so it can be communicated to the rest of the team, customers, and other stakeholders. | ||
| SE- 17 | 3.2.12.1 | Program/Project Managers shall identify and implement an ETA-approved Requirements Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of requirements throughout the system life-cycle. | This requirement ensures that the program/project and the ETA identifies how they will handle tracking and changes to the baselined set of requirements. It defines who has authority to submit and approve changes and how requirements are tracked as they flow down to other elements in the product breakdown structure. This ensures that changes to requirements are evaluated and that their impacts are understood and communicated to the rest of the team. | ||
| SE- 18 | 3.2.13.1 | Program/Project Managers shall identify and implement an ETA-approved Interface Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of the interfaces defined and generated during the application of the system design processes. | This requirement ensures that the program/project and the ETA identifies how they will manage the internal and external interfaces of their end product. This will ensure compatibility when the various parts of the system are brought together for assembly/integration. | ||
| SE- 19 | 3.2.14.1 | Program/Project Managers shall identify and implement a Technical Risk Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of the risk identified during the technical effort. | This requirement ensures that the program/project and the ETA identifies how they will handle the technical portions of the program/project risks and report them for inclusion with the cost and schedule risk portions. It ensures that the technical aspects of risks to the program/projects successful execution are captured and reported to program/project management who will be developing the overall risk posture. | ||
| SE- 20 | 3.2.15.1 | Program/Project Managers shall identify and implement an ETA-approved Configuration Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for configuration management. | This requirement ensures that the program/project and the ETA identifies how they will perform configuration management of the end products, enabling products and other work products key to the program/project. The technical products to be controlled are identified and tracked to ensure that the team knows what the configuration of their system is at all phases of the life-cycle. | ||
| SE- 21 | 3.2.16.1 | Program/Project Managers shall identify and implement an ETA-approved Technical Data Management process to include activities, requirements, guidelines, and documentation, as tailored and customized for management of the technical data generated and used in the technical effort. | This requirement ensures that the program/project and the ETA identifies how they will handle all the technical data that is generated by the program/project. This will include all data needed to manage, operate, and support the system products over the life-cycle. It ensures that the data is available and secure when needed. | ||
| SE- 22 | 3.2.17.1 | Program/Project Managers shall identify and implement an ETA-approved Technical Assessment process to include activities, requirements, guidelines, and documentation, as tailored and customized for making assessments of the progress of planned technical effort and progress toward requirements satisfaction. | This requirement ensures that the program/project and the ETA identifies how they will assess the progress of the program/project’s technical efforts, including life-cycle reviews. It ensures that the program/project team, customers, and other key stakeholders know how the effort is progressing and if additional actions are needed to resolve issues prior to becoming too costly. | ||
| SE- 23 | 3.2.18.1 | Program/Project Managers shall identify and implement an ETA-approved Decision Analysis process to include activities, requirements, guidelines, and documentation, as tailored and customized for making technical decisions. | This requirement ensures that the program/project and the ETA identify how they will make and document key technical decisions. It helps to ensure that all team members know who can make decisions, what their authority levels are, and where to go to gain an understanding of what key decisions have been made. | ||
| SE- 24 | 4.2.1 | The NASA technical team shall define the engineering activities for the periods before contract award, during contract performance, and upon contract completion in the SEMP or other equivalent program/project documentation. | It is important for both the Government and contractor technical teams to understand what activities will be handled by which organization throughout the product life-cycle. The contractor(s) will typically develop a SEMP or other equivalent program/project documentation to describe the technical activities in their portion of the project, but an overarching SEMP (or other equivalent program/project documentation) is needed that will describe all technical activities across the life-cycle whether contracted or not. | ||
| SE- 25 | 4.2.2 | The NASA technical team shall establish the technical inputs to the solicitation appropriate for the product(s) to be developed, including product requirements and Statement of Work tasks. | The technical team’s participation in the development of the solicitation is critical to enabling a successful contracted effort. Ensuring that the proper application of the common technical processes into the contracted effort will enhance the chances for success. | ||
| SE- 26 | 4.2.3 | The NASA technical team shall determine the technical work products to be delivered by the offeror or contractor, to include contractor documentation that specifies the contractor’s SE approach to the scope of activities described by the 17 common technical processes. | The technical team is in the best position to determine what kind of work products from the technical effort will need to be delivered. These products will eventually be used by the technical team to determine the suitability of the contracted effort in its ability to meet requirements, satisfy the stakeholder expectations, and perform as planned. | ||
| SE- 27 | 4.2.4 | The NASA technical team shall provide the requirements for technical insight and oversight activities planned in the NASA SEMP or other equivalent program/project documentation to the contracting officer for inclusion in the solicitation. | In addition to the work description and products to be delivered, how the technical team will gain an adequate understanding of the contracted work, what authority (if any) they will have to direct or influence the work, and their participation at key life-cycle reviews. In the end the technical team needs enough information to advise the Program/Project Manager and ETA as to the adequacy of the technical work. | ||
| SE- 28 | 4.2.5 | The NASA technical team shall participate in the evaluation of offeror proposals in accordance with applicable NASA and Center source selection procedures. | Technical personnel will need to be involved in reviewing the proposals and providing advice/guidance on their merits. These personnel may or may not be part of the technical team that will execute the program/project. | ||
| SE- 29 | 4.3.1 | The NASA technical team, under the authority of the contracting officer, shall perform the technical insight and oversight activities established in the contract including modifications to the original contract. | After the contract is awarded, the contracting officer will depend on the technical team to execute the oversight/ insight of the technical work as defined in their SEMP (or other equivalent program/project documentation) and the contract. | ||
| SE- 30 | 4.4.1 | The NASA technical team shall participate in the review(s) to finalize Government acceptance of the deliverables. | Per the agreement in the SEMP (or other equivalent program/project documentation) and the contract, the technical team will participate in the life-cycle reviews. Ultimately, this knowledge will enable the technical team to provide advice to the program/project and ETA as to the suitability of the product for acceptance. | ||
| SE- 31 | 4.4.2 | The NASA technical team shall participate in product transition as defined in the NASA SEMP or other equivalent program/project documentation. | In accordance with the SEMP (or other equivalent program/project documentation), the technical team will participate in the execution of the final aspects of the end product—either its transference in whole to the program/ project customer, its operations, and/or the final decommissioning and disposal. These activities may be performed by the same team that was involved in its development or by other technical teams. | ||
| SE- 32 | 5.2.1.1 | The technical team shall develop and document plans for life-cycle and technical reviews for use in the program/project planning process. | Each of the life-cycle reviews, as well as any other technical status reviews, needs to be identified and documented so that all stakeholders will know how the program/ projects progress will be assessed. This will typically be captured within the SEMP, in a separate Review Plan or other equivalent program/project documentation. | ||
| SE- 33 | 5.2.1.5 | The technical team shall participate in the life-cycle and technical reviews as indicated in the governing program/project management NPR. | The technical team will be responsible for generating and presenting many of the technical topics during a life-cycle and technical review. | ||
| SE- 34 | 5.2.2.1 | The technical team shall participate in the development of entrance and success criteria for each of the respective reviews. | The entrance and success criteria in Appendix G are provided as guidelines (not requirements). It is expected that they will be modified as needed by the program/project according to their size, complexity, type of end product being produced, formality, and risk acceptance posture. Specific names of documents may be provided for clarity, non-applicable products eliminated, and new products added as needed for clarity and completeness. | ||
| SE- 35 | 5.2.2.2.a (1) |
The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MCR: Baselined stakeholder identification and expectation definitions. | For an MCR one of the key products is capturing the stakeholder expectations. These may be identified as needs, goals, and objectives, or other methods for capturing their expectations. These are captured in a document or a database/model. After all comments from the MCR are dispositioned, the set of stakeholder expectations are updated with the approved comments and then baselined. | ||
| SE- 36 | 5.2.2.2. a (2) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MCR: Baselined concept definition. | Presenting one or more feasible ways of accomplishing the stakeholder expectations is a key product of the MCR. These are captured in a document or a database/model. After all comments from the MCR are dispositioned, the concept(s) are updated with the approved comments and then baselined. | ||
| SE- 37 | 5.2.2.2. a (3) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MCR: Approved Measures of Effectiveness (MOE) definition. | The MOE capture the stakeholders’ view of what would be considered the successful achievement of each expectation. These will help in the later identification of requirements, criteria for trade studies and in the success criteria for the validation efforts. | ||
| SE- 38 | 5.2.2.2. b (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: SRR: Baselined SEMP (or other equivalent program/project documentation) for projects, single-project programs, and one-step AO programs. | The SEMP is a key document for the technical effort in a similar manner that the program/project plan captures the programmatic efforts. These are captured in a document or a database/model. For projects, single-project programs, and one-step AO programs after all comments from the SRR are dispositioned, the SEMP (or other equivalent program/project documentation) is updated with the approved comments and then baselined. The SEMP (or other equivalent program/project documentation) is baselined in a later phase for the other types of programs and so will be a “Not Applicable” in this line for uncoupled, tightly coupled, and loosely coupled programs. | ||
| SE- 39 | 5.2.2.2. b (2) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: SRR: Baselined requirements. | The program/project requirements are a key product for the SRR. These are captured in a document or a database/model. After all comments from the SRR are dispositioned, the requirements are updated with the approved comments and then baselined. | ||
| SE- 40 | 5.2.2.2.c (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MDR/ SDR: Approved TPM definitions. | A key product at the SDR is the set of TPMs that the program/project has identified as the important measures to track for their efforts. These may be associated with the key driving requirements, key performance parameters, leading or lagging indicators, or other measures that are important to periodically measure and track. | ||
| SE- 41 | 5.2.2.2.c (2) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MDR/ SDR: Baselined architecture definition. | One of the key products of an SDR is the proposed architecture that will accomplish the requirements. These are captured in a document or a database/model. After all comments from the SDR are dispositioned, the architecture description is updated with the approved comments and then baselined. | ||
| SE- 42 | 5.2.2.2.c (3) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MDR/ SDR: Baselined allocation of requirements to next lower level. | Now that the overarching architecture has been defined, it is important to show how the requirements are allocated to the architecture elements of the next lower level of the product hierarchy. These are captured in a document or a database/ model. After all comments from the SDR are dispositioned, the allocation is updated with the approved comments and then baselined. | ||
| SE- 43 | 5.2.2.2.c (4) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MDR/ SDR: Initial trend of required leading indicators. | The trend is presented for the leading indicators that have been identified by the Agency as required for each program/project. These will typically be in graphical form but could also be tabular or other form appropriate for the project. At SDR this will be the initial set of trends that have been captured since SRR. Since final hardware has not been produced at this point, the trends will be on the estimated parameters. | ||
| SE- 44 | 5.2.2.2.c (5) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: MDR/ SDR: Baseline SEMP (or other equivalent program/project documentation) for uncoupled, loosely coupled, tightly coupled, and two-step AO programs. | The SEMP is a key document for the technical effort in a similar manner that the program plan captures the programmatic efforts. These are captured in a document or a database/model. For uncoupled, loosely coupled, tightly coupled, and two-step AO programs, after all comments from the MDR/SDR are dispositioned, the SEMP (or other equivalent program/project documentation) is updated with the approved comments and then baselined. The SEMP (or other equivalent program/project documentation) is baselined in an earlier phase for projects and single-project programs and so will be a “Not Applicable” in this line for those types of programs. | ||
| SE- 45 | 5.2.2.2. d (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: PDR: Preliminary design solution definition. | The key product of a PDR is the preliminary design itself. The design is captured in one or more documents, models, databases, drawings, and other means. Comments from the PDR will be captured in the final design for the next review. | ||
| SE- 46 | 5.2.2.2. e (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: CDR: Baseline detailed design. | The key product of a CDR is the final design. The design is captured in one or more documents, models, databases, drawings, and other means. The final design is updated with approved comments from the review, and the design is updated to represent the design that will be implemented. | ||
| SE- 47 | 5.2.2.2.f (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: SIR: Updated integration plan. | A key product of an SIR is the updated integration plans. These will describe how the products associated with this review will be integrated. | ||
| SE- 48 | 5.2.2.2.f (2) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: SIR: Preliminary V&V results. | Another key product of an SIR is the initial V&V results from any of the lower level products that are associated with this review. With the recursive nature of the SE engine, products will be integrated and verified/validated from the bottom of the product layer to the top. So, prior to integration into larger assemblies, lower level products will have been through their V&V activities. This ensures that, when they are assembled into the higher product layers, they will work as intended. Programs/projects may decide to perform V&V only at assembly levels—as captured in their SEMP (or other equivalent program/project documentation)—and so initial V&V results may or may not be available. | ||
| SE-49 and 50 | Deleted | See rationale in the Deleted Requirements Table. | |||
| SE- 51 | 5.2.2.2.g (3) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: ORR: Preliminary decommissioning plans. | At ORR it is important to describe how the product will ultimately be decommissioned when it has accomplished its mission. This is to ensure that decommissioning will be feasible before the product is put into use. These are captured in a document or a database/model. After all comments from the ORR are dispositioned, the plan is updated with the approved comments and then baselined. | ||
| SE- 52 | 5.2.2.2.h (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: FRR: Baseline disposal plans. | At FRR it is also important to describe how the product will ultimately be disposed of when it has accomplished its mission. This is to ensure that disposal will be feasible before the product is put into use. These are captured in a document or a database/model. After all comments from the FRR are dispositioned, the plan is updated with the approved comments and then baselined. | ||
| SE- 53 | 5.2.2.2.h (2) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: FRR: Baseline V&V results. | At FRR, the baselined V&V results for the product are presented and any remaining open work identified. This is to ensure that the product is ready for flight. Note that for some programs/projects the V&V results may need to be baselined at ORR per Center policies/procedures. Maturing and presenting a product earlier than required in the Agency NPR is at the discretion of the program/project/Center and does not require a waiver. | ||
| SE- 54 | 5.2.2.2.h (3) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: FRR: Final certification for flight/use. | The key product at the FRR is the certification that the product is ready for flight/use. This gains agreement with all key stakeholders that the product is ready to put into the operational phase. Any remaining open items are identified and plans for closure are developed. | ||
| SE- 55 | 5.2.2.2.i (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: DR: Baseline decommissioning plans. | The key product at the DR is the plan on how the product will be removed from service. The approved comments from the DR are used to baseline the plan. | ||
| SE- 56 | 5.2.2.2.j (1) | The technical team shall provide the following minimum products at the associated life-cycle review at the indicated maturity level: DRR: Updated disposal plans. | The key product of the DRR is the plan on how the product will be disposed of after it has been decommissioned. The approved comments from the DRR are used to update the plan. | ||
| SE- 57 | 5.2.2.7 | Technical teams shall monitor technical effort through periodic technical reviews. | In addition to the life-cycle reviews, the technical teams need to periodically monitor the technical progress of their program/project. These may be held formally or informally with relevant personnel. | ||
| SE- 58 | 6.2.3 | The technical teams shall define in the program/project SEMP how the required 17 common technical processes, as tailored, will be recursively applied to the various levels of program/project product layer system structure during each applicable life-cycle phase. | The SEMP is the key document that lays out the work that the technical team needs to perform and the manner in which they will perform it. This requirement ensures that each of the 17 common technical processes is addressed and how it will be applied to the various levels in the end-item product hierarchy and their associated enabling products. | ||
| SE- 59 | 6.2.5 | The technical team shall ensure that any technical plans and discipline plans are consistent with the SEMP (or equivalent program/project documentation) and are accomplished as fully integrated parts of the technical effort. | Since the SEMP is the primary planning document for the SE effort, all subsequent planning documents are in alignment and consistent with the SEMP. | ||
| SE- 60 | 6.2.6 | The technical team shall establish TPMs for the program/project that track/describe the current state versus plan. | The measures that the program/project will use to track the progress of key aspects of the technical effort are identified and documented. These TPMs will include the required leading indicators described in other requirements of this NPR and also any project-unique measures deemed necessary to track the key performance parameters. | ||
| SE- 61 | 6.2.7 | The technical team shall report the TPMs to the Program/Project Manager on an agreed-to reporting interval. | The selected TPMs need to be measured periodically and their trends reported to the program/project manager at the agreed-to interval as documented in the SEMP (or other equivalent program/project documentation). This ensures the PM and ETA are kept up to date on these key parameters so that decisions can be made in a timely manner. | ||
| SE- 62 | 6.2.8. a | The technical team shall ensure that the set of TPMs include the following leading indicators: Mass margins for projects involving hardware. | If the program/project has hardware elements, tracking of the remaining margins associated with their mass is a required leading indicator measure by the Agency. This is especially important for flight projects. For ground or other projects in which mass is not relevant, a waiver to this requirement can be documented in the SEMP or other equivalent program/project documentation. | ||
| SE- 63 | 6.2.8. b | The technical team shall ensure that the set of TPMs include the following leading indicators: Power margins for projects that are powered. | If the program/project has elements that require power, tracking of the remaining margins associated with their power consumption is a required leading indicator measure by the Agency. This is especially important for flight projects. For ground or other projects in which power consumption is not relevant, a waiver to this requirement can be documented in the SEMP or other equivalent program/project documentation. | ||
| SE- 64 | 6.2.9 | The technical team shall ensure that a set of review trends is created and maintained that includes closure of review action documentation (RIDs, RFAs, and/or Action Items as established by the project). | During life-cycle reviews, comments from the reviewers are captured on forms, databases, spreadsheets, or other manner. Depending on the program/project, these may be called RFAs, RIDs, Action Items, or other terminology. Whatever they are called, the disposition and closure of these comments—typically called their burndown—are required indicator trends by the Agency. This ensures that the approved comments are incorporated into the designs and plans in a timely manner. | ||
Submitted By: Approved By: _______________________ ___________ _______________________________ __________ Program/Project Manager Date Engineering Technical Authority Date
The following is a list of NASA Handbooks, NASA Standards, and endorsed military and industry standards that are applicable to systems engineering. These documents are available on the NASA Technical Standards System found at https://standards.nasa.gov/endorsed_standards, and should be applied as appropriate for each program or project.
| Document Number | Name |
| AE/GEIA-859 | Data Management, Revision B |
| ANSI/EIA 632 | Processes for Engineering a System |
| IEEE 1012 | Standard for System, Software, and Hardware Verification and Validation Software Reviews and Audits |
| IEEE 1028 | Standard for Software Reviews and Audits |
| IEEE 15939:2017 | Systems and Software Engineering |
| IEEE 828 | Standard for Configuration Management in Systems and Software Engineering |
| ISO/IEC 20246 | Software and Systems Engineering Work Product Reviews |
| ISO/IEC TS 24748 | Systems and Software Engineering Life Cycle Management |
| ISO/IEEE 15288 | Systems and Software Engineering - System Life-Cycle Processes |
| ISO/IEEE 16085 | Systems and Software Engineering - Risk Management |
| ISO/IEEE 29148 | Systems and Software Engineering - Requirements Engineering |
| MIL-STD-31000B | Department of Defense Standard Practice Technical Data Packages |
| NASA/SP-2010-576 | NASA Risk-Informed Decision Making Handbook |
| NASA/SP-2011-3422 | NASA Risk Management Handbook |
| NASA/SP-2016-6105 | NASA Systems Engineering Handbook |
| NASA-HDBK-2203 | NASA Software Engineering Handbook |
| NASA-HDBK-7009 | NASA Handbook for Models and Simulations |
| NASA-STD-3001 | NASA Space Flight Human System Standard |
| NASA-STD-7009 | NASA Standard for Models and Simulations |
| SAE/EIA-649-2 | Configuration Management Requirements for NASA Enterprises |
| SAE/EIA-649B | Configuration Management Standard |
| SAE/GEIA-HB-649 | Configuration Management Standard Implementation Guide |
The following requirements have been deleted from the original version of NPR 7123.1. Rather than resequence the remaining requirements, the original requirement numbering was left intact in case Centers or other organizations refer to these requirement numbers in their flow-down requirement documents. For each requirement that was deleted, the justification for its deletion is noted.
Table J-1 Deleted Requirements and Justification
| Req No | Requirement Statement | Justification for Deletion |
|---|---|---|
| [SE-01] | 2.1.4.3 Center Directors shall perform the following activities: a. Establish policies, procedures, and processes to execute the requirements of this SE NPR. | Original text was used to ensure each Center has a defined SE process. Now, 10 years after the initial SE NPR was generated, Centers have defined processes. The emphasis is now that each program/project identifies and implements SE processes that are approved by the ETA. |
| [SE-02] | 2.1.4.3 Center Directors shall perform the following activities: b. Assess and take corrective actions to improve the execution of the requirements of this SE NPR. | Original text was used to ensure each Center has a process for continuous improvement of their SE process. Now, 10 years after the initial SE NPR was generated, Centers routinely make updates and a requirement is no longer needed. |
| [SE-03] | 2.1.4.3 Center Directors shall perform the following activities: c. Select appropriate standards applicable to projects under their control. | Selection of technical standards applicable to a specific project is an ETA responsibility. |
| [SE-04] | 2.1.4.3 Center Directors shall perform the following activities: d. Complete the compliance matrix, as tailored, in Appendix H.1 for those requirements owned by the Office of Chief Engineer (OCE) and provide to the OCE upon request. | The H.1 and H.2 compliance matrices were combined into a single matrix. Responsibility for compliance matrix completion is now the responsibility of the program/project and ETA. |
| [SE-05] | 2.1.5.2 For those requirements owned by Center Directors, the technical team shall complete the compliance matrix in Appendix H.2 and include it in the SEMP. | The H.1 and H.2k compliance matrices were combined into a single matrix. Responsibility for compliance matrix completion is now the responsibility of the program/project and ETA. |
| [SE-49] | 5.2.1.7 The technical team shall provide the following minimum products at the associated milestone review at the indicated maturity level: g. ORR: (1) Updated operational plans. | Operational plans are optional and may be outside the purview of systems engineering to develop. |
| [SE-50] | 5.2.1.7 The technical team shall provide the following minimum products at the associated milestone review at the indicated maturity level: g. ORR: (2) Updated operational procedures. | Operational plans are optional and may be outside the purview of systems engineering to develop. |
The following documents were used as reference materials in the development of this SE NPR. The documents are offered as informational sources and are not evoked in this NPR, though they may be referenced.
1. NPD 7120.6, Knowledge Policy on Program and Projects.
2. NPD 8081.1, NASA Chemical Rocket Propulsion Testing.
3. NPD 8700.1, NASA Policy for Safety and Mission Success.
4. NPR 1400.1, NASA Directives and Charters Procedural Requirements.
5. NPR 2810.1, Security of Information Technology.
6. NPR 7120.10, Technical Standards for NASA Programs and Projects.
7. NPR 7120.11, NASA Health and Medical Technical Authority (HMTA) Implementation.
8. NPR 8705.4, Risk Classification for NASA Payloads.
9. NASA/SP-2010-3404, Work Breakdown Structure (WBS) Handbook.
10. NASA/SP-2014-3705, NASA Spaceflight Program & Project Management Handbook.
11. NASA-STD-7009, Standard for Models and Simulations.
12. MIL-STD-499B (draft), Systems Engineering.
13. ANSI/EIA 632, Processes for Engineering a System. Note: EIA 632 is a commercial document that evolved from the never released, but fully developed, 1994 Mil-Std 499B, Systems Engineering. It was intended to provide a framework for developing and supporting universal SE discipline for both defense and commercial environments. EIA 632 was intended to be a top-tier standard further defined to lower level standards that define specific practices. IEEE 1220 is a second-tier standard that implements EIA 632 by defining one way to practice SE.
14. AS9100: Quality Management Systems—Requirements for Aviation, Space, and Defense Organizations.
15. ISO/IEC 15288, Systems and Software Engineering—System Life-Cycle Processes.
16. ISO/IEC TR 19760, Systems Engineering—A Guide for the Application of ISO/IEC 15288 (System Life-Cycle Processes).
17. The Capability Maturity Model Integration (CMMI) ® Model.
18. Defense Acquisition University Systems Engineering Fundamentals. Ft. Belvoir, Virginia: Defense Acquisition University Press, December 2000.
19. International Council on Systems Engineering Systems Engineering Guide.