| NODIS Library | Program Formulation(7000s) | Search |

NASA Ball NASA
Procedural
Requirements 		NPR 7123.1D
Effective Date: July 05, 2023
Expiration Date: July 05, 2028
COMPLIANCE IS MANDATORY FOR NASA EMPLOYEES
Printable Format (PDF)

Subject: NASA Systems Engineering Processes and Requirements Updated w/Change 2

Responsible Office: Office of the Chief Engineer


| TOC | ChangeLog | Preface | Chapter1 | Chapter2 | Chapter3 | Chapter4 | Chapter5 | Chapter6 | AppendixA | AppendixB | AppendixC | AppendixD | AppendixE | AppendixF | AppendixG | AppendixH | AppendixI | AppendixJ | AppendixK | ALL |

Appendix G. Life-Cycle and Technical Review Entrance and Success Criteria

G.1 Overview

G.1.1 This appendix describes the recommended best practices for entrance and success criteria for the life-cycle and technical reviews required in Chapter 5 regardless of whether the review is accomplished in a one-step or two-step process. The entrance criteria do not provide a complete list of all products and their required maturity levels. Terms for maturity levels of technical products defined in the tables of this appendix are addressed in detail in Appendix F. Additional programmatic products may also be required by the appropriate governing NPRs for the project/program.

G.1.2 Tailoring and customizing are expected for projects and programs. The entrance and success criteria and products required for each review will be customized appropriately for the particular program or project being reviewed. The decision not to tailor and customize life-cycle review criteria should be justified to the ETA.

G.1.3 The recommended criteria in the following tables are focused on demonstrating acceptable program/project technical maturity, adequacy of technical planning and credibility of budget, schedule and risks (as applicable), and readiness to proceed to the next phase. Customized or tailored criteria developed by programs or projects for life-cycle reviews should also be focused on assessing these factors.

G.1.4 Programs and projects use different Appendix G tables for some life-cycle reviews. Programs (except single-project programs) use tables G-1 and G-2 for program-level SRR and SDRs. Projects and single-project programs use the tables starting with G-3.

G.2 System Requirements Review (SRR) for Programs The SRR for a program is used to ensure that the programâ??s functional and performance requirements are properly formulated and correlated with the Agency and Mission Directorate strategic objectives.

Table G-1 – SRR Entrance and Success Criteria for Programs

System Requirements Review for Programs
Entrance Criteria Success Criteria
  1. The Program has successfully completed the MCR life-cycle review (if applicable) and all RFAs and RIDs have been addressed and resolved, or a timely closure plan exists for those remaining open.
  2. A preliminary Program SRR agenda, success criteria, and instructions to the review board have been agreed to by the technical team, the program manager, and the review chair prior to the Program SRR.
  3. All planned higher level SRRs and peer reviews have been successfully conducted and RID/RFA/Action Items have been addressed with the originator or designated TA.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. Top program risks with significant technical, health and medical, system security (including cybersecurity****), safety, cost, and schedule impacts have been identified along with corresponding mitigation strategies.
  6. An approach for verifying compliance with program requirements has been defined.
  7. Procedures for controlling changes to program requirements have been defined and approved.
  8. The following primary products are ready for review:
    1. **Program requirements (including performance, health and medical, safety, human system integration, and defined external system interfaces to other programs) are ready to be baselined after review comments are incorporated.
    2. **Baselined approach for human systems integration.
  9. Other program SRR technical products have been made available to the cognizant participants prior to the review:
    1. *Preliminary traceability of program-level requirements on projects to the Agency strategic goals and Mission Directorate requirements and constraints.
    2. *Initial risk mitigation plans and resources for significant technical risks.
    3. *Preliminary cost and schedule for uncoupled, loosely coupled, and tightly coupled programs.
    4. *Preliminary documentation of Basis of Estimate (cost and schedule) for uncoupled, loosely coupled, and tightly coupled programs.
    5. *Review Plan ready to be baselined after review comments are incorporated.
    6. *Preliminary Configuration Management Plan.
    7. *Preliminary SEMP (or equivalent program documentation) for uncoupled, loosely coupled, tightly coupled, and two-step AO programs.
    8. ***RF (radio frequency) spectrum requirements have been identified.
    9. Preliminary System Security Plan.
  1. Program requirements have been defined and support Mission Directorate strategic objectives.
  2. The program requirements are adequately levied on projects of the program.
  3. Traceability of program requirements to individual projects is documented in accordance with Agency needs, goals, and objectives, as described in the NASA Strategic Plan.
  4. Definition of external system interfaces with other programs is adequately mature and approved.
  5. The program cost and schedule estimates are credible to meet program requirements.
  6. Top risk identification is complete and mitigation strategies appear reasonable.
  7. Evidence is provided that the program is compliant with NASA and implementing Center requirements, standards, processes, and procedures.
  8. To-be-determined (TBD) and to-be-resolved (TBR) items are clearly identified with acceptable plans and schedules for their disposition.
  9. The responsible Center spectrum manager at the responsible Center was notified of preliminary requirements.
  10. Proposed tailoring is appropriate and consistent with applicable Agency and Center guidance.
  11. Lessons Learned from other projects and programs have been identified and addressed.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

**** Cybersecurity requirements are found in NPR 2810.1

G.3 System Definition Review for Programs

The SDR for a program evaluates the credibility and responsiveness of the proposed program requirements/architecture to the Mission Directorate requirements, the allocation of program requirements to the projects, and the maturity of the programs mission/system definition. Programs (except single-project programs) should use the entrance and success criteria in Table G-2. For project and single-project programs, refer to Table G-5.

Table G-2 – SDR Entrance and Success Criteria for Programs

System Deinition Review for Programs
Entrance Criteria Success Criteria
  1. The program has successfully completed the previous planned life-cycle reviews and all RFAs and RIDs have been addressed and resolved, or a timely closure plan exists for those remaining open.
  2. An agenda for the program SDR, success criteria, and instructions to the review board have been agreed to by the technical team, the project manager, and the review chair prior to the review.
  3. All planned higher level SDRs and peer reviews have been successfully conducted and RID/RFA/Action Items have been addressed with the originator or designated TA.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. The following primary products are ready for review:
    1. **Approved definition of program TPMs.
    2. **Program architecture definition and a list of specific supporting projects that are ready to be baselined after review comments are incorporated.
    3. **Allocation of program requirements to the supporting projects that is ready to be baselined after review comments are incorporated.
    4. **Approval and status of technical performance related to leading indicators, margins, TPMs, and resolution of the previous review discrepancies addressing effectiveness of technical achievement and communicating the overall risk to the project.
    5. **SEMP (or equivalent program documentation) ready to be baselined for uncoupled, tightly coupled, and loosely coupled programs and for two-step AO programs.
  6. Other SDR technical products (as applicable) for hardware, software, and human system elements have been made available to the cognizant participants prior to the review:
    1. *Updated program requirements and constraints.
    2. *Traceability of program-level requirements on projects to the Agency strategic goals and Mission Directorate requirements and constraints that is ready to be baselined after review comments are incorporated.
    3. Preliminary system interface definitions.
    4. Preliminary implementation plans.
    5. Preliminary integration plans.
    6. *Preliminary verification and validation plans.
    7. *Updated cost and schedule.
    8. *Updated SEMP (or equivalent program documentation) for one-step AO programs and single-project programs.
    9. *Updated risk mitigation plans and resources for significant technical risks.
    10. *Updated cost and schedule.
    11. *Updated Documentation of Basis of Estimate (cost and schedule).
    12. *Preliminary plans for technical work to be accomplished during Implementation.
    13. *Updated Review Plan.
    14. *Configuration Management Plan that is ready to be baselined after review comments are incorporated.
    15. ***Preliminary assessment of RF spectrum requirements.
    16. *Baseline System Security Plan for uncoupled and loosely coupled programs.
  1. Evidence is provided that the program formulation activities are complete and implementation plans are credible to meet mission success.
  2. The program requirements address critical NASA needs as identified in the Mission Directorate strategic objectives.
  3. The program cost and schedule estimates are credible to meet program requirements within available resources.
  4. Program implementation plans are credible to achieve mission success.
  5. The program risks have been identified and mitigation strategies appear reasonable.
  6. Allocation of program requirements to projects has been completed and proposed projects are feasible within available resources.
  7. The maturity of the programâ??s definition and associated plans is sufficient to begin preliminary design.
  8. The program has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  9. TBD and TBR items are clearly identified with acceptable plans and schedules for their disposition.
  10. Program has clearly identified plans and schedules for applicable RF system certification data package submissions (experimental, developmental, or operational).
  11. Center spectrum manager at responsible Center was notified of preliminary requirement assessment.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

G.4 Mission Concept Review

The MCR affirms the mission/project need and evaluates the proposed missionâ??s objectives and the ability of the concept to fulfill those objectives.

Table G-3 – MCR Entrance and Success Criteria

Mission Concept Review
Entrance Criteria Success Criteria
  1. An agenda for the MCR, success criteria, and instructions to the review board have been agreed to by the technical team, the project manager, and the review chair prior to the review.
  2. All planned higher level MCRs and peer reviews have been successfully conducted and RID/RFA/Action Items have been addressed and resolved with the originator or designated TA, or a timely closure plan exists for those remaining open.
  3. The following primary products are ready for review:
    1. **Stakeholders have been identified and stakeholder expectations have been defined and are ready to be baselined after review comments are incorporated.
    2. **The concept has been developed to a sufficient level of detail to demonstrate a technically feasible solution to the mission/project needs and is ready to be baselined after review comments are incorporated.
    3. **MOEs and any other mission success criteria have been defined and are ready to be approved.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. Other technical products (as applicable) for hardware, software, and human system elements have been made available to the cognizant participants prior to the review:
    1. *Mission/project goals and objectives that are ready to be baselined after review comments are incorporated.
    2. Alternative concepts that have been analyzed and are ready to be reviewed.
    3. *Initial risk-informed cost and schedule estimates for implementation.
    4. *Preliminary mission descope options.
    5. *A preliminary assessment performed by the team of top technical, cost, schedule, system security (including cybersecurity****), and safety risks with developed associated risk management and mitigation strategies and options.
    6. *Preliminary approach to verification and validation for the selected concept(s).
    7. *A preliminary SEMP (or equivalent project documentation), including technical plans.
    8. *Technology Development Plan that is ready to be baselined after review comments are incorporated.
    9. *Initial technology readiness that has been assessed and documented with technology assets, heritage products, and gaps identified.
    10. Single Point Failure/Fault Tolerance philosophy.
    11. Preliminary engineering development assessment and technical plans to achieve what needs to be accomplished in the next phase.
    12. Conceptual life-cycle support strategies (logistics, supply chain management, manufacturing, and operation).
    13. Software criteria and products, per NASA-HDBK-2203.
    14. ***Preliminary assessment of RF spectrum needs.
    15. Preliminary Human Systems Integration approach. (For additional guidance on HSI entrance criteria refer to NASA/SP-20210010952, NASA Human Systems Integration (HSI) Handbook).
  1. Mission objectives are clearly defined and stated and are unambiguous and internally consistent.
  2. The selected concept(s) satisfactorily meets the stakeholder expectations.
  3. The mission is feasible. A concept has been identified that is technically and logistically feasible. A rough cost estimate is within an acceptable cost range.
  4. The concept evaluation criteria to be used in candidate systems evaluation have been identified and prioritized.
  5. The need for the mission has been clearly identified.
  6. The cost and schedule estimates are credible and sufficient resources are available for project formulation.
  7. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  8. TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  9. Alternative concepts have adequately considered the use of existing assets or products that could satisfy the mission or parts of the mission.
  10. Technical planning is sufficient to proceed to the next phase and includes planning for hardware, software, human systems, and data deliverables.
  11. Risk and mitigation strategies have been identified and are acceptable based on technical risk assessments.
  12. Software components meet the success criteria defined in the NASA-HDBK-2203.
  13. Human Systems Integration aspects are included in the management and technical planning following guidelines within NASA/SP-20210010952 and are sufficient to proceed to the next phase.
  14. Concurrence by the responsible Center spectrum manager that RF needs have been properly identified and addressed.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

**** Cybersecurity requirements are found in NPR 2810.1

G.5 System Requirements Review (SRR) for Projects and Single-Project Programs The SRR evaluates whether the functional and performance requirements defined for the system of interest are responsive to the programâ??s requirements and ensures the preliminary project plan and requirements will satisfy the mission. This table is used for projects and single-project programs. For other types of programs, refer to Table G-1.

Table G-4 – SRR Entrance and Success Criteria

System Requirements Review for Projects and Single-Project Programs
Entrance Criteria Success Criteria
  1. The project has successfully completed the previously planned life-cycle reviews and responses have been made to all RFAs and RIDs, or a timely closure plan exists for those items remaining open.
  2. A preliminary SRR agenda, success criteria, and instructions to the review board have been agreed to by the technical team, project manager, and review chair prior to the SRR.
  3. All planned higher level SRR and peer reviews have been successfully conducted and RID/RFA/Action Items have been addressed and resolved with the originator or designated TA, or a timely closure plan exists for those remaining open.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. The following primary technical products for hardware, software and human system elements are available to the cognizant participants prior to the review:
    1. **Requirements for system being reviewed are ready to be baselined after the review and preliminary allocation to the next lower level system has been performed.
    2. **For projects, one-step AO programs and single-project programs, the SEMP (or equivalent program/project documentation) is ready to be baselined after review comments are incorporated.
    3. **Human Systems Integration approach is ready to be baselined after review comments are incorporated. (For additional guidance on HSI entrance criteria refer to NASA/SP-20210010952, NASA Human Systems Integration (HSI) Handbook)
  6. Other SRR work products (as applicable) for hardware, software, and human system elements have been made available to the cognizant participants.
    1. *Updated concept definition.
    2. *Updated concept of operations.
    3. Updated parent requirements.
    4. *Risk management plan ready to be baselined after review comments are incorporated.
    5. *Updated risk assessment and mitigations.
    6. *Configuration management plan ready to be baselined after review comments are incorporated.
    7. Initial document tree or model structure.
    8. Preliminary verification and validation method identified for each requirement.
    9. Preliminary system safety analysis.
    10. Product certification or product acceptance data requirements.
    11. Interfaces with external systems are identified and preliminary definitions are ready to be baselined (e.g., Interface Control Documents).
    12. Preliminary MOPS and TPM and other key driving requirements.
    13. Other specialty discipline analyses, as required.
    14. *Updated cost and schedule estimates for the project implementation.
    15. *Updated documentation of Basis of Estimate (cost and schedule).
    16. *Updated Technology Development Plan.
    17. *Updated technology readiness assessment that has been reviewed and documented that includes technology assets, heritage products, and capability gaps identified.
    18. Logistics documentation (e.g., preliminary maintenance plan).
    19. *Initial Human Rating Certification Package.
    20. *System safety and mission assurance plan ready to be baselined after review comments are incorporated.
    21. *Preliminary operations concept.
    22. Preliminary engineering development assessment and technical plans to achieve what needs to be accomplished in the next phase.
    23. Software criteria and products, per the NASA-HDBK-2203.
    24. ***RF spectrum requirements have been addressed including preparing requisite data for the responsible Center Spectrum Manager for possible Stage 1 Certification.
    25. *Preliminary System Security Plan.
  1. The functional and performance requirements defined for the system are responsive to the stakeholder needs and parent requirements, reflect the system's intended operational use, and represent capabilities likely to be achieved within the scope of the project.
  2. The maturity of the requirements definition and associated plans is sufficient to begin Phase B.
  3. The project utilizes a sound process for the allocation and control of requirements throughout all levels, and a plan has been defined to complete the requirements definition at lower levels within schedule constraints.
  4. System Interfaces with external entities and between major internal elements have been identified, including system security expectations.
  5. Preliminary approaches have been determined for how requirements will be verified and validated.
  6. Major risks have been identified and technically assessed, and viable mitigation strategies have been defined.
  7. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  8. TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  9. Software components meet the success criteria defined in NASA-HDBK-2203.
  10. Human Systems Integration aspects are included in the management and technical planning following guidelines within NASA/SP-20210010952 and are sufficient to proceed to the next phase
  11. Concurrence by the responsible Center spectrum manager that the program/project has provided requisite RF system data.
  12. Proposed tailoring is appropriate and consistent with applicable Agency and Center guidance.
  13. Lessons Learned from other projects and programs have been identified and addressed.
  14. Single Point Failure/Fault Tolerance philosophy is reflected in requirements.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

G.6 Mission Definition Review/System Definition Review (MDR/SDR) for Project and Single-Project Programs

The MDR/SDR evaluates whether the proposed mission/system architecture is responsive to the program mission/system functional and performance requirements and whether requirements have been allocated to the next lower product layer and to all functional elements of the mission/system. This table is to be used for projects and single-project programs.

Table G-5 – MDR/SDR Entrance and Success Criteria
(Projects and Single-Project Program)

Mission Definition Review/System Definition Review for Projects
and Single-Project Programs
Entrance Criteria Success Criteria
  1. The project has successfully completed the previously planned life-cycle reviews and all RFAs and RIDs have been addressed and resolved, or a timely closure plan exists for those items remaining open.
  2. A preliminary MDR/SDR agenda, success criteria, and instructions to the review board have been agreed to by the technical team, project manager, and review chair prior to the MDR/SDR.
  3. All planned higher level MDR/SDR and peer reviews have been successfully conducted and RID/RFA/Action Items have been addressed with the originator or designated TA.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. The following primary technical products for hardware, software, and human system elements are available to the cognizant participants prior to the review:
    1. **Defined architecture, including major tradeoffs and options ready to be baselined after review comments are incorporated.
    2. **Allocation of requirements to next lower level is ready to be baselined after review comments are incorporated.
    3. **MOPs, TPMs, and other key driving requirements ready to be approved.
    4. **Approval and status of technical performance related to leading indicators, margins, TPMs, and resolution of the previous review discrepancies addressing effectiveness of technical achievement and communicating the overall risk to the project.
  6. Other MDR/SDR technical products listed below for both hardware and software system elements have been made available to the cognizant participants prior to the review:
    1. Supporting analyses, functional/timing descriptions, and allocations of functions to architecture elements.
    2. *Updated SEMP (or equivalent program/project documentation).
    3. *Updated risk management plan.
    4. *Updated risk assessment and mitigations (if required by the governing PM NPR, including PRA).
    5. *Updated Technology Development Plan.
    6. *Updated technology readiness that has been assessed and documented with technology assets, heritage products, and gaps identified.
    7. *Updated cost and schedule data with ranges and a basis of the estimates.
    8. *Preliminary Integrated Logistics Support Plan (ILSP).
    9. *Updated Human Systems Integration approach . (For additional guidance on HSI entrance criteria refer to NASA/SP-20210010952, NASA Human Systems Integration (HSI) Handbook).
    10. *Updated Human Rating Certification Package.
    11. Preliminary system interface definitions.
    12. Initial technical resource utilization estimates and margins.
    13. *Updated safety and mission assurance (SMA) plan.
    14. *Preliminary operations concept.
    15. Preliminary system safety analysis.
    16. Software criteria and products, per NASA-HDBK-2203.
    17. ***RF spectrum considerations assessment.
    18. *Preliminary System Security Plan.
    19. Preliminary integration plans
    20. Preliminary Verification and Validation Plans
  1. The proposed mission/system architecture is credible and responsive to program requirements and constraints, including resources.
  2. The program/project cost and schedule estimates are credible to meet program/project requirements within available resources with acceptable risk.
  3. The projectâ??s mission/system definition and associated plans are sufficiently mature to begin Phase B.
  4. All technical requirements are allocated to the architectural elements.
  5. The architecture tradeoffs are completed, and those planned for Phase B adequately address the option space.
  6. Significant development, mission, and health and medical safety risks are identified and technically assessed, and a process and resources exist to manage the risks.
  7. Adequate planning exists for the development, insertion, or deployment of any enabling new technology.
  8. The operations concept is consistent with proposed design concept(s) and is in alignment with the mission requirements.
  9. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  10. TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  11. Software components meet the success criteria defined in NASA-HDBK-2203.
  12. Human Systems Integration aspects are included in the management and technical planning following guidelines within NASA/SP-20210010952 and are sufficient to proceed to the next phase
  13. Concurrence by the responsible Center spectrum manager that RF spectrum considerations have been addressed.
  14. Procurement and supply chain risk management execution is complementary with the technical development schedule.
  15. Architecture supports the Single Point Failure/Fault Tolerance requirements.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

G.7 Preliminary Design Review (PDR)

The PDR demonstrates that the preliminary design meets all system of interest requirements with acceptable risk and within the cost and schedule constraints and establishes the basis for proceeding with detailed design.

Table G-6 – PDR Entrance and Success Criteria

Preliminary Design Review
Entrance Criteria Success Criteria
  1. The Project has successfully completed the previous planned life-cycle reviews, and all RFAs and RIDs have been addressed and resolved, or a timely closure plan exists for those remaining open.
  2. A preliminary PDR agenda, success criteria, and instructions to the review board have been agreed to by the technical team, project manager, and review chair prior to the PDR.
  3. All planned lower level PDRs and peer reviews have been successfully conducted, and RID/RFA/Action Items have been addressed with the originator or designated TA.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. The following primary products are ready for review:
    1. **A preliminary design that can be shown to meet all technical requirements and performance measures or has waivers.
    2. **Baselined integration plans
    3. **Baselined Verification and Validation Plan
  6. Other PDR technical work products (as applicable) for hardware, software, and human system elements have been made available to the cognizant participants prior to the review:
    1. Subsystem design specifications (hardware and software), with supporting trade-off analyses and data, as required, that are ready to be baselined after review comments are incorporated.
    2. Status of technical performance related to margins, TPMs, and resolution of the previous review discrepancies addressing effectiveness of technical achievement and communicating the overall risk to the project.
    3. *Updated technology readiness assessment.
    4. *Updated Technology Development Plan.
    5. *Updated risk assessment and mitigation.
    6. *Life-Cycle Cost and Integrated Master Schedule (IMS) that are ready to be baselined after review comments are incorporated. When required, the Joint Confidence Level (JCL) analysis.
    7. *Baselined Integrated Logistics Support Plan (ILSP).
    8. *Baselined Project Protection Plan.
    9. Applicable technical plans that are ready to be baselined after review comments are incorporated (e.g., technical performance measurement plan, contamination control plan, parts management plan, environments control plan, Electromagnetic Interference/ Electromagnetic Compatibility (EMI/EMC) control plan, payload-to-carrier integration plan, producibility/manufacturability program plan, reliability program plan, quality assurance plan).
    10. Applicable design standards that have been identified and incorporated.
    11. *Updated safety analyses and plans.
    12. Preliminary engineering drawing tree.
    13. Interface control documents that are ready to be baselined after review comments are incorporated.
    14. *Verification/validation plan that is ready to be baselined after review comments are incorporated.
    15. Plans to respond to regulatory requirements (e.g., Environmental Impact Statement), as required, that are ready to be baselined after review comments are incorporated.
    16. Preliminary Disposal Plan.
    17. Updated technical resource utilization estimates and margins.
    18. *Baseline operations concept.
    19. Updated SEMP (or equivalent program/project documentation).
    20. Updated Human Systems Integration approach. (For additional guidance on HSI entrance criteria refer to NASA/SP-20210010952, NASA Human Systems Integration (HSI) Handbook).
    21. *Updated Human Rating Certification Package.
    22. Software criteria and products, per NASA-HDBK-2203.
    23. ***Design and requisite data submitted to Center/facility spectrum manager for preparation of request for certification of Stage 2 spectrum support by at least 60 days prior to PDR.
    24. *Updated Preliminary System Security Plan.
    25. Procurement status including Supply Chain Risk Management (SCRM) activities (e.g., audits and assessments, GIDEP, counterfeit avoidance).
    26. List of potential single point failures.
  1. The top-level requirements, including mission success criteria, TPMs, and any sponsor-imposed constraints, are agreed upon, finalized, stated clearly, and consistent with the preliminary design.
  2. The flow down of verifiable requirements is complete and proper, or, if not, an adequate plan exists for timely resolution of open items. Requirements are traceable to parent technical requirements and to mission goals and objectives.
  3. The program/project cost, schedule, and JCL analysis (when required) are credible and within program/project constraints; are ready for NASA commitment; and are ready for the Management Agreement (for projects governed by NPR 7120.5).
  4. The preliminary design is expected to meet the requirements at an acceptable level of risk.
  5. Definition of the system interfaces (both external entities and between internal elements) is consistent with the overall technical maturity. Associated risks, including system security, have been identified and represent an acceptable level of risk.
  6. Any required new technology has been developed to an adequate state of readiness, or backup options exist and are supported to make them viable alternatives.
  7. The project risks are understood and have been credibly assessed, and plans, a process, and resources exist to effectively manage them.
  8. Safety and mission assurance (e.g., safety, reliability, maintainability, quality controls, quality verifications, supplier risk management, and Electrical, Electronic, and Electromechanical (EEE) parts) have been adequately addressed in preliminary designs and any applicable SMA products (e.g., PRA, system safety analysis, and failure modes and effects analysis) meet requirements, are at the appropriate maturity level for this phase of the program/project life cycle, and indicate that the program/project safety/reliability residual risks will be at an acceptable level.
  9. Adequate technical and programmatic margins (e.g., mass, power, memory) and resources exist to complete the development within budget, schedule, and known risks.
  10. The operational concept is technically sound, includes (where appropriate) human systems, and includes the flow down of requirements for its execution.
  11. Technical trade studies are mostly complete to sufficient detail and remaining trade studies are identified, plans exist for their closure, and potential impacts are understood.
  12. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  13. TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  14. Preliminary analysis of the primary subsystems has been completed and summarized, highlighting performance and design margin challenges.
  15. Appropriate modeling and analytical results are available and have been considered in the design.
  16. Heritage designs have been suitably assessed for applicability and appropriateness.
  17. Manufacturability has been adequately included in design.
  18. Software components meet the success criteria defined in NASA-HDBK-2203.
  19. Human Systems Integration aspects are included in the management and technical planning following guidelines within NASA/SP-20210010952 and are sufficient to proceed to the next phase
  20. Concurrence by the responsible Center spectrum manager that the program/project has provided requisite RF system data.
  21. Procurement and supply chain risk management execution is complementary with the technical development schedule.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

G.8 Critical Design Review (CDR)

The CDR demonstrates that the maturity of the design is appropriate to support proceeding with full-scale fabrication, assembly, integration, and test. The CDR determines that the technical effort is on track to complete the system development, meeting functional and performance requirements within the identified cost and schedule constraints at an acceptable risk.

Table G-7 – CDR Entrance and Success Criteria

Critical Design Review
Entrance Criteria Success Criteria
  1. The project has successfully completed the previous planned life-cycle reviews, and all RFAs and RIDs have been addressed and resolved or a timely closure plan exists for those remaining open.
  2. A preliminary CDR agenda, success criteria, and instructions to the review board have been agreed to by the technical team, project manager, and review chair prior to the CDR.
  3. All planned lower level CDRs and peer reviews have been successfully conducted, and RID/RFA/Action Items have been addressed with the originator or designated TA.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. **A baselined detailed design that can be shown to meet all technical requirements and performance measures or has waivers.
  6. Other CDR technical work products (as applicable) for hardware, software, and human system elements have been made available to the cognizant participants prior to the review:
    1. Product build-to specifications along with supporting trade-off analyses and data that are ready to be baselined after review comments are incorporated.
    2. Fabrication, assembly, integration, and test plans and procedures are being developed and are ready to be baselined after review comments are incorporated.
    3. Technical data package (e.g., integrated schematics, spares provisioning list, interface control documents, engineering analyses, and specifications).
    4. Status of technical performance related to margins, TPMs and resolution of the previous review discrepancies addressing effectiveness of technical achievement and communicating the overall risk to the project.
    5. Defined operational limits and constraints.
    6. Updated technical resource utilization estimates and margins.
    7. Acceptance plans that are ready to be baselined after review comments are incorporated.
    8. Command and telemetry list.
    9. *Updated Verification and Validation Plan.
    10. Updated integration plans.
    11. Preliminary launch site operations plan.
    12. Preliminary checkout and activation plan.
    13. Preliminary disposal plan (including decommissioning or termination).
    14. *Updated technology readiness assessment.
    15. *Updated Technology Development Plan.
    16. *Updated risk assessment and mitigation.
    17. Updated SEMP (or equivalent program/project documentation).
    18. Updated Human Systems Integration approach. (For additional guidance on HSI entrance criteria refer to NASA/SP-20210010952, NASA Human Systems Integration (HSI) Handbook).
    19. *Updated Human Rating Certification Package.
    20. Updated reliability analyses and assessments.
    21. *Updated Life-Cycle Costs and IMS.
    22. *Updated ILSP.
    23. *Updated Project Protection Plan.
    24. Subsystem-level and preliminary operations safety analyses that are ready to be baselined after review comments are incorporated.
    25. Systems and subsystem certification plans and requirements (as needed) that are ready to be baselined after review comments are incorporated.
    26. *System safety analysis with associated verifications that is ready to be baselined after review comments are incorporated.
    27. Software criteria and products, per NASA-HDBK-2203.
    28. ***Received Stage 2 (Experimental) RF system certification signed by NTIA.
    29. ***Provided measured/as-designed parameter updates to Center/facility spectrum manager for request for certification of Stage 4 (Operational) spectrum support no later than 60 days prior to CDR.
    30. *Baselined System Security Plan.
    31. Procurement status including Supply Chain Risk Management (SCRM) activities (e.g., audits and assessments, GIDEP, counterfeit avoidance, surveillance tailoring).
    32. List of all single point failures and their effects as well as rationale for acceptance.
  1. The detailed design is expected to meet the requirements with adequate margins.
  2. Interface control documents are sufficiently mature to proceed with fabrication, assembly, integration, and test, and plans are in place to manage any open items.
  3. The program/project cost and schedule estimates are credible and within program/project constraints.
  4. High confidence exists in the product baseline, and adequate documentation exists or will exist in a timely manner to allow proceeding with fabrication, assembly, integration, and test.
  5. The product verification and product validation requirements and plans are complete.
  6. The testing approach is comprehensive, and the planning for system assembly, integration, test, and launch site and mission operations is sufficient to progress into the next phase.
  7. Adequate technical and programmatic margins (e.g., mass, power, memory) and resources exist to complete the development within budget, schedule, and known risks.
  8. Risks to safety and mission success are understood and credibly assessed and plans and resources exist to effectively manage them.
  9. Safety and mission assurance (e.g., safety, reliability, maintainability, quality controls, SCRM, QA, and EEE parts) have been adequately addressed in system and operational designs, and any applicable SMA products (e.g., PRA, system safety analysis, and failure modes and effects analysis) meet requirements, are at the appropriate maturity level for this phase of the program/project life cycle, and indicate that the program/project safety/reliability/system security residual risks will be at an acceptable level.
  10. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  11. TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  12. Engineering test units, life test units, and/or modeling and simulations have been developed and tested per plan.
  13. Material properties tests are completed along with analyses of loads, stress, fracture control, contamination generation, and other analyses.
  14. EEE parts have been selected, and planned testing and delivery will support build schedules.
  15. The operational concept has matured, is at a CDR level of detail, and has been considered in test planning.
  16. Manufacturability has been adequately included in design.
  17. Software components meet the success criteria defined in NASA-HDBK-2203.
  18. Human Systems Integration aspects are included in the management and technical planning following guidelines within NASA/SP-20210010952 and are sufficient to proceed to the next phase
  19. Concurrence by the responsible Center spectrum manager that the program/project has provided requisite RF system data.
  20. Procurement and supply chain risk management execution is complementary with the technical development schedule.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

G.9 Production Readiness Review (PRR)

For projects developing or acquiring multiple systems/units (typically greater than three or as determined by the project). The PRR determines the readiness of the system developers to efficiently produce the required number of systems. It ensures that the production plans, fabrication, assembly, integration enabling products, operational support, and personnel are in place and ready to begin production.

Table G-8 – PRR Entrance and Success Criteria

Production Readiness Review
Entrance Criteria Success Criteria
  1. The significant production engineering problems and nonconformances encountered during development are resolved.
  2. The design documentation needed to support production is available.
  3. The production plans (including but not limited to critical process controls, control limits, and procedures) and preparation to begin fabrication are developed.
  4. The production-enabling products are ready.
  5. Raw materials are approved and certified.
  6. Resources are available, have been allocated, and are ready to support end product production.
  7. Updated costs and schedules.
  8. Risks have been identified, credibly assessed, and characterized, and mitigation efforts have been defined.
  9. The bill of materials is available and critical parts identified.
  10. Delivery schedules are available.
  11. In-process and end-item inspections and tests have been identified and planned.
  12. Software criteria and products, per NASA-HDBK-2203.
  13. *Spectrum (radio frequency) consideration assessments.
  14. *Updated Human Systems Integration approach.
  1. High confidence exists that the system requirements will be met in the final production configuration.
  2. Adequate resources are in place to support production.
  3. The program/project cost and schedule estimates are credible and within program/project constraints.
  4. Design-for-manufacturing considerations have been incorporated to ensure ease and efficiency of production and assembly.
  5. The product is deemed manufacturable. Evidence is provided that the program/project is compliant with NASA and Implementing Center requirements, standards, processes, and procedures.
  6. TBD and TBR items are clearly identified, with acceptable plans and schedule for their disposition. Alternate sources for resources have been identified for key items.
  7. Adequate spares have been planned and budgeted.
  8. Required facilities and tools are sufficient for end-product production.
  9. Specified special tools and test equipment are available in proper quantities.
  10. Production and support staff are qualified.
  11. Drawings and/or production models are approved/certified.
  12. Production engineering and planning are sufficiently mature for cost-effective production.
  13. Production processes and methods are consistent with quality requirements and compliant with occupational health and medical, safety, environmental, and energy conservation regulations.
  14. Qualified suppliers are available for materials that are to be procured.
  15. Software components meet the success criteria defined in NASA-HDBK-2203.
  16. Concurrence by the responsible Center spectrum manager that program/project complies with RF spectrum policy and regulation.
  17. PRR plans are mature and results to date indicate high likelihood of supplier quality control success.
  18. Production processes and methods are within acceptable risk from threats and vulnerabilities.

*Required per NPD 2570.5.

G.10 System Integration Review (SIR)

An SIR ensures that the component parts of the system of interest are on schedule and technically mature to be integrated and that integration facilities, support personnel, and integration plans and procedures are on schedule and ready to support integration. Depending on the complexity of the system of interest, a series of formal or informal SIRs may be conducted at lower levels in the product hierarchy.

Table G-9 – SIR Entrance and Success Criteria

System Integration Review
Entrance Criteria Success Criteria
  1. The project has successfully completed the previous planned life-cycle reviews, and all RFAs and RIDs have been addressed and resolved or a timely closure plan exists for those remaining open.
  2. A preliminary SIR agenda, success criteria, and instructions to the review board have been agreed to by the technical team, project manager, and review chair prior to the SIR.
  3. The following primary products are ready for review:
    1. **Integration plans baselined at PDR that have been updated and approved.
    2. **Initial V&V results from any lower tier products that have been verified.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. Status of technical performance related to margins, TPMs, and resolution of the previous review discrepancies addressing effectiveness of technical achievement and communicating the overall risk to the project.
  6. Integration procedures have been identified and are scheduled for completion prior to their need dates.
  7. Segments and/or components are on schedule to be available for integration.
  8. Mechanical and electrical interface requirements for hardware necessary to start system integration have been verified in accordance with the interface control documentation and plans for verification of remaining hardware exist.
  9. All functional, unit-level, subsystem, and qualification testing has been conducted successfully or is on track to be conducted prior to scheduled integration.
  10. Integration facilities, including clean rooms, ground support equipment, handling fixtures, overhead cranes, and electrical test equipment, and their associated quality controls are ready or will be available when required.
  11. Support personnel have been trained.
  12. Handling and safety requirements have been documented.
  13. All known system discrepancies have been identified, dispositioned, and are on schedule for closure.
  14. The quality control organization is ready to support integration effort.
  15. Other SIR technical products (as applicable) for hardware, software, and human system elements have been made available to the cognizant participants prior to the review:
    1. *Updated Life-Cycle Costs and IMS.
    2. *Updated design solution definition.
    3. Updated interface definition(s).
    4. *Updated verification and validation plans.
    5. Final transportation criteria and instructions.
    6. *Preliminary mission operations plans.
    7. Preliminary decommissioning plans.
    8. Preliminary disposal plans.
    9. Software criteria and products, per NASA-HDBK-2203.
    10. Procurement status including Supply Chain Risk Management (SCRM) activities (e.g., audits and assessments, GIDEP, counterfeit avoidance).
  1. Integration plans and procedures are on track for completion and approval to support system integration.
  2. Previous component, subsystem, and system test results form a satisfactory basis for proceeding to integration.
  3. The program/project cost and schedule estimates are credible with adequate margins and within program/project constraints.
  4. Risks are identified and accepted by program/project leadership, as required.
  5. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  6. TBD and TBR items are clearly identified with acceptable plans and schedule for their dispositions.
  7. The integration procedures and workflow have been clearly defined and documented or are on schedule to be clearly defined and documented prior to their need date.
  8. The review of the integration plans, as well as the procedures, environment, and configuration of the items to be integrated, provides a reasonable expectation that the integration will proceed successfully.
  9. All training necessary to properly integrate the system has been performed.
  10. Software components meet the success criteria defined in NASA-HDBK-2203.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

G.11 Test Readiness Review (TRR)

A TRR for each planned test or series of tests ensures that the test article (hardware/software), test facility, support personnel, and test procedures are ready for testing and data acquisition, reduction, and control.

Table G-10 – TRR Entrance and Success Criteria

Test Readiness Review
Entrance Criteria Success Criteria
  1. A preliminary TRR agenda, success criteria, and instructions to the review team have been agreed to by the technical team, project manager, and review chair prior to the TRR.
  2. The objectives of the testing have been clearly defined and documented.
  3. Approved test plans, test procedures, test environment, and configuration of the test item(s) that support test objectives are available.
  4. All test interfaces have been placed under configuration control or have been defined in accordance with an agreed to plan, and version description document(s) for both test and support systems have been made available to TRR participants prior to the review.
  5. All known system discrepancies have been identified and dispositioned in accordance with an agreed-upon plan.
  6. All required test resourcesâ?"people (including a designated test director), facilities, test articles, test instrumentation, and other test-enabling productsâ?"have been identified and are available to support required tests.
  7. Roles and responsibilities of all test participants are defined and agreed to.
  8. Test safety planning has been accomplished, and all personnel have been trained.
  9. Spectrum (radio frequency) considerations addressed.
  10. As-built hardware and software documentation defining the configuration of the item under test are released and under configuration control.
  1. Adequate test plans are completed and approved for the system under test.
  2. Adequate identification and coordination of required test resources are completed.
  3. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  4. TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  5. Risks have been identified, credibly assessed, and appropriately mitigated.
  6. Residual risk is accepted by program/project leadership as required.
  7. Plans to capture any lessons learned from the test program are documented.
  8. The objectives of the testing have been clearly defined and documented, and the review of all the test plans, as well as the procedures, environment, and configuration of the test item, provides a reasonable expectation that the objectives will be met.
  9. The test cases have been analyzed and are consistent with the test plans and objectives.
  10. Test personnel have received appropriate training in test operation and health and medical safety procedures.
  11. *Concurrence by the responsible Center spectrum manager that all tests are performed. in accordance with spectrum policy and regulation.

*Required per NPD 2570.5.

G.12 System Acceptance Review (SAR)

The SAR verifies the completeness of the specific end products in relation to their expected maturity level, requirement verification, compliance to stakeholder expectations, and ensures that the system of interest has sufficient technical maturity to authorize its acceptance for operational use or delivery to the launch site or operational environment.

Table G-11 – SAR Entrance and Success Criteria

System Acceptance Review
Entrance Criteria Success Criteria
  1. The project has successfully completed the previous planned life-cycle reviews, RFA/RIDs have been closed, and plans to complete open work are defined.
  2. A preliminary SAR agenda, success criteria, and instructions to the review team have been agreed to by the technical team, project manager, and review chair prior to the review.
  3. The following SAR technical products have been made available to the cognizant participants prior to the review:
    1. Results of the SARs conducted at the major suppliers.
    2. Product verification results.
    3. Product validation results.
    4. Documentation that the delivered system complies with the established acceptance criteria.
    5. Documentation that the system will perform properly in the expected operational environment.
    6. Technical data package that has been updated to include all test results.
    7. Final Certification Package.
    8. Baselined as-built hardware and software documentation.
    9. Updated risk assessment and mitigation.
    10. Required safety, shipping, handling, checkout, and operational plans and procedures.
    11. Software criteria and products, per NASA-HDBK-2203.
    12. *Received Stage 4 (Operational) system certification signed by NTIA.
    13. Completed planning for sustaining the system.
    14. Updated list of all single point failures and their effects.
  1. Required tests and analyses are complete and indicate that the system will perform properly in the expected operational environment.
  2. Risks are identified and mitigated to acceptable levels.
  3. System meets the established acceptance criteria.
  4. TBD and TBR items are resolved.
  5. Acceptance data package is complete and reflects the delivered system.
  6. All applicable lessons learned for organizational improvement and system operations are captured.
  7. Software components meet the success criteria defined in NASA-HDBK-2203.
  8. *Concurrence by the responsible Center spectrum manager that the Stage 4 (Operational) system certification has been obtained and the system is compliant with spectrum policy and regulation.
  9. The system hardware, software, documentation, and associated products are complete and ready for acceptance.

*Required per NPD 2570.5.

G.13 Operational Readiness Review (ORR)

The ORR ensures that all system and support (flight and ground) hardware, software, personnel, procedures, supporting capabilities, and user documentation accurately reflect the deployed state of the system and are operationally ready.

Table G-12 – ORR Entrance and Success Criteria

Operational Readiness Review
Entrance Criteria Success Criteria
  1. All planned ground-based testing has been completed.
  2. Test failures and anomalies from verification and validation testing have been resolved, and the results/mitigations/work-arounds have been incorporated into supporting and enabling operational products.
  3. All operational supporting and enabling products (e.g., facilities, equipment, documents, software tools, databases) that are necessary for nominal and contingency operations have been tested and delivered/installed at the site(s) necessary to support operations.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. Operations documentation (e.g., handbook, procedures) has been written, verified, and approved.
  6. Users/operators have been trained on the correct operation of the system.
  7. Operational contingency planning has been completed, and operations personnel have been trained on their use.
  8. The following primary products are ready for review:
    1. **Preliminary V&V results.
    2. **Baseline decommissioning plan.
    3. **Baseline disposal plans.
  9. Other ORR technical products have been made available to the cognizant participants prior to the review:
    1. *Updated cost and schedule.
    2. *Updated Project Protection Plan.
    3. Updated as-built hardware and software documentation.
    4. Baselined operations plans.
    5. Updated operational procedures.
    6. Preliminary certification for flight/use.
    7. *Updated Human Rating Certification Package.
    8. Software criteria and products, per NASA-HDBK-2203.
  10. ***Received Stage 4 (Operational) system certification signed by NTIA.
  11. ***All requisite radio frequency authorizations are in place.
  12. Updated list of all single point failures (SPF) and their effects including rationale for acceptance of any new SPFs.
  1. The system, including all enabling products, is determined to be ready to be placed in an operational status.
  2. All applicable lessons learned for organizational improvement and systems operations have been captured.
  3. All waivers and anomalies have been closed.
  4. Systems hardware, software, personnel, tools, supporting infrastructure, and procedures are in place to support operations.
  5. Operations plans and schedules are consistent with mission objectives.
  6. Mission risks have been identified, planned mitigations are adequate, and residual risks are accepted by the program/project manager.
  7. Testing is consistent with the expected operational environment.
  8. The program/project cost and schedule estimates are credible and within program/project constraints.
  9. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  10. TBD and TBR items are resolved.
  11. Software components meet the success criteria defined in NASA-HDBK-2203.
  12. Concurrence by the responsible Center spectrum manager that all necessary spectrum certification(s) and authorization(s) have been obtained.
  13. An operational Human Systems Integration capability has been established and HSI planning is in place for the remaining life-cycle phases.

*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1. ***Required per NPD 2570.5.

G.14 Mission Readiness Review/Flight Readiness Review (MRR/FRR)

The MRR/FRR examines tests, demonstrations, analyses, and audits that determine the systemâ??s readiness for a safe and successful flight or launch and for subsequent flight operations. The MRR/FRR also ensures that all flight and ground hardware, software, personnel, and procedures are operationally ready.

Table G-13 – MRR/FRR Entrance and Success Criteria

Mission Readiness Review/Flight Readiness Review
Entrance Criteria Success Criteria
  1. The system and support elements are ready and have been properly configured for flight/mission operations.
  2. System and support element interfaces have been demonstrated to function as expected.
  3. The system state supports a launch â??goâ?? decision based on the established go/no-go criteria.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. Failures and anomalies from previously completed flights, tests, and reviews have been resolved, and the results/mitigations/work-arounds have been incorporated into supporting and enabling operational products.
  6. The following primary products are ready for review:
    1. **Final certification for flight/use.
    2. **Baselined V&V results.
  7. Other MRR/FRR technical products have been made available to the cognizant participants prior to the review:
    1. *Updated cost.
    2. *Updated schedule.
    3. Updated as-built hardware and software documentation.
    4. Updated operations procedures.
    5. Updated decommissioning plan.
    6. Updated disposal plan
    7. Software criteria and products, per NASA-HDBK-2203.
  8. ***Received Stage 4 (Operational) system certification signed by NTIA.
  9. ***All requisite spectrum (radio frequency) authorizations are in place.
  10. Updated list of all single point failures and their effects.
  1. The flight vehicle/system is ready for flight/mission operations.
  2. The hardware is deemed acceptably safe for flight/mission operations.
  3. Certification that flight operations can safely proceed with acceptable risk has been achieved.
  4. Flight and ground software elements are ready to support launch and flight operations.
  5. Interfaces have been checked and demonstrated to be functional.
  6. The program/project has demonstrated compliance with applicable NASA and implementing Center requirements, standards, processes, and procedures.
  7. TBD and TBR items are resolved.
  8. Open items and waivers have been examined and residual risk from these is deemed to be acceptable.
  9. The flight and recovery environmental factors are within constraints.
  10. All open safety and mission risk items have been addressed, and the residual risk is deemed acceptable.
  11. Supporting organizations are ready to support flight/mission operations.
  12. Software components meet the success criteria defined in NASA-HDBK-2203.
  13. Responsible Center spectrum manager(s) concur that all necessary spectrum certification(s) and authorization(s) have been obtained.
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

***Required per NPD 2570.5.

G.15 Post-Launch Assessment Review (PLAR)

A PLAR evaluates the readiness of the spacecraft systems to proceed with full, routine operations after post-launch deployment. The review also evaluates the status of the project plans and the capability to conduct the mission with emphasis on near-term operations and mission-critical events.

Table G-14 – PLAR Entrance and Success Criteria

Post-Launch Assessment Review
Entrance Criteria Success Criteria
  1. The launch and early operations performance, including (when appropriate) the early propulsive maneuver results, are available.
  2. The observed spacecraft and science instrument performance, including instrument calibration plans and status, are available.
  3. The launch vehicle performance assessment and mission implications, including launch sequence assessment and launch operations experience with lessons learned, are completed.
  4. The mission operations and ground data system experience, including tracking and data acquisition support and spacecraft telemetry data analysis, is available.
  5. The mission operations organization, including status of staffing, facilities, tools, and mission software (e.g., spacecraft analysis and sequencing), is available.
  6. In-flight anomalies and the responsive actions taken, including any autonomous fault protection actions taken by the spacecraft or any unexplained spacecraft telemetry, including alarms, are documented.
  7. The need for significant changes to the system (e.g., hardware, software, or interfaces), support systems, operations (e.g., schedules, processes and procedures), and staffing has been documented.
  8. Documentation is updated, including any updates originating from the early operations experience.
  9. Plans for post-launch development have been addressed.
  1. The observed spacecraft and science payload performance agrees with prediction, or if not, is adequately understood so that future behavior can be predicted with confidence.
  2. All anomalies have been adequately documented and their impact on operations assessed. Further, anomalies impacting spacecraft health and medical, safety, or critical flight operations have been properly dispositioned.
  3. The mission operations capabilities, including staffing and plans, are adequate to accommodate the actual flight performance.
  4. Open items, if any, on operations identified as part of the ORR have been satisfactorily dispositioned.
  5. *Concurrence by the responsible Center spectrum manager that the system is compliant with spectrum policy and regulation.

*Required per NPD 2570.5.

G.16 Critical Event Readiness Review (CERR)

A CERR evaluates the readiness of the project and the flight system to execute the critical event during flight operation.

Table G-15 – CERR Entrance and Success Criteria

Critical Event Readiness Review
Entrance Criteria Success Criteria
  1. Critical event/activity requirements and constraints have been identified, including spectrum considerations.
  2. Critical event/activity design and implementation are complete.
  3. Critical event/activity testing is complete.
  4. Critical event/activity operations planning, including contingencies, is complete.
  5. Operations personnel training for the critical event/activity has been conducted.
  6. Critical event/activity sequence verification and validation is complete.
  7. Flight system is healthy and capable of performing the critical event/activity.
  8. Flight failures and anomalies from critical event/activity testing have been resolved, and the results/mitigations/work-arounds have been incorporated into supporting and enabling operational products.
  9. The following technical products have been made available to the cognizant participants prior to the review:
    1. Final certification for critical event readiness.
    2. Updated operations procedures.
  1. The critical activity design complies with requirements. The preparation for the critical activity, including the verification and validation, is thorough.
  2. The project (including all the systems, supporting services, and documentation) is ready to support the activity.
  3. The requirements for the successful execution of the critical event(s) are complete and understood and have flowed down to the appropriate levels for implementation.
  4. Any TBD and TBR items related to the critical event have been resolved.
  5. All open risk items related to the critical event have been addressed, and the residual risk is deemed acceptable.
  6. *Concurrence by the responsible Center spectrum manager that the system is compliant with spectrum policy and regulation.

*Required per NPD 2570.5.

G.17 Post-Flight Assessment Review (PFAR)

The PFAR evaluates how well mission objectives were met during a mission and identifies all flight and ground system anomalies that occurred during the flight and determines the actions necessary to mitigate or resolve the anomalies for future flights of the same spacecraft design.

Table G-16 – PFAR Entrance and Success Criteria

Post-Flight Assessment Review
Entrance Criteria Success Criteria
  1. All anomalies that occurred during the mission, as well as during preflight testing, countdown, and ascent, are dispositioned.
  2. All flight and post-flight documentation applicable to future flights of the spacecraft or the design is available.
  3. All planned activities to be performed post-flight have been completed.
  4. Problem reports, corrective action requests, and post-flight anomaly records are completed. Include spectrum (radio frequency) interference or other related factors during assessment.
  5. All post-flight hardware and flight performance data evaluation reports are completed.
  6. Plans for retaining assessment documentation and imaging have been made.
  1. Formal final report documenting flight performance and recommendations for future missions is complete and adequate.
  2. All anomalies have been adequately documented and dispositioned.
  3. The impact of anomalies on future flight operations has been assessed and documented.
  4. Reports and other documentation have been retained for performance comparison and trending.
  5. Responsible Center spectrum manager was notified of any RF spectrum interference issues.
  6. Recommendations for updates to the system design, test and operations procedures, or safety inspections have been identified and a credible plan exists to incorporate the changes.

G.18 Decommissioning Review (DR)

A DR confirms the decision to terminate or decommission the system and assesses the readiness of the system for the safe decommissioning and disposal of system assets. This review can be applied for the system that was deployed through earlier efforts of this program/project or for a legacy capability that will be replaced by the system being deployed.

Table G-17 – DR Entrance and Success Criteria

Decommissioning Review
Entrance Criteria Success Criteria
  1. The requirements associated with decommissioning are defined.
  2. Plans are in place for decommissioning and any other removal from service activities.
  3. Resources are in place to support and implement decommissioning.
  4. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  5. Health and medical, safety, environmental, and any other constraints have been identified.
  6. Current system capabilities relating to decommissioning are understood.
  7. Off-nominal operations, all contributing events, conditions, and changes to the originally expected baseline have been considered and assessed.
  8. The following primary product is ready for review:
    1. **Updated decommissioning plan.
  9. Other DR technical products have been made available to the cognizant participants prior to the review:
    1. *Updated cost.
    2. Updated schedule.
    3. *Updated disposal plan.
  1. The rationale for decommissioning is documented.
  2. The decommissioning plan is complete, meets requirements, is approved by appropriate management, and is compliant with applicable Agency safety, environmental, and health regulations.
  3. Operations plans for decommissioning, including contingencies, are complete and approved.
  4. Adequate resources (schedule, budget, and staffing) have been identified and are available to successfully complete all decommissioning activities.
  5. All required support systems for decommissioning are available.
  6. All personnel have been properly trained for the nominal and contingency decommissioning procedures.
  7. Safety, health, and environmental hazards have been identified, and controls have been verified.
  8. Risks associated with the decommissioning have been identified and adequately mitigated.
  9. Residual risks have been accepted by the required management.
  10. Any TBD and TBR items are clearly identified with acceptable plans and schedule for their disposition.
  11. Plans for archival and subsequent analysis of mission data have been defined and approved, and arrangements have been finalized for the execution of such plans.
  12. Plans for the capture and dissemination of appropriate lessons learned during the project life cycle have been defined and approved.
  13. Plans for transition of personnel have been defined and approved.
  14. Concurrence by the responsible Center spectrum manager that the decommissioning plans are compliant with spectrum policy and regulation.
*Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

**Product is required per NPR 7123.1.

G.19 Disposal Readiness Review (DRR)

A DRR confirms the readiness for the final disposal of the system assets. This review can be applied for the system that was deployed through earlier efforts of this program/project or for a legacy capability that will be disposed of and replaced by the system being deployed.

Table G-18 – DRR Entrance and Success Criteria

Disposal Readiness Review
Entrance Criteria Success Criteria
  1. Requirements associated with disposal are defined.
  2. Plans are in place for disposal and any other removal from service activities.
  3. Resources are in place to support disposal.
  4. Safety, environmental, health, and any other constraints are described.
  5. Current system capabilities related to disposal are described and understood.
  6. Off-nominal operations, all contributing events, conditions, and changes to the originally expected baseline have been considered and assessed.
  7. *Updated cost.
  8. Updated schedule.
  9. The following primary product is ready for review:
    1. **Updated disposal plan.
  1. The rationale for disposal is documented.
  2. The disposal plan is complete, meets requirements, is approved by appropriate management, and is compliant with applicable Agency safety, environmental, and health regulations.
  3. Operations plans for disposal, including contingencies, are complete and approved.
  4. All required support systems for disposal are available.
  5. All personnel have been properly trained for the nominal and contingency disposal procedures.
  6. Safety, health, and environmental hazards have been identified, and controls have been verified.
  7. Risks associated with the disposal have been identified and adequately mitigated.
  8. Residual risks have been accepted by the required management.
  9. If hardware is to be recovered from orbit:
    1. Return site activity plans have been defined and approved.
    2. Required facilities are available and meet requirements, including those for contamination control, if needed.
    3. Transportation plans are defined and approved.
    4. Shipping containers and handling equipment, as well as contamination and environmental control and monitoring devices, are available.
  10. Plans for disposition of mission-owned assets (i.e., hardware, software, facilities and data) have been defined and approved.
  11. Adequate resources (schedule, budget, and staffing) have been identified and are available to successfully complete all disposal activities.
  12. All mission and project data and documentation has been archived per disposal plan.
  13. TBD and TBR items related to system disposal have all been dispositioned.
  14. Concurrence by the responsible Center spectrum manager that the disposal plans are compliant with spectrum policy and regulation.

* Product is required for programs/projects covered by NPR 7120.5. If there is disagreement between this table and NPR 7120.5, NPR 7120.5 takes precedence.

** Product is required per NPR 7123.1.

G.20 Peer Reviews

Peer reviews provide the technical insight essential to ensure product and process quality. Peer reviews are focused, in-depth technical reviews that support the evolving design and development of a product, including critical documentation or data packages. The participants in a peer review are the technical experts and key stakeholders for the scope of the review. Another purpose of the peer review is to add value and reduce risk through expert knowledge infusion, confirmation of approach, identification of defects, and specific suggestions for product improvements.

Table G-19 – Peer Review Entrance and Success Criteria

Peer Review
Entrance Criteria Success Criteria
  1. The product to be reviewed (e.g., document, process, model, design details) has been identified and made available to the review team.
  2. Peer reviewers independent from the project have been selected for their technical background related to the product being reviewed.
  3. A preliminary agenda, success criteria, and instructions to the review team have been agreed to by the technical team and project manager.
  4. Rules have been established to ensure consistency among the team members involved in the peer review process.
  5. *Spectrum (radio frequency) considerations addressed.
  1. Peer review has thoroughly evaluated the technical integrity and quality of the product.
  2. Any defects have been identified and characterized.
  3. Results of the peer review are communicated to the appropriate project personnel.
  4. Spectrum-related aspects have been concurred to by the responsible Center spectrum manager.

*Required per NPD 2570.5.

G.21 Program Implementation Reviews (PIR) and Program Status Reviews (PSR) PIRs or PSRs are periodically conducted, as required by the Decision Authority and documented in the program plan, during the Implementation phase to evaluate the programâ??s continuing relevance to the Agencyâ??s Strategic Plan. These reviews assess the program performance with respect to expectations and determine the programâ??s ability to execute the implementation plan with acceptable risk within cost and schedule constraints.

Table G-20 – PIR/PSR Entrance and Success Criteria

Program Implementation and Program Status Reviews
Entrance Criteria Success Criteria
  1. A preliminary PIR agenda, success criteria, and instructions to the review team have been agreed to by the technical team, project manager, and review chair prior to the review.
  2. The current status of the overall technical effort is available and ready to be reviewed.
  3. Programmatic products are ready for review at the maturity levels stated in the governing program/project management NPR.
  4. Current actual and estimated costs are available and compared to the expected plan.
  5. Current schedule is available showing remaining work planned.
  6. Trending of the selected Technical Performance Parameters relevant to the current Program phase is available.
  7. Updated technical plans are available.
  8. *Spectrum (radio frequency) considerations addressed.
  1. Program still meets Agency needs and should continue.
  2. The program cost and schedule estimates are credible and within program constraints.
  3. Risks are identified and accepted by program/project leadership, as required.
  4. Technical trends are within acceptable bounds.
  5. Adequate progress has been made relative to plans, including the technology readiness levels.
  6. For technology development programs, technologies have been identified that are ready to be transitioned to another project or to an organization outside the Agency.
  7. Spectrum-related aspects have been concurred to by the responsible Center spectrum manager.

*Required per NPD 2570.5.

G.22 Design Certification Review (DCR)

This review is not depicted in the standard life-cycle review figures but has proven useful to larger projects such as human space flight. Projects/Centers may choose to add this review to their standard life cycle if they feel it is useful. The DCR ensures that the design complies with functional and performance requirements, as demonstrated in verification, validation, and qualification evidence. The certified design forms the basis from which system acceptance will be assessed. A DCR should, ideally, be held after a CDR and before a SAR.

Table G-21 – DCR Entrance and Success Criteria

Design Certification Review
Entrance Criteria Success Criteria
  1. The project has successfully completed the previous planned life-cycle reviews, RFA/RIDs have been closed, and plans to complete open work are defined.
  2. A preliminary DCR agenda, success criteria, and instructions to the review team have been agreed to by the technical team, project manager, and review chair prior to the review.
  3. The following DCR technical products have been made available to the cognizant participants prior to the review:
    1. Updated Verification and Validation Plan.
    2. As-run qualification test procedures, configurations, test environments, and test results.
    3. Product verification results.
    4. Product validation results.
    5. Documentation that the system will perform properly in the design environments.
    6. Final design certification package.
    7. Safety products (e.g., Failure Mode and Effects Analysis/Critical Items Lists (FMEA/CILs), Failure Mode, Effects, and Criticality Analysis (FMECA), Safety, Hazard Reports).
    8. All operating, production or fabrication, and maintenance constraints are documented.
    9. Updated risk assessment and mitigation.
    10. Waivers/deviations affecting the qualification articles, procedures, or environments.
  1. Qualification tests, configurations, and test environments demonstrate the system can meet functional and performance requirements across all applicable flight envelopes, configurations, and environments.
  2. Required tests and analyses are complete and indicate that the system will perform properly in the expected design environments.
  3. Design certification data package is complete and reflects the as-certified system.
  4. Waivers/deviations and non-conformance affecting the qualification test articles, procedures, or environments have been approved.
  5. Design mitigations have been appropriately implemented in response to safety products (e.g., FEMA/CILs, FMECA, Safety, and Hazard Reports) and indicate residual safety and mission success risks are acceptable for all intended uses of the system.
  6. Operating, production or fabrication, and maintenance constraints demonstrate a viable path to producing the system per the design.
  7. Risks are known and manageable.
  8. TBD and TBR items are resolved.
  9. *Concurrence by the responsible Center spectrum manager that all tests are performed in accordance with spectrum policy and regulation.


| TOC | ChangeLog | Preface | Chapter1 | Chapter2 | Chapter3 | Chapter4 | Chapter5 | Chapter6 | AppendixA | AppendixB | AppendixC | AppendixD | AppendixE | AppendixF | AppendixG | AppendixH | AppendixI | AppendixJ | AppendixK | ALL |
 
| NODIS Library | Program Formulation(7000s) | Search |

DISTRIBUTION:
NODIS

This document does not bind the public, except as authorized by law or as incorporated into a contract. This document is uncontrolled when printed. Check the NASA Online Directives Information System (NODIS) Library to verify that this is the correct version before use: https://nodis3.gsfc.nasa.gov.