Program/System Requirements Review

Entrance Criteria

Success Criteria

1.  A Formulation Authorization Document (FAD) has been approved.

2.  Program requirements have been defined that support mission directorate requirements on the program.

3.  Major program risks and corresponding mitigation strategies have been identified.

4.  The high-level program requirements have been documented to include:

a.   performance,

b.   safety, and

c.   programmatic requirements.

5.   An approach for verifying compliance with program requirements has been defined.

6.   Procedures for controlling changes to program requirements have been defined and approved.

7.   Traceability of program requirements to individual projects is documented in accordance with Agency needs, goals, and objectives, as described in the NASA Strategic Plan.

8.   Top program/project risks with significant technical, safety, cost, and schedule impacts are identified.

1.   With respect to mission and science requirements, defined high-level program requirements are determined to be complete and are approved.

2.   Defined interfaces with other programs are approved.

3.   The program requirements are determined to provide a cost-effective program.

4.   The program requirements are adequately levied on either the single-program project or the multiple projects of the program.

5.   The plans for controlling program requirement changes have been approved.

6.   The approach for verifying compliance with program requirements has been approved.

7.   The mitigation strategies for handling identified major risks have been approved.

Program/System Definition Review

Entrance Criteria

Success Criteria

1.    A Program/System Requirements Review has been satisfactorily completed.

2.    A program plan has been prepared that includes the following:

a.    how the program will be managed;

b.    a list of specific projects;

c.    the high-level program requirements (including risk criteria);

d.    performance, safety, and programmatic requirements correlated to Agency and directorate strategic objectives;

e.    description of the systems to be developed (hardware and software), legacy systems, system interfaces, and facilities; and

f.    identification of major constraints affecting system development (e.g., cost, launch window, required launch vehicle, mission planetary environment, engine design, international partners, and technology drivers).

3.    Program level SEMP which includes project technical approaches and management plans to implement the allocated program requirements including constituent launch, flight, and ground systems, and operations and logistics concepts.

4.    Independent Cost Analyses (ICAs) and Independent Cost Estimates (ICEs).

5.    Management plan for resources other than budget.

6.   Documentation for obtaining the program commitment agreement that includes the following:

a.   the feasibility of the program mission solution with a cost estimate within acceptable cost range,

b.   project plans adequate for project formulation initiation,

c.   identified and prioritized program concept evaluation criteria to be used in project evaluations,

d.   estimates of required annual funding levels,

e.   credible program cost and schedule allocation estimates to projects,

f.   acceptable risk and mitigation strategies (supported by a technical risk assessment),

g.   organizational structures and defined work assignments,

h.   defined program acquisition strategies.

i.   interfaces to other programs and partners,

j.   a draft plan for program implementation, and

k.   a defined program management system.

7.   A draft program control plan that includes:

a.   how the program plans to control program requirements, technical design, schedule, and cost to achieve its high-level requirements;

b.   how the requirements, technical design, schedule, and cost of the program will be controlled;

c.   how the program will utilize its technical, schedule, and cost reserves to control the baseline;

d.   how the program plans to report technical, schedule, and cost status to the MDAA, including frequency and the level of detail; and

e.   how the program will address technical waivers and how dissenting opinions will be handled.

8. For each project, a top-level description has been documented.

1. An approved program plan and management approach.

2. Approved SEMP and technical approach.

3. Estimated costs are adequate.

4. Documentation for obtaining the Program Commitment Agreement is approved.

5. An approved draft program control plan.

6. Agreement that the program is aligned with the Agency needs, goals and objectives.

7. The technical approach is adequate.

8. The schedule is adequate and consistent with cost, risk and mission goals.

9. Resources other than budget are adequate and available.

Mission Concept Review

Entrance Criteria

Success Criteria

1.   Mission goals and objectives.

2.   Analysis of alternative concepts to show at least one is feasible.

3.   Concept of operations.

4.   Preliminary mission descope options.

5.   Preliminary risk assessment, including technologies and associated risk management/mitigation strategies and options.

6.   Conceptual test and evaluation strategy.

7.   Preliminary technical plans to achieve next phase.

8.   Defined MOEs and MOPs.

9.   Conceptual life-cycle support strategies (logistics, manufacturing, and operation).

1.   Mission objectives are clearly defined and stated and are unambiguous and internally consistent.

2.   The preliminary set of requirements satisfactorily provides a system that will meet the mission objectives.

3.   The mission is feasible. A solution has been identified that is technically feasible. A rough cost estimate is within an acceptable cost range.

4.   The concept evaluation criteria to be used in candidate systems evaluation have been identified and prioritized.

5.   The need for the mission has been clearly identified.

6.   The cost and schedule estimates are credible.

7.   An updated technical search was done to identify existing assets or products that could satisfy the mission or parts of the mission.

8.   Technical planning is sufficient to proceed to the next phase.

9.   Risk and mitigation strategies have been identified and are acceptable based on technical risk assessments.

System Requirements Review

Entrance Criteria

Success Criteria

1.   Successful completion of the MCR and responses made to all MCR Requests for Actions (RFAs) and Review Item Discrepancies (RIDs).

2.   A preliminary SRR agenda, success criteria, and charge to the board have been agreed to by the technical team, project manager, and review chair prior to the SRR.

3.   The following technical products for hardware and software system elements are available to the cognizant participants prior to the review:

a.    system requirements document;

b.   system software functionality description;

c.    updated concept of operations;

d.   updated mission requirements, if applicable;

e.    baselined SEMP;

f.     risk management plan;

g.   preliminary system requirements allocation to the next lower level system;

h.   updated cost estimate;

i.     Technology Development Maturity Assessment Plan;

j.     updated risk assessment and mitigations (including PRA as applicable).

k.    logistics documentation (e.g., preliminary maintenance plan);

l.     preliminary human rating plan, if applicable;

m.  Software Development Plan (SDP);

n.   system safety and mission assurance plan;

o.   configuration management plan;

p.   initial document tree;

q.   verification and validation approach;

r.     preliminary system safety analysis; and

s.    other specialty disciplines, as required.

1.   The project utilizes a sound process for the allocation and control of requirements throughout all levels, and a plan has been defined to complete the definition activity within schedule constraints.

2.   Requirements definition is complete with respect to top-level mission and science requirements, and interfaces with external entities and between major internal elements have been defined.

3.   Requirements allocation and flow down of key driving requirements have been defined down to subsystems.

4.   Preliminary approaches have been determined for how requirements will be verified and validated down to the subsystem level.

5.   Major risks have been identified and technically assessed, and viable mitigation strategies have been defined.

Mission Definition Review

Entrance Criteria

Success Criteria

1.   Successful completion of the SRR and responses made to all SRR RFAs and RIDs.

2.   A preliminary MDR agenda, success criteria, and charge to the board have been agreed to by the technical team, project manager, and review chair prior to the MDR.

3.   The following technical products for hardware and software system elements are available to the cognizant participants prior to the review:

a.   system architecture;

b.       updated system requirements document, if applicable;

c.   system software functionality description;

d.       updated concept of operations, if applicable;

e.   updated mission requirements, if applicable;

f.    updated SEMP, if applicable;

g.       updated risk management plan, if applicable;

h.       Technology Development Maturity Assessment Plan;

i.    preferred system solution definition, including major trades and options;

j.    updated risk assessment and mitigations (including PRA, as applicable);

k.   updated cost and schedule data;

l.    logistics documentation (e.g., preliminary maintenance plan);

m.      Software Development Plan (SDP);

n.       system safety and mission assurance plan;

o.       configuration management plan;

p.       updated initial document tree, if applicable;

q.       preliminary system safety analysis; and

r.    other specialty disciplines as required.

1.   The resulting overall concept is reasonable, feasible, complete, responsive to the mission requirements, and is consistent with system requirements and available resources (cost, schedule, mass, and power).

2.   System and subsystem design approaches and operational concepts exist and are consistent with the requirements set.

3.   The requirements, design approaches, and conceptual design will fulfill the mission needs within the estimated costs.

4.   Major risks have been identified and technically assessed, and viable mitigation strategies have been defined.

Preliminary Design Review

Entrance Criteria

Success Criteria

1.   Successful completion of the SDR or MDR and responses made to all SDR or MDR RFAs and RIDs, or a timely closure plan exists for those remaining open.

2.   A preliminary PDR agenda, success criteria, and charge to the board have been agreed to by the technical team, project manager, and review chair prior to the PDR.

3.   PDR technical products listed below for both hardware and software system elements have been made available to the cognizant participants prior to the review:

a.   Updated baselined documentation, as required.

b.       Preliminary subsystem design specifications for each configuration item (hardware and software), with supporting trade-off analyses and data, as required. The preliminary software design specification should include a completed definition of the software architecture and a preliminary database design description, as applicable.

c.   Updated technology development maturity assessment plan.

d.       Updated risk assessment and mitigation.

e.   Updated cost and schedule data.

f.    Updated logistics documentation, as required.

g.       Applicable technical plans (e.g., technical performance measurement plan, contamination control plan, parts management plan, environments control plan, EMI/EMC control plan, payload-to-carrier integration plan, producibility/manufacturability program plan, reliability program plan, quality assurance plan).

h.       Applicable standards.

i.    Safety analyses and plans.

j.    Engineering drawing tree.

k.   Interface control documents.

l.    Verification/validation plan.

m.      Plans to respond to regulatory requirements (e.g., Environmental Impact Statement), as required.

n.       Disposal plan.

o.       Technical resource utilization estimates and margins.

p.       System-level safety analysis.

q.       Preliminary limited life items list (LLIL).

1.   The top-level requirements including mission success criteria, TPMs, and any sponsor-imposed constraints are agreed upon, finalized, stated clearly, and consistent with the preliminary design.

2.   The flow down of verifiable requirements is complete and proper or, if not, an adequate plan exists for timely resolution of open items. Requirements are traceable to mission goals and objectives.

3.   The preliminary design is expected to meet the requirements at an acceptable level of risk.

4.   Definition of the technical interfaces is consistent with the overall technical maturity and provides an acceptable level of risk.

5.   Adequate technical interfaces are consistent with the overall technical maturity and provide an acceptable level of risk.

6.   Adequate technical margins exist with respect to TPMs.

7.   Any required new technology has been developed to an adequate state of readiness, or back-up options exist and are supported to make them a viable alternative.

8.   The project risks are understood and have been credibly assessed, and plans, a process, and resources exist to effectively manage them.

9.   Safety and mission assurance (e.g., safety, reliability, maintainability, quality, and EEE parts) have been adequately addressed in preliminary designs and any applicable S&MA products (e.g., PRA, system safety analysis, and failure modes and effects analysis) have been approved.

10.     The operational concept is technically sound, includes (where appropriate) human factors, and includes the flow down of requirements for its execution.

Critical Design Review

Entrance Criteria

Success Criteria

1.   Successful completion of the PDR and responses made to all PDR RFAs and RIDs, or a timely closure plan exists for those remaining open.

2.   A preliminary CDR agenda, success criteria, and charge to the board have been agreed to by the technical team, project manager, and review chair prior to the CDR.

3.   CDR technical work products listed below for both hardware and software system elements have been made available to the cognizant participants prior to the review:

a.   updated baselined documents, as required;

b.       product build-to specifications for each hardware and software configuration item, along with supporting trade-off analyses and data;

c.   fabrication, assembly, integration, and test plans and procedures;

d.       technical data package (e.g., integrated schematics, spares provisioning list, interface control documents, engineering analyses, and specifications);

e.   operational limits and constraints;

f.    technical resource utilization estimates and margins;

g.       acceptance criteria;

h.       command and telemetry list;

i.    verification plan (including requirements and specification);

j.    validation plan;

k.   launch site operations plan;

l.    checkout and activation plan;

m.      disposal plan (including decommissioning or termination);

n.       updated Technology Development Maturity Assessment Plan;

o.       updated risk assessment and mitigation;

p.       updated reliability analyses and assessments;

q.       updated cost and schedule data;

r.    updated logistics documentation;

s.   software design document(s) (including interface design documents);

t.    updated LLIL;

u.       subsystem-level and preliminary operations safety analyses;

v.       systems and subsystem certification plans and requirements (as needed); and

w.      system safety analysis with associated verifications.

1.   The detailed design is expected to meet the requirements with adequate margins at an acceptable level of risk.

2.   Interface control documents are sufficiently matured to proceed with fabrication, assembly, integration, and test, and plans are in place to manage any open items.

3.   High confidence exists in the product baseline, and adequate documentation exists or will exist in a timely manner to allow proceeding with fabrication, assembly, integration, and test.

4.   The product verification and product validation requirements and plans are complete.

5.   The testing approach is comprehensive, and the planning for system assembly, integration, test, and launch site and mission operations is sufficient to progress into the next phase.

6.   Adequate technical and programmatic margins and resources exist to complete the development within budget, schedule, and risk constraints.

7.   Risks to mission success are understood and credibly assessed, and plans and resources exist to effectively manage them.

8.   Safety and mission assurance (e.g., safety, reliability, maintainability, quality, and EEE parts) have been adequately addressed in system and operational designs, and any applicable S&MA products (e.g., PRA, system safety analysis and failure modes and effects analysis) have been approved.

Post-Launch Assessment Review

Entrance Criteria

Success Criteria

1.   The launch and early operations performance, including (when appropriate) the early propulsive maneuver results, are available.

2.   The observed spacecraft and science instrument performance, including instrument calibration plans and status, are available.

3.   The launch vehicle performance assessment and mission implications, including launch sequence assessment, launch operations experience with lessons learned, are completed.

4.   The mission operations and ground data system experience, including tracking and data acquisition support and spacecraft telemetry data analysis, is available.

5.   The mission operations organization, including status of staffing, facilities, tools, and mission software (e.g., spacecraft analysis, and sequencing), is available.

6.   In-flight anomalies and the responsive actions taken, including any autonomous fault protection actions taken by the spacecraft or any unexplained spacecraft telemetry, including alarms, are documented.

7.   The need for significant changes to procedures, interface agreements, software, and staffing has been documented.

8.   Documentation is updated, including any updates originating from the early operations experience.

9.   Future development/test plans are developed.

1.   The observed spacecraft and science payload performance agrees with prediction, or if not, is adequately understood so that future behavior can be predicted with confidence.

2.   All anomalies have been adequately documented, and their impact on operations assessed. Further, anomalies impacting spacecraft health and safety or critical flight operations have been properly disposed.

3.   The mission operations capabilities, including staffing and plans, are adequate to accommodate the actual flight performance.

4.   Liens, if any, on operations, identified as part of the ORR, have been satisfactorily disposed.

Critical Event Readiness Review

Entrance Criteria

Success Criteria

1.   Mission overview and context for the critical event(s).

2.   Activity requirements and constraints.

3.   Critical activity sequence design description including key tradeoffs and rationale for selected approach.

4.   Fault protection strategy.

5.   Critical activity operations plan including planned uplinks and criticality.

6.   Sequence verification (testing, walk-throughs, peer review) and critical activity validation.

7.   Operations team training plan and readiness report.

8.   Risk areas and mitigations.

9.   Spacecraft readiness report.

10.     Open items and plans.

1.       The critical activity design complies with requirements.

2.       The preparation for the critical activity, including the verification and validation, is thorough.

3.       The project (including all the systems, supporting services, and documentation) is ready to support the activity.

4.   The requirements for the successful execution of the critical event(s) are complete and understood and have flowed down to the appropriate levels for implementation.