Chapter 5. Systems Engineering Technical Reviews

5.1 Life Cycle

5.1.1

NASA has four interrelated product lines: Basic and Applied Research (BAR); Advanced Technology Development (ATD); Flight System and Ground Support (FS&GS) projects; and Institutional Projects (IPs). Each product line has its own unique product-line life cycle. Figure 5-1 shows the life cycle for NASA programs. Figure 5-2 shows the life cycle for NASA projects. Figure 5-3 shows the product line technical review schedule and technical reviews mapped into the management life cycle.

5.1.2

The IP management life cycle proceeds through a capital assets life cycle in five well-defined phases. An IP project starts with a "Pre-Formulation and Proposal" phase, progresses into a "Preliminary Design" and then a "Build/Construct/Fabricate" phase, and eventually ends after "Operations and Maintenance" with an "Asset Disposal" phase. For noncapital asset projects, the last three phases are replaced by an "Execute Project Plan" phase. Typically, these projects enable all of the other NASA investment areas and product lines.

5.1.3

The two major common phases for all product lines are Formulation and Implementation. Each product line has specific phases. FS&GS projects have three variations human, robotic, and Announcement of Opportunity (AO) projects.

5.1.4

The life-cycle phases and the technical reviews of this chapter are closely linked to the management life-cycle phases of NPR 7120.5 as represented in figures 5-1 and 5-2. The application of the common technical processes within each life-cycle phase produces technical results that provide inputs to technical reviews and support informed management decisions for progressing to the next life-cycle phase.

5.1.5

The progress between life-cycle phases is marked by key decision points (KDPs). At each KDP, management examines the maturity of the technical aspects of the project. For example, management examines whether the resources (staffing and funding) are sufficient for the planned technical effort, whether the technical maturity has evolved, what the technical and nontechnical internal issues and risks are, or whether the stakeholder expectations have changed. If the technical and management aspects of the project are satisfactory, including the implementation of corrective actions, then the project can be approved to proceed to the next phase.

Figure 5.1 The NASA Program Life Cycle

Figure 5.2 The NASA Project Life Cycle

Figure 5.3 Product Line Technical Review Schedule

5.1.6

Three points are important: (1) Management reviews and the technical reviews support one another. (2) Technical reviews are completed before a KDP. (3) Technical reviews are event based and occur when the entrance criteria for the applicable review as specified in Appendix G are satisfied. They occur based on the maturity of the relevant technical baseline as opposed to calendar milestones (e.g., the quarterly progress review, the yearly summary).

5.2 Technical Review Requirements

5.2.1 Review Process and Practices

5.2.1.1

For each product line (BAR, ATD, IP, and FS&GS), technical efforts are monitored throughout the life cycle to ensure that the technical goals of the project are being achieved and that the technical direction of the project is appropriate.

5.2.1.2

Technical teams shall monitor technical effort through periodic technical reviews.

5.2.1.3

A technical review is an evaluation of the project, or element thereof, by a knowledgeable group for the purposes of:

a. Assessing the status of and progress toward accomplishing the planned activities.

b. Validating the technical tradeoffs explored and design solutions proposed.

c. Identifying technical weaknesses or marginal design and potential problems (risks) and recommending improvements and corrective actions.

d. Making judgments on the activities' readiness for the follow-on events, including additional future evaluation milestones to improve the likelihood of a successful outcome.

e. Making assessments and recommendations to the project team, Center, and Agency management.

f. Providing a historical record that can be referenced of decisions that were made during these formal reviews.

g. Assessing the technical risk status and current risk profile.

5.2.1.4

See NPR 7120.5 for major program and project reviews and independent reviews.

5.2.1.5

Technical reviews are used to evaluate the status of the technical progress and are supported by other equivalent technical discipline activities, including safety reviews.

5.2.1.6

The technical team shall ensure that system aspects represented or implemented in software are included in all technical reviews to demonstrate that project technical goals and progress are being achieved and that all NPR 7150.2 software review requirements are implemented.

5.2.2 Planning and Conduct

The technical team shall develop and document plans for technical reviews for use in the project planning process. The technical review schedule, as documented in the SEMP, will be reflected in the overall project plan described in NPR 7120.5. The results of each technical review will be used to update the technical review plan as part of the SEMP update process. The review plans, data, and results should be maintained and dispositioned as Federal records.

5.3 Minimum Required Set of Technical Reviews

5.3.1 Definition of Minimum Required Reviews

5.3.1.1

The minimum set of required technical reviews applies to all current and future NASA FS&GS and IP programs and projects as defined in section P.2 of this document (including spacecraft, launch vehicles, instruments developed for space flight programs and projects, designated research and technology developments to be incorporated by space flight programs and projects, critical technical facilities specifically developed or significantly modified for space flight systems, information systems and technology that support space flight programs and projects, and ground systems that are in direct support of space flight operations). Between each life-cycle phase, a program or project goes through KDPs preceded by one or more reviews that enable a disciplined approach to assessing programs' and projects' readiness to progress to the next phase. Allowances are made within a phase for the differences between human and robotic FS&GS projects. Additional description of technical reviews is provided in the NASA Systems Engineering Handbook (SP-6105). (For more information on program and project life cycles and management reviews, see the appropriate NPR, e.g., NPR 7120.5.)

5.3.1.2

The technical team shall address the entrance and success criteria listed in Appendix G for applicability to the respective reviews.

5.3.1.3

The technical team shall execute the required Program/System Requirements Review (P/SRR) and Program Approval Review (PAR) in accordance with the review entry and success criteria defined in tables G-1 and G-2 of Appendix G.

5.3.1.4

The technical team shall execute the required program technical reviews in accordance with the following timeline: P/SRR before KDP 0 and PAR before KDP 1.

5.3.1.5

For human FS&GS projects, the technical team shall execute the following required minimum set of technical reviews in accordance with the review entry and success criteria defined in tables G-3, G-4, G-6, G-7, G-8, and G-10 through G-18 of Appendix G: Mission Concept Review (MCR), System Requirements Review (SRR), System Definition Review (SDR), Preliminary Design Review (PDR), Critical Design Review (CDR), System Integration Review (SIR), Test Readiness Review (TRR), System Acceptance Review (SAR), Operational Readiness Review (ORR), Flight Readiness Review (FRR), Post-Launch Assessment Review (PLAR), Critical Event Readiness Review (CERR), Post-Flight Assessment Review (PFAR), and Decommissioning Review (DR). (For more information on program and project life cycles and management reviews, see the appropriate NPR, e.g., NPR 7120.5.)

5.3.1.6

For robotic FS&GS projects, the technical team shall execute and document the following minimum required technical reviews: the MCR, SRR, Mission Definition Review (MDR), PDR, CDR, SIR, TRR, ORR, FRR, PLAR, CERR, and DR in accordance with the review entry and success criteria given in tables G-3, G-4, G-5, G-7, G-8, G-10, G-11, G-13 through G-16, and G-18 of Appendix G. Robotic projects can combine the SRR and MDR based on size and level of risk. If the two reviews are conducted separately, Table G-4 will be used for the SRR and Table G-5 will be used for the MDR. If the two reviews are combined, the entrance and success criteria for both SRR and MDR will be combined for this single review.

5.3.1.7

The technical team shall also execute a Production Readiness Review (PRR) as an additional technical review for both human and robotic FS&GS projects developing or acquiring multiple or similar systems greater than three (or as determined by the project) in accordance with the review entry and success criteria defined in Table G-9 of Appendix G. Any project producing end products with three or less units will still perform the required CDR. The CDR will include production considerations when a PRR is not performed.

5.3.1.8

The technical team shall execute the required FS&GS project technical reviews in accordance with the following timelines:

a. MCR prior to KDP A.

b. Human FS&GS project SRR prior to SDR and robotic missions SRR and MDR prior to KDP B.

c. Human FS&GS project SDR prior to KDP B.

d. PDR prior to KDP C.

e. CDR prior to starting fabrication of system end products and SIR.

f. PRR prior to starting fabrication of system end products for projects requiring multiple units.

g. SIR prior to KDP D.

h. TRR prior to starting product verification and product validation testing.

i. Human FS&GS project SAR after completion of KDP D.

j. ORR after SAR or KDP D and before FRR.

k. FRR prior to KDP E.

l. PLAR after system end product launch.

m. CERR after PLAR and before KDP F.

n. Human FS&GS project PFAR at end of flight and before KDP F.

o. DR after KDP F.

5.3.1.9

The assigned technical team shall accomplish the monitoring function for flight-related ATD projects using appropriately defined and conducted periodic technical reviews (PTRs) and continuation reviews (CRs). (See Figure 5-3.)

5.3.1.10

The assigned technical team shall accomplish the monitoring function for IPs using PTR and SAR. (See Figure 5-3.)

5.3.1.11

Reviews are considered complete when the following are accomplished:

a. Agreement exists for the disposition of all Review Item Discrepancies (RIDs) and Request for Actions (RFA).

b. The review board report and minutes are complete and distributed.

c. Agreement exists on a plan to address the issues and concerns in the review board's report.

d. Agreement exists on a plan for addressing the actions identified out of the review.

e. Liens against the review results are closed, or an adequate and timely plan exists for their closure.

f. Differences of opinion between the project under review and the review board(s) have been resolved, or a timely plan exists to resolve the issues.

g. A report is given by the review board chairperson to the appropriate management and governing program management committees (PMCs) charged with oversight of the project.

h. Appropriate procedures and controls are instituted to ensure that all actions from reviews are followed and verified through implementation to closure.