Systems Engineering Management Plan 
Section 3.9 - Model Text

3.9 Formal technical reviews.

Formal Technical reviews* shall be conducted in accordance with MIL-STD-1521B* general and detailed requirements to assess the degree of completion of technical efforts related to major milestones before proceeding with further technical effort.
Note: These reviews are sometimes referred to as "Dog and Pony shows" by the way they are performed; more often now they are called joint reviews and concentrate on the prepared products in accordance with MIL-STD-1521B.

The schedule and plan for conducting formal technical reviews shall be contained in the Project Management System; Development/Engineering Program Plan and shall identify the customer involvement by milestone reviews and critical engineering dates as demanded in the SOW.

In addition to the planned milestone reviews,* ad hoc reviews can be convened should circumstances make them necessary.

The scheduling of reviews is of particular importance since holding them too early, before all documentation is available, could lead to decisions being made based on insufficient information; whereas scheduling them too late can mean that project commitments have already been made which cannot be changed without incurring heavy financial or time penalties.

It is therefore strongly recommended that the boards responsible for the various reviews delegate the actual reviewing to specialist working groups who shall perform the reviews as and when necessary, reporting their findings at the next routine meeting of the board.

These reviews shall be a joint effort between the design responsible SEM and any participating company representatives. The objective of the review shall be to satisfy senior management and the customer that the design of the system and its comprising hardware and software satisfies all aspects of the requirement.

A brief overview of the Formal Technical Reviews follows, full details of the review process involved however, are contained in the MIL-STD-1521B and the appropriate Appendix:-

Prior to a formal review an agenda shall be provided containing the following information, purpose, location, and schedule of the joint contractor/Government review required to manage the acquisition of the system, equipment, HWCI, CSCI or services. This agenda shall set forth the place, time, date, purpose, and objectives of each forthcoming review. See DI-A-7088 for an example format and content of the agenda.

The review meeting(s) will make a number of recommendations and a number of viewpoints recorded; the designer is required to take note of this advice but will retain the authority and responsibility for making the final decision. This does not alter the principle that 'the customer is always right'. If having heard all the arguments the customer insists on certain features in the design despite opposition from the design team, then this view must prevail.

Minutes of the formal reviews shall provide the documentation of technical information and data required to record the joint contractor/Government decisions and agreements reached during the formal reviews and audits. See 'Documentation Standard' for format and content of the minutes to be prepared.

Participation at the formal technical reviews shall include members of senior management, the design/development team, quality assurance, configuration control; when appropriate, they may include staff from purchasing, installation and maintenance. While there is an understanding tendency to include extra staff at the review meeting, 'just in case', large meetings can be inefficient and expensive; whenever possible the numbers attending design reviews should be restricted to ten though they needed to be well briefed.

Participants are expected to brief themselves thoroughly and probe for weak spots in the design, but at the same time to maintain self-discipline and objectivity.

CONCLUSION:

The formal technical review aims to satisfy senior management and the customer that the design will satisfy all aspects of the requirement. It is a critical, co-operative examination, first of the design concept, later of its detail and finally of its suitability for production and use.

Most attention shall be paid to areas which contain unfamiliar problems, and outside consultants may be called in to comment. The conclusions of the review shall be formally recorded as advice to the design authority who shall retain the final responsibility for the design and authority over it.

• 3.9.1 System requirements review(s).
• These reviews shall ascertain progress in defining system technical requirements, program/design requirements (PMS) and for implementing other Engineering Management activity.
  The SRR shall be conducted during the system Concept and Exploration or Demonstration and Validation phase.
  The Prime documents to be reviewed is the preliminary System/Segment Specification and project standards and procedures.
  On successful completion of SRR(s) shall establish the Design Requirements and Development Component baselines. This review may be referenced as a contract review for the development of the system.
  The number of SRR reviews shall be determined and shall be the responsibility of the acquirer agency procurement.
• 3.9.2 System design review.
• This review shall be conducted to evaluate the optimization's, correlation, completeness, and the risks associated with the allocated program/design requirements, including the corresponding test requirements in fulfilling the performance requirements specified in the System/subsystem specification, System/subsystem design description (i.e., the Functional Configuration Identification).
  Also included is a summary review of the Systems Engineering process which produced the allocated program/design requirements and of the engineering planning for the next phase of effort.
  This review shall be conducted when the system definition effort has proceeded to the point where system characteristics are defined and the allocated configuration identification has been established. This review shall be in sufficient detail to insure a technical understanding among all participants on:
• the updated or completed system and subsystem specification;
• the completed system or system/subsystem design description(s);
• the preliminary configuration item (CI) prime and critical item development specification(s);
• other system definition efforts, productions, and plans, for example., SEMP, CMP, DMP, RMP, System Test Plan, SDP(s), Documentation Standard, speciality program plans, etc.
• A successful SDR shall establish the Functional baseline.
  The SDR is the responsibility of the acquirer or an authorized representative.
• 3.9.3 Software specification review. (Software only)
• The Software Specification Review (SSR) shall be a 'formal' (customer present) review of each of the specific CSCI requirements as specified in the 'Software Requirements Specification' and the 'Interface Requirements Specification(s) and shall be held after the System Design Review but prior to the start of the CSCI Preliminary Design.
  The reviews purpose is to establish the Allocated Baseline for the specific CSCI by demonstrating to the acquirer the adequacy of the Software Requirements Specification and Interface Requirements Specification(s).
  A successful SSR and authentication of the specifications shall establish the Software Allocated Baseline for each CSCI or software contained within a HWCI.
  The SSR must be identified in a SOW as a task and is usually used as a payment milestone. The acquirer or his agent will be responsible for approving the outcome of such reviews.
  NOTE: The Allocated Baseline for HWCIs may be delayed but must be established before the Formal Critical Design Review (CDR).
• 3.9.4 Preliminary design review.
• The 'Preliminary Design Review' (PDR) shall be a formal technical review or a series of reviews of the basic design approach for each CI or aggregate of a configuration item or for a functionally related group of configuration items to:
• evaluate the process, technical adequacy, and risk resolution (on technical cost, and schedule basis) of the selected design approach;
• determine its compatibility with performance and engineering specialty requirements of the CI development specifications;
• establish the existence and compatibility of the physical and functional interfaces among the CI and other items of equipment, facilities, computer software, and personnel.
• It shall be held after the HWCI Development specification(s), Software Design Description (Top-level), Software Test Plan, and the System/hardware development Test plan are available, but prior to the start of detailed design.
  The PDR may be a combined hardware/software or may be separately held depending on the complexity of the items to be reviewed.
  The PDR shall be approved by the acquirer or his appointed agent.
• 3.9.5 Critical design review.
• The 'Critical Design Review' (CDR) shall be conducted on each Configuration Item prior to fabrication/production/coding release to insure that the detail design solutions, as reflected in the draft HWCI Product Specification, Software Design Description Database Design, Interface Design, and Engineering drawings satisfy the requirements established by the hardware Development Specification(s) (CIDS/PIDS) for the HWCIs and Software Design (Top-level) for CSCIs.
  For large/complex CIs the CDR shall be conducted on an incremental basis, i.e., progressive reviews are conducted versus a single CDR.
  The purpose of the review shall be to:
• determine that the detail design of the CI under review satisfies the performance and engineering speciality requirements of the CI development specifications;
• establish the detail design compatibility among other items of equipment, facilities, computer software and personnel;
• assess producibility and CI risk areas (on a technical, cost, and schedule basis);
• review the preliminary HWCI product specifications;
• review any deviations/waivers.
• Additional In-progress reviews may be scheduled post-CDR which address:
• Response to outstanding actions;
• Modifications to design necessitated by approved 'Change Proposals' or design/program errors;
• Updating sizing and timing data;
• Updated design information, as applicable;
• Results obtained during in-house testing, including problems encountered and solutions implemented or proposed.
• The CDR shall be approved by the acquirer or his appointed agent.
• 3.9.6 Test readiness review.
• The 'Test Readiness Review' TRR shall be a formal design review of the readiness to begin the CSCI testing phase and shall be performed on each deliverable CSCI as applicable.
  The review shall ensure the traceability of the specific CSCI software test plan/test descriptions and test procedures back to the 'Software Design Descriptions' and 'Software requirements specification'.
  The review shall be conducted after the 'Software Test Procedure' is available and the software unit and integration and testing is complete.
  The major items to be reviewed are:
• Software/Interface requirements changes (Class I);
• Design Changes (affecting the Design);
• Software Test Plan;
• Software Test description;
• Software Test procedure;
• Unit integration test cases, procedures and results;
• problem/change reports;
• deviations/waivers raised;
• any changes to previously prepared associated software products.
• A successful TRR shall lead to permission to start CSCI testing.
  The TRR shall be approved by the acquirer or his appointed agent.
• 3.9.7 Formal qualification review.
• The test, inspection, or analytical process of all the configuration items comprising the system are verified to have met the specific contractual performance requirements (specifications or equivalent).
  This review does not apply to hardware and software requirements verified at FCA for the individual configuration item.
  The FQR is the final evaluation of the design status of the software (CSCI) and hardware (HWCI) comprising the system. A comparison is made of all functional evidence available against the System/subsystem specification.
  The major documents to be reviewed are:
• Plans, validation, and test reports from all test phases;
• each individual HWCI and CSCI Functional Configuration Audit report;
• each individual HWCI and CSCI Physical Configuration Audit report;
• any open problem/change report, CPs or CRs;
• TRD and/or Product Acceptance Test Specification(s);
• Product Test Procedure(s)/Schedules(s);
• deviations/waivers raised;
• Master record index (VCM, test reports. etc.,).
• A successful FQR shall lead to the approval of the Production Baseline. This review may be incremental for large or complex systems.
  The FQR shall be approved by the acquirer or his appointed technical agent.
• 3.9.8 Production readiness review.
• The 'Production Readiness Review' (PRR) is intended to determine the status of completion of the specific actions which must be satisfactorily accomplished prior to executing a production go-ahead decision. The review is accomplished in an incremental fashion during the Engineering and Manufacturing Development phase, usually two initial reviews and one final review to assess the risk in exercising the production go-ahead decision.
  Timing of the incremental PRRs is a function of program posture and is not specifically locked into other reviews.
  The PRR is performed to decide if the system is in a suitable condition for delivery to the acquirer (customer).
  A frozen and approved Production Baseline Build Standard is mandatory before production authorization can be given.
  The major documents to be formally reviewed are:
• the FQR review report;
• the System PCA reports;
• the Production Baseline documentation (G.A, PAS/PTS(s), Product specifications, MRI, etc.,);
• the production delivery requirements as stated in the SOW;
• Master record index;
• A successful PRR shall lead to an instruction to the appropriate group within the manufacture and deliver copies to the acquirer in accordance with the Contractual requirements. The production baseline shall form part of the Physical Baseline Build Standard (PBBS).
  The PRR shall be approved by the acquirer or his appointed agent.
  Other such formal reviews for specific applications can be added, for example flight test, etc.,

Further information see  Design Review discussion.

---

LE FastCounter

Back to Home page MANAGING STANDARDS Home page

Please send any beneficial comments or identification of errors using the following form to: kenr@wysywig.airtime.co.uk

Copyright © by Ken Rigby 1996, 1997, 1998 -- 2003

Formal Technical reviews are equivalent to DEF STAN identified Design Reviews.