The 'Software Review Procedures' shall define the
minimum procedure for performing in-process and internal reviews.
External or formal design reviews will be identified and preparations
for these reviews identified.
The 'Software Review Procedures' shall establish the methods and procedures outlined in the 'Software Product Evaluation Plan' which is a functional part of the 'Software Development Plan'.
Reviews provide a timely and objective means of ensuring quality by assessing the software products during preparation for compliance with acquirer requirements and project standards.
Reviews provide a timely and objective means of ensuring compliance by assessing the products of each development phase for compliance with customer requirements and the project software development standards.
Internal In-Process reviews are used to inspect the individual software products during the development phases and shall documented in Internal In-Process Summary Reports.
The review process is used to ensure that the work of the previous phase has been satisfactorily completed by ensuring that the product is technically correct, it meets the requirements of the previous phase, it forms a suitable baseline for the subsequent phase and that the plans for the next phase will lead to a satisfactory implementation of that phase.
All reviews shall be documented in the form of a
product evaluation record (a model text should be described Documentation
Standard), which will be controlled as a Software document, thus
providing good Management visibility of the software development
process and as a record for approval and certification purposes.
This document comprises of the following sections:
This document incorporates the procedures
outlined in the SPEP and by the addition of further details establish
the methods and criteria for initiating, conducting, and reporting
Reviews as required by contractual standards
Insert here a list of documents referenced in
this document.
For example the SDP and SPEP, specific for the CSCI
or project specific system.
Requirements for in-process, internal, and
formal reviews will be discussed and procedures defined or identified.
Internal In-Process reviews shall be used to inspect individual products prepared during the software development life cycle.
A record of these reviews will be documented on Internal In-Process Summary Reports (see figure for an example) and held by the local Configuration control. These records will form part of the Product Evaluation Records for the phase in which they were generated.
The internal In-Process Summary Report(s) shall be presented at the specific end-of-phase review and attached to the review report.
An example in-process review.
Internal In-Process Reviews Participants.
The internal in-process review revolves around an inspection team which consists normally of between three and five members.
A Moderator shall be appointed and responsible for organizing the internal In-Process Review.
Two to four other members of the team shall be selected to perform the inspection.
See Section 4 for more details.
An Internal Review shall be conducted on completion of each software phase of the system/software development life cycle defined by the 'Software Development Plan' and shall take the form of a Review meeting.
All Internal Reviews shall be conducted to an agenda, the content of which shall be distributed in advance of the review meeting.
Review Participants
For a internal review to be effective the review participants must have authority and comprehension of the design process commensurate with their review role.
For an internal review it is mandatory that there is representation from the following organizations/personnel.
1: Process Control - to ensure compliance with the contracted project Software Development Standards and Procedures.
2: Quality Assurance - to ensure adherence to project Quality Standards.
3: Review Moderator as nominated by the Technical Authority or IPT for the software products under review.
4: Originator - to support the document(s) and implement changes recommended by the review team.
5: The organization responsible for the preceding phase, e.g., System/software designers, software engineers, programmers, specialist disciplines, e.g., reliability, etc.,
6: The organization responsible for
the subsequent phase, e.g., Software engineers, programmers, testers,
etc.,
Representation from other organizations
e.g., participating companies, specialist areas, is non-mandatory
and shall be determined by the Review Moderator. However, all
areas having an interface with the document(s) under review shall
be given the opportunity to comment and attend the review. The
responsibility for defining these areas and ensuring that they
are given the opportunity to comment lies with the originator
in consultation with the technical authority document under review.
Formal Technical reviews* shall be conducted in accordance with MIL-STD-1521B* general and detailed requirements to assess the degree of completion of technical efforts related to major milestones before proceeding with further technical effort.
An internal In-process review(s) is a formal method for finding errors in individual products during their preparation.
The application of the following procedure shall be adopted unless an acceptable alternative procedure exists.
Review Team.
The internal in-process review revolves around an inspection team which consists normally of between three and five members.
The Moderator
The Moderator shall be responsible for organizing the internal In-Process Review and ensuring each member has a copy of or has access to the necessary document(s) to be reviewed.
The Moderator shall act as the chairperson in the internal in-process review. He or she shall ensure that the review maintains its flow. The Moderator shall also ensure the discussion does not stray from its purpose and that the members maintain their assigned roles.
The Members
Two to four other members of the team shall be selected to perform the inspection.
The Review.
The Basic steps are:
Planning
The Moderator, on receiving the document for inspection organizes the location and time for the review and distributes the document to the members for scrutiny.
Overview
The Moderator shall prepare a short summary describing where the software product fits into the project.
This shall be provided to the members with the software product to be inspected.
Preparation
The members should work alone for up to two hours to familiarize themselves with the material and discover possible defects to present to the review.
The Review
The review shall be attended by the members and the Moderator. The Review should only last for up to two hours even if all the material has not been covered - it is the moderators responsibility to ensure the software product is covered within the allotted time.
The review involves a read through of the document and at any point the reader or other members can interject and highlight a defect. All presumed defects are noted and classified by the Moderator on a provided form.
No discussion about whether it is really a defect is allowed, or any discussions about possible solutions. The object of the review is to identify defects only. The Moderator cannot raise defects although he or she may make observations at the end of the review.
Rework
After the review, the defects list shall be assigned to someone for "repair".
Follow Up
The Moderator is responsible for ensuring that defects have been corrected, generally by a further iteration of the review process but involving only the moderator and the repairer (probably the author of the document).
Third Hour (Optional)
An additional hour may be added to allow discussion of "good ideas" not allowed by the moderator in the first two hours.
Review Reporting
The Internal In-Process Review Summary Report will
be completed at the conclusion of the review.
Preparing for internal review.
Prior to initiating an internal review, the document(s) originator(s) shall ensure the following activities are completed.
1: Software Configuration Management have promoted the data items if applicable to read-only or 'frozen' status and have satisfactorily completed Consistency Checking i.e., no errors or omissions.
2: The data items shall be annotated Draft for review only.
3: The relevant life cycle phase check-list has been completed.
4: The document and the check-list have been checked by the originator's supervisor, and have been approved by signature as suitable for review.
Initiating a review.
Having satisfied the preparation requirements the following activities shall be performed.
Originator.
1: Advise that the documents are ready for review.
2: Inform the Review Moderator.
3: Determine the review participants in conjunction with the Review Moderator.
4: Arrange a suitable venue.
5: Circulate the following documents to the review participants as determined by the Review Moderator so that they are received at least 10 working days prior to the review. A reduction of this period, especially in the case of a re-review, may be arranged by prior agreement with all recipients:-
- A copy of the data items to be reviewed.
- A copy of the completed check-list.
- An invitation (calling) memo (see fig. ) giving the time, date, location, and agenda of the meeting.
The recipients of the above documentation set must include:-
- The mandatory review participants,
- All areas with an interface with the data items being review,
- The technical authority as identified in the relevant SDP with overall responsibility for the System software under review.
6: On receipt of any comments, prepare a suitable response for presentation at the review meeting.
The originator of the document, in conjunction with the Review Moderator, may elect to present these responses in the form of a pre-determined format, see Fig for an example.
Each response shall cover a single comment, or series of related comments, along with a detailed response. The response should clearly state any proposed changes to the document which will be discussed and subsequently accepted, amended or rejected at the meeting. If the response is accepted it will be signed by the Review Moderator to record this fact.
Responses shall be completed in black
ink, have no correction fluid on them and all crossing out/revisions
shall be initialed by the review moderator.
However, it must be noted that the only mandatory requirement
is that the meeting minutes are clear, concise, unambiguous and
that they record all agreements made at the meeting, the use of
a pre-determined format to satisfy this requirement is at the
discretion of the Review Moderator.
Review Participants.
Examine the review documents, in conjunction with the relevant higher level requirements/design documentation, to establish compliance with customer requirements and evaluation criteria.
Provide written comments, highlighting deficiencies, omissions, ambiguities and feasibility to the Review Moderator within the timescale specified in the invitation memo using the proformas if applicable.
During the review.
Depending on the nature, size and complexity of the documents under review, the Review Moderator may require the originator to provide a brief overview of the system and documents, its baseline, structure, mode of operation, etc., for the benefit of the meeting.
The review participants shall complete the attendance record detailing name, signature, representing organization and security status.
The originator shall distribute copies of all received comments and responses to those comments to all review participants.
The review group shall perform a step-wise examination of the data against the check-list and written comments presented by the Review Moderator, to highlight deficiencies.
The objective of the review will be to ensure that the criteria for the documents produced during the specific life cycle phase has been achieved - see SPEP.
Documents may be rejected by the moderator during a review due to excessive numbers of changes required, this will occur at the discretion of the relevant assurance groups.
The Review meeting should not attempt to resolve design deficiencies, this should be done outside the meeting by the relevant design group.
Areas of conflict in the review which cannot be resolved by the review meeting shall be passed to the relevant technical authority for action and the review recorded as failed.
If the documentation under review is found to be unsatisfactory the reason(s) for rejection must be included in the Review Report (see para. ) and the document(s) shall fail review. A re-review shall be called with an agreed timescale to allow agreement to be reached.
The timescale for the review should be followed as closely as possible, if review is not completed within the specified time it shall only continue at the discretion of all the review participants.
- If the reasons for incomplete review are of an administrative or straightforward definitive nature, subsequent approval may be obtained via an additional Review.
- If the originator was unable to fully define, in detail, all the proposed changes and hence, as a result, they could not be agreed at the meeting, a timescale for their definition and a subsequent re-review must be agreed and included in the minutes.
- If the reasons for rejection are due to a technical inability to satisfy the requirements of the phase, then the problem must be raised immediately with project management and a timescale for re-work and re-review determined.
At the end of the review, the review findings shall be summarized for acceptance by the review group.
The Software Product Evaluation Records shall be compiled by the Originator and presented to the local Configuration Control. These records shall be passed to the Management Review and submitted to the Software Configuration Controller (SCC) along with the reviewed documents.
A record of the approval status of reviewed documentation (Product Control Report) shall be retained by the local Software Configuration Controller (SCC).
Review Reporting.
All decisions and recommendations made at the Review are to be recorded by the Review Moderator (or his nominee) in the form of a Review Report (see Documentation Standard for model text).
The Review Report will record all the decisions taken, recommendations made and actions placed during the Review meeting and as such must include the following:-
- Any deficiencies in the technical content of the design (Typographical or grammatical errors shall be noted by the originator).
- The reasons for rejection, where applicable, which must always be accompanied by a recommended course of action.
- Any actions placed, note that actions should be placed on personnel representing organizations, rather than the organizations themselves.
- If necessary, the process required for subsequent approval of rejected documents.
Following the Management Review the Review Report shall be submitted to Software Configuration Control together with the reviewed documentation.
Re-reviews.
Re-review of the design is necessary if the documentation has received an approval level of either:
1: Passed review pending incorporation of minuted changes.
In which case the need for a full re-review shall be decided by Software Management in consultation with the Review Moderator. As a minimum an Ex-Committee review group of interested parties shall be convened to ensure that the actions of the review minutes are completed. The membership of this review group shall at least include:
- The Review Moderator.
- Process control.
- QA
2: Failed review.
In which case a full review shall be
convened after the document(s) has been reworked.
All definitions, abbreviations and
acronyms contained in the Software Development Plan are applicable
to this document, however for convenience a number have been repeated
here.
The definitions contain their source; if a tailoring has been necessary it is indicated. If no source is given this is a new definition.
acceptance criteria - The criteria a software product must meet to successfully complete a test phase or meet delivery requirements. (IEEE-729)
acceptance test - Formal testing conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system. (IEEE-729)
configuration -
The functional and/or physical characteristics of hardware/software
as set forth in technical documentation and achieved in a product.
(DOD-STD-480A)
CHECK-LISTS
A check-list shall be applied for each phase of the system/software life-cycle. These check-lists are provided in Figures .
The purpose of the check-list is to assist the Review procedure by concisely covering factors that indicate the status and standard of the documents submitted for review. By answering a check-list authors of the documents may determine their state of readiness for Review.
The check-list forms a baseline to the review and is by no means an exhaustive examination of the design. It may contain points that are not particularly relevant to the item under review and omit points that are relevant. For this reason check-lists shall be regularly reviewed by the process control group for applicability.
The check-list shall be answered in full and signed by the originator of the items under review, thus ensuring that all software development standards aspects of the design have been considered. They shall be countersigned by the Review Moderator.
Any points that are considered to be 'not applicable' by the originator shall be checked and agreed by the process control
At the completion of the review the check-list shall
be re-examined for correctness, signed by the moderator and attached
to the review report.
Back to Home page MANAGING STANDARDS Home page.
Please send any beneficial comments or identification of errors
using the following form to: kenr@wysywig.airtime.co.uk
