Regulatory Guide 1.168: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
Line 15: Line 15:
| page count = 12
| page count = 12
}}
}}
{{#Wiki_filter:Regulatory guides are issued to describe and make available to the public such information as methods acceptable to the NRC staff for implementing specificparts of the NRC's  regulations, techniques used by the staff in evaluating specific problems or postulated accidents, and data needed by the NRC staff in itsreview of applications for permits and licenses.  Regulatory guides are not substitutes for regulations, and compliance with them is not required.  Methods andsolutions different from those set out in the guides will be acceptable if they provide a basis for the findings requisite to the issuance or continuance of a permitor license by the Commission.This guide was issued after consideration of comments received from the public.  Comments and suggestions for improvements in these guides are encouragedat all times, and guides will be revised, as appropriate, to accommodate comments and to reflect new information or experience.  Written comments may besubmitted to the Rules and Directives Branch, ADM, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001.  Regulatory guides are issued in ten broad divisions:  1, Power Reactors; 2, Research and Test Reactors; 3, Fuels and Materials Facilities; 4, Environmentaland Siting; 5, Materials and Plant Protection; 6, Products; 7, Transportation; 8, Occupational Health; 9, Antitrust and Financial Review; and 10, General. Single copies of regulatory guides (which may be reproduced) may be obtained free of charge by writing the Distribution Services Section, U.S. NuclearRegulatory Commission, Washington, DC 20555-0001, or by fax to (301)415-2289, or by email to DISTRIBUTION@NRC.GOV.  Electronic copies of this guideand other recently issued guides are available at NRC's home page at <WWW.NRC.GOV> through the Electronic Reading Room, Accession NumberML040410189.U.S. NUCLEAR REGULATORY COMMISSION              Revision 1February 2004REGULATORY GUIDEOFFICE OF NUCLEAR REGULATORY RESEARCHREGULATORY GUIDE 1.168(Draft was issued as DG-1123)VERIFICATION, VALIDATION, REVIEWS, AND AUDITS FORDIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMSOF NUCLEAR POWER PLANTS
{{#Wiki_filter:Regulatory guides are issued to describe and make available to the public such information as methods acceptable to the NRC staff for implementing specificparts of the NRC's  regulations, techniques used by the staff in evaluating specific problems or postulated accidents, and data needed by the NRC staff in itsreview of applications for permits and licenses.  Regulatory guides are not substitutes for regulations, and compliance with them is not required.  Methods andsolutions different from those set out in the guides will be acceptable if they provide a basis for the findings requisite to the issuance or continuance of a permitor license by the Commission.This guide was issued after consideration of comments received from the public.  Comments and suggestions for improvements in these guides are encouragedat all times, and guides will be revised, as appropriate, to accommodate comments and to reflect new information or experience.  Written comments may besubmitted to the Rules and Directives Branch, ADM, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001.  Regulatory guides are issued in ten broad divisions:  1, Power Reactors; 2, Research and Test Reactors; 3, Fuels and Materials Facilities; 4, Environmentaland Siting; 5, Materials and Plant Protection; 6, Products; 7, Transportation; 8, Occupational Health; 9, Antitrust and Financial Review; and 10, General. Single copies of regulatory guides (which may be reproduced) may be obtained free of charge by writing the Distribution Services Section, U.S. NuclearRegulatory Commission, Washington, DC 20555-0001, or by fax to (301)415-2289, or by email to DISTRIBUTION@NRC.GOV.  Electronic copies of this guideand other recently issued guides are available at NRC's home page at <WWW.NRC.GOV> through the Electronic Reading Room, Accession NumberML040410189.U.S. NUCLEAR REGULATORY COMMISSION              Revision 1February 2004 REGULATORY GUIDEOFFICE OF NUCLEAR REGULATORY RESEARCHREGULATORY GUIDE 1.168(Draft was issued as DG-1123)VERIFICATION, VALIDATION, REVIEWS, AND AUDITS FORDIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMSOF NUCLEAR POWER PLANTS


==A. INTRODUCTION==
==A. INTRODUCTION==
Line 22: Line 22:
"Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," to 10 CFR Part
"Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," to 10 CFR Part
50 describes criteria that must be met by a quality assurance program for structures, systems, and components that prevent or mitigate the consequences of postulated accidents.  In particular, besides the structures, systems, and components that directly prevent or mitigate the consequences of postulated accidents, the criteria of Appendix B also apply to all activities affecting the safety-related functions of such structures, systems, and components, such as designing, purchasing, installing, testing, operating, maintaining, or modifying. A specific requirement is contained in 10 CFR 50.55a(h)(2) that protection systems in nuclearpower plants with construction permits issued after January 1, 1971, but before May 13, 1999, must meet the requirements stated in either IEEE Std 279, "Criteria for Protection Systems for Nuclear  
50 describes criteria that must be met by a quality assurance program for structures, systems, and components that prevent or mitigate the consequences of postulated accidents.  In particular, besides the structures, systems, and components that directly prevent or mitigate the consequences of postulated accidents, the criteria of Appendix B also apply to all activities affecting the safety-related functions of such structures, systems, and components, such as designing, purchasing, installing, testing, operating, maintaining, or modifying. A specific requirement is contained in 10 CFR 50.55a(h)(2) that protection systems in nuclearpower plants with construction permits issued after January 1, 1971, but before May 13, 1999, must meet the requirements stated in either IEEE Std 279, "Criteria for Protection Systems for Nuclear  
1 IEEE publications may be obtained from the IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854.2 Revision 1 of Regulatory Guide 1.153, "Criteria for Safety Systems," endorses IEEE Std 603-1991, "Criteria for Safety Systemsfor Nuclear Power Generating Stations," as a method acceptable to the NRC staff for satisfying the NRC's regulations withrespect to the design, reliability, qualification, and testability of the power, instrumentation, and control portions of the safetysystems of nuclear power plants.1.168-2Power Generating Stations,"1 or in IEEE Std 603-1991, "Criteria for Safety Systems for Nuclear Power Generating Stations," and the correction sheet dated January 30, 1995.1,2  The NRC staffconsiders IEEE Std 279-1971 to meet the requirements in 10 CFR 50.55a(h)(2).  Protection systems in nuclear power plants with construction permits issued before January 1, 1971, are to be consistent with their licensing basis or may meet the requirements of IEEE Std 603-1991 and the correction sheet dated January 30, 1995.  Protection systems in applications filed on or after May
1 IEEE publications may be obtained from the IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854.
 
2 Revision 1 of Regulatory Guide 1.153, "Criteria for Safety Systems," endorses IEEE Std 603-1991, "Criteria for Safety Systems for Nuclear Power Generating Stations," as a method acceptable to the NRC staff for satisfying the NRC
's regulations with respect to the design, reliability, qualification, and testability of the power, instrumentation, and control portions of the s afety systems of nuclear power plants.
 
1.168-2Power Generating Stations,"1 or in IEEE Std 603-1991, "Criteria for Safety Systems for Nuclear Power Generating Stations," and the correction sheet dated January 30, 1995.
 
1,2  The NRC staffconsiders IEEE Std 279-1971 to meet the requirements in 10 CFR 50.55a(h)(2).  Protection systems in nuclear power plants with construction permits issued before January 1, 1971, are to be consistent with their licensing basis or may meet the requirements of IEEE Std 603-1991 and the correction sheet dated January 30, 1995.  Protection systems in applications filed on or after May
13, 1999, are to meet the requirements for safety systems in IEEE Std 603-1991 and the correction sheet dated January 30, 1995.Clause 4.3 of IEEE Std 279-1971 states that the quality of components is to be achievedthrough the specification of requirements known to promote high quality, such as requirements for design, inspection, and test.  Clause 5.3 of IEEE Std 603-1991 states that components and modules (of safety systems) must be of a quality that is consistent with minimum maintenance requirements and low failure rates.  Safety system equipment must be designed, manufactured, inspected, installed, tested, operated, and maintained in accordance with a prescribed quality assurance program. Note that guidance on the application of these criteria for safety system equipmentemploying digital computers and programs or firmware is found in IEEE Std 7-4.3.2-1993,
13, 1999, are to meet the requirements for safety systems in IEEE Std 603-1991 and the correction sheet dated January 30, 1995.Clause 4.3 of IEEE Std 279-1971 states that the quality of components is to be achievedthrough the specification of requirements known to promote high quality, such as requirements for design, inspection, and test.  Clause 5.3 of IEEE Std 603-1991 states that components and modules (of safety systems) must be of a quality that is consistent with minimum maintenance requirements and low failure rates.  Safety system equipment must be designed, manufactured, inspected, installed, tested, operated, and maintained in accordance with a prescribed quality assurance program. Note that guidance on the application of these criteria for safety system equipmentemploying digital computers and programs or firmware is found in IEEE Std 7-4.3.2-1993,
"Standard Criteria for Digital Computers in Safety Systems of Nuclear Power GeneratingStations,"1 which is endorsed by Revision 1 of Regulatory Guide 1.152, "Criteria for DigitalComputers in Safety Systems of Nuclear Power Plants."In Appendix B to 10 CFR Part 50, many of the criteria contain requirements closely relatedto the activities of verification and testing.  Criterion I, "Organization," in describing theestablishment and execution of a quality assurance program, specifies that applicants must (a)
"Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations,"1 which is endorsed by Revision 1 of Regulatory Guide 1.152, "Criteria for DigitalComputers in Safety Systems of Nuclear Power Plants.
 
"In Appendix B to 10 CFR Part 50, many of the criteria contain requirements closely relatedto the activities of verification and testing.  Criterion I, "Organization," in describing theestablishment and execution of a quality assurance program, specifies that applicants must (a)
ensure that an appropriate quality assurance program is established and effectively executed and (b) verify, such as by checking, auditing, and inspection, that activities affecting safety-related functions have been correctly performed.  Criterion II, "Quality Assurance Program," states, inpart, that activities affecting quality must be accomplished under suitably controlled conditions.
ensure that an appropriate quality assurance program is established and effectively executed and (b) verify, such as by checking, auditing, and inspection, that activities affecting safety-related functions have been correctly performed.  Criterion II, "Quality Assurance Program," states, inpart, that activities affecting quality must be accomplished under suitably controlled conditions.


Controlled conditions include the use of appropriate equipment, suitable environmental conditions for accomplishing the activity, and assurance that all prerequisites for the given activity have been satisfied.  It also states, in part, that the program must take into account the need for verification of quality by inspection and test.  Criterion III, "Design Control," requires, in part, that design controlmeasures provide for verifying or checking the adequacy of design.  Criterion XI, "Test Control,"requires, in part, that a test program be established to ensure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorporate the requirements and acceptance limits contained in applicable design documents.  Finally, Criterion XVIII, "Audits," requires, in part, that a comprehensive system of planned and periodic audits be  
Controlled conditions include the use of appropriate equipment, suitable environmental conditions for accomplishing the activity, and assurance that all prerequisites for the given activity have been satisfied.  It also states, in part, that the program must take into account the need for verification of quality by inspection and test.  Criterion III, "Design Control," requires, in part, that design controlmeasures provide for verifying or checking the adequacy of design.  Criterion XI, "Test Control,"requires, in part, that a test program be established to ensure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is
3 The term "safety systems" is synonymous with "safety-related systems."  The General Design Criteria cover structures, systems,and components "important to safety."  The scope of this regulatory guide is, however, limited to "safety systems."1.168-3carried out to verify compliance with all aspects of the quality assurance program and to determinethe effectiveness of the program.This regulatory guide endorses IEEE Std 1012-1998, "IEEE Standard for SoftwareVerification and Validation,"1 and IEEE Std 1028-1997, "IEEE Standard for Software Reviewsand Audits."1 IEEE Std 1012-1998, with the exceptions stated in the Regulatory Position,describes a method acceptable to the NRC staff for complying with parts of the NRC's regulationsfor promoting high functional reliability and design quality in software used in safety systems.3  Inparticular, the method is consistent with the previously cited General Design Criteria and the criteria for quality assurance programs in Appendix B, as applied to software verification and validation (V&V).  The criteria of Appendices A and B apply to systems and related quality assurance processes. If those systems include software, the requirements extend to the software elements.  IEEE Std 1028-1997 provides guidance acceptable to the NRC staff for carrying out software reviews, inspections, walkthroughs, and audits subject to certain provisions.In general, information provided by regulatory guides is reflected in the Standard ReviewPlan, NUREG-0800.  The Office of Nuclear Reactor Regulation uses the Standard Review Plan to review applications to construct and operate nuclear power plants.  This regulatory guide provides guidance that is consistent with the revised Chapter 7 of NUREG-0800; Section 7.0,
 
"Instrumentation and Controls," which was revised in June 1997.The information collections contained in this regulatory guide are covered by therequirements of 10 CFR Part 50, which were approved by the Office of Management and Budget (OMB), approval number 3150-0011.  The NRC may not conduct or sponsor, and a person is not required to respond to, a request for information or an information collection requirement unless the requesting document displays a currently valid OMB control number.
identified and performed in accordance with written test procedures that incorporate the requirements and acceptance limits contained in applicable design documents.  Finally, Criterion XVIII, "Audits," requires, in part, that a comprehensive system of planned and periodic audits be  
3 The term "safety systems" is synonymous with "safety-related systems."  The General Design Criteria cover structures, systems, and components "important to safety."  The scope of this regulatory guide is, however, limited to "safety systems." 1.168-3carried out to verify compliance with all aspects of the quality assurance program and to determinethe effectiveness of the program.This regulatory guide endorses IEEE Std 1012-1998, "IEEE Standard for Software Verification and Validation,"1 and IEEE Std 1028-1997, "IEEE Standard for Software Reviews and Audits.
 
"1 IEEE Std 1012-1998, with the exceptions stated in the Regulatory Position,describes a method acceptable to the NRC staff for complying with parts of the NRC
's regulationsfor promoting high functional reliability and design quality in software used in safety systems.
 
3  Inparticular, the method is consistent with the previously cited General Design Criteria and the criteria for quality assurance programs in Appendix B, as applied to software verification and validation (V&V).  The criteria of Appendices A and B apply to systems and related quality assurance processes. If those systems include software, the requirements extend to the software elements.  IEEE Std 1028-1997 provides guidance acceptable to the NRC staff for carrying out software reviews, inspections, walkthroughs, and audits subject to certain provisions.In general, information provided by regulatory guides is reflected in the Standard ReviewPlan, NUREG-0800.  The Office of Nuclear Reactor Regulation uses the Standard Review Plan to review applications to construct and operate nuclear power plants.  This regulatory guide provides guidance that is consistent with the revised Chapter 7 of NUREG-0800; Section 7.0,
"Instrumentation and Controls," which was revised in June 1997.The information collections contained in this regulatory guide are covered by therequirements of 10 CFR Part 50, which were approved by the Office of Management and Budget (OMB), approval number 3150-0011.  The NRC may not conduct or sponsor, and a person is not
 
required to respond to, a request for information or an information collection requirement unless the requesting document displays a currently valid OMB control number.


==B. DISCUSSION==
==B. DISCUSSION==
Line 37: Line 55:


However, conformance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assurance processes used to design safety systems.  These practices are based on experience, and they represent industry consensus on approaches used for development of such systems.Software incorporated into instrumentation and control systems covered by Appendix Bwill be referred to in this regulatory guide as safety system software.  For safety system software, software V&V, reviews, and audits are important parts of the effort to achieve compliance with NRC requirements.  Software engineering practices rely, in part, on software V&V and on  
However, conformance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assurance processes used to design safety systems.  These practices are based on experience, and they represent industry consensus on approaches used for development of such systems.Software incorporated into instrumentation and control systems covered by Appendix Bwill be referred to in this regulatory guide as safety system software.  For safety system software, software V&V, reviews, and audits are important parts of the effort to achieve compliance with NRC requirements.  Software engineering practices rely, in part, on software V&V and on  
1.168-4technical reviews and audits to meet general quality and reliability requirements in Criteria 1 and21 of Appendix A to 10 CFR Part 50, as well as Criteria II, III, XI, and XVIII of Appendix B.  Inaddition, management reviews and audits of software processes are part of a verification process consistent with Criterion I of Appendix B.General design verification processes, but not details of software V&V planning and theconduct of reviews and audits, are described by IEEE Std 7-4.3.2-1993, which is endorsed by Revision 1 of Regulatory Guide 1.152, and ASME/NQA-1-1994, "Quality AssuranceRequirements for Nuclear Facility Applications."  Two consensus standards on softwareengineering, IEEE Std 1012-1998 and IEEE Std 1028-1997 (reaffirmed in 2002), describe the software industry's approaches to software verification, validation, review, and audit activities thatare generally accepted in the software engineering community.  Meeting these standards helps to meet regulatory requirements by ensuring that disciplined software V&V, review, and audit practices accepted within the software community will be incorporated into software processes applied to safety system software.  IEEE Std 1012-1998 describes the process of software V&V,
1.168-4technical reviews and audits to meet general quality and reliability requirements in Criteria 1 and21 of Appendix A to 10 CFR Part 50, as well as Criteria II, III, XI, and XVIII of Appendix B.  Inaddition, management reviews and audits of software processes are part of a verification process consistent with Criterion I of Appendix B.General design verification processes, but not details of software V&V planning and theconduct of reviews and audits, are described by IEEE Std 7-4.3.2-1993, which is endorsed by Revision 1 of Regulatory Guide 1.152, and ASME/NQA-1-1994, "Quality AssuranceRequirements for Nuclear Facility Applications.
including elements of a software V&V plan, and describes a minimum set of V&V activities for software at different integrity levels.  IEEE Std 1028-1997 is a process standard that provides guidance on how to conduct audits, inspections, and walkthroughs, and technical and management reviews.Technical reviews, some audits, and software inspections and walkthroughs are focused onthe V&V of products of the software development process.  Management reviews and other audits are focused on ensuring that planned activities are being accomplished effectively.  Reviews and audits are closely associated with V&V activities since technical reviews and audits are frequently conducted by the V&V organization and because the V&V organization normally participates in management reviews.  Because of this close connection of the V&V activity with reviews and audits, IEEE Std 1012-1998 and IEEE Std 1028-1997 are addressed together in this regulatory guide.Additional information on conducting software reviews can be found in the annexes toIEEE Std 1028-1997.  Annex A lists different review titles and shows which of the five review types in the standard are appropriate to use with each review title.  For example, a Software Requirements Review may be carried out using the IEEE Std 1028-1997 Technical Review.
 
"  Two consensus standards on softwareengineering, IEEE Std 1012-1998 and IEEE Std 1028-1997 (reaffirmed in 2002), describe the
 
software industry
's approaches to software verification, validation, review, and audit activities thatare generally accepted in the software engineering community.  Meeting these standards helps to meet regulatory requirements by ensuring that disciplined software V&V, review, and audit practices accepted within the software community will be incorporated into software processes applied to safety system software.  IEEE Std 1012-1998 describes the process of software V&V,
including elements of a software V&V plan, and describes a minimum set of V&V activities for software at different integrity levels.  IEEE Std 1028-1997 is a process standard that provides guidance on how to conduct audits, inspections, and walkthroughs, and technical and management
 
reviews.Technical reviews, some audits, and software inspections and walkthroughs are focused onthe V&V of products of the software development process.  Management reviews and other audits are focused on ensuring that planned activities are being accomplished effectively.  Reviews and audits are closely associated with V&V activities since technical reviews and audits are frequently conducted by the V&V organization and because the V&V organization normally participates in management reviews.  Because of this close connection of the V&V activity with reviews and audits, IEEE Std 1012-1998 and IEEE Std 1028-1997 are addressed together in this regulatory guide.Additional information on conducting software reviews can be found in the annexes toIEEE Std 1028-1997.  Annex A lists different review titles and shows which of the five review types in the standard are appropriate to use with each review title.  For example, a Software Requirements Review may be carried out using the IEEE Std 1028-1997 Technical Review.


Annex B to IEEE Std 1028-1997 compares the five review types according to various characteristics of the review types and provides guidance in choosing a review type.This regulatory guide is based on current standards and describes methods acceptable forany safety system software.  This regulatory guide discusses required V&V activities.  The applicant or licensee determines how the required activities will be implemented.
Annex B to IEEE Std 1028-1997 compares the five review types according to various characteristics of the review types and provides guidance in choosing a review type.This regulatory guide is based on current standards and describes methods acceptable forany safety system software.  This regulatory guide discusses required V&V activities.  The applicant or licensee determines how the required activities will be implemented.
Line 44: Line 69:
==C. REGULATORY POSITION==
==C. REGULATORY POSITION==
IEEE Std 1012-1998, "IEEE Standard for Software Verification and Validation," providesmethods that are acceptable to the NRC staff for meeting the requirements of 10 CFR Part 50 as  
IEEE Std 1012-1998, "IEEE Standard for Software Verification and Validation," providesmethods that are acceptable to the NRC staff for meeting the requirements of 10 CFR Part 50 as  
1.168-5they apply to the verification and validation of safety system software, subject to the exceptionslisted in these Regulatory Positions.The methods in IEEE Std 1028-1997, "IEEE Standard for Software Reviews and Audits,"provide an approach acceptable to the NRC staff for carrying out software reviews, inspections, walkthroughs, and audits, subject to the exceptions listed below in Regulatory Position 8.  These methods are often used in association with software quality assurance activities.The annexes to IEEE Std 1012-1998 and IEEE Std 1028-1997 contain information thatmay be useful, but the information in these annexes should not be viewed as the only possible solution or method.  Since a consensus has not been reached in the nuclear industry regarding the use of these methods, these annexes are not endorsed by the NRC staff, except as noted below.In this Regulatory Position, the cited criteria are in Appendix B to 10 CFR Part 50 unlessotherwise noted.1.CRITICAL SOFTWAREIEEE Std 1012-1998 defines a four-level method of quantifying software criticality, inwhich level 4 is the highest and level 1 the lowest (Clause 4.1).  IEEE Std 1012-1998 requires the applicant or licensee either use the method in the standard or define another method and provide a mapping between the applicant's or licensee's method and the method defined in the standard. Software used in nuclear power plant safety systems should be assigned integrity level 4 or equivalent, as demonstrated by a mapping between the applicant or licensee approach and integrity level 4 as defined in IEEE Std 1012-1998.2.SOFTWARE RELIABILITYIn its discussion of component and integration test plans in Table 1, IEEE Std 1012-1998identifies measurement of software reliability as a criterion for determining whether software elements "correctly implement software requirements" (Activity 5.4.3, "Design V&V Activity,"Task (5), "Component V&V Test Plan Generation and Verification," and Task (6), "IntegrationV&V Test Plan Generation and Verification").Consistent with the staff's position on reliability measures for digital safety systemscontained in other regulatory guides, the NRC staff's acceptance of quantitative reliability goals for computer-based safety systems is predicated on deterministic criteria for the computer system in its entirety (i.e., hardware, system software, firmware, application, and interconnections).3.INDEPENDENCE OF SOFTWARE VERIFICATION AND VALIDATIONCriterion I, "Organization," requires that persons and organizations performing qualityassurance functions report to a management level such that sufficient authority and organizational freedom exist, including sufficient independence from cost and schedule limitation
1.168-5they apply to the verification and validation of safety system software, subject to the exceptionslisted in these Regulatory Positions.The methods in IEEE Std 1028-1997, "IEEE Standard for Software Reviews and Audits,"provide an approach acceptable to the NRC staff for carrying out software reviews, inspections, walkthroughs, and audits, subject to the exceptions listed below in Regulatory Position 8.  These methods are often used in association with software quality assurance activities.The annexes to IEEE Std 1012-1998 and IEEE Std 1028-1997 contain information thatmay be useful, but the information in these annexes should not be viewed as the only possible solution or method.  Since a consensus has not been reached in the nuclear industry regarding the use of these methods, these annexes are not endorsed by the NRC staff, except as noted below.In this Regulatory Position, the cited criteria are in Appendix B to 10 CFR Part 50 unless otherwise noted.1.CRITICAL SOFTWAREIEEE Std 1012-1998 defines a four-level method of quantifying software criticality, inwhich level 4 is the highest and level 1 the lowest (Clause 4.1).  IEEE Std 1012-1998 requires the
 
applicant or licensee either use the method in the standard or define another method and provide a mapping between the applicant
's or licensee
's method and the method defined in the standard. Software used in nuclear power plant safety systems should be assigned integrity level 4 or equivalent, as demonstrated by a mapping between the applicant or licensee approach and integrity level 4 as defined in IEEE Std 1012-1998.2.SOFTWARE RELIABILITYIn its discussion of component and integration test plans in Table 1, IEEE Std 1012-1998identifies measurement of software reliability as a criterion for determining whether software elements "correctly implement software requirements
" (Activity 5.4.3, "Design V&V Activity,"Task (5), "Component V&V Test Plan Generation and Verification," and Task (6), "IntegrationV&V Test Plan Generation and Verification
").Consistent with the staff
's position on reliability measures for digital safety systemscontained in other regulatory guides, the NRC staff's acceptance of quantitative reliability goals for computer-based safety systems is predicated on deterministic criteria for the computer system in its entirety (i.e., hardware, system software, firmware, application, and interconnections).3.INDEPENDENCE OF SOFTWARE VERIFICATION AND VALIDATIONCriterion I, "Organization," requires that persons and organizations performing qualityassurance functions report to a management level such that sufficient authority and organizational freedom exist, including sufficient independence from cost and schedule limitation


====s. Quality ====
====s. Quality ====
1.168-6assurance functions include "verifying, such as by checking, auditing, and inspection, thatactivities affecting the safety-related functions have been correctly performed."  Criterion III,"Design Control," imposes an independence requirement for the verification and checking of theadequacy of the design, requiring that those who perform the verification and checking be persons other than those who performed the design.  A method of performing independent software V&V
1.168-6 assurance functions include  
"verifying, such as by checking, auditing, and inspection, thatactivities affecting the safety-related functions have been correctly performed.
 
"  Criterion III,"Design Control," imposes an independence requirement for the verification and checking of theadequacy of the design, requiring that those who perform the verification and checking be persons other than those who performed the design.  A method of performing independent software V&V
is described in Revision 1 of Regulatory Guide 1.152.  Another acceptable method is described in IEEE Std 1012-1998 in Clause 7.4.1 and Annex C.Regardless of the approach selected for a given V&V task, ultimate responsibility for theadequacy of V&V and the quality of subsequent safety system software lies with the applicant or licensee.  This is particularly important when an external organization has performed the V&V
is described in Revision 1 of Regulatory Guide 1.152.  Another acceptable method is described in IEEE Std 1012-1998 in Clause 7.4.1 and Annex C.Regardless of the approach selected for a given V&V task, ultimate responsibility for theadequacy of V&V and the quality of subsequent safety system software lies with the applicant or licensee.  This is particularly important when an external organization has performed the V&V
tasks (e.g., a licensee acquires a commercial-grade product approved by the NRC staff for implementation in a safety-related system).  In these cases, the applicant or licensee is not relieved of the responsibility of assuring that the V&V and subsequent software quality satisfy the NRC'srequirements for reliability.  As such, the extent of independence between the organization responsible for design and the organization responsible for verification and checking of the design must be verified by the applicant or licensee to meet the NRC's requirements contained inAppendix B.  This independence is to be sufficient to ensure that the V&V process is not compromised by schedule and resource demands placed on the design process.  Criterion II,
tasks (e.g., a licensee acquires a commercial-grade product approved by the NRC staff for implementation in a safety-related system).  In these cases, the applicant or licensee is not relieved of the responsibility of assuring that the V&V and subsequent software quality satisfy the NRC
'srequirements for reliability.  As such, the extent of independence between the organization responsible for design and the organization responsible for verification and checking of the design must be verified by the applicant or licensee to meet the NRC
's requirements contained inAppendix B.  This independence is to be sufficient to ensure that the V&V process is not compromised by schedule and resource demands placed on the design process.  Criterion II,
"Quality Assurance Program," states that the program must provide for indoctrination and trainingof personnel performing activities affecting quality as necessary to ensure that suitable proficiency is achieved and maintained.  The independent verifiers must be sufficiently proficient in software engineering to ensure that software V&V is adequately implemented.  Accordingly, it is beneficial if the independent verifiers are also knowledgeable regarding nuclear applications. IEEE Std 1012-1998 provides guidance for establishing financial, managerial, andtechnical independence for software V&V.  Criterion 1 of Appendix B requires that persons and organizations performing quality assurance functions report to a management level such that they have authority and organizational freedom, including independence from cost and schedule considerations, sufficient to identify quality problems; initiate, recommend, or provide solutions;
"Quality Assurance Program," states that the program must provide for indoctrination and trainingof personnel performing activities affecting quality as necessary to ensure that suitable proficiency is achieved and maintained.  The independent verifiers must be sufficiently proficient in software engineering to ensure that software V&V is adequately implemented.  Accordingly, it is beneficial if the independent verifiers are also knowledgeable regarding nuclear applications. IEEE Std 1012-1998 provides guidance for establishing financial, managerial, andtechnical independence for software V&V.  Criterion 1 of Appendix B requires that persons and organizations performing quality assurance functions report to a management level such that they have authority and organizational freedom, including independence from cost and schedule considerations, sufficient to identify quality problems; initiate, recommend, or provide solutions;
and verify implementation of solutions.  The IEEE Std 1012-1998 guidance for financial independence provides appropriate freedom with respect to the V&V organization's budget, andthe standard's guidance for managerial independence provides appropriate freedom with respect toV&V schedules, as well as overall project cost and schedule considerations.Similarly, the IEEE Std 1012-1998 guidance for technical independence satisfies therequirements in Criterion III of Appendix B that design verification or checking be performed by individuals or groups other than those who performed the original design.  Note that Clause C.4.1 of IEEE Std 1012-1998 states that the V&V responsibility "is vested in an organization that isseparate from the development organization."  The NRC staff position is that this does not meanthat a separate company should perform independent V&V.  However, the requirements specified in Criterion I and III of Appendix B described above must be met. 4.CONFORMANCE OF MATERIALS  
and verify implementation of solutions.  The IEEE Std 1012-1998 guidance for financial independence provides appropriate freedom with respect to the V&V organization
4 Available from EPRI Distribution Center, 207 Coggins Drive, P.O. Box 23205, Pleasant Hill, CA 94523; phone 510-934-4212.1.168-7Criterion III, "Design Control," states that measures are to be established for the selectionand review for suitability of application of materials, parts, equipment, and processes that are essential to the safety-related functions of the structures, systems, and components to which Appendix B applies.  Criterion VII, "Control of Purchased Material, Equipment, and Services,"states that measures are to be established to ensure that purchased material, whether purchased directly or through contractors and subcontractors, conforms to the procurement document
's budget, andthe standard
's guidance for managerial independence provides appropriate freedom with respect toV&V schedules, as well as overall project cost and schedule considerations.Similarly, the IEEE Std 1012-1998 guidance for technical independence satisfies therequirements in Criterion III of Appendix B that design verification or checking be performed by individuals or groups other than those who performed the original design.  Note that Clause C.4.1 of IEEE Std 1012-1998 states that the V&V responsibility  
"is vested in an organization that isseparate from the development organization.
 
"  The NRC staff position is that this does not meanthat a separate company should perform independent V&V.  However, the requirements specified in Criterion I and III of Appendix B described a bove must be met. 4.CONFORMANCE OF MATERIALS  
4 Available from EPRI Distribution Center, 207 Coggins Drive, P.O. Box 23205, Pleasant Hill, CA 94523; phone 510-934-4212.
 
1.168-7Criterion III, "Design Control," states that measures are to be established for the selectionand review for suitability of application of materials, parts, equipment, and processes that are essential to the safety-related functions of the structures, systems, and components to which Appendix B applies.  Criterion VII, "Control of Purchased Material, Equipment, and Services,"states that measures are to be established to ensure that purchased material, whether purchased directly or through contractors and subcontractors, conforms to the procurement document


====s.  IEEE====
====s.  IEEE====
Line 61: Line 105:


Additional detailed information on acceptance processes is available in EPRI TR-106439,
Additional detailed information on acceptance processes is available in EPRI TR-106439,
"Guideline on Evaluation and Acceptance of Commercial Grade Digital Equipment for NuclearSafety Applications" (October 1996).45.QUALITY ASSURANCECriterion I identifies the quality assurance functions of (a) ensuring that an appropriatequality assurance program is established and effectively executed and (b) verifying, such as by checking, auditing, and inspecting, that activities affecting the safety-related functions have been correctly performed.  Criterion XVII requires that sufficient records be maintained to furnish evidence of activities affecting quality.  Criterion III requires that design changes be subject todesign control measures commensurate with those applied to the original design.  In addition to the provisions of IEEE Std 1012-1998 (in Clause 7.7.4) regarding control procedures, any V&V
"Guideline on Evaluation and Acceptance of Commercial Grade Digital Equipment for NuclearSafety Applications
" (October 1996).
45.QUALITY ASSURANCECriterion I identifies the quality assurance functions of (a) ensuring that an appropriatequality assurance program is established and effectively executed and (b) verifying, such as by checking, auditing, and inspecting, that activities affecting the safety-related functions have been correctly performed.  Criterion XVII requires that sufficient records be maintained to furnish evidence of activities affecting quality.  Criterion III requires that design changes be subject todesign control measures commensurate with those applied to the original design.  In addition to the provisions of IEEE Std 1012-1998 (in Clause 7.7.4) regarding control procedures, any V&V
materials necessary for the verification of the effectiveness of the V&V programs or necessary to furnish evidence of activities affecting quality should be maintained as quality assurance records.
materials necessary for the verification of the effectiveness of the V&V programs or necessary to furnish evidence of activities affecting quality should be maintained as quality assurance records.


The records necessary for the verification of changes must be maintained in accordance with Criterion XVII.6.TOOLS FOR SOFTWARE DEVELOPMENTTools used in the development of safety system software should be handled according toIEEE Std 7-4.3.2-1993,1 as endorsed by Revision 1 of Regulatory Guide 1.152.  IEEE Std 7-4.3.2-1993 states that "V&V tasks of witnessing, reviewing, and testing are not required for softwaretools, provided the software that is produced using the tools is subject to V&V activities that will detect flaws introduced by the tools."  If this cannot be demonstrated, the provisions of thisregulatory guide are applicable.7.VERIFICATION AND VALIDATION TASKS  
The records necessary for the verification of changes must be maintained in accordance with Criterion XVII.6.TOOLS FOR SOFTWARE DEVELOPMENTTools used in the development of safety system software should be handled according toIEEE Std 7-4.3.2-1993, 1 as endorsed by Revision 1 of Regulatory Guide 1.152.  IEEE Std 7-4.3.2-
1.168-8Table 3 of IEEE Std 1012-1998 lists "optional" V&V tasks.  These are further described inAnnex G (which is for information only) to IEEE Std 1012-1998.  These tasks are intended to provide a tailoring capability by allowing tasks to be added to the minimum set for critical software.  Exception is taken to the "optional" status of some tasks on this list; they are consideredby the NRC staff to be necessary components of acceptable methods for meeting the requirements of Appendices A and B to 10 CFR Part 50 as applied to software, regardless of whether they are performed by the V&V organization.  The following tasks are considered by the NRC staff to be part of the minimum set of V&V activities for critical software unless they are (1) incorporated into other V&V tasks in the software verification and validation plan (SVVP) or (2) performed outside the software V&V organization as part or all of the duties of some other organization.7.1Audits Criterion I of Appendix B defines quality assurance functions as including verifying, suchas by checking, auditing, and inspection, that activities affecting safety-related functions have been correctly performed.  Criterion III requires design control measures for verifying or checking the adequacy of design.  Safety system software V&V organizations may employ audits, including functional audits, in-process audits, and physical audits of software.  Although these audits are commonly considered to be the responsibility of the software quality assurance organization and the configuration management organization, they may be performed and relied upon by the V&V
1993 states that  
"V&V tasks of witnessing, reviewing, and testing are not required for softwaretools, provided the software that is produced using the tools is subject to V&V activities that will detect flaws introduced by the tools.
 
"  If this cannot be demonstrated, the provisions of thisregulatory guide are applicable.7.VERIFICATION AND VALIDATION TASKS  
1.168-8Table 3 of IEEE Std 1012-1998 lists  
"optional" V&V tasks.  These are further described inAnnex G (which is for information only) to IEEE Std 1012-1998.  These tasks are intended to provide a tailoring capability by allowing tasks to be added to the minimum set for critical software.  Exception is taken to the  
"optional" status of some tasks on this list; they are consideredby the NRC staff to be necessary components of acceptable methods for meeting the requirements of Appendices A and B to 10 CFR Part 50 as applied to software, regardless of whether they are performed by the V&V organization.  The following tasks are considered by the NRC staff to be part of the minimum set of V&V activities for critical software unless they are (1) incorporated into other V&V tasks in the software verification and validation plan (SVVP) or (2) performed outside the software V&V organization as part or all of the duties of some other organization.7.1Audits Criterion I of Appendix B defines quality assurance functions as including verifying, suchas by checking, auditing, and inspection, that activities affecting safety-related functions have been correctly performed.  Criterion III requires design control measures for verifying or checking the adequacy of design.  Safety system software V&V organizations may employ audits, including functional audits, in-process audits, and physical audits of software.  Although these audits are commonly considered to be the responsibility of the software quality assurance organization and the configuration management organization, they may be performed and relied upon by the V&V
organization.  If so, the audits should be described in the SVVP.  An acceptable method of conducting these audits is described in IEEE Std 1028-1997.7.2Regression Analysis and TestingCriterion III, "Design Control," requires that design changes be subject to design controlmeasures commensurate with those applied to the original design.  Regression analysis and testing following the implementation of software modifications is an element of the V&V of software changes.  It is considered by the NRC staff to be part of the minimum set of software V&V
organization.  If so, the audits should be described in the SVVP.  An acceptable method of conducting these audits is described in IEEE Std 1028-1997.7.2Regression Analysis and TestingCriterion III, "Design Control," requires that design changes be subject to design controlmeasures commensurate with those applied to the original design.  Regression analysis and testing following the implementation of software modifications is an element of the V&V of software changes.  It is considered by the NRC staff to be part of the minimum set of software V&V
activities for safety system software.7.3Security AssessmentA security breach of a digital system containing safety system software has the potential toprevent that software from fulfilling its safety function.  Appendix A imposes functional and reliability requirements with respect to safety systems.  According to 10 CFR 73.46, vital equipment (which includes safety system software) must be protected by physical barriers and access control.  The NRC staff considers security assessment of safety system software to be part of the minimum set of software V&V activities for such software.7.4Test EvaluationTest evaluation includes confirming the technical adequacy of test materials such as plans,designs, and results.  These materials are evaluated for consistency with Criterion II, "Quality  
activities for safety system software.7.3Security AssessmentA security breach of a digital system containing safety system software has the potential toprevent that software from fulfilling its safety function.  Appendix A imposes functional and reliability requirements with respect to safety systems.  According to 10 CFR 73.46, vital equipment (which includes safety system software) must be protected by physical barriers and access control.  The NRC staff considers security assessment of safety system software to be part of the minimum set of software V&V activities for such software.7.4Test EvaluationTest evaluation includes confirming the technical adequacy of test materials such as plans,designs, and results.  These materials are evaluated for consistency with Criterion II, "Quality  
1.168-9Assurance Program," in its requirement for controlled conditions, and with Criterion XI, "TestControl," in its requirement for the evaluation of test results.7.5Evaluation of User Documentation User documentation is important to the safe operation and proper maintenance of safetysystem software.  The requirements of Criterion III, "Design Control," for correctly translating thedesign basis of safety system software into specifications, procedures, drawings, and instructions, apply to software documentation, including user documentation.8.OTHER CODES AND STANDARDSVarious sections of IEEE Std 1012-1998 and IEEE Std 1028-1997 reference other industrycodes and standards.  These references to other standards should be treated individually.  If a referenced standard has been incorporated separately into the NRC's regulations, licensees andapplicants must comply with that standard as set forth in the regulation.  If the referenced standard has been endorsed in a regulatory guide, the standard constitutes a method acceptable to the NRC
1.168-9Assurance Program," in its requirement for controlled conditions, and with Criterion XI, "Test Control," in its requirement for the evaluation of test results.7.5Evaluation of User Documentation User documentation is important to the safe operation and proper maintenance of safetysystem software.  The requirements of Criterion III, "Design Control," for correctly translating thedesign basis of safety system software into specifications, procedures, drawings, and instructions, apply to software documentation, including user documentation.8.OTHER CODES AND STANDARDSVarious sections of IEEE Std 1012-1998 and IEEE Std 1028-1997 reference other industrycodes and standards.  These references to other standards should be treated individually.  If a referenced standard has been incorporated separately into the NRC
staff for meeting a regulatory requirement as described in the regulatory guide. If a referenced standard has been neither incorporated into the NRC's regulations nor endorsed in a regulatoryguide, licensees and applicants may consider and use the information in the referenced standard, if appropriately justified, consistent with current regulatory practice.
's regulations, licensees andapplicants must comply with that standard as set forth in the regulation.  If the referenced standard has been endorsed in a regulatory guide, the standard constitutes a method acceptable to the NRC
staff for meeting a regulatory requirement as described in the regulatory guid
 
====e. If a referenced====
 
standard has been neither incorporated into the NRC
's regulations nor endorsed in a regulatoryguide, licensees and applicants may consider and use the information in the referenced standard, if appropriately justified, consistent with current regulatory practice.


==D. IMPLEMENTATION==
==D. IMPLEMENTATION==
The purpose of this section is to provide information to applicants and licensees regardingthe NRC staff's plans for using this regulatory guide.  No backfitting is intended or approved inconnection with the issuance of this guide. Except when an applicant or licensee proposes or has previously established an acceptablealternative method for complying with the specified portions of the NRC's regulations, themethods described in this guide will be used in the evaluation of (1) submittals in connection with applications for construction permits, design certifications, operating licenses, and combined licenses involving safety system software, and (2) submittals from operating reactor licensees who voluntarily propose safety system software modifications that have a clear nexus to this guidance.
The purpose of this section is to provide information to applicants and licensees regardingthe NRC staff
's plans for using this regulatory guide.  No backfitting is intended or approved inconnection with the issuance of this guide. Except when an applicant or licensee proposes or has previously established an acceptablealternative method for complying with the specified portions of the NRC
's regulations, themethods described in this guide will be used in the evaluation of (1) submittals in connection with applications for construction permits, design certifications, operating licenses, and combined licenses involving safety system software, and (2) submittals from operating reactor licensees who voluntarily propose safety system software modifications that have a clear nexus to this guidance.
 
1 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC 20402-9328(telephone (202)512-1800); or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161; <http://www.ntis.gov/ordernow>; telephone (703)487-4650.  Copies are available for inspection orcopying for a fee from the NRC Public Document Room at 11555 Rockville Pike, Rockville, MD; the PDR
's mailing address isUSNRC PDR, Washington, DC 20555; telephone (301)415-4737 or (800)397-4209; fax (301)415-3548; email is PDR@NRC.GOV.
 
1.168-10BIBLIOGRAPHY
Hecht, H.A., T. Tai, K.S. Tso, "Class 1E Digital Systems Studies," NUREG/CR-6113, USNRC, October 1993.
 
1 Hecht, H., et al., "Review Guidelines for Software Languages for use in Nuclear Power PlantSafety Systems," NUREG/CR-6463, Revision 1, USNRC, October 1997.
 
1 Hecht, H., et al., "Verification and Validation Guidelines for High Integrity Systems,"NUREG/CR-6293, Volumes 1 and 2, USNRC, March 1995.
 
1Lawrence, J.D., "Software Reliability and Safety in Nuclear Reactor Protection Systems,"NUREG/CR-6101 (UCRL-ID-117524, Lawrence Livermore National Laboratory), USNRC,
November 1993.
 
1Lawrence, J.D., and G.G. Preckshot, "Design Factors for Safety-Critical Software," NUREG/CR-6294, USNRC, December 1994.
 
1 Seth, S., et al., "High Integrity Software for Nuclear Power Plants:  Candidate Guidelines,Technical Basis and Research Needs," NUREG/CR-6263, Volumes 1 and 2, USNRC, June 1995.


1 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC 20402-9328(telephone (202)512-1800); or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161; <http://www.ntis.gov/ordernow>; telephone (703)487-4650.  Copies are available for inspection orcopying for a fee from the NRC Public Document Room at 11555 Rockville Pike, Rockville, MD; the PDR's mailing address isUSNRC PDR, Washington, DC 20555; telephone (301)415-4737 or (800)397-4209; fax (301)415-3548; email is PDR@NRC.GOV.1.168-10BIBLIOGRAPHYHecht, H.A., T. Tai, K.S. Tso, "Class 1E Digital Systems Studies," NUREG/CR-6113, USNRC,October 1993.1Hecht, H., et al., "Review Guidelines for Software Languages for use in Nuclear Power PlantSafety Systems," NUREG/CR-6463, Revision 1, USNRC, October 1997.1Hecht, H., et al., "Verification and Validation Guidelines for High Integrity Systems,"NUREG/CR-6293, Volumes 1 and 2, USNRC, March 1995.1Lawrence, J.D., "Software Reliability and Safety in Nuclear Reactor Protection Systems,"NUREG/CR-6101 (UCRL-ID-117524, Lawrence Livermore National Laboratory), USNRC,
1  
November 1993.1Lawrence, J.D., and G.G. Preckshot, "Design Factors for Safety-Critical Software," NUREG/CR-6294, USNRC, December 1994.1Seth, S., et al., "High Integrity Software for Nuclear Power Plants:  Candidate Guidelines,Technical Basis and Research Needs," NUREG/CR-6263, Volumes 1 and 2, USNRC, June 1995.1  
1.168-11REGULATORY ANALYSISA separate regulatory analysis was not prepared for this regulatory guide.  The regulatoryanalysis prepared for Draft Regulatory Guide DG-1123, "Verification, Validation, Reviews, andAudits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants
1.168-11REGULATORY ANALYSISA separate regulatory analysis was not prepared for this regulatory guide.  The regulatoryanalysis prepared for Draft Regulatory Guide DG-1123, "Verification, Validation, Reviews, andAudits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants" (January2003), provides the regulatory basis for this regulatory guide as well.  DG-1123 was issued for public comment as the draft of this present regulatory guide.  A copy of the regulatory analysis (attached to DG-1123) is available for inspection and copying for a fee at the U.S. Nuclear Regulatory Commission Public Document Room, 11555 Rockville Pike, Rockville, MD; the PDR's mailing address is USNRC PDR, Washington, DC 20555; telephone (301)415-4737 or 1-(800)397-4209; fax (301)415-3548; e-mail <PDR@NRC.GOV>.  DG-1123 is also available in the NRC's ADAMS system at WWW.NRC.GOV under Accession Number ML030270328.
" (January2003), provides the regulatory basis for this regulatory guide as well.  DG-1123 was issued for public comment as the draft of this present regulatory guide.  A copy of the regulatory analysis (attached to DG-1123) is available for inspection and copying for a fee at the U.S. Nuclear Regulatory Commission Public Document Room, 11555 Rockville Pike, Rockville, MD; the PDR's mailing address is USNRC PDR, Washington, DC 20555; telephone (301)415-4737 or 1-(800)397-4209; fax (301)415-3548; e-mail <PDR@NRC.GOV>.  DG-1123 is also available in the


}}
NRC's ADAMS system at WWW.NRC.GOV under Accession Number ML030270328.}}


{{RG-Nav}}
{{RG-Nav}}

Revision as of 10:55, 31 August 2018

Verification, Validation, Reviews, and Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants
ML040410189
Person / Time
Issue date: 02/29/2004
From:
Office of Nuclear Regulatory Research
To:
References
DG-1123 RG-1.168, Rev 1
Download: ML040410189 (12)


Regulatory guides are issued to describe and make available to the public such information as methods acceptable to the NRC staff for implementing specificparts of the NRC's regulations, techniques used by the staff in evaluating specific problems or postulated accidents, and data needed by the NRC staff in itsreview of applications for permits and licenses. Regulatory guides are not substitutes for regulations, and compliance with them is not required. Methods andsolutions different from those set out in the guides will be acceptable if they provide a basis for the findings requisite to the issuance or continuance of a permitor license by the Commission.This guide was issued after consideration of comments received from the public. Comments and suggestions for improvements in these guides are encouragedat all times, and guides will be revised, as appropriate, to accommodate comments and to reflect new information or experience. Written comments may besubmitted to the Rules and Directives Branch, ADM, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001. Regulatory guides are issued in ten broad divisions: 1, Power Reactors; 2, Research and Test Reactors; 3, Fuels and Materials Facilities; 4, Environmentaland Siting; 5, Materials and Plant Protection; 6, Products; 7, Transportation; 8, Occupational Health; 9, Antitrust and Financial Review; and 10, General. Single copies of regulatory guides (which may be reproduced) may be obtained free of charge by writing the Distribution Services Section, U.S. NuclearRegulatory Commission, Washington, DC 20555-0001, or by fax to (301)415-2289, or by email to DISTRIBUTION@NRC.GOV. Electronic copies of this guideand other recently issued guides are available at NRC's home page at <WWW.NRC.GOV> through the Electronic Reading Room, Accession NumberML040410189.U.S. NUCLEAR REGULATORY COMMISSION Revision 1February 2004 REGULATORY GUIDEOFFICE OF NUCLEAR REGULATORY RESEARCHREGULATORY GUIDE 1.168(Draft was issued as DG-1123)VERIFICATION, VALIDATION, REVIEWS, AND AUDITS FORDIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMSOF NUCLEAR POWER PLANTS

A. INTRODUCTION

In 10 CFR Part 50, "Domestic Licensing of Production and Utilization Facilities," paragraph50.55a(a)(1) requires, in part, that structures, systems, and components be designed, tested, and inspected to quality standards commensurate with the importance of the safety function to be performed. In Appendix A, "General Design Criteria for Nuclear Power Plants," to 10 CFR Part 50,

Criterion 1, "Quality Standards and Records," requires, in part, that a quality assurance program be established and implemented in order to provide adequate assurance that structures, systems, and components important to safety will satisfactorily perform their safety functions. Appendix B,

"Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," to 10 CFR Part 50 describes criteria that must be met by a quality assurance program for structures, systems, and components that prevent or mitigate the consequences of postulated accidents. In particular, besides the structures, systems, and components that directly prevent or mitigate the consequences of postulated accidents, the criteria of Appendix B also apply to all activities affecting the safety-related functions of such structures, systems, and components, such as designing, purchasing, installing, testing, operating, maintaining, or modifying. A specific requirement is contained in 10 CFR 50.55a(h)(2) that protection systems in nuclearpower plants with construction permits issued after January 1, 1971, but before May 13, 1999, must meet the requirements stated in either IEEE Std 279, "Criteria for Protection Systems for Nuclear

1 IEEE publications may be obtained from the IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854.

2 Revision 1 of Regulatory Guide 1.153, "Criteria for Safety Systems," endorses IEEE Std 603-1991, "Criteria for Safety Systems for Nuclear Power Generating Stations," as a method acceptable to the NRC staff for satisfying the NRC

's regulations with respect to the design, reliability, qualification, and testability of the power, instrumentation, and control portions of the s afety systems of nuclear power plants.

1.168-2Power Generating Stations,"1 or in IEEE Std 603-1991, "Criteria for Safety Systems for Nuclear Power Generating Stations," and the correction sheet dated January 30, 1995.

1,2 The NRC staffconsiders IEEE Std 279-1971 to meet the requirements in 10 CFR 50.55a(h)(2). Protection systems in nuclear power plants with construction permits issued before January 1, 1971, are to be consistent with their licensing basis or may meet the requirements of IEEE Std 603-1991 and the correction sheet dated January 30, 1995. Protection systems in applications filed on or after May

13, 1999, are to meet the requirements for safety systems in IEEE Std 603-1991 and the correction sheet dated January 30, 1995.Clause 4.3 of IEEE Std 279-1971 states that the quality of components is to be achievedthrough the specification of requirements known to promote high quality, such as requirements for design, inspection, and test. Clause 5.3 of IEEE Std 603-1991 states that components and modules (of safety systems) must be of a quality that is consistent with minimum maintenance requirements and low failure rates. Safety system equipment must be designed, manufactured, inspected, installed, tested, operated, and maintained in accordance with a prescribed quality assurance program. Note that guidance on the application of these criteria for safety system equipmentemploying digital computers and programs or firmware is found in IEEE Std 7-4.3.2-1993,

"Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations,"1 which is endorsed by Revision 1 of Regulatory Guide 1.152, "Criteria for DigitalComputers in Safety Systems of Nuclear Power Plants.

"In Appendix B to 10 CFR Part 50, many of the criteria contain requirements closely relatedto the activities of verification and testing. Criterion I, "Organization," in describing theestablishment and execution of a quality assurance program, specifies that applicants must (a)

ensure that an appropriate quality assurance program is established and effectively executed and (b) verify, such as by checking, auditing, and inspection, that activities affecting safety-related functions have been correctly performed. Criterion II, "Quality Assurance Program," states, inpart, that activities affecting quality must be accomplished under suitably controlled conditions.

Controlled conditions include the use of appropriate equipment, suitable environmental conditions for accomplishing the activity, and assurance that all prerequisites for the given activity have been satisfied. It also states, in part, that the program must take into account the need for verification of quality by inspection and test. Criterion III, "Design Control," requires, in part, that design controlmeasures provide for verifying or checking the adequacy of design. Criterion XI, "Test Control,"requires, in part, that a test program be established to ensure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is

identified and performed in accordance with written test procedures that incorporate the requirements and acceptance limits contained in applicable design documents. Finally, Criterion XVIII, "Audits," requires, in part, that a comprehensive system of planned and periodic audits be

3 The term "safety systems" is synonymous with "safety-related systems." The General Design Criteria cover structures, systems, and components "important to safety." The scope of this regulatory guide is, however, limited to "safety systems." 1.168-3carried out to verify compliance with all aspects of the quality assurance program and to determinethe effectiveness of the program.This regulatory guide endorses IEEE Std 1012-1998, "IEEE Standard for Software Verification and Validation,"1 and IEEE Std 1028-1997, "IEEE Standard for Software Reviews and Audits.

"1 IEEE Std 1012-1998, with the exceptions stated in the Regulatory Position,describes a method acceptable to the NRC staff for complying with parts of the NRC

's regulationsfor promoting high functional reliability and design quality in software used in safety systems.

3 Inparticular, the method is consistent with the previously cited General Design Criteria and the criteria for quality assurance programs in Appendix B, as applied to software verification and validation (V&V). The criteria of Appendices A and B apply to systems and related quality assurance processes. If those systems include software, the requirements extend to the software elements. IEEE Std 1028-1997 provides guidance acceptable to the NRC staff for carrying out software reviews, inspections, walkthroughs, and audits subject to certain provisions.In general, information provided by regulatory guides is reflected in the Standard ReviewPlan, NUREG-0800. The Office of Nuclear Reactor Regulation uses the Standard Review Plan to review applications to construct and operate nuclear power plants. This regulatory guide provides guidance that is consistent with the revised Chapter 7 of NUREG-0800; Section 7.0,

"Instrumentation and Controls," which was revised in June 1997.The information collections contained in this regulatory guide are covered by therequirements of 10 CFR Part 50, which were approved by the Office of Management and Budget (OMB), approval number 3150-0011. The NRC may not conduct or sponsor, and a person is not

required to respond to, a request for information or an information collection requirement unless the requesting document displays a currently valid OMB control number.

B. DISCUSSION

The use of industry consensus standards is part of an overall approach to meeting therequirements of 10 CFR Part 50 when developing safety systems for nuclear power plants.

Conformance to such standards does not guarantee that regulatory requirements will be met.

However, conformance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assurance processes used to design safety systems. These practices are based on experience, and they represent industry consensus on approaches used for development of such systems.Software incorporated into instrumentation and control systems covered by Appendix Bwill be referred to in this regulatory guide as safety system software. For safety system software, software V&V, reviews, and audits are important parts of the effort to achieve compliance with NRC requirements. Software engineering practices rely, in part, on software V&V and on

1.168-4technical reviews and audits to meet general quality and reliability requirements in Criteria 1 and21 of Appendix A to 10 CFR Part 50, as well as Criteria II, III, XI, and XVIII of Appendix B. Inaddition, management reviews and audits of software processes are part of a verification process consistent with Criterion I of Appendix B.General design verification processes, but not details of software V&V planning and theconduct of reviews and audits, are described by IEEE Std 7-4.3.2-1993, which is endorsed by Revision 1 of Regulatory Guide 1.152, and ASME/NQA-1-1994, "Quality AssuranceRequirements for Nuclear Facility Applications.

" Two consensus standards on softwareengineering, IEEE Std 1012-1998 and IEEE Std 1028-1997 (reaffirmed in 2002), describe the

software industry

's approaches to software verification, validation, review, and audit activities thatare generally accepted in the software engineering community. Meeting these standards helps to meet regulatory requirements by ensuring that disciplined software V&V, review, and audit practices accepted within the software community will be incorporated into software processes applied to safety system software. IEEE Std 1012-1998 describes the process of software V&V,

including elements of a software V&V plan, and describes a minimum set of V&V activities for software at different integrity levels. IEEE Std 1028-1997 is a process standard that provides guidance on how to conduct audits, inspections, and walkthroughs, and technical and management

reviews.Technical reviews, some audits, and software inspections and walkthroughs are focused onthe V&V of products of the software development process. Management reviews and other audits are focused on ensuring that planned activities are being accomplished effectively. Reviews and audits are closely associated with V&V activities since technical reviews and audits are frequently conducted by the V&V organization and because the V&V organization normally participates in management reviews. Because of this close connection of the V&V activity with reviews and audits, IEEE Std 1012-1998 and IEEE Std 1028-1997 are addressed together in this regulatory guide.Additional information on conducting software reviews can be found in the annexes toIEEE Std 1028-1997. Annex A lists different review titles and shows which of the five review types in the standard are appropriate to use with each review title. For example, a Software Requirements Review may be carried out using the IEEE Std 1028-1997 Technical Review.

Annex B to IEEE Std 1028-1997 compares the five review types according to various characteristics of the review types and provides guidance in choosing a review type.This regulatory guide is based on current standards and describes methods acceptable forany safety system software. This regulatory guide discusses required V&V activities. The applicant or licensee determines how the required activities will be implemented.

C. REGULATORY POSITION

IEEE Std 1012-1998, "IEEE Standard for Software Verification and Validation," providesmethods that are acceptable to the NRC staff for meeting the requirements of 10 CFR Part 50 as

1.168-5they apply to the verification and validation of safety system software, subject to the exceptionslisted in these Regulatory Positions.The methods in IEEE Std 1028-1997, "IEEE Standard for Software Reviews and Audits,"provide an approach acceptable to the NRC staff for carrying out software reviews, inspections, walkthroughs, and audits, subject to the exceptions listed below in Regulatory Position 8. These methods are often used in association with software quality assurance activities.The annexes to IEEE Std 1012-1998 and IEEE Std 1028-1997 contain information thatmay be useful, but the information in these annexes should not be viewed as the only possible solution or method. Since a consensus has not been reached in the nuclear industry regarding the use of these methods, these annexes are not endorsed by the NRC staff, except as noted below.In this Regulatory Position, the cited criteria are in Appendix B to 10 CFR Part 50 unless otherwise noted.1.CRITICAL SOFTWAREIEEE Std 1012-1998 defines a four-level method of quantifying software criticality, inwhich level 4 is the highest and level 1 the lowest (Clause 4.1). IEEE Std 1012-1998 requires the

applicant or licensee either use the method in the standard or define another method and provide a mapping between the applicant

's or licensee

's method and the method defined in the standard. Software used in nuclear power plant safety systems should be assigned integrity level 4 or equivalent, as demonstrated by a mapping between the applicant or licensee approach and integrity level 4 as defined in IEEE Std 1012-1998.2.SOFTWARE RELIABILITYIn its discussion of component and integration test plans in Table 1, IEEE Std 1012-1998identifies measurement of software reliability as a criterion for determining whether software elements "correctly implement software requirements

" (Activity 5.4.3, "Design V&V Activity,"Task (5), "Component V&V Test Plan Generation and Verification," and Task (6), "IntegrationV&V Test Plan Generation and Verification

").Consistent with the staff

's position on reliability measures for digital safety systemscontained in other regulatory guides, the NRC staff's acceptance of quantitative reliability goals for computer-based safety systems is predicated on deterministic criteria for the computer system in its entirety (i.e., hardware, system software, firmware, application, and interconnections).3.INDEPENDENCE OF SOFTWARE VERIFICATION AND VALIDATIONCriterion I, "Organization," requires that persons and organizations performing qualityassurance functions report to a management level such that sufficient authority and organizational freedom exist, including sufficient independence from cost and schedule limitation

s. Quality

1.168-6 assurance functions include

"verifying, such as by checking, auditing, and inspection, thatactivities affecting the safety-related functions have been correctly performed.

" Criterion III,"Design Control," imposes an independence requirement for the verification and checking of theadequacy of the design, requiring that those who perform the verification and checking be persons other than those who performed the design. A method of performing independent software V&V

is described in Revision 1 of Regulatory Guide 1.152. Another acceptable method is described in IEEE Std 1012-1998 in Clause 7.4.1 and Annex C.Regardless of the approach selected for a given V&V task, ultimate responsibility for theadequacy of V&V and the quality of subsequent safety system software lies with the applicant or licensee. This is particularly important when an external organization has performed the V&V

tasks (e.g., a licensee acquires a commercial-grade product approved by the NRC staff for implementation in a safety-related system). In these cases, the applicant or licensee is not relieved of the responsibility of assuring that the V&V and subsequent software quality satisfy the NRC

'srequirements for reliability. As such, the extent of independence between the organization responsible for design and the organization responsible for verification and checking of the design must be verified by the applicant or licensee to meet the NRC

's requirements contained inAppendix B. This independence is to be sufficient to ensure that the V&V process is not compromised by schedule and resource demands placed on the design process. Criterion II,

"Quality Assurance Program," states that the program must provide for indoctrination and trainingof personnel performing activities affecting quality as necessary to ensure that suitable proficiency is achieved and maintained. The independent verifiers must be sufficiently proficient in software engineering to ensure that software V&V is adequately implemented. Accordingly, it is beneficial if the independent verifiers are also knowledgeable regarding nuclear applications. IEEE Std 1012-1998 provides guidance for establishing financial, managerial, andtechnical independence for software V&V. Criterion 1 of Appendix B requires that persons and organizations performing quality assurance functions report to a management level such that they have authority and organizational freedom, including independence from cost and schedule considerations, sufficient to identify quality problems; initiate, recommend, or provide solutions;

and verify implementation of solutions. The IEEE Std 1012-1998 guidance for financial independence provides appropriate freedom with respect to the V&V organization

's budget, andthe standard

's guidance for managerial independence provides appropriate freedom with respect toV&V schedules, as well as overall project cost and schedule considerations.Similarly, the IEEE Std 1012-1998 guidance for technical independence satisfies therequirements in Criterion III of Appendix B that design verification or checking be performed by individuals or groups other than those who performed the original design. Note that Clause C.4.1 of IEEE Std 1012-1998 states that the V&V responsibility

"is vested in an organization that isseparate from the development organization.

" The NRC staff position is that this does not meanthat a separate company should perform independent V&V. However, the requirements specified in Criterion I and III of Appendix B described a bove must be met. 4.CONFORMANCE OF MATERIALS

4 Available from EPRI Distribution Center, 207 Coggins Drive, P.O. Box 23205, Pleasant Hill, CA 94523; phone 510-934-4212.

1.168-7Criterion III, "Design Control," states that measures are to be established for the selectionand review for suitability of application of materials, parts, equipment, and processes that are essential to the safety-related functions of the structures, systems, and components to which Appendix B applies. Criterion VII, "Control of Purchased Material, Equipment, and Services,"states that measures are to be established to ensure that purchased material, whether purchased directly or through contractors and subcontractors, conforms to the procurement document

s. IEEE

Std 1012-1998 provides guidance for retrospective V&V of software that was not verified under the standard. Specifically, Clauses 1.2, 1.4, and 4.1, Task 1 of Activity 5.6.1 in Table 1, and Annex D, "V&V of Reusable Software," discuss V&V of pre-existing (e.g., commercial off-the-shelf) software during the operation and maintenance phase of the software lifecycle. The use of this guidance for the acceptance of pre-existing safety system software not verified during development to the provisions of this regulatory guide or its equivalent is not endorse

d. Revision

1 of Regulatory Guide 1.152 provides information on the acceptance of pre-existing software.

Additional detailed information on acceptance processes is available in EPRI TR-106439,

"Guideline on Evaluation and Acceptance of Commercial Grade Digital Equipment for NuclearSafety Applications

" (October 1996).

45.QUALITY ASSURANCECriterion I identifies the quality assurance functions of (a) ensuring that an appropriatequality assurance program is established and effectively executed and (b) verifying, such as by checking, auditing, and inspecting, that activities affecting the safety-related functions have been correctly performed. Criterion XVII requires that sufficient records be maintained to furnish evidence of activities affecting quality. Criterion III requires that design changes be subject todesign control measures commensurate with those applied to the original design. In addition to the provisions of IEEE Std 1012-1998 (in Clause 7.7.4) regarding control procedures, any V&V

materials necessary for the verification of the effectiveness of the V&V programs or necessary to furnish evidence of activities affecting quality should be maintained as quality assurance records.

The records necessary for the verification of changes must be maintained in accordance with Criterion XVII.6.TOOLS FOR SOFTWARE DEVELOPMENTTools used in the development of safety system software should be handled according toIEEE Std 7-4.3.2-1993, 1 as endorsed by Revision 1 of Regulatory Guide 1.152. IEEE Std 7-4.3.2-

1993 states that

"V&V tasks of witnessing, reviewing, and testing are not required for softwaretools, provided the software that is produced using the tools is subject to V&V activities that will detect flaws introduced by the tools.

" If this cannot be demonstrated, the provisions of thisregulatory guide are applicable.7.VERIFICATION AND VALIDATION TASKS

1.168-8Table 3 of IEEE Std 1012-1998 lists

"optional" V&V tasks. These are further described inAnnex G (which is for information only) to IEEE Std 1012-1998. These tasks are intended to provide a tailoring capability by allowing tasks to be added to the minimum set for critical software. Exception is taken to the

"optional" status of some tasks on this list; they are consideredby the NRC staff to be necessary components of acceptable methods for meeting the requirements of Appendices A and B to 10 CFR Part 50 as applied to software, regardless of whether they are performed by the V&V organization. The following tasks are considered by the NRC staff to be part of the minimum set of V&V activities for critical software unless they are (1) incorporated into other V&V tasks in the software verification and validation plan (SVVP) or (2) performed outside the software V&V organization as part or all of the duties of some other organization.7.1Audits Criterion I of Appendix B defines quality assurance functions as including verifying, suchas by checking, auditing, and inspection, that activities affecting safety-related functions have been correctly performed. Criterion III requires design control measures for verifying or checking the adequacy of design. Safety system software V&V organizations may employ audits, including functional audits, in-process audits, and physical audits of software. Although these audits are commonly considered to be the responsibility of the software quality assurance organization and the configuration management organization, they may be performed and relied upon by the V&V

organization. If so, the audits should be described in the SVVP. An acceptable method of conducting these audits is described in IEEE Std 1028-1997.7.2Regression Analysis and TestingCriterion III, "Design Control," requires that design changes be subject to design controlmeasures commensurate with those applied to the original design. Regression analysis and testing following the implementation of software modifications is an element of the V&V of software changes. It is considered by the NRC staff to be part of the minimum set of software V&V

activities for safety system software.7.3Security AssessmentA security breach of a digital system containing safety system software has the potential toprevent that software from fulfilling its safety function. Appendix A imposes functional and reliability requirements with respect to safety systems. According to 10 CFR 73.46, vital equipment (which includes safety system software) must be protected by physical barriers and access control. The NRC staff considers security assessment of safety system software to be part of the minimum set of software V&V activities for such software.7.4Test EvaluationTest evaluation includes confirming the technical adequacy of test materials such as plans,designs, and results. These materials are evaluated for consistency with Criterion II, "Quality

1.168-9Assurance Program," in its requirement for controlled conditions, and with Criterion XI, "Test Control," in its requirement for the evaluation of test results.7.5Evaluation of User Documentation User documentation is important to the safe operation and proper maintenance of safetysystem software. The requirements of Criterion III, "Design Control," for correctly translating thedesign basis of safety system software into specifications, procedures, drawings, and instructions, apply to software documentation, including user documentation.8.OTHER CODES AND STANDARDSVarious sections of IEEE Std 1012-1998 and IEEE Std 1028-1997 reference other industrycodes and standards. These references to other standards should be treated individually. If a referenced standard has been incorporated separately into the NRC

's regulations, licensees andapplicants must comply with that standard as set forth in the regulation. If the referenced standard has been endorsed in a regulatory guide, the standard constitutes a method acceptable to the NRC

staff for meeting a regulatory requirement as described in the regulatory guid

e. If a referenced

standard has been neither incorporated into the NRC

's regulations nor endorsed in a regulatoryguide, licensees and applicants may consider and use the information in the referenced standard, if appropriately justified, consistent with current regulatory practice.

D. IMPLEMENTATION

The purpose of this section is to provide information to applicants and licensees regardingthe NRC staff

's plans for using this regulatory guide. No backfitting is intended or approved inconnection with the issuance of this guide. Except when an applicant or licensee proposes or has previously established an acceptablealternative method for complying with the specified portions of the NRC

's regulations, themethods described in this guide will be used in the evaluation of (1) submittals in connection with applications for construction permits, design certifications, operating licenses, and combined licenses involving safety system software, and (2) submittals from operating reactor licensees who voluntarily propose safety system software modifications that have a clear nexus to this guidance.

1 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC 20402-9328(telephone (202)512-1800); or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161; <http://www.ntis.gov/ordernow>; telephone (703)487-4650. Copies are available for inspection orcopying for a fee from the NRC Public Document Room at 11555 Rockville Pike, Rockville, MD; the PDR

's mailing address isUSNRC PDR, Washington, DC 20555; telephone (301)415-4737 or (800)397-4209; fax (301)415-3548; email is PDR@NRC.GOV.

1.168-10BIBLIOGRAPHY

Hecht, H.A., T. Tai, K.S. Tso, "Class 1E Digital Systems Studies," NUREG/CR-6113, USNRC, October 1993.

1 Hecht, H., et al., "Review Guidelines for Software Languages for use in Nuclear Power PlantSafety Systems," NUREG/CR-6463, Revision 1, USNRC, October 1997.

1 Hecht, H., et al., "Verification and Validation Guidelines for High Integrity Systems,"NUREG/CR-6293, Volumes 1 and 2, USNRC, March 1995.

1Lawrence, J.D., "Software Reliability and Safety in Nuclear Reactor Protection Systems,"NUREG/CR-6101 (UCRL-ID-117524, Lawrence Livermore National Laboratory), USNRC,

November 1993.

1Lawrence, J.D., and G.G. Preckshot, "Design Factors for Safety-Critical Software," NUREG/CR-6294, USNRC, December 1994.

1 Seth, S., et al., "High Integrity Software for Nuclear Power Plants: Candidate Guidelines,Technical Basis and Research Needs," NUREG/CR-6263, Volumes 1 and 2, USNRC, June 1995.

1

1.168-11REGULATORY ANALYSISA separate regulatory analysis was not prepared for this regulatory guide. The regulatoryanalysis prepared for Draft Regulatory Guide DG-1123, "Verification, Validation, Reviews, andAudits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants

" (January2003), provides the regulatory basis for this regulatory guide as well. DG-1123 was issued for public comment as the draft of this present regulatory guide. A copy of the regulatory analysis (attached to DG-1123) is available for inspection and copying for a fee at the U.S. Nuclear Regulatory Commission Public Document Room, 11555 Rockville Pike, Rockville, MD; the PDR's mailing address is USNRC PDR, Washington, DC 20555; telephone (301)415-4737 or 1-(800)397-4209; fax (301)415-3548; e-mail <PDR@NRC.GOV>. DG-1123 is also available in the

NRC's ADAMS system at WWW.NRC.GOV under Accession Number ML030270328.