ML20237K942
| ML20237K942 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 01/24/1986 |
| From: | TEXAS UTILITIES ELECTRIC CO. (TU ELECTRIC) |
| To: | |
| Shared Package | |
| ML20237J194 | List:
|
| References | |
| FOIA-87-87, FOIA-87-A-14 PROC-860124-04, NUDOCS 8708270355 | |
| Download: ML20237K942 (10) | |
Text
. _ _ _ _ _ _ _ - _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _
COMANCHE PEAK RESPONSE TEAM ACTION PLAN ISAP VII.a.4
Title:
Audit Program and Auditor Qualification Revision No.
0 1
Description Original Issue Ok P1 k
~
Prepared and Recotmnended by:
Review Team Leader j
Nfz,lltf$~
/lgqlg Date z
l Approved by:
Senior Review Team Q, k 1 k J, [,
y
'7 toh1/tr i/2.+/sc Nce 1
l 8708270355 070819 h A-14 PDR 1
Rsvisient 1
Pcg3 1 of 9 ISAP VII.a.4 Audit Program and Auditor Qualification
1.0 DESCRIPTION
OF ISSUE IDENTIFIED BY NRC The Comanche Peak SSER 11, Appendix P, Section 4.7, pages P-31 through P-34, describes the NRC concens in the areas of audit and reporting. The concerns pertaining to the audit program and audit personnel qualification have been extracted and are presented here:
" Region IV found that TUEC's audit procedures did not etsply with NRC requirements, and that the program was not implemented in accordance with procedures. The lack of an established audit program was also substantiated by Region IV.
For example, Region IV Report No. 50-445/84-32 cited TUEC for j
failure to establish and implement a comprehensive system of l
planned and periodic audits. Non-compliances identified were:
annual audits were not adequately addressed by audit implementation procedures; planning and staffing to perform 1983 audits were inadequate; The Westinghouse site l
l organization performing Nuclear Steam Supply System (NSSS) l engineering services was not audited by TUEC from 1977 through 1981; and audits of vendors that manufacture or fabricate -
parts, components, and equipment for safety related systems i
were not conducted in compliance with annual or other applicable requirements dating back to August 1978.
Assessments by the Miscellaneous and Mechanical and Piping
- i l
Groups concurred with the QA/QC Group that the audit frequency of vendors did not comply with ANSI N45.2.12 requirements.
I Review of past administration of the audit program disclosed that during 1981 and 1982, the height of construction, the i
j audit staff consisted of four auditors.
From 1982 to 1984, j
l the audit staff has increased from 4 to 12.
Also, on I
occasions, individuals participating on the audit teams were not QA auditors. As such, a potential existed to compromise i
their independence. The TRT reviewed the technical background, experience, and training of auditors, as well as l
l the quality of audit reports. The TRT determined auditor staffing and qualifications to be questionable, which rendered l
the audit results for 1981 through 1983 potentially ineffective.-
l The TRT and Region IV reviewed the scope of the QA Program audited during 1983. Of approximately 650 safety related procedures, 165 (25% ovstall) were audited.
In looking at quality procedures, TUEC audited 24% of TUGCO's implementing procedures and 39% of Brown & Root procedures for a composite 32% audit rate. Although audits on a sampling basis are j
acceptable, there was no evidence that all safety related I
areas were audited. The audits did not encompass all aspects of the QA Program in order to determi:.e effectiveness.
l l
l Rovision:
1 Page 2 of 9 i
ISAP VII.a.4 (Cont'd) j
1.0 DESCRIPTION
OF ISSUE (Cont'd)
With respect to audit corrective action fellow up, it was learned that TUEC QA had not been, verifying that corrective action on previous audit findings was accomplished.
For example, audit TCP-111, initiated.co veri'fy corrective actions on previous audit findings, was started prior to the TRT's review..TUEC emphasized that itF-111 be considered a "Pu'ch List of Completion Tasks" to verify that corrective actiot. had been implemented and not an attempt to rewrite or change previous audit findings. Another specific example of ineffective follow up action was found that pertained to a deficiency identified in audit TCP-23, performed in SepteLber, 1981. Audit TCP-68, cenducted in March, 1983, attempted to verify corrective actim of TCP-23's audit finding, but logs that would document the corre'tive action had been destroyed.
e A new deficiency was written at that time and the response was accepted, but the corrective action implementation is still l
unverified.
j Following the Lobbin Report, the NRC performed a CAT inspection (IR 445/83-18;446/83-12, dated April 11,1983) and i
included a review of the TUGC0 audit program at the corporate offices. The inspection included a review of 18 audits (conducted between 1978 end early 1983), auditor qualifications, audit p;anning and scheduling, audit reporting
.and follow up, and audit program effectiveness. The report concluded that weaknesses existed in the established QA audit program which included the scheduling and frequency of audits, the lack of effective monitoring of the construction program, i
and the lack of effective resolution of certain audit findings. The inspection also indicated that the QA Program should have been more effective.
I During the TRT's evaluation of allegations and concerns, it was observed that the audit function did not always identify QA Program breakdowns, or if reported, effective corrective action was not instituted to prevent recurrence Based on its findings and observations, the TRT concludes that the QA audit
. program has had and continues to exhibit deficiencies.
Over a significant period cf time, recurring deficiencies include: inadequate staffing during peak periods;
... procedura.: and implementation inadequacies; questionable j
qualifications and capabilities; incomplete assessment of the i
QA program on an annual basis... In summation, the QA/AC i
Group finds the past audit... system less than adequate, and the audit program at the time of the TRT review was questionable."
l l
1 R2 vision:
1 P ge 3 of 9 1
ISAP VII.a.4 i
(Cont'd)
I i
i i
2.0 ACTION IDENTIFIED BY NRC j
j Evaluate the TRT findings and consider the implications of l
these findings on construction quality. "...examirrrion of tne potential safety implications should include, but r be l
limited to the areas or activities selecte'd by the Tyr."-
1
" Address the root cause of each finding and its genteic implications..."
" Address the collective significance of these deficiencies..."
" Propose an action plan... that will ensure that auch problems do not occur in the future."
j
3.0 BACKGROUND
l In addition to the TRT issue and SSER 11, the NRC issued a notice of violation (445/8432-03; 446/8411-03):
j Contrary to requirements, the following examples were identified
]
which demonstrate the failure to establish and implement a j
comprehensive system of planned and periodic audits of safety related activities as required, as noted below:
Annual audits were not adequately addressed by the audit implementation procedures.
TUGC0 Procedure DQP-CS-4, Revision 0, dated August 9, 1978, only required two audits of vendors fabricating reactor coolant pressure boundary components, parts, and equipment; one audit of vendors fabricating engineered safeguards components, parts, and equipment; and audits of balance of plant (safety related) as required by the quality assurance manager.
TUGC0 Procedure DQP-CS-4, Revision 2, dated April 16, 1981, required only that organizations will be audited on a regularly scheduled basis.
TUGC0 Procedure DQP-CS-4, Revisions 2 and 10, did not specify auditing frequencies for design, procurement, construction, and operations activities.
I i
i
R vision:
1 Pego 4 of 9 ISAP VII.a.4 (Cont'd)
3.0 BACKGROUND
(Cont'd)
TUGC0 Procedure DQP-CS-4, Revision 10, based audit requirements on Regulatory Guide 1.33, Revision 2, Feb ruary, 1978. This commitment did not fully address the requirements of the Construction Quality Assurance Program.
The above procedure and subsequent revisions failed to describe and require annual audits in accordance with commitments and requirements. Earlier audit procedures were not available to determine if they met requirements.
l Planning and staffing to perform 1983' audits was inadequate to 1
assure that a comprehensive system of audits was established and implemented to verify compliance with all aspects of the j
Quality Assurance Program, in that, of 656 safety related i
procedures (which control safety related activities) the NRC review revealed that the applicant sampled only 165, or 25 percent, during the 1963 audit program.
Consequently, significant aspects of the safety related activities were not adequately audited.
The Westinghouse site organization, established in 1977 to perform Nuclear Steam System Supply (NSSS) engineering services, was not audited by TUGC0 during the years of 1977, 1978, 1979, 1980, and 1981.
Audits.of vendors that manufacture or fabricate parts, components, and equipment for reactor coolant pressure boundary and engineered safeguards systems have not been conducted annually dating back to August 9,1978.
l
)
l
Rsvision:
1
~
Pegs 5 of 9 ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN 4.1 Scope and Methodology 4.1.1 The scope of this action plan is to, evaluate the adequacy of the.TUGC0 QA Audit Program from its l
inception to the present, determine the effect of any identified inadequacies du the Quality Assurance i
Program and/or the physical plant, and to recomunend appropriate corrections and/or improvements to the
-(
current program. This evaluation will be accomplished j
through a review which will address audit planning and
{
scheduling, preparation, performance, reporting, follow i
up and closecut, and audit personnel qualification.
4.1.2 The specific methodology is described below.
4.1.2.1 All revisions of the program and procedures i
i pertaining to the QA Audit Program that have been in effect at CPSES will.be evaluated to identify commitments and'the degree to which the written program conformed to these commitments.
Included in this evaluation l
will be the CPSES PSAR/ TSAR (Appendices lA(N) l and 1A(B), Chapter 17.1, and QA branch l
l questions and answers); TUGC0 Corporate Quality Assurance Program; CPSES Project l
Quality Assurance Plan (Design and i
Construction); Dallas Quality Procedures /
Instructions manual.
1 4.1.2.2 Reports, documentation, and data generated l
during the implementation of the program will be reviewed on a selective basis to evaluate the effectiveness of implementation. The selection of specific items to be reviewed will be based on concerns identified by the NRC; significant revisions to commitments, program description, and/or organization; and to pursue questionable areas identified during.the review. Specific copics to be addressed include the following:
Audit Planning Criteria Published and As-run Schedules
R; vision:
1 Pcgo 6 of 9 4
ISAP VII.a.4 1
(Cont'd) 4.0 CPRT ACTION PLAN (Cont'd)
Audit Plans and Checklists Audit Reports Audit Deficiency Follow up-
. Audit Team Members, Including Qualifications and Staffing Levels Organizations Performing Audit Activities
' Application of Audit Activities to Hardware l
vs Program / Procedures l
l The object of.this evaluation is to develop a conclusion concerning the adequacy of program assessment provided by the Audit Program.
This information will be used as an input _for recommendations for revision of the current j
program, as appropriate.
4.1.2.3~
Should audit program deficiencies or j
veaknesses related to construction activities be identified, they will be evaluated to determine whether action beyond that-specified in ISAP VII.c is required to identify potential areas of concern regarding I
I construction quality. Should such actions be l
I required, a detailed plan will be developed and this ISAP revised to describe the methodology.
In addition, should any identified audit program' deficiencies apply j
also to off-site TUGC0 suppliers, a program will be developed to determine the acceptability of the suppliers' quality assurance programs for.the applicable equipment and services during the period in question. This program, if required,'will utilize external sources of information such as other utility or, architect-engineer audits, the Coordinated Agency for Supplier Evaluation, and the NRC " White Book".
If suppliers are identified for which the
Re.visicn:
1' Pega 7 of 9 ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION Pl.AN (Cont'd) adequacy of the QA program cant.ot be determined, a detailed plan will be: developed to resolve the concern and this ISAP revised l
to describe the methodology.
l l
4.1.2.4 The current audit program, including auditor i
qualification requirements,.will be evaluated against licensing commitments contained in l
the FSAR.
i A determination will be made concerning the I
adequacy of the TUGC0 written program and the i
qualifications of the audit staff and i
staffing levels for the remaining construction phase of.Unic.2 and for the l
operations phase.
l l
4.1.2.5 The Results Report will provide l
recommendations for corrective action and/o,r program improvements as appropriate.
4.1.2.6 Copies of the Results Report for this ISAP l
will be provided to TUGC0 for their consideration in responding to'the NRC Notice of Violation (445/8432-03; 446/8411-03).-
4.2 Participant's Roles and Responsib111 ties 4.2.1 TUGC0 4.2.1.1 Scope TUGC0 will assist in identifying and locating applicable information and documentation to support the Review Team activities.
4.2.1.2 Personnel Mr. D. McAfee, Dallas QA Manager, will ensure effective coordination between the Review Team and TUGCO.
I 1
(
\\
l 1
Reviiion:-
1 Page-8 of 9 3
.c ISAP' VII'.a.4
..~
7 (Cont'd) 4
.,f 4.0 CPRT ACTION PLAN (Cont'd) 4.2.2 ERC
.i l
4.2.2.1 Scope
/
ERC will be responsible for reviewing applicable TUGC0 documentation-to'datermine the adequacy of.the program.
4.2.2.2 Personnel Mr.'J. Hansel Review Team Leader-4
-l Mr.'J. Gelzer Issue Coordinator l-l
-1 4.3 Qual'1fications of Personnel l
4.3.1 Participants.will be qualified to the requirements of
]
the CPSES Quality Assurance Program or to the specif-ic i
requirements of the CPRT Program Plan.
4.4 Procedures.
Program Plan and Issue-Specific Action. Plans.
4.5 Standards / Acceptance Criteria Audit activities shall be in compliance with 10CFR50, Appendix B, Criterion XVIII'and.theLepplicable ebdes and standards relating to CPSES FSAR paragraph 17.1.18.
Specifically such activities are acceptable if:
4.5.1 Audits to assurts that. procedures and ' activities comply i
with the everall QA program are performed by:
I
.s 4.5.1.1 The QA organization to provide a i
comprehensive independent verification an'd j
evaluation of. quality-related procedures and activities.
i i
4.5.1.2 The applicant to verify and evaluate the QA progrses, procedures, and activities of suppliers.
4.5.2 An audit plan is prepared identifying audits to be.
performed, their frequencies, and. schedules. Audits should be regularly scheduled based upon the status and i
i
Rsvision:
1 Pags 9 of 9 ISAP VII.a.4 (Cont'd) t 4.0 CPRT ACTION PLAN (Cont'd)
I safety importance of the activities being performed and are initiated early enough to assure effective QA during design, procurement, manufacturing, construction, installation, inspection, and testing.
4.5.3 Audits include an object'i~ve evaluation of. quality-related practices, procedures, instructions; activities and items; and review of documents and records to
(
ensure that the QA program is effective and properly implemented.
l 4.5.4 Provisions are established requiring that audits be performed in all areas where the requirements of Appendix B to 10CFR Part 50.are applicable. Areas which are often neglected but should be included are activities associated with:
4.5.4.1 Indoctrination and training programs.
j 1
4.5.4.2 Interface control among the applicant and the principal contractors.
4.5.4.3 Correctiva action, calibration, and non-
{
conformance control systems.
4.5.4.4 SAR commitments, if 4.5.5 Audit data are analyzed and the resulting reports indicating any quality problems and the effectiveness of the QA program, including the need for reaudit of deficient areas, are reported to management for review and assessment.
4.5.6 Audits are performed in accordance with pre-established written procedures or checklists and conducted by trained personnel having no direct responsibilities in the areas being audited.
4.6 Decision Criteria Decisions concerning the number of individual reports, records, files, etc., to be reviewed, and the level of detail to which they will be reviewed, will be determined by the quantity and quality of data obtained as implementation of the ISAP proceeds.
This item will be considered closed when sufficient data has been evaluated to support a firm conclusion es to the acceptability of the program against the acceptance criteria contained in Paragraph 4.5.
i
-(-
i
/g i
4 4
- COMANCHE PEAK RESPONSE TEAM ACTION PLAN l
ISAP Vll.a.4 1
Title:
Audit Program and Auditor Qualification j
l I
l l
i Revision No.
0 1
Description Original Issue Reflects Cocunents
~
On Plan Prepared and Recommended by:
Review Team Leader j
Date S 7-/
'If
/f2 aff I
Approved by:
Senior Review Team Q, k / _ _hd, /
Date
& f2.j lg f" lly/pg i
CXilIBIT 2B i
I 9
y-y (v V V 5 QJ J 4
i i
(
I 6
Revision:
1 Page 5 of 9 ISt.P VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN 4.1 Scope and Methodology 4.1.1 The scope of this action plan is to evaluate the l
adequscy of the TUCCO QA Audit Program from its inception to the present, determine the effect of any identified inadequacies on the Quality Assurance Program and/or the physical plant, and to recommend appropriate corrections and/or improvements to the current program. This evaluation will be accomplished through a review which will addrese audit planning and scheduling, preparation, performance, reporting, follow I
up and closecut, and audit personnel qualification.
4.1.2 The specific methodology is described below.
4.1. '.1 2
All revisions of the,pnogram an@scedureD Rt0J11ning to_ the _QA.Attdit Program.the,Lhav,e been in effect at CPSES(vill be evaluated t identify commitmenttiaj L the degree to y ch' the written program conformed to these commitments.
Includhd in thTs W luation will be the CPSES PSAR/FSAR-(Appendices IA(N) and 1A(B), Chapter'17.1, and QA branch questions and answers); TUGC0 Qsrporate Quality Assurance Program; CPSES Project Quality Assurance Plan (Design and Construction); Dallas Quality Procedures /
Instructions manual.
4.1.2.2 Reports, documentation,and.. data. generated l
2 during the implementation..of_the program,will be reviewed on a selective basis (n. evaluatt the effectiveness of.1Epiementatiqn,.
The selection of specific items to be reviewed villbebasedonconceris,'NenQf[11[y3e NRC; significant revisions,to commitments, program description, and/or organization; and to pursue questionable areas identified during the. review. Specific. topics to be j
r addressed include the following:
N Audit Plannin6 Criteria Published and As-run Schedules t
1
}
0
,f 1:
t Revision:
1 Page 6 of 9 i
ISAP Vll.a.4 (Cont'd) s 4.0 CPRT ACTION PLAN (Cont'd) 4
. Audit Plans and Checklists
\\
Audit Reports Audit Deficiency Follow up.
.i Audit Team Members. Including Qualifications sy l
and Staffing Levels Organizations Performing Audit Activities I
Application of Audit Activit%es to Hardware 3
vs Program / Procedures 1
The object of this i
conclusion concerningcluation is to develop a assessmentjprovidedbythe ads _qua~cy'~ sf"ficham e Audit Program.
This information.will be used as an,ingutJot 1
recommendations,for revision of the current program, as appropriate.
i 4.1.2.3 Should audit program deficiencies or weaknesses related to construction activities 4
'gj '
3 be identified, they vill he, evaluated to determine whether action. bey,ond-that,
l p p ','
spe,cified in ISAP VII.c is. required to j
identify potential areas of concern regarding s,..
construction quality.
Should such actions be and this ISAP revised to describe therequired, I
i methodology.i in addition, ghquid any identified audit program (gf.tcie 41es, spply
{
also to off-site TUGCO..supplietr, a,..
will be developed to determine _Ch; _ program acceptability of the suppliers' sual.it,y assurance, pro' grams for the applicable equipment and services during the period in question.
This program, if required, will utilize external sources of information'such as other utility or architect-engineer audits, the Coordinated Agency for Supplier Evaluation, and the NRC " White Book".
suppliers are. identified. for which the, If -
a f
i l
l
{
-- --~ R
- 1
'u kevision:
1 Page 7 of 9 ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN (Cont'd) j adequacy of the QA program..cappot_be determined, a detailed plan.will be* develop'ed to resolve the conc ~ern in3Ms[151FFised to describe the methodology.
4.1.2,4 The current audit program, including auditor I
4 qualification requirements,_will. bt_gyalgte.d t-against licensing commitments contained in the FSAR.
A determination will.be made concerning the I
i, adequacy of the TUGC0 vritten program and the qualifications of the audit staff and d
staffing levels'for the remaining construction phase of Unit 2.an'd for the operations phase.
4 1.2.5 The.Resulta_ Report will provide l
recommendations for corrective action and/o.
i r
program' improvements as appropriate.
l
~
4.1.2.6 Copies of the Results Report for this ISAP will be provided to TUGC0 for their consideration in responding to the NRC Notice l
of Violation (445/8432-03; 446/8411-03).
j 4.2 Participant's Roles and Responsibilities 4.2.1 TUCCO 4.2.1.1 Scope TUGC0 will assist in identifying and locating f
applicable information and' documentation to support the. Review Team activities.
4.2.1.2 Personnel Mr. D. McAfee, Dallas QA Manager, will ensure effective coordination between the Review Team and TUCCO.
l j
D 1
1 Revision':
1 Page 8 of 9 ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN (Cont 'd) 4.2.2 ERC 4.2.2.1 Scope ERC will be responsible for reviewing applicable TUGC0 documentation to determine the adequacy of the program.
4.2.2.2 Personnel 1
Mr. J. Hansel Review Team Leader Mr. J. Celzer Issue Coordinator 4.3 Qualifications of_ Personnel 4.3.1 Participants will be qualified to the requirements of the-CPSES Quality Assurance Program or to the specific requirements of the CPRT Program Plan.
4.4 Procedures Program Plan and Issue-Specific Action Plans, i
4.5 Standards / Acceptance Criteria i
Audit activities shall be in compliance with 10CFR50, Appendix B, Criterion XVIII and the applicable codes and standards relating to CPSES FSAR paragraph 17.1.18.
act2vities are acceptable if:
Specifically such 4.5.1 Audits to assure that i
procedures and activities comply with the overall QA program are performed by:
4.5.1.1 The QA organization to provide a comprehensive independent verification and evaluation of quality-related procedures and activities, i
4.5.1.2 The applicant'to verify and evaluate the QA programs, procedures, and activities of
. suppliers.
4.5.2 A'n audit plan is prepared identifying audits to be performed, their frequencies, and schedules.
i should bs regularly scheduled based upon the status and Audits,
i
.l l
i
I.
Revision:
1 i
Page 9 of 9 ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN (Cont'd) safety importance of the activities being performed and are initiated early enough to' assure effective QA during design, procurement, manufacturing, construction, installation, inspection,~and testing.
' Audits include an objective evaluation of quality-4.5.3 related practices, procedures, instructions; activ,ities and items; and review of documents and records to
)
ensure that the QA program is effective and properly implemented.
j 4.5.4 Provisions are established requiring that audits be performed in all areas where the requirements of i
Appendix B to 10CFR Part 50 are applicable. Areas which are often neglected but should be included are I
activities associated with:
4.5.4.1 Indoctrination and training programs.
4.5.4.2 Interface control among the applicant and the principal contractors.
4.5.4.3 Corrective action, calibration, and non-conformance control systems.
4.5.4.4 SAR commitments.
4.5.5 Audit data are analyzed and the resulting reports indicating any quality problems and the effectiveness of the QA program, including the need for teaudit of deficient areas, are reported to management for review and assessment.
4.5.6 Audits are performed in accordance with pre-established written procedures or checklists and. conducted by trained personnel having no direct responsibilities in the areas being audited.
4.6 Decision Criteria Decisions concerning the number of individu/21 reports,-
records, files, etc., to be~ reviewed, and the level of detail to which they will be reviewed, will be det; ermined by the quantity and quality of data obtained as inaplementation of the ISAP proceeds.
I This item vill be considered closed whea sufficient data has been evaluated to support a firm conclusion as to the acceptability of the program against the acceptance criteria contained in Paragraph 4.5.
i
~Ao r
g Il L-Q C...
tb
\\h '
COMANCHE PEAK RESPONSE TEAM RESULTS REPORT J
ISAP:
VII.a.4 1
Title:
Audit Program and Auditor Qualification REVISION 1 Sk V I?lA I
e Coordinator /
f Date
/
14.
,J l$
Revi Tea'm Leader' Date/
/
s). lS_e
+//e/rc John. Beck, Chairman CPRT-SRT Date
, -' ~
l 6
EXHIBIT 29 e
i Rev10 ion:
1 Psgs 1 of 53 i
RESULTS REPORT ISAP VII.a.4 Audit Program and Auditor Qualification
1.0 DESCRIPTION
OF ISSUE i
=
The Comanche Peak SSER 11, Appendix P, Section 4.7, pages P-31 through P-34, describes the NRC concerns in the areas of Quality Assurance auditing and audit reporting. The concerns pertaining to the audit program and audit personnel qualification have been extracted and are presented here:
j
" Region IV found that TUEC's audit procedures did not comply with NRC requirements, and that the program was not implemented in accordance with procedures. The lack of an established audit program was also substantiated by Region IV.
For example, Region IV Report No. 50-445/84-32 cited TUEC for
{
failure to establish and implement a comprehensive system of" planned and periodic audits. Non-compliances identified were:
annual audits were not adequately addressed by audit implementation procedures; planning and staffing to perform
{
1983 audits were inadequate; the Westinghouse site organization performing Nuclear Steam Supply System (NSSS) engineering services was not audited by TUEC from 1977 through
)
1981; and audits of vendors that manufacture or fabricate parts, components, and equipment for safety-related systems were not conducted in compliance with ennual or other applicable requirements dating back to August 1978.
Assessments by the Miscellaneous and Mechanical and Piping Croups concurred with the QA/QC Group that the audit frequeny" C
of vendors did not comply with ANSI N45.2.12 requirements.
Review of past administration of the audit program disclosed that during 1981 and 1982, the height of construction, the audit staff consir.ted of four auditors. From 1982 to 1984, the audit staff has increased from 4 to l't.
Also, on occasions, individuals participating on the audit teams were not QA auditors. As such, a potential existed to compromise their independence. The TRT reviewed the technical background, experience, and training of auditors, as well as the quality of audit reports. The TRT determined auditor p staffing and qualification.s_to be_ questionable, which rendered the audit results for 1981 through 1983 potentially ineffective.
The TRT and Region IV reviewed the scope of the QA Program audited during 1983. Of approximately 650 safety-related procedures, 165 (25% overall) were audited. In looking at quality procedures, TUEC audited 24% of TUGCO's implementing procedures and 39% of Brown & Root procedures for a composite 32% audit rate. Although audits on a sampling basis are
Revisient 1
Pcg2 2 of 53 i
RESULTS REPORT ISAP VII.a.4 (Cont'd)
1.0 DESCRIPTION
OF ISSUE (Cont'd) acceptable, there was no evidence that all safety-related areas were audited. The audits did not encompass _!Ll of the QA Program in ordei to determine effectiveness @s.ect,s n
43 With respect to audit so_rrective action _ follow-up, it was learned that TUEC QA had nbt been verifying that corrective action on previous audit findings was accomplished. For example, audit TCP-111, initiated to verify corrective actions on previous audit findings, was started prior to the TRT's review. TUEC emphasized that TCP-111 be considered a " Punch List of Completion Tasks" to verify that corrective action had been implemented and not an attempt to rewrite or change previous audit findings. Another specific example of ineffective follow-up action was found that pertained to a deficiency identified in audit TCP-23, performed in September, 1981. Audit TCP-68, conducted in March, 1983, attempted to verify corrective action of TCP-23's audit finding, but logs that would document the corrective action had been destroyed.
A new deficiency was written at that time and the response was accepted, but the corrective action implementation is still j
unverified.
Following the Lobbin Report, the NRC performed a CAT inspection (IR 445/83-18;446/83-12, dated April 11, 1983) and included a review of the TUGC0 audit program at the corporate offices. The inspection included a review of 18 audits (conducted between 1978 and early 1983), auditor qualifications, audit planning and scheduling.. audit reporting and follow-up, and audit program effectiveness. The report concluded that weaknesses existed in the established QA audit program which included the scheduling and frequency of audits, e the. lack _of effective m nitoring of the construction program, and the lack of effective resolution of certain audit findings. The inspection also indicated that the QA Program should hcve been more effective.
During the TRT's evaluation of allegations and concerns, it was observed that the audit function did not always identify QA Program breakdot;ns, or if reported, effective corrective action was not instituted to erevent recurrence..
Based on its findings and observations, the TRT concludes that the QA audit... program has had and continues to exhibit deficiencies. Over a significant period of time, recurring Deficiencies include: inadequate staffing during peak periods;
.. procedural and implementation inadequacies; questionable
Revision:
1 Paga 3 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
1.0 DESCRIPTION
OF ISSUE (Cont'd) qualifications and capabilities; incomplete assessment of the QA Program on an annual basis;
. In summation, the QA/QC Group finds the past audit system less than adequate, and the audit program at the time of the TRT review was questionable."
2.0 ACTION IDENTIFIED Evaluate the TRT findings and qonsider the implications of~
{
-f>
these findings _on_ construction quality. "... examination of the potentiWsafety implicationsWulT include, but not be limited to the areas or activities selected by the TRT."
" Address the root cause of each finding and its generic implications..."
" Address the collectiv,e significance of these deficiencies..."
".P_Iop2EC.E Laction ple... that will ensure that such problems do not occur in the future."
3.0 BACKGROUND
i In addition to the TRT issue and SSER 11, the NRC issued the I
following notice of violation (445/8M 2-03; 446/8411-03) on j
February 15, 1985:
" Contrary to requirements, the following examples were identified which demonstrate the failure to establish and implement a Yrelatedactivitiesasrequired,asnotedbelow: comprehensive system of planned an Annual audits were not adequately addressed by the audit implementation procedures.-
TUGC0 Procedure DQP-CS-4, Revision 0, dated August 9, 1978, only required two eudits'of vendors fabricating reactor coolant pressure boundary components, parts, and equipment; one audit of vendors fabricating engineered safeguards components, parts, and equipment; and audits of balance of plant (safety-related) as required by the quality assurance manager.
- \\A(, { ()
s
- V 1A
Revisient i
l' Pr.g3 4 of 53 RESULTS REPORT
?
ISAP VII.a.4 (Cont'd) i l
l i
3.0 BACKGROUND
(Cont'd) i TUGC0 Procedure DQP-CS-4, Revision 2, dated April 16, j
1981, required only that organizations will be audited on a regularly scheduled basis.
l TUGC0 Procedure DQP'-CS-4, Revisions 2 and 10, did not i
specify auditing fr'equencies for design, procurement, construction,_and operations activities.
TUGC0 Procedure DQP'-CS-4, Revision 10, based audit requirements on Regulatory Guide 1.33 Revision 2, February, 1978. This commitment did not fully address the requirements of the Construction Quality Assurance Program.
1 The above procedure and subsequent revisions failed to describe and require annual audits in accordance with commitments and requirements. Earlier audit procedures were not available to determine if they met requirements.
4 Planning and staffing to perform 1983 audits was inadequate to assure that a comprehensive system of audits was established i
and implemented to verify compliance with all aspects of the Quality Assurance Program, in that, of 656 safety-related procedures (which control safety-related activities) the NRC review revealed that the applicant sampled only 165, or 25 l
l percent, during the 1983 audit program. Consequently, l
significant aspects of the safety-related activities were not adequately audited.
f The Westinghouse site organization, established in 1977 to 3
perform Nuclear Steam System Supply (NSSS) engineering j
services, was not audited by TUGC0 during the years of 1977, j
1978, 1979, 1980, and 1981.
i i
1 Audits of vendors that manufacture or fabricate parts, j
components, and. equipment for reactor coolant pressure i
boundary and engineered safeguards systems have not been conducted annually dating back to August 9, 1978."
1 i
I l
i 1
l l
l
'W y
,;y my p p l
~
1
Rmrision:
1 Pago 5 of 53 RESULTS REPORT ISAP VII.a.4 l
(Cont'd) l 4.0 CPRT ACTION PLAN l
4.1 Scope and Methodology 4.1.1 The scope of this action plan was to evaluate the adequacy of the TUGC0 QA Audit Program from its inception to the present, determine the effect of any identified inadequacies on the Quality Assurance Program and/or the physical plant and to recommend appropriate corrections and/or improvements to the l
current program. This evaluation was accomplished I
through a review which addressed audit planning and scheduling, preparation, performance, reporting, follow-up and closcout and audit personnel qualification.
4.1.2 The specific methodology is described below, i~
4.1.2.1
@ a~1TaFJp revisions of the,ppogram and Jrocedurie.Jertaining to the QA Audit Program y
for CPSES were reviewed to identify licensing j
commitments and the degree of conformance to l
these commitments in the written program.
Included in thir review were the CPSES PSAR/FSAR (Appendixes lA(N) and 1A(b),
Chapter 17.1, and QA Branch questions and answers), TUGC0 Corporate Quality Assurance i
Program, CPSES Project Quality Assurance Plan (Design and Construction)-and Dallas Quality Procedures / Instructions Manual.
l
' N 4.1.2.2 R porta. dac"m a tation and data generated J
during the implementation of the QA audit program were selected for, review on the basis of concerns identified by the NRC; significant revisions to comm1twents, program description and/or organization; anl to pursue questionable areas identified during the review. Specific topics addressed include the following:
l t
Audit Planning Criteria',
Published and As-run Schedules.
Audit Plans and Checklists.
G R cLLO d ue 2 m @ j g n[,.
Ch u ' : W
- (g lf?/
L___________-----
l Revisien 1
Pcgs 6 of 53 RESULTS REPORT l
i 1 SAP VII.a.4
{
(Cont'd) j
)
1 4.0 CPRT ACTION PLAN (Cont'd)
Audit Reports, y
Audit Deficiency Follow-up, i
Audit Team Members, Including Qualifications and Staffing Levels, Organizations Performing Audit Activities l
and 6
Application of Audit Activities to Hardware I
vs Program / Procedures.
i From this evaluation, conclusions were drawn concerning the adequacy of program assessment provided by the audit program. This information was used as input for the l
recommendations for revisions to the current j
program.
4.1.2.3 Should audit program der'iciencies or weaknesses related to construction activities be identified, they will be evaluated to determine whether action beyond that ap3s1 Qe_d Jn iSAP VII.c is required to
%identifypotentialareasofconcernregarding
/ construction quality. Should such actions be required, a detailed plan will be developed and this ISAP revised to describe the methodology.f (This conditional step was ]
etsHlned to be not required and therefore was not performed. SeeSection6.0forj discussion.)fInaddition,shouldany i
identifieif audit program deficiencies apply also to off-site TUGC0 suppliers, a. prpgram l
will be developed to determine the (c~ceptability_olfilhg aupplierst-quality I
assurance programa for the applicable equipment and services during the period in question. This program, if required, will utilize external sources of information such as other utility or architect-engineer audits, the Coordinated Agency for Supplier Evaluation and the NRC " White Book." If suppliers are identified for which the 1
(
l f
a
'A.
7 RLvision:
1 Pcgo 7 of 53 i
i RESULTS REPORT l
ISAP VII.a.4 j
(Cont'd)
{
4.0 CPRT ACTION PLAN (Cont'd) adequacy of the QA program cannot be determined, a__ detailed plan will be develoged to resolve the concern and this ISAP revised' I - 4,to de. scribe the methodology.
' j(This con'ditional step was determined to be,,'
.... -. ~. _.... -_. _ _
)
not required and therefore was not erformed. See Section 6.0 for discussion.),.!
4.1.2.4 The surrent audit erogram, including qudit.or s4 qualification requirements, was evaluated 4
against licensing commitments contained in the FSAR.
The qualifications of the current audit staff were evaluated. A determination was made concerning the adequacy of the TUGC0 vritten program and the qualifications of the audit staff for the remaining construction phase of Unit 2 and for the operations phase.
T 4.1.2.5 QA/QC Program deviations were identified for NN correctionland suggested program improvemen_tp j
are presented.
I 4.1.2.6 Copies of the results report for this ISAP l
l 1
will be provided to TUGC0 for their
\\)
{
consideration in responding._to_.the_NRC Notice j
3 iof. Violaeion.(445/8.432-02;._446/8411 03).
l 4.2 Participants Roles and Responsibilities t
4.2.1 TUGC0 i
4.2.1.1 Scope TUGCC assisted in identifying and locating applicable information and documentation to support.the Review Team activities.
]
4.2.1.2 l'ersonnel i
t Mr. D. McAfee, Dallas QA Manager, provided coordination between the Review Team and TUGC0 QA staff personnel.
I i
~
_j
Revicion:
1 Pcge 8 of 53 RESULTS REPORT i
ISAP VII.a.4 (Cont'd) o I
4.0 CPRT ACTION PLAN (Cont'd) 4.2.2 ERC 4.2.2.1 Scope ERC was responsible for reviewing applicable TUGC0 documentation to determine the adequacy of the program.
4.2.2.2 Personnel i
Mr. J. Hansel Review Team Leader i
Mr. J. Gelzer Issue Coordinator
~
4.3 Qualifications of Personnel Participants were qualified to the specific requirements of the CPRT Program Plan.
4.4 Standards / Acceptance Criteria Audit activities shall be in compliance with 10CFRSO, Appendix B, Criterion XVIII and the applicable codes and I
standards relating to the CPSES PSAR/FSAR sections pertaining j
to the QA audit program as interpreted in Section 7.1 of the NRC Standard Review Plan (NUREG 0800). Specifically such
_ activities are in compliance if (quoting from NUREG 0800):
o" Audits to assure that procedures and activities comply with the overall QA program ge_ performed by:
e y
The QA_ organization to provide a
.-1 1* 4 comprehensive ?nde @ verification and
- f-3 q Y "'4 V evaluation of quality-related procedures and activities.
The applicant to verify and evaluate the QA programs, procedur Q ud activities of suppliers.
An audit plan is prepared identifying audits to be Eerforned, tEeir freq(heduled based upon the statu cies, and gshedules. Audits should be regularly sc safety importance of the activities being performed and are initiated early enough to assure effective QA during design, procurement, manufacturing, construction, installation, inspection, and testing.
i
e 4
($
R3 vision:
1 Pega 9 of 53 RESULT 5 REPORT x
L ISAP VII.a;4
}-
(Cont'd) 4.0 C?RT ACTr0N PLAN (Cont'd)
,,);
Audits incluje an Gi6ctive3evaluat_ ion of quality-m t
related practices, procedures, instructions; activities and items; and revfew of documents and. records to ensure that the QA program is effective and properly implemented.
Provisions are established requiring that audits be performed in,harkas where the requirements of" g
Appendix B to 10CFR Part 50 are applicable. Areas which are often neglected but should be included are l
activities associated with:
Indoctrination and training programs.
i
- [,
Interface, control among the applicant and the principa): contractors.
l)! '
Corrective action, calibration, and non-conformance control systems.
\\,.
L >
SAR commitments.
s p*
Audit dna..are...analyzgd and the resfLU1ng_ reports indicating any quality problems aol the effectiveness 5
of the QA program, including the need for reaudit of deficient areas, are reported ta =aggraj: for review i
and assessment.
e Audits are_ performed in sccordance with pre-established written prociidures or.phecklists and _ conduct $d by en in g ersonnel havig no direct responsibilities in the areas be.tng audited."
i
{
4.5 cision Criteria s
Decisions concerning the number of individual reports, f
records and files to be reviewed, and'she level of detail to
}
4
^
which they were reviewed, were based on, the quantity and j
, quality of data obtained as implementation of the ISAl' g
proceeded.
Sufficient data vere evaluated to supon t_the
{
dttglusions, reached U to the_ ganaptabilityithtsggm, j
a A st the acceptance briteria contained in J rc E9Ph 4s.4..
)
a g
',T j
j y
3 r' 1 r s
4 i
l 2
4
~
s m
'1 I
i
)
) t s
a n
8
-j i,
- 3., 3' 3
s s
1 y
i t
).
e s,.
i l
4.y
\\
1 3
s i
1 3
.n:
Rsvision:
1 Pcan 10 of 53 RESULTS REPORT' i
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS The TUGC0 QA Audit Program adequacy was evaluated from its inception to the__present. The specific TRT findings in the SSER 11 and the February 15, 1983, Notice of Violation are addressed in this section.
The implementation of this action plan was accomplished in two phases:
Audit P_rogram,(Section 5.1) and, Audit, Personnel Qualification (Section 5.2).
These sections are~sub-divided as follows to address the various elements of the Audit Program and f
1 the Audit Personnel Qualification Program:
Section 5.1 Audit Program 5.1.1 Organization s
5.1.2 Staffing e
i 5.1.3 i
Audit Planning and SchedulinA s
- culQ M t p.u M' 5.1.4 d'sdit Preparation 9(
3 5.1.5 Audit Performance r.
E 5.1.6 Audit Reporting L
C. A s
g 5.1.7 AFollow-up and Closeout it i
,e Section 5.2 Audit Personnel Qualification 5.2.1 Qualification Requirements 5.2.2 Training / Qualification Records a-c l
Implementation consisted of a review of program definition and implementation documents, a review of records and other related documents and interviews with personnel involved in the program.
',,3 gp The historical PSAR/FSAR was reviewed to identify the standards and regulatory guidance to which TUGC0 was committed, pertaining to the audit program and qualification of audit p'ersonnel.
This review determined that:
The basis for the audit program from December 1973 (Amendment
- 2 of the PSAR) through the present is ANSI N45.2.17-1213 (Qritt_3..Restision 0. May.jL 1973) as' contained in " Guidance on Quality Assurance Requirements During Design and I
'c.j b
o
M* - '
Rsvision:
.1 =
Pegs 11 of 53 i
RESULTS REPORT ISAP VII.a.4 (Cont'd) i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) l Procurement Phase of Nuclear Power Plants" (Gray Book), dated i
June 7, 1973, and as amplified by the " Regulatory Staff
)
Cocments and Supplementary Guidance on N45.2 Standards" contained in Section D.2.b of the Gray Book.
The only exception to this commitment is that since August i 1984 (Amendment #52 of the'FSAR) TUGCO elected to adopt, as an alternative for scheduling external audits, the guidance pre,vided by Regulatory Guide 1.144. Revision 1, September 1980, Paragraph C.3(b).
-lThebasisforauditpersonnelqualificationswasANSI
- 7. N45.2.12-1973 until June 1, 1983, which was the effective date
.of TUGC0 commitment to Regulatory Guide 1.146, August 1980, jwhich endorses ANSI N45.2,23-1978.
The documents and record files reviewed included the following:
W
.j/'
TUGC0 Corporate Quality Assurance Program, d
f gst$y W 's CPSES Project Quality Assurance Plan for Design and g g #0M Construction,
%v-- -
)
_ Dallas Quality Procedures / Instructions Manual, j
~~
Audit Report Files, Audit Report Notes Files.
Equipment Release Files (Vendor Compliance Release Inspections),
Vendor Correspondence Files, Vendor Reevaluation Files.
Vendor Performance Evaluation Files and Audit Personnel Training / Qualification Files.
UCLL.:L N/ug[R.qj 'l'$ P D FPe<4i sf 7
..~
Rsvisien:
1 Pegs 12 of 53 RESULTS REPORT i
ISAP VII.a.4 (Cont'd)
I 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Findings pertinent te each topic will be discussed within the appropriate sub-section. At the end of each sub-section a
% statement regarding possible advJrse effects of the findings on the i
audit program will be made.,fTnere is no direct connection between the audit program and the quality of installed hardware; therefore, no conclusions regarding the st'atus of the physical plant have been/
- d*'% %e. >
p N Recommended corrective action for QA/QC Program Deviations 4
identified in the body of this report are summarized in section 5.3, " Evaluation of Findings".
5.1 Audit Program Documents and files were reviewed to evaluate the adequacy and effectiveness of the TUGC0 audit program including organizatiet., staffing, planning and scheduling, preparation, performance, reporting and deficiency follow-up and closecut.
Initially. 69 audit files were reviewed to assist in reaching a conclusion pertaining to the performance of the TUGCO QA Q
audit personnel. Internal audits were selected to provide X
da frnm cha incepr4nn nf the nregram to the eresent, as well as a sampling of the various disciplines and major topics of interest such as document control and nonconformance/ corrective action systems. Vendor audits were selected primarily to review TUGC0 activities pertaining to prime contractors (B&R, E, G&H) as well as on-site-contractors. This review was not intended to be a comprehensive evaluation of the scope, content, conclusions, etc., of the audit; rather, it was intended to determine if the required steps in the audit process had been properly i
performed. The review addressed the following items:
Audit Preparation - To determine if the audit scope was documented, if an audit team leader had been designated, if checklists had been prepared and if they appeared to contain an appropriate level of detail.
Audit Performance - To identify the audit team, to determine if pre-audit and post-audit conferences were conducted, if it appeared that objective evidence had been examined and if audit personnel appeared to evaluate program elements to soma depth.
~
f l
1
r Revisicn:
1 1
Pegs 13 of 53 j
i RESULTS REPORT i
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Audit Reporting - To determine if audit reports had been prepared that would provide appropriate management with sufficient information concerning the adequacy and j
effectiveness of the audited QA program elements.
Audit Deficiency Follow-up and Closecut - To determine if responses to audit deficiencies were evaluated, if 7
an acceptage cTitirst and schedule for corrective action had been agreed on hnd_if implementation of corrective
_ action was verified.
/#+ ~
~
h.
[, -
Later, as the rNu~1't 5K,1pn'ii [ cpi d from othbr QA/QC Review Team Issue Coordinators, nin (9)' audit files' vere reviewed in depth in an attempt to reach
.onclusion'a's to why particular prograt=atic problem areas (e.g., non-conformance control and corrective action systems, inspector training, qualification, and certification programs), which were identified by External Sources had not been identified and corrected through implementation of the TUGCO QA audit program. The review of these nine (9) files included a re-review of four (4) of the i
sixty-nine (69) previously reviewed and were selected on the j
basis of thWTftTis~in the audit index as being the total available that addressed these subjects. This review included the following activities:
A detailed review of the audit checklist for content and to attempt to determine if the checklist was developed from requirements documents or from the existing written program, A review of auditor field notes to attempt to determine the depth of examination, sample size (when applicable), type of objective evidence evaluated, whether program documents were evaluated against requirements, and if apparent deviations had been flagged to be included in the audit report, A review of the audit report to determine if the findings documented in the auditor field notes were accurately reficcted in the report and A review of documentation subsequent to the audit report in an attempt to determine the adequacy of auditor evaluation of responsis to audit deficiencies, dialogue leading to agreement on proposed corrective action and verification of completion of corrective action.
Q 4
6.
/
%W _Wh2.wfn[, /(A s.
)
ridhd%hvM u
( pg '
i
?
y
Revision:
1 Pege 14 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
In response to the NRC Notice of Violation (445/8432; 446/8411), the audit files representing the 29 internal audits of the design / construction phase performed during 1983 were reviewed to identify the major activities, organizations, and y C(
Appendix B criteria which were addressed, to enable an evaluation of the audit program coverage for the year. These f
J9 files included djye~)whith had been previously reviewed.
b The Design Adequacy Review Team provided a list of design fC [~
specifications which they plan to evaluate for design adequacy
~
3 and Architect-Engineer / Vendor interface. From the. vendors identified, twelve were selected for this review based on considerations such as frequency of audits performed, awareness of problems with a particular vendor, apparent time frame of vendor activity and type of product. Additional rationale for selection is contained in the working files for this ISAP.
A review was performed of TUGCO QA Audit.and Vendor Compliance files (audit reports, audit notes, equipment releases, vendor correspondence, vendor performance evaluation and vendor reevaluation files) for these twelve vendors to evaluate the degree of awareness by TUGC0 QA of the adequacy, effectiveness and status of the QA programs of their vendors despite the fact that annual vendor audits were not scheduled.
~.-- _
The results of these reviews are summarized in the following paragraphs.
5.1.1 Organization ANSI N45.2.12-1973, Section 1.4, " Organization",
states, "The organizational structure, functional responsibilities, levels of authority, and lines of internal and external communication for management direction of audits of the quality assurance program shall be documentv ".
Although Section 17.1.18 (Audits) of the CPSES FSAR states that TUCCO will perform audits, nowhere in Section 17.1.1 (Organization) is the responsibility for the audit program delineated.
hW
(&
L i
- t..
v, g 4,,
1
a Revision:
1 Page 15 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Within section 17.1.1.1, the list of principal duties and responsibilities of the Director, Quality Assurance includes " monitoring and surveillance of the quality assurance activities conducted by TUGCO, prime contractors, sub-contractors and vendors." However there is no specific mention (nor has there been since the original docketing of the PSAR) of responsibility for the audit program.
During the review of the QA program and records, it wc:
determined that the QA organization was in fact responsible for the audit program and that TUGC0 management recognized and supported this responsibility.
It was noted that on December 30, 1985, TUGC0 QA initiated a request for an FSAR revision to include the audit program among the principal duties and responsibilities of the Director, Quality Assurance.
The TUGC0 QA Audit Program is administered sud implemented by the Dallas-based Quality Assurance organization. The implementing procedures for the audit program are contained in the Dallas Quality Procedures / Instructions Manual. Since 1982, procedures in this manual have assigned responsibilities for execution of portions of the QA audit program to the position of Supervisor, QA Audits. This position was not officially r6 cognized in published organizational descriptions and organization charts until the issue of procedure DQP-QA-16, " Dallas Quality Assurance Organization" in Eeptember 1985. During the review of the audit program procedures and records, it was determined that the position was filled during this period and was recognized and supported by QA management.
Although ANSI N45.2.12-1973 and the accompanying staff comments and guidance pertain primarily to the auditing
{
organization, the requirements of the ANSI standard and l
the associated supplementary. guidance listed below apply as well to the audited organization as an 4
Rsvision:
1 l
Pega 16 of 53 1
1 RESULTS REPORT l
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) integral part of the overall audit program. Therefore, these requirements should be included in QA program documents applicable to the audited organization as well as the auditing organization (e.g., Corporate QA Program, Project QA Plan) to ensure compliance by all responsible organir.ations.
An effective audit system shall include provisions for reasonable and timely access of audit personnel to facilities, documents, and personnel necessary in the planning and performance of the audits.
At the post audit conference an effort shall l
be made to establish a tentative course and l
l schedule of corrective action for non-I conformances. Where it is not possible to provide such information at the post audit conference, the management of the audited organization should commit to a specific date for the determination of the course of corrective action and the schedule for l
implementation.
l l
Management of the audited organization or activity shall review and investigate the
)
audit findings to determine and schedule appropriate corrective action. They shall respond to the report in writing, within 30 days after receipt.
The review of these files indicated that these requirements had been met in that:
There was no evidence to indicate that audit personnel had been denied reasonable and timely access as required to perform their duties.
There was no evidence to indicate that problems had been encountered pertaining to the establishment of the tentative course and schedule of corrective actions and schedule for implementation.
I a.-
2
Rsvision:
1 Pago 17 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
There was evidence that management of the audited organization reviewed and investigated audit findings and determined and scheduled corrective action. Written 4
responses were received by the auditing organization within a reasonable time of the i
requested date (on time, or within one week) in approximately eighty (80) percent of the files reviewed. This ratio is not unusual in the experience of the RTL.
The first.two of,these requirements listed above are ncLt,,a_d_ dressed in the.._TU_GC0_QA pronram documents. The" t,hird is addressed..to_,,the. audited organization only through. the transmittal-of-tha aurH r rep _ ort.
o Conclusions
/
/
It is concluded that the failure to formally document I
the responsibility for the audit program in the PSAR/FSAR resulted in no adverse effect on the audit k
program because sub-tier program documents were written and management acted as if the responsibility had been formally documented.
It is concluded that the failure to formally describe the position and responsibilities of the Supervisor, QA Audits, in organization descriptions resulted in no
'q adverse effect on the audit program because audit
(~/
program documents were written and QA management acted as if the position had been formally documented.
I/
e It is concluded that the failure to include the I
~,'
responsibilities of the audited organization in appropriate program documents resulted in no adverse effect on the audit program because review of the audit files revealed that the requirements had been met by s
the audited organization.
5.1.2 Staffing The adequacy of the TUGC0 tudit staff (number and qualifications) has been questioned: by the NRC. When considering the adequacy of an audit staff, one must include not only personnel assigned, full time to the audit function, but also the qualified personnel
{
available on call to the audit organization.
i t
I Rsvisient 1
)
Pags 18 of 53 RESULTS REPORT i
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
The applicable standards and regulations contain no requirement for an audit staff, per se; only that management shall establish an audit system that includes manpower, funding and facilities to implement the system. This allows management maximum flexibility to allocate resources, even to the extreme of assigning a full-time audit administrator responsible for planning and scheduling of audits and tracking open items, who has free access to qualified audit personnel as required for individual audit preparation, conduct, reporting and follow-up, but who has no permanently assigned audit staff.
Therefore, judgement on the adequacy of the size of an audit staff must be based on factors such as assigned duties and responsibilities, outside resources available on a timely basis and the ability to accomplish the scheduled audit activities (providing schedules are not dictated by manpower availability).
The years 1981 - 1982 were cited by the TRT as a period when the numbers and qualifications of the TUGC0 audit staff were questionable. In addition, Region IV Inspection Report 445/84-32 cited 1983 as a period when the audit staff was inadequate. A review was perforced of training and qualification files for TUGC0 Quality Assurance personnel (designated audit staff and others) who participated in the audit program as Lead Auditors or auditors during these periods. The following tabulation summarizes the characteristics of these personnel.
l l
Ravision:
1 Page 19 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
\\l 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) t 1981 1982 1983 g
Designated Audit Staff 1
14 (Including Supervisor)
-h Total Participants 20 g-Qualified Lead Auditors 0
16 18 Education
- Masters Degree 0
1 1
[
Bachelors (Engineering)
)
6 8
Bachelors (Other) 2 2
5 Associates (Technical) 0 1
2*
Associates (Other) 0 1
1 l
Some College 1
2 3
NOTE:
Four of the engineers were registered f
Professional Engineers, and one was a
/
certified Quality Engineer. One holder of an Industrial Management degree was also a certified Quality Engineer.
Since the formation of a designated audit staff in 19'i9, through the present, it has been the practice to supplement the staff as needed with other qualified members of the QA organization to function as auditors and lead auditors.
It is therefore concluded that, based on the evaluation of the years 1981 through 1983 b
and the other files reviewed, the formally designated
/
_ audit staff could be considered deficient in numbers i
\\, and technical qualifications, but hat the effective j
audit staff of Quality Assurance personne..~was, and i
- )
'/
~
coiifin'tieT to.be,_adequ'a'telJn both number ~a~nT
~
~~
%./
qualifications.
19 t
Conclusions W.
i Ja s
Based on the reviews performed, it is concluded that j
gg individuals did not audit activities which they were f'~
g responsible for performing.
It is concluded that audit staffing was ad' equate to implement the audit program and schedules during the periods of interest.
I Personnel holding multiple degrees are listed only once, in the highest applicable category.
l Revision:
1 1
Page 20 of 53 I
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION.OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 5.1.3 Audit Planning and Scheduling This section addresses the following topics:
Scheduling requirements, Vendor audit scheduling and performance, NRC Notice of Violation pertaining to Westinghouse site organization, l
- Planning and scheduling process for internal audits and
- NRC Notice of Violation pertaining to 1983 audit program coverage.
The requirements of ANSI N45.2.12-1973 pertaining to audit planning and scheduling can be paraphrased by saying that audits are performed by or for an organization participating in activities affecting quality and that elements of the Quality Assurance program affecting these activities shall be audited at least annually or at least once within the life of the j
activity, whichever is shorter. Within this context, the following terms are defined:
Organization - Plant owner, contractor, etc.,
l Activities - As listed in the last sentence of ANSI N45.2.12-1973, ?aragraph 1.2, and Elements - The 18 criteria of 10CFR50, Appendix B.
1 The TUGC0 vritten program (i.e., PSAR/FSAR, QA Program.
I QA Plan and implementing procedures) reflected the requirement for scheduling internal audits on an annual I
basis only from August 1973 through April 1981. The l
annual requirement was not reintroduced until November 1984. Additionally, the program eve has required that vendors be audited on an annua asis.
I i
1 J
q
}
i Revision:
1 Page 21 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Not until the issuance of DQP-CS-4, Revision 11, in November 1984, was the TUGC0 vritten program in full compliance with applicable planning and scheduling com=itments. This revision reflected the options permitted by Regulatory Guide 1.144 to audit vendors on a triennial basis, provided annual assessments are performed. This revision also contained the requirement to audit other applicable organizations (e.g., TUGCO, site contractors) at least annually.
I
'This lack of requirements in the written program for
- scheduling annual internal audit and supplier audits,
3 kcorstituted a noncompliance with licensing commitments.
Vendor Audit Program In evaluating the effect of this noncompliance on the vendor audit program, the following evolution of TUGC0 QA audit program responsibilities was identified:
From 1973 to 1977, TUGC0 was responsible for auditing only the major contractors: Gibbs &
Hill (G&H), Brown & Root (B&R), and Westinghouse (H). During this same time period G6H, B&R, and E had prime responsibility for vendor preaward surveys, audits, and source inspections. However, TUGC0 did participate in these activities.
Early in 1978, although E retained its original role concerning NSSS vendors and B&R g
retained full responsibility for its ASME j
g' _. _, suppliers, TUCC0 assumed more direct control, ghs JM -
in_the other vendor creaward survey, audit, and source inspection activities previously delegated to G&H and B&R. This included balance of plant vendors (safety-related and non-safety-related), on-site contractors and non-ASME material suppliers. Additional control was exercised through increased involvement in the scheduling process and the conduct of more joint audits.
Revision:
1 Page 22 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0
~
IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 4 y
Early in 1981, TUGC0 began performing g
T*
l vendor-related activities without G&H g A
.y b o c
- involvement. Also during 1981, TUGC0 assumed 4
full responsibility for non-ASME material i
suppliers from B&R. This level of I
responsibility has continued to the present.
Based on the review *of the audit index, it was I
determined that TUGC0 QA performed audits of the major contractors during the period from 1973 to 1977 as follows:
, 1973 1974 1975 1976 1977 G6H N/A 4
1 1
2 W
1 0
1 6
4 B&R (Houston) N/A 1
0 2
1 B6R (Site)
N/A N/A 2
1 1
The PSAR was docketed in 1973 and the Construction Permit was issued in December 1974. The tabulation indicates that with two exceptions, required audits were performed during this period in accordance with j
the annual requirements of ANSI N45.2.12-1973.
From 1978 (when TUGC0 became more involved in the administration of the audit program) through 1984, the performance of contractor / vendor audits sta's not in secordance with applicable requirements for timelfless o_r_fre g y." T6F' example, for rive of the twelve vendors reviewed, the initial audit was performed after the first release inspection (in three cases more than a year afterward) which was not in compliance with the ANSI requirement that auditing be initiated as early in the life of the activity as practicable to assure timely implementation of quality assurance
(' requirements.
I_t was also..noted_thatt.anywhere.from,
1, fp.ur months.to six years..alagaed between audi_ts of i these vendors. No single vendor was audited on_a3 annual. basis.through'ou_t. thia _tiat_reriod, which constituted a violation of the ANSI requirement for annual audits.
The foregoing raises a concern that the TUGCO QA organization was not cognizant of the effectiveness of the vendor's QA programs.
In assessing this concern, l
the following was disclosed:
_ _. _ ~
b
Revision:
1 Page 23 of $3 1
RESULTS REPORT s
ISAP VII.a.4
.l (Cont'd) 4 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
In October 1978, TUGC0 QA instituted a Vendor Rating program whereby numerical ratings were assigned to vendors based on type, severity, and quantity of defects identified during I
source inspections..These ratings were utilized to adjust the scheduling of inspections and audits. In January 1981, a Vendor Performance Evaluation system was instituted which enhanced the Vendor Rating system by also considering the number of items submitted for. inspection in calculating the rating.
It was noted during the review ~
that various audit notifications and/or reports documented that some of these audits had been initiated as a result of unsatisfactory source inspection reports.
g V
I In May 1979, a Joint Inspection / Audit program
\\
was instituted whereby auditors would pf accompany inspectors on release inspection f'[
trips to investigate specific program areas based on problems identified during previous source inspections.
TUGC0 QA also utilized " Requests for Corrective Action" which were correspondence with the vendors requesting corrective actio.
for specific, identified problems when it was concluded that the problems did not warrant an audit investigation or did not appear to indicate an edverse trend.
Internal documents known as " Yellow Flag Sheets" were also utilized.by the TUGC0 QA staff to ensure that applicable QA personnel were aware of status and/or problems with particular vendors, e.g., a hold placed on shipments from a vendor until after an audit was performed.
While it is recognized that source inspections and limited-scope audits performed in response to
'3 identified problems might not be considered adequate substitutes for regularly scheduled program audits, the did provide data which were utilized to Qs by TUGC0 to evaluate the;effectivere:s of vendor xten g programs.
Rsvision:
1 Page 24 of 53 RESULTS REPORT I
l ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It is concluded that during the period prior to 1978, the frequency of performance of contractor / vendor audits, with the exception of the two cases noted, was satisfactory, but for the period from 1978 through 4
November 1984 the frequency was not satisfactory.
Q it is further concluded that as a result of the source inspection activities, the limited audit activity, vendor ratings, corrective action requests j(/4l AAMA
< ~ ^ '
and internal QA communications, that sufficient
/'
g information was available to,_and__used to some extent g
i Ay, the QA organization to evaluace the effectiveness f
4, of vendor QA programs even though audits of the vendq s A
were not performed on an annual basis.
The NRC notice of violation (445/8432-03; 446/8411-03) identified that the Westinghouse site organization had
)
not been audited from its inception in 1977 through 1981. To provide background, the description of the Westinghouse site organization in the NSSS contract between TUGC0 and Westinghouse includes the following
)
in Section 5.0 -(paragraphs A and B are not applicable to this discussion):
"C.
Construction Site Services - Beginning 21 months prior to the arrival of the first of Vendor's equipment at the job site (or such later date as may be mutually agreed upon),
Vendor shall establish an organization with representation at the job site to the extent necessary in the judgement of Purchaser to coordinate construction site activities for Vendor's equipment. This organization will 2
be retained with representation at the job site until fuel loading of the second NSSS is completed.
D.
Field Testing and Startup Services - Vendor shall establish an organization to provide advice and consultation on tasting and startup. Special startup and testing equipment (or its then current equivalent) identified in Exhibit A shall be provided at the job site by Vendor on a cost-free loan basis. Testing and startup procedure preparation assistance shall also be provided,"
L C d-( lb,
, [hg
@ h b % c uw% *!(tMO Q
(,./lm % M
, m -:. w.a... a.u s w g h p p
j Revision:
1 Page 25 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Further, the safety-related activities relating to erecting, installing, inspecting, testing, etc., of the
{
NSSS were the responsibility of TUGC0 and were' performed by TUGC0 br Brown & Root organizational i
elements. The activities performed by the Westinghouse cite organization were primarily liaison and did not includw the categories of safety-related activities requiring periodic audits as listed in Section 1.2 of ANSI N45.2.12-1973.
This understanding of the involvement of the Westinghouse site organization in the site safety-related activities was confi'med by TUGCO QA during th,e r
performance of an audit of Westinghouse site activities pertaining to work on the reactor internals (audit TWH-23) in March 1982. This audit confirmed that the development of work travelers was performed by Texas Utilities Services, Inc. engineering and that construction and inspection activities were performed by Brown & Root.
The audit also revealed that Westinghouse-Pensacola supplied drawings and performed overview surveillance of TUGC0 and Brown & Root activities related to the reactor internals. After concerns were expressed by the NRC, an audit of Westinghouse site activities was scheduled and performed in November 1984 (audit TWH-30). This audit confirmed the TUCCO understanding of the Westinghouse site organization scope of work.
It was determined that the W site organization scope of work did not include the categories of safety-related activities requiring periodic audits per ANSI N45.2.12-1973. This position was confirmed by TUGC0 i
through the performance of the above-mentioned audits.
It is therefore concluded that there was no requirement o audit the Westinghouse site organization.
i Internal Audit Program As previously discussed, ANSI N45.2.12-1973 requires that applicable program elements affecting an activity be audited annually.
This means that the activities to l
be audited must be identified along with the pregram i
elements affecting these activities. Audits of thest
Revision:
1 Page 26 of 53 1
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 1MPLEMENTAIION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) activities are to be scheduled on the basis of their status and safety importance and performed to evaluate the adequacy and effectiveness of the program as developed and implemented by the participating organizations.
Evaluating the adequacy of the TUGC0 QA internal audit program at the construction site is complex. Prior to 1978, site audits were scheduled and conducted as vendor audits. Since that time, except for contractors operating under their own QA programs, site audits have been scheduled and conducted as internal audits.
The following paragraphs describe the evolution of the audit planning process during the Design / Construction phase of CPSES.
When the TUGCO internal audit program was first implemented at the construction site in 1978, audit planning and scheduling appeared to be primarily activity-oriented. However, by 1982 audit planning appeared to have become primarily organization-oriented. During the review of the audit files, observations related to the structure of checklists, task assignments of team members during the audit, structure and content of audit reports, is well as audit titles indicates that audit planning and scheduling continued to be organization-oriented.
This approach to audit planning evolved to accommodate the organizational structure and procedure systems that developed as TUGC0 assumed additional responsibility for site activities.
The written program has always required that audit planning and scheduling activities be performed, and the reviews of the files revealed that audit schedules had been prepared. However, it was determined that
.qntil_ the ise.uance of procedure DQP-AG-4, "CPSES, t
Construction Phase Audit Program," Revision 0, on September 4, 1985,_ther_e was no procedure that detailed the audit planning..and_ schedulinGroctas., A review of this procedure revealed that although audits were to be I
e g
1 1
Rsvision:
1 Pcgs 27 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) performed annually, activities to be audited would be determined from existing programs and procedures, rather than from an analysis of site activities e
affecting quality and their importance to safety. The procedure also implied that; audit planning was organization-oriented.
However, as a result of diset.ssions with TUGC0 QA management and a review of audit schedules, it is concluded that the audit planning and scheduling process has generally considered project activities as an input. Procedure DQP-AG-4 has since been revised (Revision 1, dated January 27, 1986), and now describes an adequate audit planning and scheduling process.
Procedure DQP-AG-4 also provides for the development of an annual audit schedule which is subject to periodic review and adjustment based on project status, program 1
changes, outside reviews, etc. As-run schedules for past years have been prepared to document audits j
scheduled but not performed (and the reasons why they were not performed), and additional audits which had been added to the schedule.
It was noted that the proposed 1986 audit schedule includes a short scoping statement for each scheduled audit. This enhancement to the schedule provides a more couplete description of the planned audit program coverage.
Also related to audit planning, the NRC Notice of Violation (445/8432-03; 446/8411-03) stated that TUGC0 audit planning for 1983 was inadequate because only 25 percent of the safety-related procedures were audited.
This use of procedures as an audit planning baseline is appropriate only when the procedures are pre-defined as the program elements. Accountability for audit coverage would then be simplified because a procedure listing is very specific as opposed to developing a list of activities, applicable criteria, and participating organizations. Attractive as it may appear, this method of audit; planning contains weaknesses and is not recommended because auditing strictly by:the procedures could result in overlooking an activity for which a procedure was required but had not been developed.
ALC-sWdAd x
,(.<. 74d I
(
Revision:
1 Page 28 of 53 RESULTS rep 0RT ISAP VII.a.4 (Cont'd) 4 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
By calculating the percentage of procedures addressed by the audit program, an inference could be drawn as to the actcancy of audit program coverage. However, unless this information is correlated with actual project activity, previous audit and/or surveillance
/.
results, importanes of the activities represented by procedures not audited, etc., no judgement should be made of the adequacy of audit program coverage based solely on percentage of procedures audited.
A review of the 1983 Audit Program revealed that TUGC0 performed twenty-nine (29) internal audits of design / construction phase activities as well as eight -
onsite audits of construction contractors.
From data gathered during the review of the twenty-nine
<3(29) internal audit files, a matrix was developed to evaluate the activities audited, the program elements (Appendix B criteria) addressed, and the organizations through which they were addressed. In addition, the list of activities audited (contractor and internal) was compared to the major site activities in process during the year as identified by project control personnel. These evaluations revealed that thg_198jL audit program addressed the, major on-going site activities, that all site organizations were audited, and that all program elements (Appendix B criteria) were addressed. Therefore, it is concluded that the audit program coverage of the Design / Construction Phase j
of CPSES during 1983 was adequate. It was also noted that the audit sch4dule was adjusted throughout the year in response to organizational and program changes, to address problems identified by outside organizations and to reflect changes in the project schedules.
Conclusions
)
It is concluded that the failure to perform vendor audits on an annual frequency'ranulted in_ng adverme effect_on the audit program because of the additional j
activities implemented by the QA organization to l
(
supplement the audit activity.
(, -)
,s}
/'
w -e 4
4 Revision:
1 j
Page 29 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It is concluded that there was no requirement to audit the Westinghouse site organization.
It is concluded that the lack of a formalized methodology for internal audit planning and scheduling resulted in no adverse effect on the audit program
[9[>(
7 because it was established that audit planning was accomplished and audit schedules were prepared to implement the aucTit prdiffam.
3M f g.
f c,
-r 5.1.4 Audit Preparation yn. & W & g Y
\\
The audit preparation phase consists primarily of accumulating pertinent documents and data from internal and external sources from which the Audit Team Leaders would develop the final detailed audit scopes and audit checklists.
The reviews of the audit files revealed that while checklists had been prepared for the conduct of the audits and that sources such as TUGC0 site surveillance, INFO evaluations, CAT reports, etc., had been utilized to address topics of special interest, the majority of the checklists reviewed in detail appeared to have been developed from existing procedures rather than from governing requirements. As discussed previously, utilizing procedures as the baseline could preclude verification that the procedures, individually or collectively, adequately implement the requirements.
It can be an acceptable practice to perform only compliance audits for those portions of a program which remain basically unchanged. However, there was no Tvidence in the TUGC0 QA audit files reviewed to indicate that a comprehensive assessment of the written program had previously been performed in the areas of Non-conformance Control / Corrective Action or QC Training / Qualification / Certification to ve'rify that the program procedures were in compliance with j
governing requirements.
m.
In addition, the checklists reviewed did not provide for verification that procedures in different CPSES procedure systems that address the same or similar activities vsrtegnaistterLt in approach, level of detail or control of the activity.
Ravicion:
1 Paga 30 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
During 1985, the QA Audit Group initiated the use of an
" Appendix B Checklist." Applicable portions of this -
standard checklist are now utilized by the audit team to verify that the procedures within the scope of the audit are in compliance with the applicable criteria of j
(,gg Y'i Conclusions a y V y It is concluded that the n dit preparation activities were not adequate. This resulted in an s'dverse effect
?
en the audit prog' ram, in that most audits reviewed did
/
)
not evaluate the adequacy of the written program to j
implement the governing requirements.
y 5.1.5 Audit Perfomance ANSI N45.2.12-1973, to which TUGC0 was committed N
through May 1983, contains no specific criteria for an Il Audit Team Leader.
~
From February 1982 through February 1985, as a portion i
y of on-the-job training, the TUGC0 program contained d
\\, '
provisions for a lead auditor candidate to be
. X t e
designated as the acting Audit Te g gade_r, to perform
\\
under the. direct supervision of a qualjfin Q,ead y
Auditor.
(p This practice did not constitute a noncompliance prior to the June 1983 commitment to ANSI N45.2.23-1978.
However, from that time until the issue of procedure DQP-CS-3, TUGC0 QA Audit Program," Revision 12, dated February 15, 1985, the pratetice did constitute a technical noncompliance with the requiremenYa'~57 ANSI N45.2.21 1978 It is not uncommon for an audit team leader to delegate functions to team members during the course of an audit, for training or other reasons, while retaining Fultimate responsibility therefore. However, in the TUGC0 program, full responsibility for an audit was
, assigned to other than a qualified Lead Auditor, which was contrary to the requirements of ANSI N45.2.23-1978.
a
I Rsvicion:
1
\\
'y.
Pega 31 nr 54 RESULTS REPORT I
1 ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION 0F RESULTS (C I
i ANSI N45.2.12-1973 requires that auditors be independent of any direct responsibility for the activities they will audit. The reporting level for the Director, Quality Assurance provides adequate independence of the QA staff from the design and construction activities.
The NRC raised a question in SSER 11 concerning the Yj
.P.bjectivity of surve,illance personnel performing audits MLV
\\ of activitie.s_over which th_ey.had surveillance responsibili.t,y. This practice is not uncommon'in the
~
industry and, in fact, can be beneficial to the audit program in that the knowledge and experience gained by personnel from surveillance activities can be utilized as input to the audit planning process and through participation on the audit team. The only time this situation might constitute a conflict would be the case where the purpose of the audit was to evaluate the adequacy and effectiveness of the surveillance program itself.
No audits of this nature were identified during this review.
I It wee deect-Ser8 tbn* pre-nudit and. post c.'idit conferences were conducted by the audit team and that appropriate personnel were in attendance.
The review of the audit files revealed that objective evidence had been examined when appropriate to support the audit activity as defined by the audit checklists.
It appears that the auditors evaluated program elements to an appropriate depth to support the audit checklist s.
Conclus'.ons It is concluded that the practice of designating a lead auditor candidate as an acting team leader resulted in no adverse effect on the audit program because the
?
F Q*
acting team leader, performed. the Activities under.,t;he dirzet supervi' mion_oLa.-qu=14Nd lshuditor who was a member..oi..the Ap4dit.feam.
' ~'
e c
t a concluded that the practic'e of utilizing site surveillance personnel as audit team members had no adverse effect on the audit program because these personnel did not perform audits of activities which i
they were responsible for performing.
i, l
f 4
a l
e
Ravision:
1
- s Page 3? of *2 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSIO 5.1.6 Audit Reporting The review of audit has not always complied with the requirements of ANSIf N45.2.12-1973.
Early in the program, audit reports did not consistently include items such as a description of the audit scope, persons contacted, a s'ummary of audit i
results and an evaluation statement regarding the effectiveness of the QA Program elements which were audited.
However, the information not included in the j
reports, with the exception of the summary and the i
evaluation statement, can be found in the audit files.
The content and quality of audit reports have gradually improved over time.
Currently, the audit reports provide the reader with an adequate understanding of j
the scope and results of the audit.
The current N45.2.12-1973. procedures adequately reflect the requirements of ANSi During the detailed review of the nine (9) audit files in the training and nonconformance control areas (aec ?"Fe 12), several devictions were observe'd that indicate weaknesses in the implementation of the audit program related to audit reporting:
4 There were approximately twenty instances where the published audit reports did not appear to accurate *y reflect the auditors' i
field notes and/or checklists.
items in the notes that appeared to beFor example, deficiencies were reported as concerns or comments, and in one case, an item identified as "unsat" on the audit checklist was not reflected in the audit report at all. The items identified did not pertain directly to any items of hardware.
the RTL that additional data could have been(It is re obtained which could have altered the conclusions and was not reflected in the notes / checklist).
i i
i a
I
.___.w
Revision:
1 Page 33 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
In many of these cases, the accompanying notes seemed to indicate that the auditor either judged the "importance" of the finding in the decision to document, or may have had additional information which was not included in the audit files. There was no discernible pattern for not documenting identified unsatisfactory conditions.
l It was noted that the category of " concern" was deleted from audit reports in April 1984 with the issue of revision 7 of DQI-CS-4.6.
All discrepancies are now categorized as deficiencies in the TUCCO audit program.
l It is the responsibility of the Audit Team Leader to ensure that the audit report accurately reflects the audit findings as documented by the audit team, and that j
pertinent backup data is included in the i
audit files.
There were occasions when major sections of DstN VMnat cbliipleted wh1DF e'ffec't, altered the scope of the audit. This was not documented in the audit report. This could have provided an inaccurate view of audit program coverage.
In some cases, the reason for the omission was noted on the checklist (e.g., " time constraints", " covered by another checklist",
"would have required verification of other procedures", etc.), but in some cases was not.
There was no discernible pattern in the stated reasons for not completing checklists.
In March 1985, the QA Audit Supervisor instituted a " Checklist Review Verification Form" (nct procedurally define D which is used by the Audit Team Leader to document the extent of completion of the audit checklist.
This form is now utilized in the audit j
planning process to ensure that incomplete items are rescheduled as required.
I i
a
~
Ravision:
1 s
Page 34 of 53
_RESULTS R/ PORT ISAP VII,a.4 (Cont'd) 5.0 IMptpiENTATION or ACTION PLAN AND DISCUSSION OF RES It was noted that the TUGC0 audit reports do not include a formal Recommended Corrective I
Action for identified deficiencies.
The language of ANSI N45.2.12-1973 can ba interpreted either way as to whether er not I
this is a requirement.
The RTLplupports the TUGC0 position, although it is neutrary to experience, because the responsible organization is better able to determine the proper course of corrective action and is ultimately responsible for the final determination and implementation.
l The QA organization remains responsible for l
evaluation and approval of the course of
,}
corrective action as well as verification of effective implementation.
Audit 1
report distribution did not appear to be consistent in that the reports in some cases were addressed to the supervisor responsible for the activity requiring corrective action rather than to project canagement, although project management did receive copies.
Although this is technically 1 !
not a violation of requirements, it can caure project management to be omitted from direct participation in the reporting, investigation, and corrective action l
activities of the audit process.
l In addition, original recipients did not always receive subsequent correspondence related to i
follow-up and closecut.
Conclusions audit reports (e.g., audit scope, summary results, evaluation statement)_could hays.Jesulted in h ee contributAdlo_a lackan adverse effect on the audit pro SLlull understanding by
,(.
reader,of the adequacy and effectiveness of the QAthe progran as evaluated by the QA organization.
.)
P 4
?
e m e4 i
R1 vision:
}
Feg2 35 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) ~
5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It is concluded th,2t the failure to properly document audit deficiencies could have result,ed in an adverse tifect on the adequ'ic'y'sf the audit pr6 gram in that
'adiopriatc~managementmay'notlemadeawareofall items reguiring corrective action or of their
,77 m,
~
is cone'rded thak,the failure to document deviations It I
f rom the audit check 11:sts' could have resulted in an adverse effect on the audit program in that appropriate m'anagendlit"nilghY not be.made aware of program activities / elements which were not evaluated.
It is concluded that.the failure of the auditor to recommend effective corrective action for audit deficiencies resulted in no adverse effect on the audit program because the organization responsible for the activity has the ultimate responsibility for determination of corrective action.
It is concluded that the lack of consistent distribution of audit reports and subsequent correspondence cou.jf, have.. result.eLin.An. p.$ verse ef f ect f
on the adequacy of the audit program in that project management would not be involved in the activities pertcining to investigation, corrective action and closcout of audit deficiencies.
5.1.7 Foll)v-up and Closeout The final major step in the audit process is timely follow-up, by both the audited organization and the auditing organization.
The review of the sudit report files indicated that, throughout the program, the early steps of the follow-up process appeared to have been completed in a timely manner by both the audited organization and the 4
auditing organization. This included determination of.
and schedule for comp).etion of corrective action by the q
audited organization, evaluation of the proposed corrective action by the auditing organization and additional correspondence between the two, as required, until resolution was reached.
\\
a f
Pavision:
1 Peg 2 36 of 53 RESULTS REPORT ISAP Vll.a.4 (Cont'd) i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULT The timeliness of implementation of corrective action by the audited organization cannot be ascertained from the audit files for reasons that are discussed below.
Interviews with the Assistant QA Manager indicate that the general feeling within the auditing organization was that implementation of corrective action was timely. However, the timeliness of verification of implementation of corrective action by the auditing organization, as well as closeout of the audit deficiencies and audit reports was often unsa tis f ac to ry,.
This situation was eventually recognized and addressed by TUCCO QA through the performance of audit TCP-111 "Closecut of Open Construction Audit Findings" in l
August 1984 This audit was performed to verify implementation of corrective action for and._22._c.otteernm_e of which had been,83 defief enefsf.
open far-nver csix yearED During the audit, all but four (4) items (2 deficiencies and 2 concerns) were found to have been completed.
These four items were reissued as three deficiencies against audit TCP-111, which were then closed in a follow-up two monthe later.
The NRC addressed the inability of TUGC0 QA to close an audit finding from TCP-23 during the conduct of audit TCP-68.
The deficiency in question was subsequently identified as audit TCP-40, Deficiency 2.
It was determined that this deficiency was closed during the conduct of audit TCP-99 in March 1984 1
i 4 Contribution'gtca6's'eh to the lack of timely follow-up 41d i
werz a combination'of (1) the practice of each lead esditor being responsible for scheduling applicable follow-ups during subsequent audits, (2) related audits being scheduled up to a year later, and (3) lack of a formal tracking and status system for audit deficiencies.
The apparent lack of urgency for follow-up may have been influenced in part by the general feeling that implementation was indeed timely, and therefore additional action was not required to expedite implementation.
5 Procedure DQI-AG-1.8, " Internal Audit Deficiency 1
Follow-up/Closcout" issued in June 1985, addressed i
these problems by instituting a tracking system for internal audit deficiencies.
i described in the procedure is adequate.The tracking system l
g:
P.evision:
1 j;
}
)Page 37 of 53
\\'
y s
s RESULTS REPORT ISAP VII.a.4 (Cont'd) i 5.0. ~IMPLDENTATION OF ACTION PLAN Ado LISCUSSION OF RFSULTS (Cont'd)
~
i, ;
Conclusions
),'
si 1
It is concluded that the failure to tarfcrm timely 9'
verification of implementation of corrective action for audit deficiencies gould have resulted in_an adverse V
effeci on.the.aduluncy nf the audit progrgp fo that one 3
I' of the major gos14 of 4t sudit program is tp) identify
).
and ensup corrective accion for program dei k tencies.
s
[
y}
gs 5.2 Audit Persontril _ Qualification
)
i'
,s\\
3 Pugram documendfand audit personnel tra'ning/qudNfication s
fnes were rev.i' hid to evaluate the adequacy and effet tivenet:s t
I of the 1UGC0.progrrr for qualification and certit Leation of audit personnel. The results of these reviews are st.ti.rrrized r
in the succeeding paragraphs.
\\
y,
5.2.1 Qualification Requirements e
tt u
i i
, i
- i
,,t
!/
Prior ed June ?i83, TUGCO was committed to t.be requirer' ente et) ANSI N45.2.12-1973 for aucir, persennel trainind and qualification. The TUGC0 wtitden program t
f or this period was in com;11coce with the ANSI
\\;*
requirements.
i Ir was noted,from reviewing the training and..
s qualification files,for this period that, beginning in
,o 1977, a form sim*,iat 'to the form contained in MSI F :'
I S45.2.23 was' best g umai to document certifications for s
lead auditor. The use/bf the modified form vas
!! u of this form (for which there was yet 'no commitier,t) proceduralized in 1980.
It was noted that fn the use
+
e TUCCO was awarding qualification cre68ts f(d attributes in addition to those specified in ann)N45.2.23 and was 5
,i not requf;?ius the stie number of qualification credits x
l's v*
as was um ANSI.
i t a
[ *)
Revisnn 5 of CQI-QS'2.h " Qualification of Audit
'\\
Personpel," issued June 1, 1983, resulted in the 5'
\\
'w t.
writted progran being in compli, tace with ANST
[>
3 N45.I.23-M 78.
At this thA,5 r.udit personnel who had
- ,*i N
been qual u ted and certified under the previous s
\\
pr.ocedut : avision were reevaluated and new
\\
ed cific.ations issued. This t rocedces has since been uuelled and the appropriate reterial lusorporated into DQP-QA-2, Revision 11. The curce.nt precedures are in coapliance with governing requirements.
s
-~
l Ravision:
1 Pags 38 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Conclusions It is concluded that the audit personnel qualification program adequately reflected the requirements of the appropriate governing standards and regulatory guidance and therefore resulted in no adverse effect on the audit program.
1 5.2.2 Training / Qualification Records t
i of approximately fifty (50)' personnel identified as having performed as auditors or lead auditors since 1978, the personnel qualification files of 41 of these individuals were reviewed for evidence of education, training and experience used as a basis for qualification and/or certification. The personnel reviewed were all members of the TUGC0 QA organization, though not necessarily assigned to the audit staff.
Included in this total are all who performed as f
auditors or lead auditoro since 1981, as well as some who had participated earlier.
All files, except as discussed below, contained adequate documentation (e.g., resume,~ training records, i
reading lists, certifications, etc.)
appropriate to the time of certification, including
. evaluation and decertification of personnel on-board on June 1, 1983 (Date of Commitment to ANSI N45.2.23-1978).
Files for six personnel contained no resume or other independent supporting documentation for lead auditor certifications issued in 1977, 1978, and 1981. None of these personnel have participated in the program since 1982. Although no judgement of competency of these personnel can be made from the records reviewed, others certified during this period were determined to be adequately qualified, based on their records.
Review of Lesd Auditor qualification files indicates that all personnel currently certified as Lead Auditor, except as noted below, have satisfied the requirements of ANSI N45.2.23-1978.
j Although there is no regulatory requirement to verify the education, experience, etc., for audit personnel, it was noted that TUGC0 QA is in the process of performing these verifications for the current staff and has incorporated such a! requirement in DQP-QA-2.
e e
i T
Revision:
1 Pass 39 of 53 RESULTS REPORT ISAP VII a.4 (Cont'd) 5.0 IMPLDENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Con ANSI N45.2.23-1978, Section 4.2 states that the development and administration of the examination for lead auditor is the responsibility of the employer.
It also states that this activity may be delegated to an independent certifying agency but that the employer chall retain responsibility for conformance of the examination and its administration to the standard.
The applicable TUGCO procedure does not specifically reflect this requirement, although it could be interpreted to do so.
During the review of qualification files of current staff members, it was observed that at least three
~
personnel had been granted certification as lead auditor who had no record of being administered an examination by or for TUGCO.
They had been given credit for examinations administered by previous employers.
It is believed that this action has not resulted in the certification of unqualified personnel, but it constitutes a violation of the requirements of ANSI N45.2.23-1978.
Based on the review of audit personnel training and I
qualification files, it was concluded that with the exception pertaining to the examination of Lead Auditors, the TUCCO program for qualification and certification of audit personnel was satisfactorily implemented.
Conclusions It is concluded that independent backup information not being in the files for auditors in the past resulted in no adverse effect on the audit program in that the files did contain evidence that QA management had performed an evaluation of the personnel and found them to be qualified.
It is concluded that the failure of TUGC0 to administer lead auditor examinations to particular individuals resulted in no adverse effect on the audit program in that the personnel are otherw1se qualified and are believed capable of satisfactorily performing as lead auditors.
Revision:
1 Page 40 of 53 REStTLTS REPORT ISAP VII.a.4 (Cont'd) 1 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 5.3 Evaluation of Findings V:
This section summarizes the deviations and weaknesses identified throughout this report. Each item includes the 7
z.
determination of whether or not it was considered to be a Og' QA/QC program deviation, state,. a recommended corrective action if appropriate, or describes corrective action which has already been taken.
~
The corrective actions describe below are intended to preclude recurrence of the identified deviations.
It has been i
determined that remedial corrective aution would provide no additional benefit to the QA Audit Program.
Audit Program I
)'-
10CFR50, Appendix B, Criterion I, etates in part, f
the authority and duties of persons nad f
organizations performing activities affecting the safety-related functions of structures, systems, and j
components shall be clearly established and delineated in writing... " Criterion XVIII states in part, "A comprehensive system of planned and periodic audits shall be carried out..."
Contrary to the above, the CPSES FSAR does not address l
the organizational responsibility for the definition and implementation of the audit program.
This is a QA/QC Program Deviation.
Therefore, it is recommended that the CPSES FSAR be revised to define the responsibilities for the audit program. A proposed change to the FSAR has been submitted to TUGC0 licensing, has been reviewed by the RTL and found to be acceptable.
p' 10CFR50, Appendix B, Criterion I, states in part,
"... the authority and duties of persons and organizations performing activities affecting the 4
safety-related functions of structures, systems, and components shall be clearly established cnd delineated in writing..." Criterion XVIII states in part, "A comprehensive system of planned and periodic audits shall be carried out... "
Contrary to the above, the QA program documents did not i
delineate responsibilities for the Supervisor, QA Audits.
~
Ravision:
1 Pags 41 of 53 I
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLDIENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
This is a QA/QC Program Deviation.
Corrective action was taken in DQP-QA-16, Revision 0,
)
issued in September 1985. This has been reviewed and found acceptable.
3-10CFR50, Appendix B, Criterion V, states in part,
" Activities affecting quality shall be prescribed by documented instructions, procedures... of a type appropriate to the circumstances..."
ANSI N45.2.12-1973, paragraph 3.2, states in part, "An effective audit system...shall include.
provisions for reasonable and timely access of audit personnel to facilities, documents, and personnel necessary in the planning and performance of the audits...."
l ANSI N45.2.12-1973, paragraph 4.5.1, states in part,
. management of the audited organization or activity shall review and investigate the audit findings to determine and schedule appropriate corrective action. They shall respond to the report in i
y writing, within 30 days after receipt."
The " Gray Book", paragraph D.2.b, states in part.
.during the [ post-audit] conference an effort i
shall be made to establish a tentative course and schedule of corrective action for non-conformances.
Where it is not possible to provide such information at the post-audit conference, the management of the audited organization should commit to a specific date for determination of the course of corrective action and the schedule for implementation."
Contrary to the above, neither the TUGCO QA Program nor the CPSES QA Plan, the documents in the written program which would provide direction to the management of the audited organizations, addresses the requirements.
This is a QA/QC Program Deviation.
It is recommended that appropriate QA program documents (e.g., Corporate QA Program, Site QA Plan) be revised to include these requirements.
The TRT concluded that the QA audit program had linadequate staffing during peak periods.
I i
r[
R;vicion:
1 Paga 42 of 53 y.. ez-l "%,
'4g RESULTS REPORT
@.<., kp4 ISAP VII.a.4
,y.k-z[,
(Cont'd)
+
,. sq L IT) i i
~
5.0 1 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
', s i
!ThisNRCfindingwasevaluatedbyconsideringboththe jdesignated audit staff and other members of the QA organization who performed as auditors. The total number of participants in audits shows that the audit staff was adequate in numbers and qualifications.
,Accordingly, this NRC Finding was not substantiated.
ANSI N45.2.12-1973, paragraph 3.4.2, states in part,
" Applicable elements of the Quality Assurance Program shall be audited at. least annually or at least once within the life of the activity, whichever is shorter."
Contrary to the above, internal annual audits were not specified in procedure from'1981 to 1984.
This is a QA/QC Program Deviation. The NRC finding in this area was substantiated.
Corrective action was taken in DQP-CS-4, Revision 11 issued in November, 1984. This has been reviewed and found acceptable.
?
ANSI N45.2.12-1973, paragraph 3.4.2, states in part,
" Applicable elements of the Quality Assurance Program shall be audited at least annually or at least once within the life of the activity, whichever is shorter."
Contrary to the above, vendor annual audits were not specified in procedures.
This is a QA/QC Program Deviation. The NRC finding in this area was substantiated.
Corrective action was takeh in PSAR Amendment #52 issued in August 1984 and DQP-CS-4, Revision 11 issued in November, 1984 This has been reviewed and found acceptable.
4 *-
ANSI N45.2.12-1973, paragraph 3.4.2, states in part.
" Applicable elements of the Quality Assurance Program shall be audited at least annually or at least once within the life of the activity, whichever is shorter."
Contrary to the above, annual audits were not performed for contractor organization in two (2) instances out of sixteen (16) required audits from 1973 to 1977.
M h, f U; i..)
Rsvision:
I Pega 43 of 53
)
ea
..x..
j RESULTS REPORT
.4
't
.e
.r.
- , I*
h.
ISAP VII.a.4 g
(Cont'd) m.,$
t 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
This is a QA/QC program deviation. The NRC finding in this area was substantiated.
Corrective action was taken by QA management in that for the remainder of that phase of the TUGC0 audit program, audits of major contractors were performed at least annually.
ANSI N45.2.12-1973, paragraph 3.4.2, states in part, 7
" Applicable elements of the Quality Assurance Program shall be audited at least annually or at least once within the lif e of the activity, whichever is shorter."
Contrary to the above, annual vendor audits were not performed in that indtiation was not timely and/or intervals were too large in all twelve (12) organizations examined between 1976 and 1984.
This is a QA/QC program deviation. The NRC finding in this area was substantiated.
Corrective action was taken in DAP-CS-4, Revision 11, issued in November 1984. This has been reviewed and found acceptable.
gi -
[ NRC Notice of Violation (445/8432-03, 446/8411-03) stated that the E site organization was not audited between 1971 and 1981.
It was determined that the nature of the work activities for the E site organization did not involve l
control of safety-related activities and therefore is not required to be included in the audit schedule.
Accordingly, this NRC Pinding was not substantiated.
ANSI N45.2.12-1973, paragraph 5.3, states in part,
" Planning of the audit system shall be conducted to assure covera'ge of the applicable quality assurance program... and in paragraph 3.4.2, " Audits shall
... assure conformance with the developed and implemented program."
Contrary to the above, audit checklists were developed from existing procedures rather than governing requirements for the majority of audits reviewed. For example, no assessment of the written program adequacy was performed in areas of non-conformance or corrective action or QC Training / Qualification / Certification.
.MMW
.j R:vicion:
1 Pags '44 of 53 RESULTS REPORT ISAP VII.a.4 i
(Cont'd)
?
LAMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
This is a QA/QC Program Deviation.
Corrective action consists of use of an " Appendix B checklist". This has been reviewed and found acceptable. However, for program enhancement, DQP-AG-2 should be revised to reflect use of this checklist,
~
y, Checklists did not provide verification that procedures in different systems were consistent for the same
- g 7
activities performed by different organizations.
This is not a QA/QC Program Deviation.
Cf' -
ANSI N45.2.12-1973, paragraph 1.5.1, states in part,
"[An] Audit [1s] a formal documented activity performed in accordance with written procedures or checklists to verify, by evaluation of objective evidence that a quality assurance program has been... implemented in 1
accordance with applicable requirements of ANSI N45.2.
Contrary to the above, the audit preparation phase was less than adequate because audits did not provide i
sufficient checking of program adequacy for the activities being audited and resulted in incomplete evaluation of the adequacy of the overall QA/QC
]
prograra.
This is a QA/QC Program Deviation.
Corrective action consists of the use of an " Appendix B Checklist." This has been reviewed and found acceptable. However, for program enhancement, it is recommended that procedure DQP-AG-2 be revised to reflect the use of this checklist.
10-
^"SI "'5.2.12-1973, paragraph 4.2.2, states in part, responsibilities (of the audit team leader) include orientation of the team,..., assuring communications within the team and with the organization being audited, participation in the audit performance, and coordinating the preparation and issuance of reports."-
i 4
a 7
.m.
,-e.
Rsvision:
1
's Paga 45 of 53 2
V, ;
?
.,4 RESULTS REPORT S"
p ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION, PLAN AND DISCUSSION OF RESULTS (Cont'd) l
/
ANSI N45.2.23-1978, paragraph 1.1, states in part
^
" requirements... for the qualification of audit team leaders, henceforth identified as a " Lead Auditors". who organizes and directs audits, report audit findings, and evaluates corrective action."
Contrary to the above, Lead Auditor candidates were l
designated as acting team leaders (with signature j
authority).
This is a QA/QC Program Deviation.
l Corrective action was taken in DQP-CS-3, Revision 12,'
issued in February 1985. This has been reviewed and found acceptable.
((' -
ANSI N45.2.12-1973, paragraph 4.4, states in part "An audit report shall be written and signed by the audit team leader which provides:
Description of the audit scope l
A summary of audit results including an evaluation statement regarding the effectiveness of the QA program elements which were audited."
Contrary to the above, audit reports did not include scope description and evaluation statements.
This is a QA/QC Program Deviation.
i Corrective action was taken in CQI-CS-4.6, Revision 0, issued in July 1981. This has been reviewed and found acceptable.
Audit files contain instancec of:
~~
Field notes on identified conditions that are not properly addressed in the final report.
t Checklists that are not completed.
I
_.__.._____________i
Revision:
1 Page 46 of 53 i
{
RESULTS REPORT i
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Qg This is not a QA/QC Program deviation, but indicates
.6 d implementation weaknesses in that the documented record C
I kesnotreflectallactionsoftheauditteam.
ji-ANSI N45.2.12-1973, paragraph 4.4, states in pert,
" Distribution of the [ audit] report shall include responsible management."
Contrary to the above, original recipients did not always receive subsequent correspondence related to follow-up and closecut of audit reports.
This is a QA/QC Program Deviation.
Corrective action was taken in DQP-AG-5, Revision 0, dated September 9, 1985. This has been reviewed and found acceptable. However, for program enhancement, applicable procedures should be revised to define the addressecs and minimum distribution for audit reports.
/-Q -
ANSI N45.2.12-1973, paragraph 4.5, states in part,
" Follow-up action shall be performed by the audit team leader or management of the auditing organization to:
. confirm that corrective action is accomplished as scheduled."
Contrary to the above, QA verification of timeliness of closecut was unsatisfactory although audit personnel believed that actual implementation of corrective action was timely.
This is a QA/QC Program Deviation. The NRC finding in this area was substantiated.
Corrective action was taken in DQI-AG-1.8, Revision 0, issued in June 1985. This has been reviewed and found acceptable.
Audit Personnel Qualifications i
The procedure for qualification of audit personnel contained weaknesses in that, for axample, i
OJT could.be performed under qualified auditors in addigion to Lead Auditors.
l i
-~R Rsvision:
I 4
,. 7 Paga 47 of 53 1
RESULTS REPORT y
%~'
?
ISAP VII.a.4 j
$b (Cont'd) lG l? -
jy q
5.0 IMPLEMENTATION OF ",CTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
The required reading list was incomplete.
There was no stated requirement to retain information pertaining to examinations.
r This is not a QA/QC Program Deviation.
When procedure DQP-QA-2, Revision 11 was issued in yecember1985,theweaknesseswerealiminated.
[Six(6)auditpersonnelqualificationfilesinthe l period prior to June 1983 are incomplete.
This is not a QA/QC Program Deviation.
(y-ANSI N45.2.23-1978, Paragraph 4.2, states in part, "The development and administration of the examination for lead auditor... is the responsibility of the
{
employer.
Contrcry to the above, at least three members of the l
current audit staff have been certified as Lead Auditors by TUGCO, but have not been administered an examination by, or for. TUGCO.
This is a QA/QC Program Deviation.
It is recommended that these personnel, and any others i
currently certified who did not meet the requirements, be administered an examination in accordance with the requirements of the standard.
5.4 Trend Analysis A total of fourteen (14) QA/QC Program Deviations were identified. These findings are characterized as failures to fully translate audit program requirements into the procedures used to implement the program. This can also be characterized as a lack of specificity in the CPSES implementing procedures. This therefore resulted in a failure to properly implement the audit program.
Because each of the deviations can be traced to inadequacy of procedures, this has been determined to constitute a trend.
Therefore a trend analysis has been' performed in accordance with Appendix E of the CPRT Program Plan.
.4 Ih.
R1 vision:
I
- .:.d.
g
- ,p Pcgs 48 of 53 y
RESULTS REPORT I
ISAP VII.a.4 (Cont'd) n 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) i None of the deviations described in Section 5.3, taken individually, is believed to represent a significant breakdown j
in the audit program. However, taken collectively, there was a potential for breakdown in the audit program. Therefore this is considered to be a trend adverse to the audit program.
As noted previously, there is no direct connection between an audit program and the quality of installed hardware.
For this reason, it cannot be concluded that this trend is an " adverse j
trend" as defined by Section 5 of Appendix E of the CPRT j
Program Plan. However, an inadequate audit program can fail to identify weaknesses in the prescribed QA/QC programs that, in turn could allow hardware problems to go undetected.
For this reason the RTL has determined that a root cause and generic implications analysis should be performed.
5.5 oot C E D and Generic Implication Evaluation The NRC findings led to two hypothesized root causes which
- 1) TUEC's audit procedures did not comply with NRC vere:
requirements and 2) the program was not implemented in I
accordance with procedures. The CPRT evaluation also found instances of failure to implement governing requirements properly, but in each case this was traced to a lack of
(
specificity in the implementing procedures for the CPSES Quality Assurance Audit Program. Therefore, all s' substantiated findings can be traced to the h of NRC'_s hypothesize _d root fthe lack of~ procedural compliance with NRC requirements w causes.
'\\
f caused by lack of a management system which would provide for overall coordination and control of project activities and provide a consistent mechanism for translation of governing I
requirem nts into hoplementing procedures and by failure to l
periodically check program procedures against program j
requirements and commitments. This lack of procedural compliance remained uncorrected because of the apparent lack f of full appreciation by Brevioiib TUEC management of the role of an effective QA Audit Program in ensuring the overall effectiveness of the CPSES QA Program and therefore is considered to.be.the_ root cause. [
o The generic implications ~of this root cause will be evaluated during the conduct'oT~6ther ISAPs and will De considered
-~
during the collective evaluation phase of the QA/QC Program.
e
Revision:
1
.J.
Paga 49 of 53 a
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 1 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS _(Cont'd)
A @ antive contributing em @ for the lack of a full
~
assessment or the completeness of QA Audit Program implementing procedures is determined to be the lack of a procedural hierarchy for CPSES. A procedural hierarchy would provide for at least one additional e4*r of pr Q which would nrovide a-br.idge._from tite recuirQs doments to the h ieA.nring procedures, and in addition would provid C vehicle for management to describe the overall coordination and control of project activities and organizations for topics such as the following:
Interpretation and application of commitments to codes, standards and regulations, l
Definition and application of self-imposed requirements, t
Standardization of terminology and definitions.
Delineation of responsibilities, j
1 Description and responsibility of organizational i
interfaces and
)
l Structure and content of sub-tier procedures.
A(potTn'tial_Reneric_1mplient,iog)is that the types of t
(
l procedural inadequacies identified in the audit program may
{
exist in other procedure systems at CPSES. The lack of an l
upper tier of procedures covering the topics identified above J
1eads in prortduye__ development by each organization without f_ormal c_orp. orate guidance or forma 1_ coordination between j
organizations. Further investigation is required to determine if this lack of standardization contributed to similar procedure inadequacies in other areas or organizations.
Further investigation is beyond the scope of this action plan and will be addressed as part of Collective Evaluation of the Construction QI/QC Program using information from this and
~
other ISAPs.
i corrective actions related to the adequacy of the current CPSES QA Audit Program have been identified. Additional actions to prevent future occurrence of similar deviations or programmatic weaknesses are discussed in Section 8.0 below.
t i
6,,
Ravision:
1 Pago 50 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S Section 4.1.2.3 of ISAP VII.a.4 describes conditional steps that were to be performed depending on the results of reviews and evaluations. Although audit program deviations and weaknesses related to construction activities were identified, it has been determined that no action is required beyond that specified in f
ISAP VII.c, which addresses all areas of safety-related hardware and is intended to provide confidence that any currently j
unidentified concerne related to the quality of construction of the CPSES hardware will have been identified, evaluated and resolved.
Therefore, no additional programs or plans were developed to I
address this area. Audit program deviations applicable to off-site TUCCO suppliers were also identified. During the investigation and evaluation of the impact of these deviations, sufficient information was obtained to provide confidence that the acceptability of the suppliers' quality assurance programs for the applicable equipment and services could be. and was, evaluated b'y~
T11GC0 during the periods irtcuestion. Therefore. nn adaftinngJ programs or plans wetc._dey_ eloped en determine the acceptability of l
tht._ suppliers' quality _ assurance programs based on the efforts 6f l
qt.heL%
l As a result of this review of the TUGC0 QA audit program, it has y
been determined that at no time has the written program been in (ull compliaru;Le with RpXsrning stancaras and regulatory guidatice, and in addition has exhibited other weaknesses as described in'this report. This determination substantiated the TRT finding that
. TUEC's audit procedures did not comply with NRC requirements."
The ov,eral. Effectiveness of the_ audit nrogram han_ha* @
f,ully adequate. Specific examples are: it has failed to identify and cause corrective action of inadequacies in such areas as QC inspector training, qualification, and certification; and control of non-conforming items and corrective action. It is concluded, from the evidence observed, that the failure to identify and cause corrective action in these two areas was due primarily to the practice of auditing to existing procedures while not performing verification of the adequacy of existing procedures to implement program requirements.
It is further concluded that the cause of the deviations and weaknesses in the QA audit program which have been identified in this report are the result of inadequate procedures. This resulted from the lack of a procedure hierarchy which would have provided for overall coordination and control of project activities and organizations by senior management for topics such as the following:
?
l Rsvision:
1 c
P:g2 51 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 1
6.0 CONCLUSION
S (Cont'd)
Interpretation and application of commitments to codes, standards and regulations, 1
Definition and application of self-imposed requirements, f
Standardization of terminology and definitions, i
(
Delineation of responsibilities, Description and responsibility of organizational interfaces j
and 1
Structure and content of sub-tier procedures.
Based on the written record the overall competency of the audit f
personnel appears to be adequate to perform the required audits for the design, construction, and operation phases of CPSES. Except as previously noted, the program for qualification of audit personnel 4
is considered to have been adequate, although TUGCO was slow to i
recognize and incorporate applicable standards and guidance available to the nuclear industry.
i i
The NRC TRT and Region IV findings concerning auditor staffing and qualifications during 1981, 1982,'and 1983 could be considered i
valid if the program had been totally dependent on the personnel j
assigned to the permanent audit staff. However, in light of the i
total number and_qualificattorys of personnel available to._ and 4 (-
u.tiliz e d,_inn the_ audit. program,_the audit staff is considered bojt) adequate and qualified.
qg D
~
Audit planning and scheduling, though in the past not in compliance regarding frequency and not formally systematized until recently, j
did appear to be well thought out in the context of the TUGC0 concept of the audit program requirements at any particular time.
j However, due to the lack of a procedure hierarchy which would i
provide a firm basis for the development of implementing j
i procedures, the effectiveness of the audit program to determine the adequacy of the construction QA program was less than desirable, There is evidence that appropriate consideration has been given to j
input from sources such as NRC inspections, previous audits, and i
surveillance.
J Except as noted, the findings pertaining to scheduling of audit I
frequency are valid.
Appropriate procedures have been revised, which now properly address scheduling of audit frequency.
1 1
I i
_ _____..__-A
R: vision:
1 Paga 52 of 53 d'
s RESULTS REPORT ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S (Cont'd)
The current written program for auditing and audit personnel qualification is in compliance with applicable requirements except j
in the following areas l
1
- Definition of responsibility for the audit program in the FSAR l
l Definition of responsibilities of the audited organization In addition, at least three currently certified lead auditors have l
not been administered an examination by or for TUCCO.
Appropriate corrective action should be taken in these areas.
7.0 ONGOING ACTIVITIES i
The Audit Program and procedures discussed in this report are applicable to the remaining construction phase on Unit 2 and for
]
the operating phase of the plant.
8.0 ACTION TO PRECLUDE OCCURRENCE IN THE FUTURE To prevent future occurrence of the same or similar deviations and weaknesses in the QA audit program (or other portions of the CPSES program) management, particularly senior management (Vice President and above), must actively participate in the program and be kept appraised of audit program results.
This may be accomplished through development of a hierarchy of procedures as described in Section 6.0 of this report. This would provide the necessary standards and management controls under which the overall program can be effectively defined and implemented.
In addition, the procedures should provide for direct senior management participation in the audit program in areas such as audit planning and scheduling, and resolution of programmatic problems.
Management at all levels must utilize the output of the audit program (audit reports, summaries, trend analyses, etc.)
to evaluate the adequacy and effectiveness of their portion of the QA This should be accomplished by developing requirements program.
for appropriate participation in the development of corrective action for identified deficiencies, and follow-up to ensure that the corrective action has been effectively implemented.
I i
~
b hm--_____
Revision:
I g7 Paga 53 of 53 i
RESULTS REPORT l
l ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S (Cont'd) 4 1
In addition, an independent, periodic assessment of the audit program is essential to provide Senior management with timely I
identification and corrective action of deficiencies or weaknesses in the audit program. Such a program will be defined in ISAP j
VII.a.5.
I i
i l
I I
I