ML103440443
| ML103440443 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 06/09/2010 |
| From: | Price D, Colville K, Rajkowski L Public Service Enterprise Group |
| To: | Office of Information Services |
| References | |
| FOIA/PA-2010-0334 70109827 | |
| Download: ML103440443 (35) | |
Text
SALEM ASME SECTION XI IN-SERVICE INSPECTION (ISI) IMpLEMENTATION 701098271 Root Cause Evaluation RCE Team Lead:
Sponsoring Managers:
WUMaM in tw rewO I*,t dFWW i
.a:,:o*,y 7
Kerry Ann Colville Leonard Rajkowski Devon Price Page 1 of 35
TABLE OF CONTENTS.,
1.0 EXECUTIVE
SUMMARY
3 2.0 EVENT DESCRIPTIO N...................
...................................... 4 3.0 ANALYSIS.........................................................................
4 4.0 EV A LU A T IO N.....................................................................................................
5 5.0 EXTENT O F C O ND ITIO N.....................................................................................
8
/
6.0 RISK ASSESSMENT........................................................
10 7.0 PREVIO US EVENTS 10....
10 8.0 CORRECTIVE" CTIONSTO PREVENT RECURRENCE(CAPRs)..................... 11 9.0 CO RRECTIVE ACTIO NS.......................................
11 10.0 EFFECTIVENESS REVIEW S (EFRs).................................................................. 13 11.0 ORGANIZATIONAL AND PROGRAMMATIC ISSUES........................................ 14 12.0 SAFETY CULTURE ANALYSIS 15 13.0 OTHER ISSUES.................................................
15 14.0 COMMUNICATION PLAN.....................................
16 ATTACHMENT 1 DETAILED TIMELINE OF, EVENTS........................
17 ATTACHMENT 2:
EVENT AND CAUSAL FACTORS............................
19 ATTACHMENT 3:
BARRIER ANALSIS.........
.............................. 20 ATTACHMENT 4:
TASK ANALYSIS................................................................................
21 ATTACHMENT 5:
MISSED OPPORTUNITIES................................................................
22 ATTACHMENT 6:
EXTERNALOPEX..........
.............................. 27 ATTACHMENT 7:
ROOT CAUSE INVESTIGATION CHARTER.......................................
31 ATTACHMENT 8
SUMMARY
OF RESPONSE TO CHARTER ITEMS:........
33 ATTACHMENT 9:
CHECKLIST.............................................
34 Page 2 of 35
.1
~
Title:
Salem ASME Section X1 In-service Inspection (ISI) Implementation Unit(s): Salem Units I and 2 Event Date: (See report details for individual event dates)
Event Time:.
Action Tracking Item Number: 20459689 70109827 Report Date: 06/09/2010 Sponsoring Managers:
Len Rajkowski - Director, Salem Engineering Devon Price - Manager, Outage Services Investigators:
Full Time:
Tim Giles - SA ISI Program Manager William Treston - Outage Services Dan Maxey - SA Engineering Programs Erin West - Licensing Part Time:
Kerry Ann Colville-Manager, SA Engineering Programs (Root Cause Qualified)
Mark Wagner-Human Performance Ed Maloney - Engineering Services, Programs (ISI)
Harry Smith-Exelon Peer (ISI) 1.0 EXECUTIVE
SUMMARY
On 5 April 2010, during the Salem Unit I Refueling Outage (1R20), a portion of the Auxiliary Feedwater System (AFW) buried piping was excavated and inspected in accordance with the Buried Pipe Program. The inspection revealed general corrosion on several sections of the piping supplying AFW to 12 and 14 Steam Generators.
A review of AFW piping inspection history to support past operability was initiated. Through this investigation it was determined that periodic pressure testing required under ASME Section XI IWA-5244, Buried Components, had not been performed during the first, second, or third ten-year inspection intervals. Additionally, no approved.
alternative testing method had been requested in accordance with IOCFR50.55a.
As a result, it was determined that a root cause evaluation (RCE) would be conducted to determine why the ASME Section XI periodic pressure testinigsfor buried piping Was not conducted on the AFW system, why no.
relief request was submitted for an alternative, and why this p6rtion of pipe was not identified as buried during previous Visual Test (VT-2) for leakage examinations.
After an investigation of documents spanning. more than:30 years plant history andpersonnel interviews, the RCE team determined that the root cause for this event is Shortcomings' in technical human.performance practices resulted in incomplete initial and subsequent ISI program plans; this coupled with a lack of questioning attitude by ISI inspectors resulted in the failure to perform the required'ASME Section XI testing.
Page 3 of 35
Corrective actions include requiring a formal pressure testplan including hand over hand walk downs, -boundary validation, and~fornmal acceptance'criteria for buried piping tests during each 10 year interval update and completing the current 10 year update to these requirements.
An extent of condition review ws peroed. Both Unit. and Unit buried iping aot een tested in accordance with ASME Section XI requirements. During 1R20, Unit I AFl W buried pipingwasreplaiced and-pressure tested in accordance with ASME Section XI requirements. Arisk assessment for Salem Unit 2 missed,,
pressure test was conducted. The results of that assessment concluded that deferral of the pressure test for a period of up to the end of the current operating cycle is acceptable and results in a negligible increasleinirisk.. During 2R18, Unit 2 AFW buried piping will be pressure tested. Additionally, pressure test boundary drawings were reviewed to identify other potential buried piping that would fall under the same pressure test. Other piping identified includes Service Water (SW) headers on both units. Review and evaluation of the buried portion of the Service Watersystem determined that due to the excessive leakage of the SW21 valves validated by In-Service Test (IST) data the system is determined to be non-isolable. Therefore, the requirements of I.WA-5244(b)(2) are applicable "The systein pressure test for non-isolable buried components shall consist of a test.to confirm that flow during operation is not impaired*
IS.ST data has shown that the:sral integrity of the SW systefinis satisfactorY. The extent of condition review determined that no other piping syset*
stwere a identified as buried at Salemnunits i and2.where.the condition identified would apply.
A review of the applicable Operating Experience did not identify plants that had similar experiences such' that the corrective actions would have prevented this event. Through a reviewof NRC integrated inspection reports the team identified that other plants received non-cited violations for issues similar to those identified in this report.
The NRC has identified a preliminary green non cited violation for this event. Though past Owner Acceptance Reports (OAR) had incorrectly stated that the AFW system had been tested in accordance with ASME Section XI testing requirements, this event has been determined to be non reportable.
2.0 EVENT DESCRIPTION:-
On`5 April 2010, during the Salem Unit 1 REfueling Outage (IR20), a portion of the Auxiliary Feedwater System
. -(AFX)buried pipig was excavatedatd inspected inf oi:daiae with the BuneieWaieeProgram.The inspection revealed genieral corrosion on several sections of the piping supplying AFW to 12 and 14 Steam Generator.
Salem Unit 1 AFW buried piping was replaced and pressure-tested during the outazge.
A review of AFW piping inspection history to support past operability was initiated. Through this investigation it was, determined that periodic pressure testing required under ASME Section XI.IWA-5244, Buried Components, had not been' performed during the first, second, or third ten-year inspection intervals. Additionally, no approved alternative testing method had been.requested in accordance with I OCFR50.55a.
Detailed Time Line (see Attachment #1)
3.0 ANALYSIS
LS-AA-125-1001, Revision,7, RootCause Analysis Manual.was reviewedcrtoi dwtrine whichimethods were most, appropriate for the analysisof tllis-eVent and details to conduct were outli'ned in LS-AA-125-l001..Eve6its anidCausal Factors Charting, TapRooT, Bariier Analysis and Task Analysis were chjosen as the best tools forthis event-.-
Basis for Using the Event and Causal Factors Chart (E&CF)Technique (See Attachmenit#2'for the completed E&CF)
This technique was used to provide atimeline from the beginning of the first interval long term plan until discovery of the missed inspection of the AFW buried piping. Results of utilizing this method provided for a detailed review of the event and the identification of causal factors.
Basis for Using the TapRoot Technique Thisý"techhniquei was used to provide aA innter-viewing and investigation strategy used inthe development of the root and contributing taiises. Results of utilizihg this method provided the cause codes for trendingpurposes, Page 4 of 35
Basis fgorUsifig the Barrier Analysis Technique (See Attachment #3 for the con'pleted Barrier Analysis)
This" t'echIniLue was used to identify failed or ineffective bairriers, how the barrier failed, why.the barrier failed and the corrective actions to strengthen the barrier. This process provided tile appropriate structure for analyzing and determining corrective actions.
Basis for Using the Task Analysis Technique (See Attachment #4 for the completed Task Analysis)
This technique was used to identify the tasks involved in implementing the ISI program system pressure tests. This process helped identify causal factors and areas for improvement.
Additionally, the MRC approved RCE charter (Attachment #7) identified other aspects that required investigation.
The'team investigated each item in the charter. Attachment #8 contains a summary response to the findings of each of the charter items.
4.0.'
EVALUATION:
.. Cause (describe the cause and Problem Statement identify whether it is a root Basis for Cause Determination
".cause oreconlrititing cause)
Periodic pressure tests have never been performed on the buried portions of AFW system piping at Salem as required by ASME Section XI code.
Shortcomings in technical human performance practices resulted in incomplete initial and subsequent ISI program plans; this coupled with a lack of questioning attitude by ISI inspectors resulted inthe failure to perform the required ASME Section Xl testing.
ROOT CAUSE Management, System (3M)
SPAC NI (4SL)
Technical Error (5TE)
Management System (3M)
SPAC Not Used (4SN)
Accountability NI (5AL)
Work Direction.(3D)
Selection of Worker (4SW)
Team, Selection (5TS)
Work Direction (3D)
Supervision During Work (4SD)
Program development and updates (5TS, SAL, STE):
PSEG has historically relied heaivily on the use of vendors to implement the ISI Programs at both Hope Creek and Salem. The first ten-year ISI interval Long Tenn Plans at Salem were developed and implemented by South West Research Institute (SWRI). This vendor was contracted to perforn the end of interval close out reviews during the close out of the first interval long term plan which ensures all code requirements are being met. The vendor was an expert in Class I and 2 pipe, but not on pressure tests for Class 3 piping.
The Manager in charge of the ISI groupduring the first and second intervals had assigned two separate, ISI engineers to develop the second ten-year long term plans for both Salem units. The assigned ISI engineers were not experienced in developing ISI long term plans and relied heavily on vendor support throughout the update process to ensure code requirements were being met. During the second ten-year interval close out and development of the third interval, vendors were used for the interval close outs as well and the development of the third ten-year intervals. Each program update was an opportunity to identify the missed pressure teIst since each update requires.the reviewer to create a plan for testing sUch~as system pressure tests for the entire interval to ensure program requirements are met.
Therew as not an appropriate level of verification and validation being conducted of the vendors work.
Under our current procedures, HU-AA-1212 is being utilized to determine proper level of oversight and review of the current updates/revisions. No other corrective actions will be assigned.
Pressure test walk downs (5TS, 5AL, 5TE):
The site pressure test procedure provides clear direction on the requirements for testing buried components although during' first; second, and third Isi Inter vals the pressure test visualin'st'ection for leakage did riot identify it**tportion of the A17W piping as buried. Procedures used to'coiimplete Page.5 of 35
2
~
VT72 examinations dui-ing systein pressure tests of AFW piping were not followed with* sufficient technical rigor, questioning attitude, or attention to'detai. hIn preparation for system pressure tests work packages for the test were not reviewed in sufficient detail and without sufficient technical rigor as required by the applicable procediure (see Attachment #4 Task Analysis). A thorough review of the system pressure test drawing for AFW Piping-would have
.identified a portion of the piping as yard pipe. Walk downs in the, field affter review of the w6rk package, with sufficient tebchnical-rigor and atteitionto detail would'have identified that the section of theý 4VW piping denoted as "yar#d' in the P&ID as buried. The procedure used for VT-2 examination clearly states different.requirements for buried components for system pressure tests. Technical rigor was insufficient during system pressure test walk downs since it is clearly stated that a reView of work packages is required and the system pressure test is different for buried piping than non-buried piping.
AFW1 pipe was noted as buried in 1V interval relief request (E)It wa identified in the relief request subihitted t Q
{eNRC in, 1988 for Unit I by the eHyrostatic Test grofupi tsepaatefrom ie iS I group)to perform the
-hydrosiatic test using.gan alternative method and during the liydrostatic test pefo0rmed in 1987 that a portion of the AFW piping was identified as buried. The AFW piping for Unit 2 was also identified as buried iii the test report d6ciimented in 1992 for the hydrostatic test. The periodic pressure tests.were noit performed properly for the buried section of AFW piping prior to the two tests noted above.
Prior to the hydrostatic tests conducted for the AFW buried piping it was not identified that a pressure drop test was required every period by ASME Section X1.
This is a legacy,ssu'Suand no corrective actions will be assigned to addrsess.
4
+
'Peri6dic pressure tes'tsha'e*
never been performed on the buried portions of AiW system pip,i*
ng at Salem as required by ASME Sec'tion' XI code.
ineffective Assessments
'Contributing Cause #1 Management System (3M)
Oversight / Employee Relations (40E)
ýAudits anid Evaluations Lack Depth (5LD)
,Basis CC#1 FASA of pressure tet (vertical slice) (5LD): From July 21 through August8, 2003;,Innovative Technology Solutions Corporation (ITS Corporation)conducted an independent a'ssessmient of PSEG Nuclear's Salem and Hope Creek ASME Section X1 System Pressure Testing (SPT) Program. As part of the assessment, a detailed vertical slice analysis of Salem Unit 2AFW system was conducted.,The verticalslice entailed reviewing itie ISI Classificationis, Test Boundary Drawings, Schedules,'and Test Packages freach classed compo.neiit.withi'n tlhesei systems. Both. test preparation and test *coseout praciices were evaluated in detail: Documenitation.was, reviewed and interviews conducted. The results of the review were benchmarked against-tIhe assessment lead's knowledge of similar.industry programs, and: utilities as well as current activities and initiatives within the governing regulations, codes, and standards. Observations and recominendations based on this entire review process were included in the report.
Page 6 of 35
A detailed review of this report was made. Nowhere in the report is there any mention that a portion of the AFW piping at Salem Unit 2 was buried. There was additional supporting information on marked up drawings that could not be located; however, if it had been discovered that there were portions of the AFW that were buried it would have been highlighted in the report not just on the drawings.
There was no external peer for this FASA. Current processes govern external peer requirements. This is a
______.__'I ilgacy issue and no corrective action will be assigned.
Ineffective 'Corrective Action Basis CC #2 Implementation Review of various PSEG Salem and Hope Creek assessment and audit reports consisted of In-service Contributing Cause #2 Inspection, Engineering Reorganization, Nuclear Business Unit, Outage Performance Assessments and Outage Management System (3M)
Preparation Assessments. A review of the assessments Corrective Actions (4CA) revealed a trend related to the long term plant update such Corrective Action Needs as lack of updating, revising, and referencing correct ASME Improvement (SCL) codes, code cases, and commitments. A review conducted in,. 1996 of assessments.and audits for corrective actions, corrective action closed out, and corrctie actions being nimplemented effectively to preventreoccurence indicated that the corrective action program usage was poorly implemeented'and determined tihe IS'Group s 1'-reluctance to self initiate Action Requests for issues related to progranmiatic deficiencies, many assessment recommendations were discounted with out evaluation. The Pressure Testing Program was addressed in several of the assessments. As an example one third party assessment where there were 32 recommendations and no documentation of actions taken. (See Attachment ft 5 -
Missed Opportunities). AR 00960321219 could not be located in the data base. The Root Cause team, QV, and records management performed searches. Therefore, a review of those referenced recommendation could not be conducted for current applicability.
This is a legacy issue. No corrective actions will be assigned.
Periodic pressure tests have High turnover in ISI Program Basis CC #3 never been performed on ownership adversely impacted Significant turnover and lapses in program ownership buried AFW piping at ability for ISI program resulted in missed opportunities to catch the missed system Salem as required by ASME intrusiveness pressure test of the buried section of piping. (See Section XI code Attachment #5, Missed Opportunities)
Contributing Cause #3 ISlt Program continuity has been difficult to maintain Management System (3M) throughout the operation of the units. The ISI Program has SPAC Not Used (4SN) seen many organizational changes. The ISI group started Acc untabiliity NI (SAL) with the&Salem Maintenance Detariment and then to
..Nuclea.r Site Maintenance, Site Services, Specialty Engineering, and finally Engineering Programs. Up until 1997, the core group was relatively stable with the Program Manager and field supervisioný but there were numerous contractors and other people working on the ISI program plans.
Prior to 2006 there was no dedicated ISI program manager for Salem. The responsible peisbnhad comrneting priorities Page 7 of 35
iesuting in reviews not being performed in any meaningful dýepth.
Frdin 1997 to 2005 the organization changed to having one 1SI Programn Managr forial! three units, Again*ekeping track of all the work involved in impleImentting these plans the Program Managers had to rely on contract support along with inexperienced ISI persomnel In 2006 after the NOSA, there was another change where one ISI Manager was assigned to Salem and another for Hope Creek. After 2006, the ISI Program manager was vacant for a period of time, and then changed hands two other timies prior to the current ISI program, manager assuming the position in 2008.,The turnover process gives the Program"Managers a checklist; however, does not give theinI thie in-depth knowledge of everything intheoprogram
'includ ng' te systems and thus not the'abilit 'to be intrfsive right away.
Periodic pressure tests have never been performed on buried AFW piping at Salem as required by ASME Section XI.code Pressure test requirements were
- not fully understood by responsible program implementers; however procedures did exist to identify
,special testing requirements on buried piping. Theprocedures did not idezitifyw vhich sections
.of system piping are buried.
Contributing Cause #4 Management'System (3M)
SPAC NI(4SL)
Confusing or incomplete (5CL)
Procedures (3P)
Followed Incorrectly (4F)
Details NI (5DE)
'Basis CC #4 The ASME Section XI Code requirements for testing class 3 AFW buried piping during the Salem's first ten-year inspection interval utilized 1974 Edition 1975 Summer Addenda Code. Periodic pressure testing of the buried portion of AFW piping was not performed when pressure testing and leakage walk down was done on the above ground AFW piping. The requirement for periodic system pressure tests'on buried piping required, the system to be isolated and ipressurized and the pressure in the test boundary Awas moditbred forchange (pressure drop test). T here weresignificaantdiffcrences between the pressure test requiriements between Class 1, 2 and 3 components'in this code edition. The code requirements for class I and 2 pressure testing changed significantly in later edition but had niiinimalchanges for class 3. The class 3 requirements.for the once per interval hydrostatic test were identified during the first interval and performed as required. The periodic pressure testing for all the above ground AFW:ipoing wa p6erforned'as required during-all past interyals'.' Pe~rsonnel ineriervwsiniedi6tQ
- ethat, examinerswwere not aware the piping was buried and that they were unaware of the differient requirement Iregardless.
5.0 EXTENT OF CONDITION/CAUSE:
Cause being addressed Extent of Condition Review Shortcomings in technicafl An extent of condition review was performed on all Salem Unit I human performane prctices and Unit 2 pressure testing boundary dirawings to identify potential.
human perfoirmance practices t
resultced' 2in complete initial buried piping., Unit LAF.W buried pipingliad. not,been tes ted a n d su bse q u e nt 1, p,.
rsim iilarly to U n it2. A Il U n it I b u ried p ip in g w as rep laced du e t6 its
'and subseque~nt ISI program pi plans; this coupled
.with.a lack degraded condition during buried piping inspections in'the Spring, of qtestimnig attitudelby Si.
2010.
inspectors 'resulted ina tle fai"
'Other buried piping identified: includes Service Water headers on A rE to f tesn %
both units. Review and evaluation of the buried portion ofthe AService Water system determined that due to the excessive Page 8 of 35
Cause being addressed Extent of Condition Review leakage of the SW21 valves validated by In-Service Test (IST) data the system is determined to be non-isolable. Thcrefore, the requirements of IWA-5244(b)(2) are applicable "The system pressure test for non-isolable buried components shall consist of a test to confirm.that flow during operation is not impaired." IS'l" data has shown that the structural integrity of the SW system is satisfactory. An evaluation to document our testing methodology will be conducted (ACIT #7)
An extent of condition for all ISI'and NUREG pressure test scoping requirements needs t ttb ec6inpited and will be accomplished when
- a. for-al pressure test pl*a'n inclu'ding hand over hand walkdowns, boundary validation, aido:ial acceptance criteria for buried piping tests during each 10 yeainterval update. (Ref.' CAPR #2),
Hope Creek does not have an AFW system. Pressure testing on systems with buried piping includes service water and core spray, reviewed these systems related ISI summary 11s 401600, 402300, 402400, 402500 and 402600. The buried portions of piping on these systems were identified and had been tested although the test documentation is not clear on what type of test was performed on all tests.
Identification of deficiencies in the ISI program plan have been noted historically and during the development of the third interval update due to the magnitude of the entire ISI program plan.
Ineffective Assessments Historically assessments have been ineffective in identifying specific issues with the ISI program plan. The ISI program plan includes inspection of welds, snubbers, system pressure testing, IWE/IWL, and repair/replacement. Each of these components has a specific test method requiring review, Assessments often do not go into enough detail to identify all program deficiencies. An in depth review of all aspects of the IS1 progrfamn would not be feasible in the amount of time allotted for a:
dassdssment.
Ineffective Corrective Action This was deemed to bea legacy issue. Current processes and culture Implementation dictate focus on Cor rective Action implementation.
High turnover in ISI Program No other ISI program leficiencies have been identified in SAP as a ownership adversely impacted result of high turnover; however deficiencies in other programns have ability for ISI program been attributed to high turnover. Examples include AOVs, Check intrusiveness Valves and IST programs Pressure test requirements were The SW system pressure test procedure was also noted that it did not not fully understood by identify a portion of the piping as buried. This also led to confusion responsible program on what testing is required per ASME Section Xl code. It has been impl~ementers; however determined by IST test data that an alternative test which verifies procedures did exist to identify unimpaired flow could-be credited for the SW pressure test during special testing requirements ont the current interval. This alternative test method-was not documented buried piping. The procedures as being performed nor, was there a-relief request submitted to the.
did not identify which sections
-NRC for pennission. T heahternative test method which-verifies floW.
of system piping are buried, is not impaired can only be used if piping system is not isolable. This S
tern isolable is not c learly defined in the currentmteri'akcode of record. There is howev'ever code" interpretations and ai n-approved code case that clarifies that butterfly valves (which are-used in Salems SW system) do not provide isolation and the system could be considered non-isolable. No other portions of piping were identified in the scope of ASME"Sectiun XI code,
.Page 9 of 35
6.0 RISK ASSESSMENT Plant-specific risk conSequence, Basis for Determination.
RISK = Low:
- '?-'*:::'
','*A risk assessment for Salem 2 addressifg'a rised surNeillance ofthe AFW discharge
.ineuindergr pindpipgIpre,ssure tsting'was conducted under RM Document No. SA-
'.URV-2010-00,Re,..This 'assessment.waasconducted if accordaice with' procedtire ER-AA-600-4045' Ris* Assessment of Missed or Deficient Sureillances.
"This assessment evaluated two poientialconditionis, AFW'discharge piping. failure resulting in loss of invertory,'as well as the potential for obstruction of theAFW supplypiping. The results of that assessment concluded that deferral ofthe missed surIveillance for a period of up to the end of the current operating cycle is acceptable and resultis in a negligible increasevin risk.
During the entire period of time'when the required pressure testing of the AFW buried piping was not perforned,,there were no known failures identified.,Therefore,*there
' wereno challenges to tlhe operabilityýor availability of the AFW system as'a result of posttilated degradation of these, piping segments."
The'AFW buiied systemii potential degradation of piping is'noftincluded in the PRA model at Salem and, except for several specific cases, piping failures are not modeled,..
As a rtesult; ifthefailure probazbiity of the buried AFW piping segments,was.
"6 ' tlated to be more likely as a Iesult f improper performace othesysim pressuare testing, there would be no impact to: the PRA model.
It is therefore concluded that the risk resulting from improper performance of the system pressure testing on the buried segments of the AFW piping is low.
7.0 PREVIOUS EVENTS Alreview of industry operating experience was performed usinglkeyword searches in the INPO OE database. The followiing oEwvere identified forfurther, review (See Attachment #6 f@ detailed'OPiEX rcview):
OE08665: 11-25-1997 inadequwate.iiýimpeientation ofISI Progr-m, ram-Quad Cities oEi2844:10-17-2001 Underground Lak kof afe~ty Related 14" l
SW Line OysteriCreek OE14702: 10-02'2002 External Corrosion of Auxiliary Fee'dwater Carbon Steel Piping - TurkeyPoint 3.,
OEi 8596: 05-22-2004 Auxiliary Feed,Water Pump Recirculationh Pipe Failure - Surry 2_
OE22036: 02-06-2006 Self Assessment Identifies Possible Missed Surveillance due to Inspection of ESW, ECW, and
-HPSW.Piping:Not Per ASME Section X1 Code Inspection Requirements - Peach Bottom OE2445 i 1.0-1 7-2006.Piping Degradation Discovered Downstream of the Restricting Orifices for all Three AFW Pumps Millstone,3.
0E3,1 192: 05-20-2010 ASME Specified Containment Concrete Examination Missed, Results in SR 3.0.3 Entry-Diablo Canyon 1Y
- Analysis, The purpose of the review was to deterifine,whetherany industry OE existed that could have helped JPSEG in early identification and prevention of ipiroperly performed pressure testing on the buried portion of AW piping.
The review of these OE itemnsrevealed several instances of buried piping failures and degradation similar to that recently experienced on Salem, Unit.1. Additionally, several.instances of failure to satisfy ASME Section XI testing or examination requirements were Ireeald.:However, the review did not reveal-any instances of failure to identify buried portions of ASME class piping. subject to thle testing requirements of rWA-5244.
The Peach Bottom OE discussed a case where the testing performed on the buried portions-of their ESW, ECW, and HPSW discharge piping did.not fully, satisfy the ASME Code requirements. -However, -this issue revolved: around an unused maintenance valve on open~ended discharge lines, and whether the existence of that valve made the discharge Pagle,1O :of 3,5-
piping isolable. This case does not match the configuratibn of the Salemn AFW discharge line. No other OE was identified where the testing performed on the buried portions of AFW ýdischarge piping did not fully satisfy the ASME Code requirements.
During the recent (May, 2010) ASME Code Committee'lmeetings, manry 6f the ISI program owners were asked whether their PWR's had portions of the AFW discharge piping buried. Every plant representative questioned advised that all of their AFW discharge piping was routed within the plant facilities. None of these plants had buried portions for the subject piping, Therefore, it was concluded that industry OE would not have helped Salem ISI personnel in early identification of
- buried portions of the AFW system, or that the isolable buried portions of the Salem AFW discharge piping were not properly tested in accordance with the requirements of IWA-5244.
Through a review of NRC integrated inspection reports the team identified that other plants received non-cited violations for issues similar to those identified in this report. Corrective actions from those violations could have mitigated this issue earlier than it had been identified.
8.0 CORRECTIVE ACTIONS TO PREVENT RECURRENCE (CAPRs):
Root Cause Being Addressed Corrective Action to Prevent Recurrence (CAPR)
Owner Due Date Shortcomings in technical human Revise ER-AA-330-001, 'SECTION X1 PRESSURE performance practices resulted in TESTING', to require that formal pressure test plan for incomplete-initial and subsequent all applicable systems including hand over hand walk K. Colville IST program plans; this coupled downs, boundary validation, and formal acceptance M. Ambrosino 11/01/10 with a.lack of questioning attitude criteria during each 10 year interval update.
by 1Si inspeetoirs resulted in the (CAPR #1) failure to, perform the required ASME SectioniXl testing.
For thelc'urrenit 10 year.update, develop and implement a formal pressuire testplan which includes T. Giles 12/30/10 Root Cause hand ovetr hand walk downs,.boundary yalhidation, and formal acceptance criteria (Already in process)
(CAPR #2) 9.0 CORRECTIVE ACTIONS:
Cause Being Addressed Corrective Action (CA) or Action Item (ACIT)
Owner Due Date Shortcomings in technical human Implement performance management~for individuals Price Complete performance practices resulted in th't failed to identify the burie&dprt.iio of piping by
,incomplete initial and subsequent, exhibiting a questioning attiiudeeduring theconduct of ISI program plans; this coupled the, system pressure test.
with a lack of questioning attitude (ACIT #1) by ISI inspectors resulted in the NDE Specialists will perfbrm inorefrequent Sheets 8/26/10 failure to perform the required observations on routine exams and document these in ASME Section XI testing.
the FMS system. This will help to reduce errors and improve inspection performanice through objective and Root Cause critical FMS entries.
Each NDE Specialist should perform at least I field observation per week during non-'outage time and one per day.during refueling outages. FMS entries will be monitored and recorded in each',NDE Services Perfonnance partriershipstarting with 2010 year 3 d quartertupdate.
(ACIT #2)
Page 11 of 35
Discuss thisrbbt cause ad~iuniderstand the gaps that led to the failure to identify this piping as buried with all..
PSEG certified VT-2 examiners.
Oliveri 9/1/10 (ACIT #3)
Review and update SPT IDDEAL instruction with
-. Mora 4/1/11 additionalinforniation such as;.
. Location of pressure device
- Locations of key piping and components
- Special instruction as applicable (CRCA41)
After Hope Creek R16 and Salem 2R18 perform Malney.
11/15/10 samiple of data sheet quality to'verify that all, the' information is provided to allow-for proper recreation, and understanding of th6':systemý pressurtest performed. This shouldbe performed by Corporate' Engineering.
(CRCA #2)
,Conduct a-tabletop review of.HU-AA-1212 "Technical Oliveri 5/1/11 Task Risk/Rigor Assessment, Pre-job Brief, Independent Third Party Red,iew, and Post-job Brief' to ensure that Technical IHmiumdn PerfOrmance*
principles and practices are understood'and implemented by1SI inspectors (CRCA #6)____
Reviise ER-AA-330-001 'SECTION XI PRESSURE M. Ambrosino A 11f//10 TESTING' to identifg the portion ofAFW and SW as buried'.
(CRCA #3)
Revise OU-AA-335-015, 'VT-2 VISUAL T. Oliveri 11/1/10 EXAMINATION', to identify the portion of AFW and SW~as buried.
(CRCA #4).
Revise Pressure Testing program drawings.SP'1'-336, SPTi-236, SPT-342-1, SPT-)42-2, SPT-242-l and, SPT-242-2 to indicate the portions of AFW and SW piping as buried.
(CRCA #5j T. Giles 10/12/10 Ineffective Assessments Perform a FASA to:
T.Gil12/30/10 Cointribiuting;Cause,#1.-
Rev.iew Boundary Drawings for adequacy, Evaluateif the ISI 10,year updates were complete including Relief Requests
Revise ISI Audit Template E1S-1, Inservice Inspection J. DeFebo
'11/15110 (ISI), Section E1 S-1.1(4) (i)
To read........ system pressure tests, "including buried Section XI piping,"
(ACIT #5)
Ineffective Corrective Action This'is a legacy issue. No corrective actions are".
Implementation required:
Contributing Cause #2 i
Highturnover in ISI Program.
ihrviI g Identify back up ISI program managers within each K. Colville 8/26/10 ownership adversely impacted ability fownershiprogrm avruseim abn l station to help alleviate dependence on contractor M. Pfizenmaier for ISI program intrusiveness support during program development in accordance Contiributing Cause #3 with the K'I&R program.
(CRCA #6)
Page, 12 of 35
Implement ER-AA-l 100 IMPLEMENTING AND T. Giles 8/26/10 MANAGING ENGINFEERING PROGRAMS' W. Brammeier requirement for a program notebook specifically for Pressure Testing, currently pressure testing is included in the ISI program notebook.
(CRCA #7)
Pressure test requirements were not Implemented, OEM certification guide T. Giles Complete fully understood by responsible NEGENANPG03Q,
'E... ei.....i"..g NE...
'Engiieenng-,iTrainig progmam implementers; however "Cerification Guide for Pressure Testing' for ISI procedures did exist to identify program managers.
special testing requirements on buried piping. The procedures did (CRCA #8) 9/16/10 not identify which sections of system Implement an interface agreement between outage E. Maloney 9/16/10 piping are buried.
services and engineering. The purpose of the interface Contributing Cause #4 agreement is to define the roles and responsibilities of NDE examiners and ISI Program (CRCA #9)
Develop program/procedure(interfaceprocedure)
E. Maloney 9/16/10 enhancement to provide additional:direction to examiners and provide rollout to-the examiners (ACIT #6)
Bring Pressure test requirements to Engineering and T. Oliveri Outage Services CRC to determine if training could be T. Giles a solution (ACIT H7) 1'0.0 EFFECTIVENESS REVIEWS (EFRs):
CAPR CA being addressed Effectiveness-Review.Action Owner Due Date Revise ER-AA-330-001, Verify by document reviews that formal M. Ambrosino 12/1/2010 TSECTING, to PREqURE m
pressure test plans have been developed TESTING', to require that formal which include the specifi cdethails outlined in pressure test plan including hand the CAPR. In addition, validate that ER-AA-over hand walkdowns, boundary 330-001, 'SECTION XI Pressure Testing' validation, and formal acceptance and OU-AA-335-015, 'VT-2 Visual criteria for buried piping tests Examination' have been revised to identify during each 10 year interval AFW piping as buried. Results to measure update.
effectiveness will be measured by "0" ISI (CAPR #1) pressure testing program deficiencies or findings.
Page,,l 3 of 35
I 11.0 Organizational:and Programmatic Issuesý Evaluation and Analysis.,
The purpbse of this evaluation and analysis is to detennine any Latent,Organizati6nal Weaknesses (LOWs) that created precursors to the event, or degraded the defenses designed to prevent the event.
Programmatic and Organizational Issues Corrective Action'
- Owner, Due (CA) or Action Item Date
,*(ACIT) 11.1 Are, there organizatioinal or programmatic factors that CRCA #6 lead to negative conditions prior to'the inappropriate CAPR #2 actions or problem occurrenc6(prevention)?
Yes - Prior to the impler*nentation of the OEM, certificajion' guides were not required for the ISI Program manager.
Inexperienced PSEG personnel were at times assigned to develop Long Term Plans for Salem. This resulted in an over reliance on contractors during Long Tern Plan development. There was minimal. vendor oversight and program document reviews as a result of the use of inexperienced personnel.
11.2 Are there organizational or programmatic factors that CAPR #1 created negative conditions or actions in the course of the ACIT #1.
inappropriate actions orI pr'blem occurrence (prevention'
,CIT #2
- and detection)?
'.ACIT'#3,
- CRCA #I,'-:i Yes
- Inattention to detail and, lack'of technical rigor f
RCA,#
YesýC
-CCA
ý#2 created negative conditions in the colirse of VT-2 system
- CICA3
.pressure test examination walk downs Program ACI,#,
impleinenters did not review work'packages with sufficient CRCA #
.technical rigor and attention to deiail, The walk downs in the field after reviewing the work packages with attention to detail should have identified an issue with the AFW piping.
Lack of a thorough review created negative conditions during the development of Long Term Plan interval update§.Inexperienced PSEG personnel assisted and reviewed w
he Long.TertmPlans developed by the contractor.
"The PSEG personnel did not have the experience necessary
'to be as'intrusive, in the developmeni'[f the Long Termn Plan as necessary* Each program update was an opportunity to identify the missed press'ui'e test`sirnce each update requires'the reviewer to create a plan fortesting such as system pressure tests for thetentire interval to ensure program requirements are met.
11.3 Are there organizational or programmatic factors that created conditions or actions following the inappropriate actions or problem occurrence (detection) that didn't allow the organization to detect the problem?
No 11.4 Are there organizational',or programmatic factors that CRCA #6 degraded the barriers which'could'have prevented or ACIT #2 minimized the problem occurrence?
Yes - Program turnover and periods with.no program manager. The ISI program has seen many organizational changes. To keep track of all the work involved in implementing the Long Term Plans during organizational Page 14 of 35
4
.~-----~-
changes, program managers had to rely on contract support and inexperienced PSEG personnel. (See Attachment U 5 -
Missed Opportunities). More Management oversight could have helped prevent or minizmize the problem occurrence 11.5 Is the event the result of repeat minor failures'?
No 11.6 Did the organization not adequately address precursors ACIT r5 to the event?
Yes - Numerous QA audits and assessments had multiple recommendations that were not implemented. One of the assessments noted a third party assessment where there were 32 recommendations and there was no documentation of action taken.
11.7 Were error precursors to the event adequately evaluated for risk and were appropriate defenses in place when the event occurred?
No 12.0 SAFETY CULTURE ANALYSIS:
In accordance with step 3.1.4 and the note under step 3.4, of LS-AA-125-1001, Root Cause Manual, a safety culture analysis is not required since this event is not a focal point or follow-up to; a NRC 95002 or 95003 inspections..
Additionally, it is not required based on-r'ecently completed safety culturelassessments (Nuclea.r Safety Culturee Principles FASA 70098602 completed on 1/08/10 and the Multiple Chiller Failures Root Cause Evaluation 70106293 completed on 4/05/10).
13.0 OTIIER ISSUES:
Other Issues identified during Corrective Action (CA) or Action investigation Item (ACIT)
No documentation has been initiated Perform an evaluation to document the T. Giles 10/15/10 to address system pressure testing of justification for the alternative test buried service water piping.
method for service water (SW) system piping and determine if Relief Request is warranted (ACIT #8)
Ensure regulatory compliance Review and determine if a report is B. Thomas Complete required since past Owner Activity Reports had incorrectly stated that AFW was tested IAW ASME XL requirements.
(70109827 CLRC Op 180)
ISI Program Health report does not Review ISI Program Health Procedures E. Maloney/ T.
12/14/10 reflect issues with implementation and Indicators (ER-AA-330 & SC.RA-Giles/ B.
AP.ZZ-0021) to determine Br-mmeier adequacy/completeness of evaluating overall ISI Program Health.
(ACIT #9)
Page 15 of 35
14.0:
COMMUNCATIONSPLAN:
L essons Leared, tobeý,:"
'°
- -z Lessos Leaned t be Communication Plan Action Owner Due Date" Communicatedý
- `
Shortcomings in technical human Provide a communication brief to the K Colville '
8/30/10 performance practices resulted in Nuclear Communications Department, incomplete initial and subsequent for inclusion in Outlook. (ACIT P10)
ISI program plans; this coupled with a lack of questioning attitude Review Root Cause Report and Lessons by ISI inspectors resulted in the Learned at next TAC meeting to failure to perform the required determine whether additional ASME Section XI testing.
Departmental corrective actions are L. Rajkowski 8/30/10 required. (ACIT #11)'
Communicate organizational lessons learned at Department Brief.
L. Rajkowski-k 8/30/10
_ "(ACIT#12)
Communicate the results of the Provide the root cause to the CFAM for root cause to the appropriate the Functional Area of the group individuals so lessons learned can sponsoring this Root Cause Evaluation to' K. Colville 8/30/10 be communicated at Hope Creek review for applicability to the opposite station and
.Communicate the same to the SFAM for
,that station. (ACIT#13)
Page 16 of'35
Attachment-Detailed Time Line of eventskeading to April 2010,event':
FIRST INTERVAL REQUIREMENTS
'First ISI Interval (Unit1l& 2) applicable code is.ASME SectioznXI 1974E-S78A' PerASME Section XI: 1974E-S75A Class 3 buried piping requires a pressure drop test once every period (IWD -2410 anidIWD.-2600).
A Hydrostatic testlis also required for Class 3 buried piping once during the interval. (IWD-5200) 1977: Beginning of the Salem Ul First ISI Interval Southwest Research' Institut& (SwRI)developed, the,first intervatllong term planfori Ul Class I and 2 components. Contracted scope; included components andssystesi,;subject to surffice and volunietric exams.
The long term plan for visual exams (includes class J,; 2 and 3 piping) was developed by PSEG>,..
1981: Beginning of the Salemi U2 FirstS 5I Intelval'ý-,,
SwRI developed the first interv~al, long term plan for U2 Class 'I and 2 compo'neftsZiContractcd scope.
included componcentsand systemsusbj ect to surface and2nvt dpuxetric exam3s. The long term p.la. for visual exams (includes class 1,, 2 an d 3, piping) was developed by-PSEG 1987: Hydrostatic test performed on Unit 1 Hydrostatic test is performed on 11/12/1987 as required by ASME Section XI 1974E-S75A for the 12 and 14 AFW linesburied piping (Orders 870321044 and 870324058). The pressure drop test failed so a permitted alternative was perfonred successfully.
1988: Relief Req uest to use alternative Hydrostatic' test method subm itted for UUnit 1
Relief Request submitt6d to use an alternative hydrostatic test rnei d hecaus ehrevious test failed. Re'lie6f.
Request specifically'discusses,*p otions of the AFWIiping. (Relief Request submitted because df open issue/
from NRC resident Item #'272-87T32-01, Report #.272-87-32). Date submitfedi June6, 1r988Dote'appir6'ved:.-
March 11, 1991 1988: End of Unit 1.First Interval/ Beginning of the Salem Ul Second ISI Interval SECOND INTERVAL REQUIRMENTS-UNIT I 198: PEG represUni 1Second Interval Long -Term Plan Update Developed by an Engineer in.the ISI group who was unfamiliar with the program plan and had to0 rely heavily on contract support. It was this engineer's first time doing this update.. Class 3 Aui'Ili rFeed portion of the buried piping-was not identified as buriedin dtelongteftn plan. The&engiheer was respoisibe for the-.
entire program update nciuding parts which had' been p'eifomed e
However, the contractor's (SWý I) expertise was :iiihClas I and 2 'pip'ing not systeIn pressure tests.
1991: Unit 2 Hydrostatic Test Program for Class 1 and 2 IAW Code Case N498 and' SER (Docket No.
50-311) no longer required 1992: Hydrostatic test is performed on Unit 2 Test performed on 3/7/1992:as required by ASME Section XI.1 974E-S75A for the 22 AFW line buried piping (Order 900302162). The drop test failed.sojeakige imand leakage.,out-were measured while performing the hydrostatic test: No relief request was submitted for Unit 2 to use'a different test methodology.
1 Reference Attachment #5: Missed Opportunities matrix for details of ISI, program manager turnover, VT-2 examinations and program audits.
Page 17 of 35
SECOND INTERVAL REQUIREMENTS-UNIT 2 1992: End of Unit 2 First Interval/ beginning of Unit.2 second, interval Developed by another Engineer" in heAISi group who \\was. unffmiliar with the program plan and had to rely heavily on contract support. It' was this engineer's first time dding this update.
- Class 3 Auxiliary Feed portion of the, buried piping was not identified as buried in the long term plan The engineer was responsible f6r the entire program update including parts which had been performed by a contractor during the 1S interval. However, the contractor's (SWRI) expertise was in Class 1 and 2 piping, not system pressure tests.
1994: HYDROSTATIC TEST NO LONGER REQUIRED FOR UNITS I & 2 SER received for code case N-498-1 (Docket Nos. 50-272, 50-311, and 50-354)
Hydrostatic tests for class 3 piping were no longer required 1995: Hlydrostatic Test Program End: Group disbanded 1998: First dedicated ISI programnmanager (common to site). See Attachment #5: Missed Opportunities Matrix for program manager turnover.
2001: End of Second Interval/ Beginning of Third ISI Interval for Unit I THIRD INTERVAL REQUIREMENTS FOR UNIT I & 2 2001: UNIT 1 LONG TERM PLAN FOR THIRD, INTERVAL UPDATED DEVELOPED BY CONTRACTOR Class 3 Auxiliary Feed buried piping was not included as a separate item in the long term.plan 2003: ISI System Pressure Testing Assessment ýperformed by Contractor Assessment focused on system pressure tests conducted:for AFW Piping conducted by Innovafive Technology Solutions Corp. (ITS) 2003: Nuclear Operational Services Agreement (NOSA) 2003: ISI system pressure test self assessment performed by contractor.
It evaluated pressure testing on AFW buried piping. It did not mention any findings on missing pressure tests.
2003: QUAL cards implemented for program:manager position.
2003: End: of secondinterval/IBeginning of Third,]ISIInterval for.Unit 2 2003: LONG TERM PLAN FOR THIRD INTERVAL FOR UNIT 2 DEVELOPED BY CONTR1ACTOR (ITS)
Class 3 AFW buried piping portion was not identified as buried in the long tern plan 2010: IDENTIFIED MISSED SECTION XI PRESSURE TEST OF BURIED PIPING DURING PROGRAM INSPECTION FOR UNIT I Piping was discovered to be below minimum wall thickness allowable and not have the required coating 4/9/10 Test data was reviewed related to the Unit 1 AEW corrosion 4/21/10 Notification 20459689 - Discovered missed section XI pressure test of Unit I and 2 buried piping.
The AFW piping has never been tested on a periodic basis.
Page 18 of 35
AITTACHMENT # 2:E&CFC Root Cause Evaluation - Event & Causal Factor Chart Order #70109827..
The required ASME Section XI Pressure Testing for the buried Auxiliary Feedwater piping was not performed.
UI beinn U2: beinn tes pefre
$0 lntrap NR s
ubmit e
to us intyPrEal/beoyoowing Sintrval plan oVfOpd biontra str PSEG developed plan.f t
.n tPSEG developtd planm f
- for,
, oU" I-A for isua, s faimlntiond tat-time-d
.v olut o e~~~iama~fo fale 0drognertatrle Di~d not include a formnal*
,'id not Include aformaT"'
was r*
avity on Contractor of en o
-pen ~issu esfrm tgo X~lan for pressuro tostinl J)u.
Ia for r....s..r te
__i ua) n wn
- perf orm dduing te st ing w
val**t *,
3
/
or*-
=A K
reviewed. updato...x 1
only reviewed update only interval after.This error went
,,_.fC~bntractor's oxporti5*
required some expeorience
- rqired some undetected and-was carried---*
was class 1 & 2 pningn->"
and ~ ~ ~
~
~
~
~
~
~
R reaident (Fyusný rsur.ets
'rqueprinean.
o pesro notvol de~re 1.u POIK l
rf a ad
1 I zn'- """o"itra"
.=
=
1 9
l
- =U2:.end of interval U 1.9 5U:
end of second I"
Pro)grams Eng L
U2: beginning No Ihydrostatic test no
-o-Hydro group so inevl g'"sat digSstrn
/
ýy
-^
st
,;t
, I 6'SIof first interva l::
l onger required for disbanded.eom o.r~o*
i 3
ii fhr
~at~
prs Hie of thirdttm HeathReprt L*Aplc thern plnpdt 6dfern
)
- .on9 rom U'upde.
L Exoeo moorI'nige I
/'T.I *lsystem presrtos',
self assessment erformd by contracto Qual cards ino rer Evaluated pressure K for rogra m
anae Lon burled piping l
2000 2 003 206Mse Uf2:
end of second new M
w e
ur e eest inter.va.lbeginning Fo ln...ddntietfposition establishedi Iof third I"
Iat Salem duigprga pressure tests 30 far 0
e 7_4o cn19-01-hr
- posi!tion wstomprsi y
LTplan update done by pjpingwas brie4" contractorsCn y
lmqabro Page 1 of I Page 19 of 35
ATTACHMENT #3 BARRIER ANALYSIS FAILED OR INEFFECTIVE CORRECTIVE ACTION TO RESTORE BAILEDRORINERFECTIVEHOW BARRIER FAILED WHY BARRIER FAILED BARRIER BARRIER TO EFFECTIVNESS Latent shortcominigs in technical human Personnel did nor identify that The long term plans developed (CAPR #1) performance practices.
the buried AFW Pipingpequired by contractors were not reviewed (CAPR #2) different te~stinggmethods'every 3 in, sufficient detaii by the ISI (CRCA #9) 1/3 y tears in thefirst* second, or ogram manager.
~~~pro gr m ma me.+....
(ACIT N6),...
+;.*..... "....
third inervallonig term plans.
Communication between groups working This'informatin Was not passed Lack of communication between (CRCA #7) under the 1S1 program manager.
on to PSE&Gimplementing the "Hydro" group and the (CRCA #9) personnel performing the periodic examination personnel.
periodic examinations.
ISI Program manager's knowledge of The Class 3 Auxiliary Feed Updates were completed by (CAPR #1) program requirements.
piping was not identified as, contractors-with review by IS"
ýburied in the long tenn plan.
personnel inexperienced with program updatesl.
Questioning attitude and attention to By not recognizing the AFW Lack-0of aquestioning attitude (ACIT #1)"
detail of1St program i mplementers.
piping was buried in a certain.
and attentiointo detail by (ACIT #2) aI A e` anid would require a special inspection personnel and the (ACIT #3) test.
AFW piping not being (CRCA;#I) specifically addressed in the (CRCA#4)
___________procedure.
(CRCA,#5)
Auditsof, the ISI system pressure test Bfuried piping Was not part of the, Buried piCingwas not t1(ACI
- 5) specifical'lyaddressed in the Audt plan;:..
Assessments of ISI system pressure test The buiried AFW piping was not Over conifidence by the assessor.
(CRC-A#1) program.*spifically addressed in the The 'results 6fothe program (CRCA#4) assessment scope.
review were benchmarked (CRCAr105)'
-agains
-t the aissessment, lead's
. -. -know ledge of sim ilar industry programs and'utilities as well as current aotivities and initiatives within the governing regulations,
___:_codes, and standards.
+
Page 20 of 35
ATTACHMENT #4 TASK ANALYSIS SC.RA-.IS.ZZ-0006(Q) Rev 3 VW-2 SYSTEM LEAKAGE VISUAL EXAMINATION FOR NUCLEAR CLASS 1., 2, AND 3 SYSTEMS 2.2,1 Work Packages to be used in performing examinations The P&ID was-not reviewed-in sufficient detail to have been reviewed and any supporing documents, Work identify that a portiofi of the AFW, line was buried, Order(s), System P&IDs and other inforimatizoi as appropriate
-Drawing 205236 sheet 0OLclearlyidentifies a portiln of are included in the work package. '
the AFW line as "Yafrd"pipipig.-Aquestioiing, att'tud' was not cultivated duiing thý VT-2 exam wyalk downs
'which should have identified~a portion of the pipe as buried when comparing the P&ID jo the field.
5.3.4 Perform system pressure test:per Work Order, as The AFW line was never notedtobe buried in any of the directed by the Inspection & Test Supervisor designee, and exam data sheets on file: Theprocedure clearly states that the following guidelines:
a pressure drop test or change in flow measurement is D. In systems where the buried components are isolable by required for buried portions of piping. No portions of the means of valves, the examination shall be performed:
AFW line were identified to be buried in the exam data sheets for all periodic pressure tests. The requirements for o
By determining the rate of pressure loss, or the change in t-ti p
onrtiof.fthe system pressure test were also flow between the ends of the buried, components.
'nidntif.ed as stated inthe procedurefor VT-2 Examinationis. The.onnectidnýwhs,snevir made thaCithe AFW~line, wAý'bturiedand that it reqjuired a ifrnt type
- 6fpressure test.
5.6.1 Exam Coordinator, PER'FORi the following:
A thoroughi review of th"eex*amination package could A. Review completed examinfation packagesto ensure all have noted that,4iportionof the AFW line as denoted as areas have been examined and'areas which were not examined "yard'* piping in th3 P&ID, The exam coOrdinator could have been identified and incorporated into new examination have implemented a questioning attitude when 1packages.
performing the review of the examination package.
Page 21 of 35
ATTACHMENTf#&,
MISSED OPPORTUNITIES VT-2 inspection's Year Unit 1993 Unit 2 Unit 1 1994 Unit 2 1995 Unit 2 1999 Unit I 2000 Unit 2 2002 oUnit I 2005 Unit 1 2007 Unit I 2008 Unit I 2009 Unit 2 Program Managers Year Details 1998-2000 first dedicated PM-common PM 2001-2006 common PM 2006 positibnPvacani for 6 months 2006-2007 new PM, dedicated to Salem 2007-2008 new PM 2008-present new PM Year Audit Overall Summary Pressure Testing Relevant Corrective Actions CA effectiveness 1.996 Audit #96-011 The assessment Assessment was An identified corrective action Two action requests Inservice identified 4 areas for conducted through was to improve the ISI group's were generated to Inspection improvement:
review and reluctance to initiate an action address these findings
- Program, improvements in comparison of request for programmatic AR'00960322146 and Salem & Hope procedures and procedure deficiencies.
AR'0096032.12. The Creek programmatic requirements and.
response to AR Generating maintenance, ASME X1 Sections 00960322'146 was Stations contractor oversight IWA, IWB, IWC found'acceptable by and control, and IWD. A the Salem QV Group corrective action sample of Reference CRQV, 02.
programmatic usage completed system AR 00960321219 and communication pressure tests were could not be located of management's selected for review in the data base. The expectations.
and assessed to Root Cause team, determine the QV, and records adequacy of management inispection, results, performedý searches. It and documentation.
is Pbssible that this The audit team. '
AR was rejected or reviewed the ISI.
not processed. If that Group's inservice is'ithe case no record system pressure
- _would, exist.
Page 22 of 35
p
~"'~
.~.',
testing examination"",
procedures andi "
Maintenance Department's Repair Replacement pressure test procedures. The procedures wcre found to meet the requirements of ASME XI and contained 'sufficient guidance to
conduct inspections.
1998 Nuclear Business Unit Inservice Inspection and Engineering Reorganization Audit 98-011 The audit determined that the ISl gr6up has been performing self-assessments; however they have not been done in accordance with a formal program,, The team.
reviewed.tie' SI group's informal self-assessme,nts activities" from the preceding 15 months. The' group.was a part of either the maintenance or engineering organizations.
Although maintenance and.
engineering have established self-assessment programs, both failed io include ISI as part of tffiir self-assessiment-population. As a result, self-assessments and:
corrective' actions performed to address problems identified during these self-assessments were not tracked; trended or entered into the corrective action program.
The audit team
reviewed a contractor's assessment of System Pressure, Test Programn. The teamdetermined that the con'tractor's assessment was*'
'thorough and '.
- comprehensive.
However the audit team noted that the group did not create tracking mechanisms to fix any of the thirty-two program recommendations..
Three instances of initially, ineffective corrective actions were identified: the allocation table printout had missing data: the Salem 1 Long Term Plan (LTP) was not updated,in accordance with the "
'programniaiic standard; and ASME B7F welds in SaleinI LTP manual'were not updat ed.'
-Te iheso ution'of other"'
concerns idxitified during the previous audit xwas not timely.
Howe'vertc6rnsidering th&e.
significance of the issuesISl workload, and':available ISI resources the scheduled completion dates are adequate.
qTherewcr 'no corrective action tasks assigned to this condition. As noted, "However, considering the'
ý"signiificance of the,
,issuesi,'i Si' workload,.
and available ISI resourcesthe scheduledlcornpletion
-.dates are adeq*uate."
'No further action or
.follow up was necessary for this observation Page 23 of 35
2001 PSEG Nuclear LLC QA Assessment Report 2001-0285 Minor administrative deficiencies were identified where the Long Term Plans were. not updated when Code Case information changed and Relief Requests were approved,
,Aithdrawn or denied.
The Salem Unit I LTP was reviewed.
QA reviewed Pressure Testing exemptions identified in the inspection plans foi-. r
.Salem 1, lnterval.,3;.
Salemn2, Intal.2.
The,feview was, made to verify that.
the exemption criteria was consistent with the Code Edition and addenda required by 10CFR 50.55a for each plant.
Based upon this review it was concluded that.the exemption criteria..
- was consiste nt with the required Code :/
editions and addenda, No issues identified.,
Audit team recommended a program enhancement to maintain a book or file that
,,contains all relief requests,
,.with supporting approval
,documentation, filed by relief
,.request number.
The books were developed.
2006 Salem Audit NOSA-SLM-06-07 SAP fl 80090367 Based on interviews, document reviews, and ISI examination record reviews the team concluded the' inservice Inspection.
meets expectations, with three deficiencies and no enhancements identified. The deficiencies (1) RHR Spring Can Hanger VT-3 Examinations; (2) Changes made to the Risk Informed ISI Program were incomplete, and evaluation of the changes could not be located; and (3)
Tables in the ISI Basis Document were not consistent with the same table in the ISI Program Long Term Plan, Not in scope of Audit No relevant issued identified.
2008 NOSPA-SS-Based on the results Not in scope of Review of ISI Program Plan Procedure revision 08-IQ of the assessment, Audit determined the plan was not was made to change NOS has concluded revised at least annually as the frequency of the that the Station is required by program update to periodically vulnerable in the area procedure. There were nine instead of annually.
of incomplete separate pen and ink changes documentation and added to the plan and it has Pag'e", 24. of 35
,annual program plan been approximately.4 years
," updaie'and it could since the last revision be expected thatthe (20359640).
NRC ziiightt find similar issues during their upcoming inspection.
2007 Salem Focused The focus of this FASA noted that Not all PSEG procedures with, All operations Area Self assessment was the Pressure Test Exelonequ'iiialents have been associatedwith Assessment transition to the Procedure ER-AA-deleted resulting in two sets.of NOTF's20337886/700 (FASA) report, Exelonprocedure.
335-015 allows use active procedures for the same 74667 and Exelon The conclusions of of site specific subject matter. (NOTF#
20337903/70074688 ISI/NDE the assessment were procedures in lieu 20337886 - ISI procedures, have been completed.
Procedure that the current of it and the NOTF11 20337903 - NDE In addition, A review Transition conduct of Inservice procedures'are procedures) was made of the Effectiveness; Inspection (ISI) similar in their IDDEAL data sheets SAP 70067554 progranh andNon generic guidance associatedwith all'the Destructive such that either Surface and Visual Examination (NDE) procedure, would E1Xmiiitioil activities isgenerally imeet the intent-of procedures used by.
consistent with the the other, but there the Outage Services Exelon programmatic was a breakdown NDE Group. This and implementing in communication review was to assure procedures (ER-AA-about which that all the governing 330-xxx and ER-AA-procedures should procedures and code 335-xxx series be used for forms requirements were procedures and and generic captured on these T&RMs).
guidance. Several reports., One area was reconmuendations identified was the were made by the documenting of team to improve.illumination checks the performance on the dataisheets.
going forward in Thiswas identified in implementing these order 70105644 act procedures and 0080 and is being forms and ensure tracked in order procedure 70107436 act 0010.
compliance.
2009 Hope Creek The conclusions of Not in scope of It was identified that the Procedure OU-AA-Focused Area the assessment.
Audit evaluation and reporting 335-018 was revised Self revealed that the responsibility of.the.
making the position Assessment overall performance Responsible Individual (RI) of Responsible (FASA) report, of ISI program was has not been adequately Individual (RI). The Hope Creek found to meet documented. Code relies on RI position is.
.Containment Expectations. The an engineer, identified, as a associated with Inservice team, identified three Responsible Individual for.
inspections of metal Inspection (3) deficieincies, and development, implementation containment's and RE Program nine (9) and evaluation of the program is associated with the (CISI); SAP.
recommendations, and examination results. A inspection of concrete 70095692 The deficiencies review of datasheets containments, All centered around associated with visual operations in Order responsibilities of the inspections of the I1C 70095692 have been Responsible Containment revealed that completed except for Individual (RI),
certain datasheets were not 0150 to "Perform a adequacy of details completed. with enpugh closure review as per and completeness of information to adequately.
4.6,7 of LS-AA-126-NDE data reports, addess' or define' the,,.
1001 of tthis SAP and docurnent control
insptcionbisl)eing rpored.
' order for ihe above
____of various progra m All,ýthese deficiences andc.
focused area self-Page 25 of 35
support drawings, performance improvement recommendations are being tracked toward completion in operations under order number 70095692.
assessment following the process outlined in LS-AA-126, Self-Assessment Program, and LS-AA-126-1001, Focused Area Self-Assessments."
.1 Page-26 of 35
ATTACHMENT #6 (b)(4)
Page 27,of 35
(b)(4)
F r
T 1
Inspection Reports Date Plant Issue A n n Iv,*i*
Rors Date Plant IseA-n~.;.,
The inspectors identified a Green Non-Cited Violation NRC (NCV) of 10 CFR 50.55a(g)(4) for failure to perforrn INTEGRATED periodic leakage testing of buried piping sections of the
[NTECRTION Failure to perform RN (raw water) system as required-by Section XI of the SECTOR periodic leakage testing of ASME Code for the second 10-yeai" IS interval for Units REPORT 05000413/2007 2008 Catawba buried'piping sections
.1 and 2.
005 AND durifig previous Although this report pertains to failure to perform leakages 05000414/2007 inspection interval, test in accordance with'iWA-5244, implementing 005 corrective actions based on these issues would not have facilitated identification of the buried AFW piping segments.
Page 28ý of 35
An NRC-identified Green, non-cited violation of 10 CFR 50.55a(g)(4) for the I
failure to perform periodic leakage testing of buried NRC IDENTIFICATI ON AND I
RESOLUTION OF PROBLEMS INSPECTION REPORT 05000321/2007 006 AND 05000366/2007 006 2007 Edwin I.
Hatch-
-PIJtpng -CtL'[Ui Uo ttic Hiigh.Pressure Coolant
-Injection (HPCI).and Standby Diesel Service
-Waier (SBDSW) systems
.a.required by Section XI of the'ASME Code for the third 10-year In-service Inspection (ISI) interval Dpuring the Unit 2,
- refueling outage in February 2007, NRC inspectors identified that the licensee had not performed the required periodic pressure drop test or change in flow rate test
- for buried portions of the lHPCI and SBDSW systems piping during, the third ISI interval in, accordance with the J989
- Edition. of Section XI, Article IWA-5244.,The:
licensee was committed to this Code Edition for the third IS interval. This Code required testing was for buried piping that was non-redundant and isolable.
Prior to the discovery by NRC inspectors, the licensee had incorrectly considered the buried portions of HPCI and SBDSW systems to be redundant and isolable, a classification which would have exempted them from testing. Since the third IST interval ended December 31, 2005, it was unclear'ifithe licensee was :within the grace period to request relief from this requirement which, if granted,would have alleviated the, Coderequirement.
S ubsequeint to the Februar outage, the licensee concluded that they wereunable to requestrelief and"that there was no avenue towreconcile thiis~missed exanjinationi.
The licensee has included this Code requirement in their fourth 10-year lSl.interval testing program. Although'this report pertains to failure to perform leakage test in accordance with IWA-5244, implementing corrective actions based on these issues would not have facilitated identification of the buried AFW piping segments.
I
[
NTEGRATED INSPECTION REPORT 05000269/2007
- 005, 05000270/2007
.005, 05000287/2007 005 2008 Oconee 10 CFR 50.55a(g)(4),
requires, in part, that, components classified as ASME Code Class Ill
,must meet the requirements set forth in Section XI of the ASME Code; The 1998 Edition of Section XI, Article IWA-5244, "Buried Components", requires that in non-redundant systems where the buried components are isolable by means of valves, the visual examination for leakage (VT-2) shall
-'cnsistof a leakage test:,
thtatdetermines the rate of pressure loss.
Aliernatively, the test may determine the change in flow between the ends of the buried components.
I
/
Contrary to the above, the licensee had not niet:these' requirements during'their third ISI interval which ended in 2005. The licensee r'ecently identified this issue in' their corrective action program as PIP 0-07-06007. The licensee generated corrective actions to identify all code class buried piping and update the ISI program in order to support testing of the affected piping. This finding is of very low safety significance because it was not a'design or qualification deficiency resulting ina lo ss of 0operability,,did not represent an actual loss of a safety function, di*d n..rsult in exceedin. g aTS alow:ed otiiage',
time, and didnot-affect external event mitigatioi:.
Although this report pertainsto failue to' performi leakage test in accordance with IWA-5244, implementing corrective actionsbased on these issues would not have facilitated identification of the buried AFW. piping
- segments, Page 29 of 35
INTEGRATED INSPECTION REPORT 05000250/2007 004 AND 05000251/2007 004 2007 Turkey Point The inspectors identified a Green non-cited violation (NCV) of 10 CFR 50.55a(g)(4) having very low safety significance for failure to perform periodic leakage testifng of buried piping sections of the ICW system as required by Section XI of the ASME Code for the third 10-year iSI interval for Units 3 and 4.
-The inspectors identified that the licensee had not p crforned the required periodic pressure drop test or ch~'iige in flow rate test for buried piping portions:of the ICW system during the third 10-year ISI interval for Units 3 and'4 in accordance with the 1989 Edition of the ASME Code,'Section XI, Article IWA-5244. Both units are cui-rently in the fourth ISI interval and the one-year period allowed to submit for regulatory relief following the third interval had expired at the time of this inspection. The failure to perform the requirements of IWA-5244 constitutes a violation of the ASME Code. The Code required testing for buried piping that is isolable by means of valves. The licensee had not performed this leakage
.testing because they did not consider the butterfly' isolation valves in the system to be'sufficierit for use as isolation Vlves.
Althouigh this report pertains to failure to perform leakage test in accordance with IWA-5244, implementing corrective actions based on these issues would not have facilitated identification of the buried AFW piping segments.
Page 30 of 35
ATTACHMENT #7 Root Cause Investigation Charter.
...ORDER /,NOTIFICATION NUMBER:,"
70,109827.
SPONSORING, MANAGERS:
Len Rajkowski-Director, Salem Engineering Devon Price - Manager,.Outage Services QUALIFIED ROOT CAUSE INVESTIGATOR:
Kerry Colville-Manager, Salem Engineering Programs (Root Cause Lead)
TEAM INVESTIGATOR(S):
Tim Giles - SA ISI Program Manager Williim Treston - Outage Services Bill Brammeier - HC ISI Program Manager Ed Maloney - Engineering Services, Programs (ISI)
Dan Maxey - SA Programs Engineering Erin West - Licensing SamSpeer-License Renewal Connor Lynch-Operations Harry Smith-Exelon Peer (IS I)
SCOPE:
On April 21, 2010, it was identified that the surveillance for in-service pressure testing of buried piping in the Auxiliary Feedwater (AF) System was not performed.
The Root cause will evaluate the following aspects of the event and identify corrective actions to prevent reoccurrence:
Missed Section XI Pressure Testing for Buried Piping for theAuxiliary Feedwater system:
o Evaluate why the Section XI pressure testing for Buried Piping for the Auxiliary Feedwater system was not conducted or Relief Request submitted.
o Evaluate why the previous examinations that had been conducted on the Auxiliary Feedwater system did not identify that portions of the piping system were buried.
o Evaluate whether ISI is in compliance with the Operational Excellence Model (OEM) with respect to Organization Structures and Roles and Responsibilities.
o Evaluate if enough key program guidance is being given to inspection personnel to conduct inspections.
o Review response to 2007 Violation on Missed Relief Requests o
Review ISI Program Health Procedures and Indicators (ER-AA-330 & SC.RA-AP.ZZ-0021).to determine adequacy'/completeness of evaluating overall ISI Program Health..
o Review 2007 Salem ISi,,FASA (70067554) and 2009 Hope Creek Contaihment IS! FASA"(70095692) to determine if there NVas a mis sed.opportunity to identify areas tha"t iebdedeid improveinent."
o Evaluate if Documentation Management/Record retention is being adequately implemented.
o Evaluate CAP effectiveness for these issues if identified previously.
Extent of Conditions:
o Determine Extent of Condition for other Buried Piping which falls under the Section Xl requirements including verifying proper inspection methodologies.
o Review Boundary Drawings for adequacy.
o Evaluate if the ISI 10:year updates were complete including Relief Requests o
Review past Owner Activity.Reports to determine if revisions are'required.
o Communicate results.:with Hope.Creek.
..Page,31 of 35.
INTERIM CORRECTIVE ACTIONS:
For the Missed Section XI Pressure Testing on Aux Feed *Buried Pipe: Salem Unit 1 piping was replaced and tested to code requirements prior to being placed in service. A risk assessment was approved to demonstrate that when the surveillance is extended to the start of the next Salem Unit 2 refueling outage, the risk increase is negligible. A Relief Request is in the process of being considered. Extent of condition was evaluated for Salem U I and U2 Service Water piping.
ROOT CAUSE REPORT MILESTONES:
Event Date:
04/21/10 Screening Date:
04/28/10 Charter Prepared:
04/29/10::
Charter Briefing to MRC:
04/30/10 Final Team Assignments:
04/30/10 Team Kickoff:
04/30/10 Two Week Update:
05/14/10 Sponsoring Managers Approval:
06/14/10 Final Root Cause Investigation Due Date:
06/14/10 Presentation to MRC:
06/28/10 Page 32 of 35
Attachmenht #8 Summary of Responses to RC Charter Items 0
Evaluate why the Section Xl pressure testing for Buried Piping for the Auxiliary Feedwater system was not conducted or Relief Request submitted.
Lack of overall ISI program intrusiveness and ineffective assessments allowed gaps in technical human performance (rigor) to go undetected since plant start up.
o Evaluate why' the previous, examinations that had been conducted on the Auxiliary Feedwater systern did not identifythat portions of the pipinggsystem were buried.' 'ij'
- Procedures 'used to complete, V.2 'examiniations during syste'm pr'essire tests `of AF' piping were not.
followed-with sufficient technical riigor :or attention to detail.
PSEG has historically relied heavily on-the use of'vendorsto develop IST ProgrAIms'at both H0opeCreek and Salem. The result was less than adequate oversight and review (V&V) of their. products.
o Evaluate whether IS[ is in compliance-with the Operational Excellence Model (OEM) with respect to Organization Structures and Rolesand Responsibilities.
ISI is in compliance with the OEM, 'however an "interface, agreement" between the Outage Services NI)E Group and ISI Programs engineering Will be developed to better define roles, and responsibilities.
o Evaluate if enough keyprograifm guidance is being. given to inspection personnel to cbdut'ianspections.
Not enough guidance was provided. The AFW line was never noted to be buriedain anyof the e.xa data sheets on'file. No portions of the AFW line were identified to be buri :d in th exam data sheets for all.
periodic pressure tests. A questioning attitude was also not cultivated during the VT-2 exam walk downs, which should have identified a portinof the pipe as buried. Pressure test procedures will be revised to highlight the buried portions of piping systems at both Salem and Hope Creek. Pressure test diagrams will also be revised.
o Review response to 2007 Violation on Missed Relief Requests Reference 70068999 late submittal 'of an ISI relief request. Activity 0010 was dispositioned, approved, and peer reviewed by Reg Assurance. Activity 0020 determined that the.extent of condition was isolated to this event by Reg assurance. All Corrective actions are closed o
Review ISI Program Health Procedures and Jndicators'(ER-AA-330 & SC.RA-AP.ZZ-0021) to determine adequacy/completeness ofevaluating,;overallISIProgram, Health At this time the' ISI Pr6grnamHealth'report does not reflectissues *with implementaitibin.ACIT #X was written to Review ISI Progr HeIith' P uocedIires an IJndicator0s 0(E&-AA330&C.RAS-AP.ZZ-0021) to determine adequacy/completeness of evaluating oveirall ISIProgriamHealth.- "I o
Review 2007 Salem ISI FASA (70067554) and 2 009 ;Hope: eek Contai-ment'I.,FASA (70095692) to.
determine if there was a inissed opportunity to identify areas that needed improvemn
- t.
Salem 2007. Not all PSEG procedures with Exelon equivalents were deleted resulting i two sets. of' active procedures for the same subject matter. These procedures have been resolved reference 70074667 and 70074688.
Hope Creek 2009. It was identified that the evaluation and reporting responsibility of the Responsible Individual (RI) has not been adequately documented. In addition data,sheets did not contain enough information to adequatel' ajddress or define the inspections being perfornimed Alfthese. deficiencies and performance improvement recommendations are complete under order number 70095692 with the exception of the closure review.
Evaluate if Docunentaition Mariagement/Record retention is being, adequatey'implemented.
Documentation= Management/Record retention is beingadequately implemented.'This'was shown by locating all therecords or objective evidence -necessIary.ffor this RCE.
o Evaluate CAP effectiveness* for these issues if identif led previously...<..
Issues identified in the CAP associated with thi at this time. ReferccecA,' Effectiveness in, the RCE Page 33 of 35
Attachment:
- 9 RC Quality Checklist Th.~ ~
A.
Critical Content Attributes YES NO
- 1. Is the condition that requires resolution adequately and accurately X
identified?
- 2.
Are inappropriate actions and equipment failures (causal factors)
X identified?
- 3.
Are the causes accurately identified, including root causes and X
contributing causes?
- 4.
Are there corrective actionsto prevent recurrende identified for each root cause and do they tie DIRECTLY to the root.caus'e? AND, are there
.-.. X corrective actions for contributing cause and.do they tie DIRECTLY to the.
contributing cause?
- 5.
Have the root cause analysis techniques been appropriately used and X
documented?
- 6. Was an Event and Causal Factors Chart properly prepared?
X
- 7.
Does the report adequately and accurately addressthe extent of condition in accordance with the guidance provided in Attachment 4 of LS-AA-1 25-X 1003?
- 8.
Does the report adequately and accurately address plant specific risk consequences?*
- 9. Does the repo0.adequateiy and accurately address behavioral, programmatic and organizational issues?.,.
- 10. Have previous similar events been evaluated? Has.an Operating Experience database search, been performed to. determine useful lessons X
learned or insights for development of CAs?
- 11. If required, does the report adequately address the NRC's Safety Culture Components in accordance with the guidance provided in Att. 20 & 21?
B.
Important Content Attributes
- 1.
Are all of the important facts included in the report?
X
- 2.
Does the report explain the logic used to arrive at the conclusions?
X
- 3.
If appropriate, does the report explain what root causes were considered, but eliminated, from further consideration and the: bases for X
their elimination frm' onsideration?
- 4.
Does the report identify contributing causes, if applicable?
X
- 5.
Is it clear what conditions the corrective actions are intended to create?
X
- 6.
Are there unnecessary corrective actions that do not address the root X
causes or contributing. causes?
- 7.
Is the timing for completion of each corrective action commensurate with the importance or risk associated with the issue?
X Pageo34off 35
C.
Miscellaneolbs Items YES NO
- 1.
Did an individual who is qualified in Root Cause Analysis prepare the X
report?
- 2.
Does the Executive Summary adequately and accurately describe the significance of the event, the event sequence, root causes, corrective X
actions, reportability, and previous events?
- 3.
Do the corrective actions include an effectiveness review for corrective actions to prevent recurrence?
X
- 4.
Were ALL corrective actions entered and verified to be in Action X
Tracking?
- 5.
Are the format, composition, and rhetoric acceptable (grammar, X
typographical errors, spelling, acronyms, etc.)?
- 6.
Have the trend codes been added or adjusted in SAP to match the X
investigation results?
- 7.
Has entire Root Cause Evaluation report been sent to records?
X Page 35 of 35