IR 05000382/2015010

Ir 05000382/2015010, March 16 Through 19, 2015, Waterford Steam Electric Station, Unit 3 - Supplemental

ML15097A550
Person / Time
Site:	Waterford
Issue date:	04/09/2015
From:	Pruett T W NRC/RGN-IV/DRP
To:	Chisum M R Entergy Operations
Lantz R
References
EA-13-233 IR 2015010
Download: ML15097A550 (18)

Text

April 09, 2015

EA-13-233 Mr. Michael R. Chisum, Vice President Entergy Operations, Inc. 17265 River Road Killona, LA 70057-0751

SUBJECT: WATERFORD STEAM ELECTRIC STATION, UNIT 3 NRC SUPPLEMENTAL INSPECTION REPORT 05000382/2015010

Dear Mr. Chisum:

Prior to May 26, 2013, your staff failed to establish an adequate test program to demonstrate that the train B emergency diesel generator ventilation exhaust fan would perform satisfactorily in service. This performance deficiency resulted in a failure to identify that the exhaust fan could not perform its function because it disengaged from the fan motor in April 2013. Consequently, the train B emergency diesel generator was determined to be inoperable for a period of 30 days, exceeding the Technical Specification 3.8.1 allowed outage time of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. On December 19, 2013, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at the Waterford Steam Electric Station, Unit 3. Based on the results of this inspection, documented in NRC Inspection Report 05000382/2013008 on January 30, 2014 (ADAMS ML14030A616), and the final significance determination documented in NRC Inspection Report 05000382/2014009 on March 28, 2014 (ADAMS ML14086A768), the NRC assigned a White finding Action Matrix input to the mitigating systems cornerstone in the fourth quarter of 2013. In response to this Action Matrix input, the NRC completed a supplemental inspection using or Two White Inputs in a Strategic Performance on October 10, 2014, and documented the results in NRC Inspection Report 05000382/2014011 (ADAMS ML14364A412). The NRC determined that your extent of cause review was insufficient; therefore not all inspection objectives were met, and the White finding remained open and continued to receive consideration as an Action Matrix input. On March 3, 2015, you informed the NRC that the Waterford Steam Electric Station, Unit 3, was ready for a supplemental re-inspection. On March 19, 2015, the NRC completed a supplemental re-inspection and discussed the results of the inspection with you and other members of your staff. On March 24, 2015, Mr. Ryan Lantz, Chief, Reactor Projects Branch E, - 2 - NRC Region IV, conducted a Regulatory Performance Meeting with Mr. John Jarrell, Regulatory Assurance Manager. from the revised root cause and extent of cause evaluations. The result of the inspection is documented in the enclosed inspection report. The NRC performed this supplemental inspection to determine if (1) the root and contributing causes for the significant issues were understood, (2) the extent of condition and extent of cause for the identified issues were understood, and (3) your completed or planned corrective actions were sufficient to address and prevent repetition of the root causes and contributing causes. Your identified that the two primary root causes of the White finding were that an engineering change, modification, or temporary alteration process was not effectively used to control the fastener configuration of the emergency diesel generator (Root Cause 1); and that requirements for monitoring of supporting systems for plant components were not adequate to verify operation (Root Cause 2). Specifically, an engineering evaluation was not performed to assess the impact of adding four additional set screws to the emergency diesel generator hub and spanner nut assembly in 1999, and emergency diesel generator operating procedures did not require that the emergency diesel generator exhaust fan differential pressure be monitored to verify adequate fan performance. The NRC determined that your staff identified appropriate corrective actions to revise the emergency diesel generator system operating and surveillance testing procedures to include verifying proper operation of the ventilation exhaust fan, as well as verifying that current station guidance clearly defines criteria for implementing an engineering change process. The NRC also determined that adequately reviewed operating and testing procedures for other safety-related systems, as well as an adequate assessment of reworked threaded connections or changes in configurations of fastening components in other safety-related systems. Based on these determinations, the NRC concluded that all inspection objectives were satisfied. The NRC has determined that completed or planned corrective actions were sufficient to address the performance that led to the White finding. Therefore, the finding is closed and is not considered as an Action Matrix input as of April 1, 2015. No findings were identified during this inspection. -

Sincerely,/RA/ Troy W. Pruett, Director Docket No.: 50-382 License No.: NPF-38

Enclosure:

Inspection Report 05000382/2015010

w/Attachment:

Supplemental Information cc w/encl: Electronic Distribution

-

Sincerely,/RA/ Troy W. Pruett, Director Docket No.: 50-382 License No.: NPF-38

Enclosure:

Inspection Report 05000382/2015010

w/Attachment:

Supplemental Information cc w/encl: Electronic Distribution ADAMS ACCESSION NUMBER: ML15097A550 X SUNSI Review By: TJF ADAMS X Yes No X Publicly Available Non-Publicly Available X Non-Sensitive Sensitive Keyword: OFFICE OE:DRS/OB C:DRP/E D:DRP NAME TJFarina RELantz TWPruett SIGNATURE /RA/ /RA/ /RA/ DATE 04/08/15 04/09/15 04/10/15 OFFICIAL RECORD COPY

SUBJECT: WATERFORD STEAM ELECTRIC STATION, UNIT 3 NRC SUPPLEMENTAL INSPECTION REPORT 05000382/2015010 DISTRIBUTION: Regional Administrator (Marc.Dapas@nrc.gov) Deputy Regional Administrator (Kriss.Kennedy@nrc.gov) DRP Director (Troy.Pruett@nrc.gov) DRP Deputy Director (Ryan.Lantz@nrc.gov) DRS Director (Anton.Vegel@nrc.gov) DRS Deputy Director (Jeff.Clark@nrc.gov) Operations Engineer (Thomas.Farina@nrc.gov) Senior Resident Inspector (Frances.Ramirez@nrc.gov) Resident Inspector (Chris.Speer@nrc.gov) WAT Administrative Assistant (Linda.Dufrene@nrc.gov) Branch Chief, DRP/D (Mike.Hay@nrc.gov) Senior Project Engineer, DRP/D (Robert.Hagar@nrc.gov) Senior Project Engineer, DRP/E (Cale.Young@nrc.gov) Public Affairs Officer (Victor.Dricks@nrc.gov) Public Affairs Officer (Lara.Uselding@nrc.gov) Project Manager (Michael.Orenak@nrc.gov) Branch Chief, DRS/TSB (Geoffrey.Miller@nrc.gov) RITS Coordinator (Marisa.Herrera@nrc.gov) ACES (R4Enforcement.Resource@nrc.gov) Regional Counsel (Karla.Fuller@nrc.gov) Congressional Affairs Officer (Jenny.Weil@nrc.gov) Technical Support Assistant (Loretta.Williams@nrc.gov) RIV Congressional Affairs Officer (Angel.Moreno@nrc.gov) RIV/ETA:OEDO (Michael.Waters@nrc.gov) ROPassessment.Resource@nrc.gov ROPReports Enclosure U.S. NUCLEAR REGULATORY COMMISSION REGION IV Docket No.: 05000382 License No.: NPF-38 Report No.: 05000382/2015010 Licensee: Entergy Operations, Inc. Facility: Waterford Steam Electric Station, Unit 3 Location: Dates: March 16 through 19, 2015 Inspector: T. Farina, Operations Engineer Approved By: Troy W. Pruett, Director Division of Reactor Projects Enclosure

SUMMARY

IR 05000382/2015010; 3/16/2015 3/19/2015; Waterford Steam Electric Station, Unit 3; Supplemental Inspection Inspection Procedure (IP) 95001 This supplemental inspection was conducted by a reactor operations Region IV office. No findings were identified. --The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process." the two primary root causes of the white finding to be that an engineering change, modification, or temporary alteration process was not effectively used to control the fastener configuration of the emergency diesel generator 1); and that requirements for monitoring of supporting systems for plant components were not adequate to verify operation (Root Cause 2). Specifically, an engineering evaluation was not performed to assess the impact of adding four additional set screws to the emergency diesel hub and spanner nut assembly in 1999, and emergency diesel generator operating procedures did not require that the emergency diesel generator exhaust fan differential pressure be monitored to verify adequate fan performance. This resulted in a mechanical failure of the fan in April, 2013, which was not detected during a surveillance test run of the . The train B was determined to have been inoperable for a period of 30 days, until the condition was corrected by replacing the emergency diesel generator exhaust fan hub assembly with new parts in their original configuration. The inspector determined that the licensee identified appropriate corrective actions to revise the emergency diesel generator system operating and surveillance testing procedures to include monitoring of the ventilation exhaust fan flow indications, as well as verifying that current station guidance clearly defines criteria for implementing an engineering change process. generator and deficient testing procedures, the White finding associated with this issue is closed and is not considered in assessing plant performance as of April 1, 2015, in accordance with the -

REPORT DETAILS

.01 Inspection Scope

This inspection was conducted in accordance with Inspection Procedure 95001, Supplemental a White finding, which affected the mitigating systems cornerstone in the reactor safety strategic performance area. The inspection objectives were to: provide assurance that the root causes and contributing causes of risk-significant performance issues are understood; provide assurance that the extent of condition and extent of cause of risk-significant performance issues are identified; and -significant performance issues are sufficient to address the root and contributing causes and to prevent recurrence. A 95001 supplemental inspection was completed on October 10, 2014, and documented in NRC Inspection Report 05000382/2014011. The NRC determined that objectives were met, and the White finding remained open and continued to receive consideration as an Action Matrix input. On March 3, 2015, the licensee informed the NRC that they were ready for the supplemental re-inspection. In preparation for the inspection, the licensee performed a root cause evaluation under Condition Report CR-WF3-2013-2530. Revision 9 of the root cause evaluation report, dated February 23, 2015, was provided to the inspector for review. The NRC inspector reviewed the results of the licensee Pre-NRC 95001 Inspection Snapshot Assessment, which was completed in February 2015. evaluations conducted in support of the root cause evaluation. The inspector reviewed corrective actions that were taken or planned to address the identified causes. The inspector also held discussions with licensee personnel to determine whether the root and contributing causes as well as the contribution of safety culture components were understood, and whether corrective actions taken or planned were appropriate to address the causes and preclude repetition. The inspector performed two system walkdowns and observed an operational surveillance from the control room.

.02 Evaluation of the Inspection Requirements

02.01 Problem Identification a. Determine whether the evaluation documented who identified the issue (i.e., licensee-identified, self-revealing, or NRC-identified) and under what conditions the issue was identified. the train B emergency diesel generator (EDG) ventilation exhaust fan was identified on May 22, 2013, through the licenseets, prompted by an abnormally high room temperature while the train B EDG was operating on May 20, 2013. The high room temperature was evident to licensee personnel stationed at the EDG while it was being operated. An alarm in the control room for high temperature in the train B EDG room also alerted plant operators to the high temperature condition. The issue was therefore self-revealing. identified the issue and under what conditions the issue was identified. b. Determine whether the evaluation documented how long the issue existed and prior opportunities for identification.

determination of when the exhaust fan failure occurred. Based on a detailed historical review of parameters including EDG room temperature, exhaust fan motor current, and exhaust fan differential pressure, the licensee concluded that the fan failure had occurred at the start of the previous operation of the train B EDG on April 25, 2013. Therefore, the issue had existed for 27 days prior to identification. The testing of the train B EDG on April 25, 2013, during which the fan failure occurred was a prior opportunity for identification of the issue. The failure was not identified at the time it occurred due to inadequate monitoring of exhaust fan operating indications while in service. The operating and testing procedures associated with the EDG system had never required operators to monitor these indications. The inspector long the issue existed and prior opportunities for identification. c. Determine whether the evaluation documented the plant-specific risk consequences, as applicable, and compliance concerns associated with the issue. -specific risk-based safety significance evaluation of the issue. The licensee determined that the train B EDG exhaust fan failure resulted in the train B EDG being in an inoperable condition during the time that the failed condition of the fan existed, and that this condition resulted in the plant being without one of its sources of emergency AC power required by the planspecifications for greater than the outage time allowed by the technical specification.

concluded that the overall risk significance of the issue was of low to moderate (White) significance, which was consistent with the result of the NRCInspection Report 05000382/2013008. The inspector concluded that the licensee appropriately documented the risk consequences and compliance concerns associated with the issue.

d. Findings

No findings were identified. 02.02 Root Cause, Extent of Condition, and Extent of Cause Evaluation a. Determine whether the problem was evaluated using a systematic methodology to identify the root and contributing causes. The inspectors determined that the licensee conducted a root cause evaluation in which three primary evaluation methods were used: Failure Mode Analysis, Event and Causal Factors Chart, and Why Staircase. two root causes and two contributing causes associated with this issue. The root causes were identified to be that an engineering change, modification, or temporary alteration process was not effectively used to control the fastener configuration of the train B EDG exhaust fan hub (Root Cause 1); and that requirements for monitoring of supporting systems for plant components were not adequate to verify operation (Root Cause 2). Specifically, an engineering evaluation was not performed to assess the impact of adding four additional set screws to the train B EDG exhaust fan hub and spanner nut assembly in 1999, and emergency diesel generator operating procedures did not require that the EDG exhaust fan differential pressure be monitored to verify adequate fan performance. This resulted in a mechanical failure of the fan in April, 2013, which was not detected during a surveillance test run of the EDG. The contributing causes identified by the licenseeinadequately-implemented engineering modification to the configuration of the fan hub connection, as well as a maintenance activity affecting the threaded connection, both of which occurred in 1999. These issues are further discussed in Section 02.02.b below. The inspector concluded that the licensee evaluated the issue using a systematic methodology to identify the root and contributing causes. b. Determine whether the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem.

c. Determine whether the root cause evaluation included consideration of prior occurrences of the problem and knowledge of prior operating experience. The licensee evaluation included a review of internal and external operating experience. The licensee conducted a fleet-wide search of the Entergy corrective action program for any previously documented conditions involving ventilation fan failures similar to the failure experienced at Waterford 3 in April 2013. This review identified that a failure of the same component (train B EDG ventilation exhaust fan) occurred at Waterford 3 in 1999. This failure occurred following a maintenance that involved the replacement of the fan motor. The failure was attributed to the motor electrical leads having been inadvertently reversed, resulting in a separation of the fan hub from the hub sleeve due to a reverse starting torque that was applied when the motor was started. This event initiated the maintenance and engineering design modification activities discussed in Section 02.02.b above. The licensee also conducted an industry-wide search for operating experience involving ventilation fan failures. This review identified no prior operating experience involving a failure of the hub and sleeve threaded connection associated with this type of ventilation operating plants within the industry that have similar ventilation system components. This review concluded that fan failures at other facilities were not of the same nature or connection configuration as that of the failure experienced at Waterford 3. Ts evaluation included a consideration of prior occurrences of the problem of similar equipment failures. The inspector also concluded prior operating experience. d. Determine whether the root cause evaluation addressed the extent of condition and the extent of cause of the problem. The licensee reviewed the extent of condition associated with the failure of the train B EDG ventilation exhaust fan to determine whether other similar plant components may that the failure of the fan hub/sleeve connection was applicable to a specific configuration that is unique to this type of fan (a hydramotor-controlled variable blade pitch fan). The only two applications of this type of fan in the plant are the train A and train B EDG ventilation exhaust fans. Thus, the only other component in the plant that may be susceptible to a similar failure mechanism would be the train A EDG exhaust fan.

train B fan hub sleeve had a weakened threaded connection due to a 1999 maintenance activity in which the threads were reworked, as well as a design change to increase the number of set screws used to secure the hub-to-sleeve spanner nut, which increased the stress on the threaded connection. These actions were performed following a previous failure of the train B fan hub connection in 1999 that was the result of bumping the fan in the reverse rotating direction due to the motor leads having been inadvertently reversed during motor replacement, thereby damaging the hub sleeve threads and requiring rework.

iew of the maintenance history for the train A fan, which determined that the fan motor was replaced in 2003. The review further determined that: 1) the motor had not been reverse-wired, and thus the fan hub had not experienced a reverse rotation starting torque; 2) the threaded hub sleeve connection was not removed or reworked; and 3) the attachment of the spanner nut to the hub sleeve was unchanged from the original configuration of 2 set screws. An additional tion was the fact that, due to differences in the ventilation system arrangements between the two trains, the train A fan motor is of a different size than the train B fan motor and produces approximately 40 percent less starting torque. A review of system engineering computer data showed that train A EDG exhaust fan differential pressure increases as expected during EDG operation. Based does not exist on the train A exhaust fan. The licensee also evaluated all fans and blowers that if failed could render an EDG inoperable. Two EDG electrical panels contain blowers for forced cooling. Of these, the train A Generator Control Panel (EG ECP6852 A) had two fans fail in December 2014, due to mechanical binding of one fan causing fuses for both fans to blow. This resulted in a prompt determination of inoperability for train A EDG. The licensee has initiated CR-WF3-2014-6086 to further evaluate the impact of these fans on EDG operability and perform a causal analysis. The licensee conducted an extent of cause evaluation for the first identified root cause, effectively used to control the fastener Specifically, when the licensee increased the number of set screws from 2 to 6 on the fan hub assembly in 1999, it failed to perform a specific engineering evaluation of the design change, as required by the procedure in effect at the time. The licensee conducted a search of plant records for prior engineering changes involving set screw configuration to determine if any similar conditions may exist in other plant components. No other similar conditions were identified. The licensee also conducted a search of plant records for threaded fastener configuration changes. This search yielded seven relevant hits; each was determined to have been implemented using approved design change processes, including appropriate engineering evaluations where applicable. The licensee further determined that current fleet procedures would not allow a change like the one performed in 1999 to be implemented without an engineering evaluation. The licensee conducted an extent of cause evaluation for the second identified root generator operating and testing procedures did not require the EDG exhaust fan differential pressure to be monitored, which would have identified improper operation of the exhaust fan. The licensee conducted a search of plant records over an 8 year period for events involving inadequate equipment monitoring. Twenty three condition reports were determined to be relevant to the search criteria, but none had root or apparent causes related to testing and monitoring supporting systems for plant components. The licensee then conducted a review of all Operations Technical Specification surveillances to determine if inadequate monitoring existed on other plant systems and their support systems (over 80 procedures). This review identified 14 testing and operating procedures for revision to improve monitoring of primary or supporting systems. These revisions included improved monitoring of parameters such as room cooler fan operation, pump discharge pressure, and Dry / Wet Cooling Tower fan operation, among others. The licensee conducted an extent of cause evaluation for the first contributing cause, The licensee evaluated other safety related systems and determined that there were no other fans with a similar hub design as the EDG exhaust fans. The A EDG exhaust fan motor was replaced in 2003, and B EDG exhaust fan rotating assembly was replaced in 2013. Neither work order required the disassembly or reassembly of the hub assembly; both A and B EDG exhaust fans now have hub assemblies which are factory assembled in the original configuration. Both work orders were determined to contain appropriate work instructions for installation. The licensee further determined that the current fleet work planning procedure provides adequate guidance for the level of detail required in work package instructions based on the complexity and safety significance of the activity. The licensee conducted an extent of cause evaluation for the second contributing cause, -related components where fastener threads may have been reworked, which could have resulted in a reduction of thread strength or a change in thread fit clearances. This search yielded 19 relevant examples, all of which were determined by the licensee to have been resolved in an acceptable manner. Interviews with maintenance and engineering personnel by the inspector showed a sensitivity to maintenance practices involving more than minor reworking of threaded connections, with a documented preference for replacing vice reworking damaged connections. The inspector ed the extent of condition and extent of cause of the problem. e. Determine whether the root cause, extent of condition, and extent of cause evaluations appropriately considered the safety culture components as described in IMC 0310. a review of whether a weakness in any safety culture component contributed to any causes of the issue. on identified weaknesses in two safety culture components that were related to the identified second root cause of the performance deficiency (requirements for monitoring of supporting systems for plant components is not adequate to verify operation). Weaknesses in the components of resources (H.1) and documentation (H.7) within the area of human performance were identified associated with procedure quality. Procedures did not include guidance to ensure proper operation of the EDG exhaust fans during operation and testing of the EDGs. The extent of cause review further identified a total of 14 procedures requiring revision to verify operability of primary and supporting systems during testing. included an appropriate consideration of safety culture components.

f. Findings

No findings were identified. 02.03 Corrective Actions a. Determine whether appropriate corrective actions are specified for each root and contributing cause or that the licensee has an adequate evaluation for why no corrective actions are necessary. a number of corrective actions. The principal corrective actions to address the root causes were to revise the system operating Procedure OP-009-Procedure OP-903-erential pressure when the EDG system is operating. Additional corrective actions included the replacement of the train B EDG exhaust fan hub assembly with new parts in the original configuration (which was completed within four days of the discovery of the failed condition of the fan in order to restore the EDG to an operable condition); verification that current engineering procedures would require an engineering analysis for a similar system modification; a review of all safety-related surveillance procedures to determine if an evaluation of auxiliary system parameters should be added; and the evaluation of maintenance practices on the EDG exhaust fan components and discussion with the vendor to determine whether all appropriate maintenance activities were being performed. The inspector concluded that the identified corrective actions were appropriate and addressed the root and contributing causes. b. Determine whether corrective actions have been prioritized with consideration of risk significance and regulatory compliance. condition in order to restore compliance with plant technical specifications promptly upon discovery of the failure. The inspector determined that the licensee adequately prioritized the remaining corrective actions with consideration of the risk significance of the EDG system and regulatory compliance. This included appropriate actions to address a notice of violation issued by the NRC and restore compliance (see Section 02.03.e below). c. Determine whether a schedule has been established for implementing and completing the corrective actions. The inspector determined that the licensee adequately established a schedule for implementing and completing the corrective actions. The inspector noted that, as of March 19, 2015, 93 of 98 corrective actions were either complete or had a due date within 6 weeks. Further, all corrective actions to prevent recurrence (CAPRs) have been completed. d. Determine whether quantitative or qualitative measures of success have been developed for determining the effectiveness of the corrective actions to prevent recurrence. The inspector determined that the licensee had developed an effectiveness review plan to determine the method, attributes, acceptance criteria, and schedule for effectiveness reviews of the CAPRs. The inspector reviewed this plan as captured in the corrective action program as LO-WLO-2013-00149. The corrective actions that had been identified as CAPRs were the revisions to the system operating Procedure OP 009-Procedure OP 903- Relay Operability Verification, replacement of the train B EDG exhaust fan hub assembly with new parts in the original configuration; and verification that current engineering procedures clearly define the threshold for implementing Engineering Change processes. The measures for determining effectiveness included the verification that the EDG system operating procedure log sheets have been completed with appropriate flow data and acceptance criteria; review of B EDG exhaust fan hub assembly replacement documentation for proof that a factory-assembled unit was installed; and interviews with Engineering personnel to evaluate knowledge of implementation of Engineering Change processes for proposed design changes. The inspector concluded that adequate measures of success had been developed for determining the effectiveness of the corrective actions to prevent recurrence. e. Determine whether the corrective actions planned or taken adequately address the Notice of Violation that was the basis for the supplemental inspection. The NRC issued a Notice of Violation to the licensee on March 28, 2014, for the failure to perform adequate testing on the train B EDG exhaust fan to demonstrate that the exhaust fan would perform satisfactorily in service (NRC Inspection Report 05000382/2014009, ADAMS ML14086A768). During this inspection, the inspector determined that the licensee restored compliance by revising the system operating Procedure OP-009-and surveillance testing Procedure OP-903-068the ventilation exhaust fan differential pressure during testing. The inspector concluded that the corrective actions taken or planned were adequate to address the Notice of Violation that was the basis for the supplemental inspection.

f. Findings

No findings were identified. 02.04 Evaluation of IMC 0305 Criteria For Treatment Of Old Design Issues The licensee did not request credit for self-identification of an old design issue; therefore, the risk-significant issue was not evaluated against the IMC 0305 criteria for treatment of an old design issue.

4OA7 Licensee-Identified Violations

None.

A-1 Attachment

SUPPLEMENTAL INFORMATION

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

None

Closed

05000382/2013008-01 VIO Failure To Establish an Adequate Test Program to Demonstrate that the train B Emergency Diesel Generator Exhaust Fan Would Perform Satisfactorily In Service (Section 4OA4)

Discussed

None

LIST OF DOCUMENTS REVIEWED

-- -- -- -- -- -- -- -- -- -- -- -- -- --- --- --- --- --- --- ---

CR-WF3-2013-2530

CR-WF3-2013-2530 ---

WA-01177294