IR 05000254/2007007
| ML070890547 | |
| Person / Time | |
|---|---|
| Site: | Quad Cities  |
| Issue date: | 03/30/2007 |
| From: | Satorius M Division Reactor Projects III |
| To: | Crane C Exelon Generation Co, Exelon Nuclear |
| References | |
| EA-06-112 IR-07-007 | |
| Download: ML070890547 (19) | |
Text
March 30, 2007EA-06-112Mr. Christopher M. CranePresident and Chief Nuclear Officer Exelon Nuclear Exelon Generation Company, LLC Quad Cities Nuclear Power Station 4300 Winfield Road Warrenville, IL 60555SUBJECT:QUAD CITIES NUCLEAR PLANT, UNIT 1NRC SUPPLEMENTAL INSPECTION REPORT 05000254/2007007
Dear Mr. Crane:
On February 23, 2007, the U.S. Nuclear Regulatory Commission (NRC) completeda supplemental inspection at your Quad Cities Nuclear Plant, Unit 1. The enclosed report documents the inspection results which were discussed on February 23, 2007, with T. Tulon and other members of your staff.The NRC performed this supplemental inspection to assess your evaluation ofthe White finding identified in Nuclear Regulatory Commission (NRC) InspectionReport 05000254/2006014, issued June 29, 2006. The White finding was also a violation of 10 CFR 50, Appendix B, Criterion III, "Design Control," and involvedthe failure to establish measures to ensure that the Unit 1 electromatic relief valves remained suitable for operation prior to implementing the 2002 extended power uprate.
We conducted this inspection in accordance with Inspection Procedure 95001,
"Inspection For One Or Two White Inputs In A Strategic Performance Area," and examined activities conducted under your license as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your license. The objectives of this inspection were to provide assurance that: (1) the root causes andcontributing causes of risk significant performance issues were identified, (2) the extent of condition and extent of cause of the issues were identified, and (3) your corrective actions were sufficient to address the root causes and contributing causes so that recurrence is prevented. Based on the results of this inspection, no findings of significance were identified and weconcluded that you understood the root causes and contributing causes of the issues, that you identified the extent of condition and extent of cause of the issues, and that your corrective actions were sufficient to address the causes and to prevent recurrence of the issues.
C. Crane-2-In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of thisletter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/Mark A. Satorius, DirectorDivision of Reactor ProjectsDocket Nos. 50-254License Nos. DPR-29Enclosure:Inspection Report 05000254/2007007 w/Attachment: Supplemental Informationcc w/encl:Site Vice President - Quad Cities Nuclear Power StationPlant Manager - Quad Cities Nuclear Power Station Regulatory Assurance Manager - Quad Cities Nuclear Power Station Chief Operating Officer Senior Vice President - Nuclear Services Senior Vice President - Mid-West Regional Operating Group Vice President - Mid-West Operations Support Vice President - Licensing and Regulatory Affairs Director Licensing - Mid-West Regional Operating Group Manager Licensing - Dresden and Quad Cities Senior Counsel, Nuclear, Mid-West Regional Operating Group Document Control Desk - Licensing Vice President - Law and Regulatory Affairs Mid American Energy Company Assistant Attorney General Illinois Emergency Management Agency State Liaison Officer, State of Illinois State Liaison Officer, State of Iowa Chairman, Illinois Commerce Commission Chief Radiological Emergency Preparedness Section, Dept. Of Homeland Security D. Tubbs, Manager of Nuclear MidAmerican Energy Company
SUMMARY OF FINDINGS
IR 05000254/2007007; 2/20/2007 - 2/23/2007; Quad Cities Nuclear Plant Unit 1; SupplementalInspection; Inspection Procedure 95001, "Inspection For One Or Two White Inputs In A
Strategic Performance Area."A Reactor Engineer from Region III performed this supplemental inspection. No findings ofsignificance were identified. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.
Cornerstone: Mitigating SystemsThe Nuclear Regulatory Commission (NRC) performed this supplemental inspection toassess the licensee's evaluation of a
White finding identified in NRC InspectionReport 05000254/2006014, issued June 29, 2006. The White finding was also a violation of 10 CFR 50, Appendix B, Criterion III "Design Control," and involved the failure to establishmeasures to ensure that the Unit 1 electromatic relief valves remained suitable for operation prior to implementing the 2002 extended power uprate. This inspection was conducted inaccordance with Inspection Procedure 95001, "Inspection For One Or Two White Inputs In A Strategic Performance Area." The inspector concluded that: (1) the licensee understood the root causes and contributing causes of the risk significant performance issues that resulted in the White finding; (2) the licensee identified the extent of condition and the extent of cause of the issues; and (3) the licensee's corrective actions were sufficient to address the root causes and contributing causes, and to prevent recurrence.A.Inspector-Identified and Self-Revealed FindingsNo findings of significance were identified.
B.Licensee-Identified Violations
None.
3
REPORT DETAILS
01INSPECTION SCOPEThe NRC performed this supplemental inspection to assess the licensee's evaluationof a White finding identified in Nuclear Regulatory Commission (NRC) InspectionReport 05000254/2006014, issued June 29, 2006. Increased vibrations experienced while operating at extended power uprate (EPU) power levels resulted in the degradation of multiple electromatic relief valves (ERV) actuator components which rendered the valves inoperable.
The inspectors determined that the licensee implemented the Unit 1 EPU in November 2002, but failed to verify that the ERV actuator design was suitable for operation at the increased vibration levels experienced at EPU power levels. Organizational weaknesses at the station and corporate levels contributed to the licensee's failure to identify this issue prior to, or immediately following, EPU implementation.Events Preceding the Notice of Violation:
- In December 2006, station personnel determined that an intermittent 125 VDC groundwas present on the 3D ERV actuator circuitry on Unit 2. *In response to this issue, an at-power drywell entry was conducted on December 30. During this entry, the licensee identified that several of the actuator components were significantly degraded.
- Unit 2 was shut down on December 31, 2005, to allow inspections of the remaining ERVactuators. Varying levels of degradation were identified on the remaining Unit 2 ERV actuators.
- Due to the amount of degradation identified, the licensee reduced Unit 1 reactor power topre-extended power uprate (EPU) power levels pending an inspection of the Unit 1 ERV actuators. During a planned Unit 1 maintenance outage conducted on January 6, 2006, the licensee discovered that the Unit 1 ERV actuators were also degraded.*On January 9, 2006, the NRC initiated a Special Inspection to assess the licensee'sefforts in identifying and correcting the cause of the ERV actuator degradation. The inspectors were also tasked with determining the safety significance of the ERV actuator degradation for both units. The results of this inspection were documented in Inspection Report 05000254/2006009; 05000265/2006009. However, the inspectors were unable to fully assess the effectiveness of the licensee's root cause efforts, and the significance of the ERV actuator degradation, during the inspection since the licensee's root cause efforts were not complete.*On April 15, 2006, the licensee issued the ERV actuator root cause report. Theinspectors examined the root cause report, compared the report information to the information gained during the Special Inspection, and discussed the report results with licensee personnel to assess the adequacy of the licensee's efforts. The inspectors also reviewed the results of multiple ERV actuator tests and inspections to determine the safety significance of this issue for both units. The finding was determined to be an apparent violation of 10 CFR 50, Appendix B, Criterion III, "Design Control," and was documented in 4Inspection Report 05000254/2006012; 05000265/2006012, issued May 30, 2006. Thefinal significance determination and notice of violation letter was issued on June 29, 2006, as Inspection Report 05000254/2006014.The licensee's root cause evaluation, extent of condition assessment, and corrective actionplan were documented in Issue Report (IR) 435858, "U2 D ERV Declared INOP/Unplanned14 Day LCO." The objectives of this inspection were to provide assurance that:
- (1) the rootcauses and contributing causes of risk significant performance issues were identified,
- (2) the extent of condition and extent of cause of the issues were identified, and
- (3) the corrective actions were sufficient to address the root causes and contributing causes so that recurrence is prevented. The inspector reviewed inspection reports, reviewed licensee procedures anddocumentation,and interviewed licensee staff to accomplish the inspection objectives.02EVALUATION OF INSPECTION REQUIREMENTS02.01Problem Identification a.Determine that the root cause evaluation identifies who (i.e., licensee, self-revealing, orNRC) and under what conditions the issue was identified.The inspector determined that the root cause evaluation adequately identified who andunder what conditions the issue was identified. Although the licensee's root cause report does not explicitly state that the issue wasself-revealing, it does accurately describe the sequence of events leading to identification of the issue. Additionally, the licensee's functional area self-evaluation of the root cause report and associated issues, documented in IR 552100, acknowledges this documentation oversight and identifies the issue as self-revealing through investigation of the December 2005, Unit 2 125 VDC ground on the 3D ERV valve control circuit (IR 43857, 435227, and 427819). The Root Cause report and associated documentation provided a detailed description of the equipment condition, investigation performed, and sequence of events. b. Determine that the evaluation documents how long the issue existed, and prior opportunitiesfor identification. The inspector determined that the root cause evaluation adequately identified how long theissue existed and prior opportunities for identification.With regard to how long the issue existed, the Root Cause evaluation documented that thesubject ERVs were installed during the original design and construction of the plant and the first vibration analysis took place in 1978 in response to a failure of the 3E ERV on Unit 2.
The report acknowledged that work and event histories with the ERVs on each Quad Cities unit indicated vibration of the Main Steam Lines (MSLs), coupled with loose rebuild and installation tolerances, affected the performance and operating life of these valves from initial construction; however, multiple concurrent failures did not occur until EPU was implemented.
5With regard to prior opportunities for identification, the evaluation identified a shortcoming inresolution of issues identified in the 1978 analysis and also missed opportunities to correct the problem in 1993, 2000, 2002, 2003, 2004 and 2005. Quad Cities assessment of the historical performance of these valves resulted in identification of 24 equipment related events that could be tied to vibration issues. Examples of equipment issues that could beattributed to or made worse by vibration included abnormal wear of mechanical parts, failed ERV components, through seat leakage of the ERVs, failure of the valve to open both during testing and when required following a reactor scram, and failure of the valve to close during testing. c.Determine that the evaluation documents the plant-specific risk consequences (asapplicable) and compliance concerns associated with the issue.The inspectors determined that the licensee's evaluation adequately documented theplant-specific risk consequences and compliance concerns associated with the White finding related to the failure to establish measures to ensure that the Unit 1 ERVsremained suitable for operation prior to implementing the 2002 EPU
.As documented in IR 435858, the evaluation of risk was performed in Exelon's probalisticrisk assessment (PRA) calculation SA-1425, Revision 3, and concluded that the SDP Phase 3 color would be Green. Subsequent discussion with NRC Region III Senior RiskAnalysts resulted in changes to the calculation that were documented in SA-1551, Revision 0 and concluded that the SDP color was White.The licensee's evaluation stated that compliance concerns with the ERVs themselves wereresolved following installation of the acoustic side branch (ASB) devices and a modified actuator. EPU operation began in 2002 (17 percent power uprate). The higher EPU power level increased the steam velocity in the MSLs, which contributed to higher MSL vibration loads. Quad Cities testing identified the source of the vibrations to be from the effects of steam flow past the inlet standpipes to the four ERVs and eight MSL safety valves. The ASB modification changed the inlet configuration associated with the ERVs and safety valves. The modified inlet configuration was designed to reduce the acoustic coupling between the forcing function (i.e., vortex shedding at the valve inlet) and the acoustic frequency of the valve inlet volume at EPU conditions. Following installation on each unit, a monitoring plan was implemented to validate ASB performance. The startup plan was designed to verify that plant and equipment performance remained within established acceptance criteria. Data was collected using installed strain gauges and accelerometers located on the MSLs and vibration-sensitive components (additional instrumentation was installed on both Unit 1 and Unit 2 to facilitate testing). The inspector reviewed the most current data and determined that the data continued to support the licensee's position that the ASB modification is performing as designed. Acoustic vibration levels in the range of concern (i.e., 140Hz to 160Hz) were significantly reduced. The data provided by the licensee indicated acoustic vibration loads as measured by installed strain gauges and accelerometers located on vibration sensitive equipment have been reduced to levels that were equivalent to or below those experienced at the original licensed thermal power level.An upgraded ERV design was installed in Units 1 and 2 to further improve performance andprovide further assurance of continued compliance. The actuators were designed to improve performance in this environment due to utilization of stellite guides and guideposts, 6stainless steel pivot pins, new spring configuration (non-buckling spring arrangement),improved vibration isolation, and specified component tolerances. In addition, actuator rigidity was enhanced through the addition of an upper angle bracket to fix the guide post alignment.02.02Root Cause and Extent of Conditiona. Determine that the problem was evaluated using a systematic method(s) to identify rootcause(s) and contributing cause(s).The inspectors determined that the licensee's evaluation of the ERV failures that resulted inthe White finding, as well as the organizational contributors to the issues, were adequately evaluated using systematic methods to identify root and contributing causes.Licensee personnel completed a root cause evaluation that reviewed the circumstancessurrounding the ERV failures identified in December 2005 and January 2006. The root cause evaluation performed for IR 435858, "Unit 2 D ERV Declared INOP/Unplanned 14 Day LCO," utilized several systematic methodologies to identify the root cause and contributing causes for the failures that resulted in the White finding. The root cause evaluation used Events and Causal Factor flowcharting, Barrier analysis, Task Analysis, Cause and Effect Analysis, Multi-Linear Events Sequencing, TapRoot, and Kepner-Tregoe (K-T) Problem Solving techniques.Further, the licensee performed a self-assessment of the root cause and corrective actionswhich was documented in IR 552100, "FASA-Preparation for Supplemental Inspection ERV Root
Cause.
" This self-assessment identified some minor issues with the documentation for the root cause evaluation and recommended some additional actions. While these additional recommendations were not required to prevent recurrence of the issues, they were needed to provide complete documentation of the corrective action plan. The inspector reviewed the documentation provided for the systematic tools used to identifythe root cause and contributing causes and determined that the identified causes were appropriate and that these evaluations adequately addressed the individual equipment and organizational issues that were identified to exist. b.Determine that the root cause evaluation was conducted to a level of detail commensuratewith the significance of the problem.The inspector reviewed licensee documents, reviewed test data and conducted interviewswith station personnel to assess the licensee's product. The inspector determined that the root cause evaluation was conducted to an adequate level of detail commensurate with the significance of the issue.The inspector determined that the licensee investigations of the ERV failure, the planthistorical performance, and the organizational mismanagement of the vibration issues that resulted in the White finding were sufficiently detailed, adequately identified the root causes of the failures, and prescribed appropriate corrective actions to prevent reccurrence.
7c. Determine that the root cause evaluation included a consideration of prior occurrences ofthe problem and knowledge of prior operating experience.The inspector determined that the root cause evaluation adequately considered prioroccurrences and prior operating experience. The inspector determined that the root cause evaluation included a thorough evaluation ofthe historical performance of the valves as far back as 1978, including previous failures and corrective actions through issue reports and work packages. The evaluation also considered information gathered through investigation of steam dryer failures and other impacts of extended power uprate. The evaluation also included a documented review of related industry experience. The inspector also determined that the root case evaluation documented involvement of the General Electric Co. and other vendors in identification of the problems and proposed resolutions.
d.Determine that the root cause evaluation included consideration of potential commoncauses and the extent of condition of the problem.The inspector determined that the root cause evaluation adequately addressed potentialcommon causes and the extent of cause of the issue.Extent of condition evaluations addressed potential issues with units at Quad Cities andDresden, large bore and small bore piping on main steam, feedwater, and condensate piping systems, associated instrumentation ,and pipe supports/hangers. Problems with steam dryer operation were addressed in a different root cause evaluation (IR 472321),
but the organizational contributors to that issue were found to be the same as those identified in the ERV root cause. Additional reviews were performed of historical vendor studies, the original EPU Vulnerability Study, and maintenance procedures for ERVs to identify other issues that may not have been resolved or documented appropriately.
Potential common causes were evaluated in the K-T analysis, TapRoot evaluation and Barrier
Analysis.
No new issues were identified.e.Determine that the root cause evaluation, extent of condition and the extent of causeappropriately considered the safety culture components as described in IMC 0305.The inspector determined that the root cause evaluation, extent of condition, and the extentof cause appropriately considered the safety culture components as described in IMC 0305. The organizational aspects of the issue were performed independently from the rootcause team and incorporated into the evaluation and corrective action plan. However, the inspector observed that for two safety components in areas where causal factors and corrective actions for the organizational contributing factors were identified, the facility appeared to rely on programs and processes without having or establishing a clear plan for monitoring to ensure continuing compliance. While not rising to the level of a finding or a significant weakness, these observations identified opportunities for the licensee to act to prevent potential vulnerabilities from developing that could result in inadequate engineering product development in the future. The two safety culture components are:
8*Accountability - Management defines the line of authority and responsibility for nuclearsafety. Specifically the workforce demonstrates a proper safety focus and reinforces safety principles among their peers.
The licensee identified examples in the history of the ERV issue and contractmanagement of the EPU project where appropriate focus and accountability were not maintained. Additionally, discussions in the evaluation documented significant deficiencies in vendor products with little discussion of project manager/lead engineer's accountability for product specifications, compliance with specifications, or acceptanceof inadequate response from vendors. While the evaluation and subsequent self-assessment were self-critical and the changes provided process improvement, it was not clear to the inspector how thoroughly the accountability and ownership is addressed on the Engineer level. Licensee corrective actions relied on revisions to corporate level process checklists andprogram procedures to address this component of safety culture. The message that this approach sends, unless carefully managed, is that the processes were the problem rather than the ineffective implementation of the processes. Unfortunately, the potential exists to undermine the accountability within the engineering organization.While procedures and checklists are important human performance improvement tools,reinforcement of expectations with specific examples of inadequate performance and active monitoring of work execution are critical to reinforcing organizational and peer to peer accountability. These actions may be implemented by other programs at Quad Cities, but the root cause evaluation did not document these actions or reference the tools that would accomplish them.
- Continuous Learning Environment - The licensee effectively communicates informationlearned from internal and external sources about industry and plant issues. The corrective action plan clearly contained actions that supported a continuouslearning environment. Communications with the industry on the events lessons learned, implementation of a detailed monitoring plan to evaluate effectiveness of the vibration mitigation actions, and training of site and corporate engineers in the application of OP-AA-106-1006, "Operational and Technical Decision Making Process," support this important component of safety culture. However, in other areas where corrective actions were made to revise checklists and programs, opportunities to incorporate information sharing into the processes may not be effective.Supervisors are expected to review effectiveness of pre-job briefs (post job briefs maybe intended to accomplish that function in part, but the words in HU-AA-1212 are not specific) and provide feedback to the engineers involved. Personnel interviewed indicated that it was the individual supervisors' responsibility to identify training needs or performance weaknesses input to the training organization to incorporate lessons learned in this area into continuing training. As a result, individual engineers may seethis feedback as only applicable to themselves and opportunities may be missed when there are organizational lessons that should be addressed.
9Individuals interviewed indicated that HU-AA-1212 post job briefs were performed asdetermined by the supervisor for engineering activities rather than for all tasks. Lessons learned from these critiques are shared by the supervisor at the Engineering Plan of the Day meeting on a rotating basis. The supervisor attending the meeting determines which of the lessons learned would then shared with the individual engineers. There is no other mechanism to ensure that all engineers who need the information receive it.
Additionally, there is no tool to ensure that the engineers receive a complete and consistent message. As a result, the message may be inconsistent and the effort may not provide the appropriate reinforcement of expectations. 02.03 Corrective Actionsa.Determine that appropriate corrective action(s) are specified for each root cause, or thatthere is an evaluation that no actions are necessary
.The inspector determined that appropriate corrective actions were specified for each root cause.The following specific corrective actions were implemented to prevent recurrence for theERV equipment and organizational failures:*Station Design Engineering working with Exelon corporate personnel shall provide aUnit 1 design change to reduce overall main steam line vibration to a level to support safe and reliable operation of the MSLs and attached components during future operating cycles at EPU.*Station Design Engineering working with Exelon corporate personnel shall provide aUnit 2 design change to reduce overall main steam line vibration to a level to support safe and reliable operation of the MSLs and attached components during future operating cycles at EPU.*Project Management working with Outage Planning shall implement a Unit 1 designchange to reduce overall main steam line vibrations.*Project Management working with Outage Planning shall implement a Unit 2 designchange to reduce overall main steam line vibrations.*Perform an investigation of the organizational issues as captured in IR 448773 andissue supplemental report to IR 435858 action 07 (licensee's root cause evaluation for the ERV issue.)*Present the supplemental root cause report to the Management ReviewCommittee/Plant Oversight and Review Committee for review and approval.While the organization corrective action was worded somewhat open-ended, the licenseeinvestigation documented organizational issues and went on to specify corrective actions which the licensee treated as actions to prevent recurrence.
10b.Determine that the corrective actions have been prioritized with consideration of the risksignificance and regulatory compliance.The inspectors determined that the corrective actions were adequately prioritized withconsideration of risk significance and regulatory compliance. All corrective actions associated with the previous ERV failures had been completed withthe exception of one action to develop a new test control program. Preliminary work required to determine process and methodology was completed and the new program is to be housed in a new corporate procedure, CC-AA-107-1002, "Test Requirements Control."
That action is scheduled to be completed in March of 2007 and the information provided by the licensee supported completion with enough information included in the corrective action documentation to allow the inspector to determine the intended action will be completed as intended.c.Determine that a schedule has been established for implementing and completing thecorrective actions
.The inspectors determined that an acceptable schedule was established for theimplementation and completion of corrective actions in accordance with the licensee's corrective action program. With the exception of implementing the new test control program, all corrective actions have been accomplished. For the remaining actions including the identified effectiveness measures, the inspector determined that an appropriate schedule had been established.d.Determine that quantitative or qualitative measures of success have been developed fordetermining the effectiveness of the corrective actions to prevent recurrence.The inspectors determined that adequate qualitative and quantitative measures ofsuccess were developed for determining the effectiveness of corrective actions to prevent recurrence. Effectiveness measures have been established for each unit and closure dates coincide with scheduled plant outages to allow physical inspections of the valves and associated components.03MANAGEMENT MEETINGSExit Meeting SummaryThe inspector presented the inspection results to Mr. Tulon and other members of licenseemanagement and staff at the conclusion of the inspection on February 23, 2007. The licensee acknowledged the information presented. Regulatory Performance MeetingFollowing the inspection exit meeting, Mark Ring, Branch Chief RIII-Branch 1, conducteda Regulatory Performance Meeting with Mr. Tulon and other members of the licensee management and staff. Mr. Ring acknowledged that the inspection concluded that the licensee understands the root causes and contributing causes of the ERV issues, that thelicensee identified the extent of condition and extent of cause of the issues, and that the 11licensee's corrective actions were determined to be sufficient to address the causes and toprevent recurrence of the issues. Per the guidance in Manual Chapter 0305, inspection findings are normally considered in the assessment program for four quarters providing the issues have been addressed and corrected. Thus, the White finding will remain active for consideration of further agency action, per the 0305 Action Matrix, through the second quarter of 2007. The licensee acknowledged the information presented with no questions.
1
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee
- T. Tulon, Site Vice President, Exelon
- R. Gideon, Plant Manager, Exelon
- B. Adams, Director Engineering/QC, Exelon
- R. Svaleson, Director Operations, Exelon
- D. Craddick, Director Maintenance, Exelon
- S. Darin, Manager Systems Engineering, Exelon
- W. Beck, Manager Regulatory Assurance, Exelon
- D. Moore, Manager Nuclear Oversight, Exelon
- K. Moser, Manager training, Exelon
- J. Trettin, Engineering, Exelon
- T. Fuhs, Regulatory Assurance, Exelon
- T. Wojcik, Engineering, Exelon
- D. Tubbs, MidAmerican Energy
LIST OF ITEMS
OPENED, CLOSED AND DISCUSSEDOpened None.ClosedVIO 254/2006014-01Failure to establish measures toEnsure that the U1 ERV actuators
Remained suitable for operation at EPU
Discussed
None.
AttachmentAttachment
2
LIST OF DOCUMENTS REVIEWED
The following is a list of documents reviewed during the inspection.
- Inclusion on this list doesnot imply that the NRC inspector reviewed the documents in their entirety but rather that