IR 05000254/2007007

IR 05000254-07-007; 2/20/2007 - 2/23/2007; Quad Cities Nuclear Plant Unit 1; Supplemental Inspection; Inspection Procedure 95001, Inspection for One or Two White Inputs in a Strategic Performance Area.

ML070890547
Person / Time
Site:	Quad Cities
Issue date:	03/30/2007
From:	Satorius M Division Reactor Projects III
To:	Crane C Exelon Generation Co, Exelon Nuclear
References
EA-06-112 IR-07-007
Download: ML070890547 (19)

Text

rch 30, 2007

SUBJECT:

QUAD CITIES NUCLEAR PLANT, UNIT 1 NRC SUPPLEMENTAL INSPECTION REPORT 05000254/2007007

Dear Mr. Crane:

On February 23, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Quad Cities Nuclear Plant, Unit 1. The enclosed report documents the inspection results which were discussed on February 23, 2007, with T. Tulon and other members of your staff.

The NRC performed this supplemental inspection to assess your evaluation of the White finding identified in Nuclear Regulatory Commission (NRC) Inspection Report 05000254/2006014, issued June 29, 2006. The White finding was also a violation of 10 CFR 50, Appendix B, Criterion III, Design Control, and involved the failure to establish measures to ensure that the Unit 1 electromatic relief valves remained suitable for operation prior to implementing the 2002 extended power uprate.

We conducted this inspection in accordance with Inspection Procedure 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area, and examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.

The objectives of this inspection were to provide assurance that: (1) the root causes and contributing causes of risk significant performance issues were identified, (2) the extent of condition and extent of cause of the issues were identified, and (3) your corrective actions were sufficient to address the root causes and contributing causes so that recurrence is prevented.

Based on the results of this inspection, no findings of significance were identified and we concluded that you understood the root causes and contributing causes of the issues, that you identified the extent of condition and extent of cause of the issues, and that your corrective actions were sufficient to address the causes and to prevent recurrence of the issues. In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Mark A. Satorius, Director Division of Reactor Projects Docket Nos. 50-254 License Nos. DPR-29 Enclosure: Inspection Report 05000254/2007007 w/Attachment: Supplemental Information cc w/encl: Site Vice President - Quad Cities Nuclear Power Station Plant Manager - Quad Cities Nuclear Power Station Regulatory Assurance Manager - Quad Cities Nuclear Power Station Chief Operating Officer Senior Vice President - Nuclear Services Senior Vice President - Mid-West Regional Operating Group Vice President - Mid-West Operations Support Vice President - Licensing and Regulatory Affairs Director Licensing - Mid-West Regional Operating Group Manager Licensing - Dresden and Quad Cities Senior Counsel, Nuclear, Mid-West Regional Operating Group Document Control Desk - Licensing Vice President - Law and Regulatory Affairs Mid American Energy Company Assistant Attorney General Illinois Emergency Management Agency State Liaison Officer, State of Illinois State Liaison Officer, State of Iowa Chairman, Illinois Commerce Commission Chief Radiological Emergency Preparedness Section, Dept. Of Homeland Security D. Tubbs, Manager of Nuclear MidAmerican Energy Company

SUMMARY OF FINDINGS

IR 05000254/2007007; 2/20/2007 - 2/23/2007; Quad Cities Nuclear Plant Unit 1; Supplemental

Inspection; Inspection Procedure 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area.

A Reactor Engineer from Region III performed this supplemental inspection. No findings of significance were identified. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process,

Revision 3, dated July 2000.

Cornerstone: Mitigating Systems

The Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess the licensees evaluation of a White finding identified in NRC Inspection Report 05000254/2006014, issued June 29, 2006. The White finding was also a violation of 10 CFR 50, Appendix B, Criterion III Design Control, and involved the failure to establish measures to ensure that the Unit 1 electromatic relief valves remained suitable for operation prior to implementing the 2002 extended power uprate. This inspection was conducted in accordance with Inspection Procedure 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area. The inspector concluded that: (1) the licensee understood the root causes and contributing causes of the risk significant performance issues that resulted in the White finding; (2) the licensee identified the extent of condition and the extent of cause of the issues; and (3) the licensees corrective actions were sufficient to address the root causes and contributing causes, and to prevent recurrence.

A. Inspector-Identified and Self-Revealed Findings No findings of significance were identified.

Licensee-Identified Violations

None.

REPORT DETAILS

INSPECTION SCOPE

The NRC performed this supplemental inspection to assess the licensees evaluation of a White finding identified in Nuclear Regulatory Commission (NRC) Inspection Report 05000254/2006014, issued June 29, 2006. Increased vibrations experienced while operating at extended power uprate (EPU) power levels resulted in the degradation of multiple electromatic relief valves (ERV) actuator components which rendered the valves inoperable.

The inspectors determined that the licensee implemented the Unit 1 EPU in November 2002, but failed to verify that the ERV actuator design was suitable for operation at the increased vibration levels experienced at EPU power levels. Organizational weaknesses at the station and corporate levels contributed to the licensees failure to identify this issue prior to, or immediately following, EPU implementation.

Events Preceding the Notice of Violation:

• In December 2006, station personnel determined that an intermittent 125 VDC ground was present on the 3D ERV actuator circuitry on Unit 2.

• In response to this issue, an at-power drywell entry was conducted on December 30.

During this entry, the licensee identified that several of the actuator components were significantly degraded.

• Unit 2 was shut down on December 31, 2005, to allow inspections of the remaining ERV actuators. Varying levels of degradation were identified on the remaining Unit 2 ERV actuators.

• Due to the amount of degradation identified, the licensee reduced Unit 1 reactor power to pre-extended power uprate (EPU) power levels pending an inspection of the Unit 1 ERV actuators. During a planned Unit 1 maintenance outage conducted on January 6, 2006, the licensee discovered that the Unit 1 ERV actuators were also degraded.

• On January 9, 2006, the NRC initiated a Special Inspection to assess the licensees efforts in identifying and correcting the cause of the ERV actuator degradation. The inspectors were also tasked with determining the safety significance of the ERV actuator degradation for both units. The results of this inspection were documented in Inspection Report 05000254/2006009; 05000265/2006009. However, the inspectors were unable to fully assess the effectiveness of the licensees root cause efforts, and the significance of the ERV actuator degradation, during the inspection since the licensees root cause efforts were not complete.

• On April 15, 2006, the licensee issued the ERV actuator root cause report. The inspectors examined the root cause report, compared the report information to the information gained during the Special Inspection, and discussed the report results with licensee personnel to assess the adequacy of the licensees efforts. The inspectors also reviewed the results of multiple ERV actuator tests and inspections to determine the safety significance of this issue for both units. The finding was determined to be an apparent violation of 10 CFR 50, Appendix B, Criterion III, Design Control, and was documented in Inspection Report 05000254/2006012; 05000265/2006012, issued May 30, 2006. The final significance determination and notice of violation letter was issued on June 29, 2006, as Inspection Report 05000254/2006014.

The licensees root cause evaluation, extent of condition assessment, and corrective action plan were documented in Issue Report (IR) 435858, U2 D ERV Declared INOP/Unplanned 14 Day LCO. The objectives of this inspection were to provide assurance that:

(1) the root causes and contributing causes of risk significant performance issues were identified,

(2) the extent of condition and extent of cause of the issues were identified, and

(3) the corrective actions were sufficient to address the root causes and contributing causes so that recurrence is prevented. The inspector reviewed inspection reports, reviewed licensee procedures and documentation,and interviewed licensee staff to accomplish the inspection objectives.

02 EVALUATION OF INSPECTION REQUIREMENTS 02.01 Problem Identification a. Determine that the root cause evaluation identifies who (i.e., licensee, self-revealing, or NRC) and under what conditions the issue was identified.

The inspector determined that the root cause evaluation adequately identified who and under what conditions the issue was identified.

Although the licensees root cause report does not explicitly state that the issue was self-revealing, it does accurately describe the sequence of events leading to identification of the issue. Additionally, the licensees functional area self-evaluation of the root cause report and associated issues, documented in IR 552100, acknowledges this documentation oversight and identifies the issue as self-revealing through investigation of the December 2005, Unit 2 125 VDC ground on the 3D ERV valve control circuit (IR 43857, 435227, and 427819). The Root Cause report and associated documentation provided a detailed description of the equipment condition, investigation performed, and sequence of events.

b. Determine that the evaluation documents how long the issue existed, and prior opportunities for identification.

The inspector determined that the root cause evaluation adequately identified how long the issue existed and prior opportunities for identification.

With regard to how long the issue existed, the Root Cause evaluation documented that the subject ERVs were installed during the original design and construction of the plant and the first vibration analysis took place in 1978 in response to a failure of the 3E ERV on Unit 2.

The report acknowledged that work and event histories with the ERVs on each Quad Cities unit indicated vibration of the Main Steam Lines (MSLs), coupled with loose rebuild and installation tolerances, affected the performance and operating life of these valves from initial construction; however, multiple concurrent failures did not occur until EPU was implemented.

With regard to prior opportunities for identification, the evaluation identified a shortcoming in resolution of issues identified in the 1978 analysis and also missed opportunities to correct the problem in 1993, 2000, 2002, 2003, 2004 and 2005. Quad Cities assessment of the historical performance of these valves resulted in identification of 24 equipment related events that could be tied to vibration issues. Examples of equipment issues that could be attributed to or made worse by vibration included abnormal wear of mechanical parts, failed ERV components, through seat leakage of the ERVs, failure of the valve to open both during testing and when required following a reactor scram, and failure of the valve to close during testing.

c. Determine that the evaluation documents the plant-specific risk consequences (as applicable) and compliance concerns associated with the issue.

The inspectors determined that the licensees evaluation adequately documented the plant-specific risk consequences and compliance concerns associated with the White finding related to the failure to establish measures to ensure that the Unit 1 ERVs remained suitable for operation prior to implementing the 2002 EPU.

As documented in IR 435858, the evaluation of risk was performed in Exelons probalistic risk assessment (PRA) calculation SA-1425, Revision 3, and concluded that the SDP Phase 3 color would be Green. Subsequent discussion with NRC Region III Senior Risk Analysts resulted in changes to the calculation that were documented in SA-1551, Revision 0 and concluded that the SDP color was White.

The licensees evaluation stated that compliance concerns with the ERVs themselves were resolved following installation of the acoustic side branch (ASB) devices and a modified actuator. EPU operation began in 2002 (17 percent power uprate). The higher EPU power level increased the steam velocity in the MSLs, which contributed to higher MSL vibration loads. Quad Cities testing identified the source of the vibrations to be from the effects of steam flow past the inlet standpipes to the four ERVs and eight MSL safety valves. The ASB modification changed the inlet configuration associated with the ERVs and safety valves. The modified inlet configuration was designed to reduce the acoustic coupling between the forcing function (i.e., vortex shedding at the valve inlet) and the acoustic frequency of the valve inlet volume at EPU conditions. Following installation on each unit, a monitoring plan was implemented to validate ASB performance. The startup plan was designed to verify that plant and equipment performance remained within established acceptance criteria. Data was collected using installed strain gauges and accelerometers located on the MSLs and vibration-sensitive components (additional instrumentation was installed on both Unit 1 and Unit 2 to facilitate testing). The inspector reviewed the most current data and determined that the data continued to support the licensees position that the ASB modification is performing as designed. Acoustic vibration levels in the range of concern (i.e., 140Hz to 160Hz) were significantly reduced. The data provided by the licensee indicated acoustic vibration loads as measured by installed strain gauges and accelerometers located on vibration sensitive equipment have been reduced to levels that were equivalent to or below those experienced at the original licensed thermal power level.

An upgraded ERV design was installed in Units 1 and 2 to further improve performance and provide further assurance of continued compliance. The actuators were designed to improve performance in this environment due to utilization of stellite guides and guideposts, stainless steel pivot pins, new spring configuration (non-buckling spring arrangement),improved vibration isolation, and specified component tolerances. In addition, actuator rigidity was enhanced through the addition of an upper angle bracket to fix the guide post alignment.

02.02 Root Cause and Extent of Condition a. Determine that the problem was evaluated using a systematic method(s) to identify root cause(s) and contributing cause(s).

The inspectors determined that the licensees evaluation of the ERV failures that resulted in the White finding, as well as the organizational contributors to the issues, were adequately evaluated using systematic methods to identify root and contributing causes.

Licensee personnel completed a root cause evaluation that reviewed the circumstances surrounding the ERV failures identified in December 2005 and January 2006. The root cause evaluation performed for IR 435858, Unit 2 D ERV Declared INOP/Unplanned 14 Day LCO, utilized several systematic methodologies to identify the root cause and contributing causes for the failures that resulted in the White finding. The root cause evaluation used Events and Causal Factor flowcharting, Barrier analysis, Task Analysis, Cause and Effect Analysis, Multi-Linear Events Sequencing, TapRoot, and Kepner-Tregoe (K-T) Problem Solving techniques.

Further, the licensee performed a self-assessment of the root cause and corrective actions which was documented in IR 552100, FASA-Preparation for Supplemental Inspection ERV Root

Cause.

This self-assessment identified some minor issues with the documentation for the root cause evaluation and recommended some additional actions. While these additional recommendations were not required to prevent recurrence of the issues, they were needed to provide complete documentation of the corrective action plan.

The inspector reviewed the documentation provided for the systematic tools used to identify the root cause and contributing causes and determined that the identified causes were appropriate and that these evaluations adequately addressed the individual equipment and organizational issues that were identified to exist.

b. Determine that the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem.

The inspector reviewed licensee documents, reviewed test data and conducted interviews with station personnel to assess the licensees product. The inspector determined that the root cause evaluation was conducted to an adequate level of detail commensurate with the significance of the issue.

The inspector determined that the licensee investigations of the ERV failure, the plant historical performance, and the organizational mismanagement of the vibration issues that resulted in the White finding were sufficiently detailed, adequately identified the root causes of the failures, and prescribed appropriate corrective actions to prevent reccurrence.

c. Determine that the root cause evaluation included a consideration of prior occurrences of the problem and knowledge of prior operating experience.

The inspector determined that the root cause evaluation adequately considered prior occurrences and prior operating experience.

The inspector determined that the root cause evaluation included a thorough evaluation of the historical performance of the valves as far back as 1978, including previous failures and corrective actions through issue reports and work packages. The evaluation also considered information gathered through investigation of steam dryer failures and other impacts of extended power uprate. The evaluation also included a documented review of related industry experience. The inspector also determined that the root case evaluation documented involvement of the General Electric Co. and other vendors in identification of the problems and proposed resolutions.

d. Determine that the root cause evaluation included consideration of potential common causes and the extent of condition of the problem.

The inspector determined that the root cause evaluation adequately addressed potential common causes and the extent of cause of the issue.

Extent of condition evaluations addressed potential issues with units at Quad Cities and Dresden, large bore and small bore piping on main steam, feedwater, and condensate piping systems, associated instrumentation ,and pipe supports/hangers. Problems with steam dryer operation were addressed in a different root cause evaluation (IR 472321),but the organizational contributors to that issue were found to be the same as those identified in the ERV root cause. Additional reviews were performed of historical vendor studies, the original EPU Vulnerability Study, and maintenance procedures for ERVs to identify other issues that may not have been resolved or documented appropriately.

Potential common causes were evaluated in the K-T analysis, TapRoot evaluation and Barrier

Analysis.

No new issues were identified.

e. Determine that the root cause evaluation, extent of condition and the extent of cause appropriately considered the safety culture components as described in IMC 0305.

The inspector determined that the root cause evaluation, extent of condition, and the extent of cause appropriately considered the safety culture components as described in IMC 0305.

The organizational aspects of the issue were performed independently from the root cause team and incorporated into the evaluation and corrective action plan. However, the inspector observed that for two safety components in areas where causal factors and corrective actions for the organizational contributing factors were identified, the facility appeared to rely on programs and processes without having or establishing a clear plan for monitoring to ensure continuing compliance. While not rising to the level of a finding or a significant weakness, these observations identified opportunities for the licensee to act to prevent potential vulnerabilities from developing that could result in inadequate engineering product development in the future. The two safety culture components are:

• Accountability - Management defines the line of authority and responsibility for nuclear safety. Specifically the workforce demonstrates a proper safety focus and reinforces safety principles among their peers.

The licensee identified examples in the history of the ERV issue and contract management of the EPU project where appropriate focus and accountability were not maintained. Additionally, discussions in the evaluation documented significant deficiencies in vendor products with little discussion of project manager/lead engineers accountability for product specifications, compliance with specifications, or acceptance of inadequate response from vendors. While the evaluation and subsequent self-assessment were self-critical and the changes provided process improvement, it was not clear to the inspector how thoroughly the accountability and ownership is addressed on the Engineer level.

Licensee corrective actions relied on revisions to corporate level process checklists and program procedures to address this component of safety culture. The message that this approach sends, unless carefully managed, is that the processes were the problem rather than the ineffective implementation of the processes. Unfortunately, the potential exists to undermine the accountability within the engineering organization.

While procedures and checklists are important human performance improvement tools, reinforcement of expectations with specific examples of inadequate performance and active monitoring of work execution are critical to reinforcing organizational and peer to peer accountability. These actions may be implemented by other programs at Quad Cities, but the root cause evaluation did not document these actions or reference the tools that would accomplish them.

• Continuous Learning Environment - The licensee effectively communicates information learned from internal and external sources about industry and plant issues.

The corrective action plan clearly contained actions that supported a continuous learning environment. Communications with the industry on the events lessons learned, implementation of a detailed monitoring plan to evaluate effectiveness of the vibration mitigation actions, and training of site and corporate engineers in the application of OP-AA-106-1006, Operational and Technical Decision Making Process, support this important component of safety culture. However, in other areas where corrective actions were made to revise checklists and programs, opportunities to incorporate information sharing into the processes may not be effective.

Supervisors are expected to review effectiveness of pre-job briefs (post job briefs may be intended to accomplish that function in part, but the words in HU-AA-1212 are not specific) and provide feedback to the engineers involved. Personnel interviewed indicated that it was the individual supervisors responsibility to identify training needs or performance weaknesses input to the training organization to incorporate lessons learned in this area into continuing training. As a result, individual engineers may see this feedback as only applicable to themselves and opportunities may be missed when there are organizational lessons that should be addressed.

Individuals interviewed indicated that HU-AA-1212 post job briefs were performed as determined by the supervisor for engineering activities rather than for all tasks. Lessons learned from these critiques are shared by the supervisor at the Engineering Plan of the Day meeting on a rotating basis. The supervisor attending the meeting determines which of the lessons learned would then shared with the individual engineers. There is no other mechanism to ensure that all engineers who need the information receive it.

Additionally, there is no tool to ensure that the engineers receive a complete and consistent message. As a result, the message may be inconsistent and the effort may not provide the appropriate reinforcement of expectations.

02.03 Corrective Actions a. Determine that appropriate corrective action(s) are specified for each root cause, or that there is an evaluation that no actions are necessary.

The inspector determined that appropriate corrective actions were specified for each root cause.

The following specific corrective actions were implemented to prevent recurrence for the ERV equipment and organizational failures:

• Station Design Engineering working with Exelon corporate personnel shall provide a Unit 1 design change to reduce overall main steam line vibration to a level to support safe and reliable operation of the MSLs and attached components during future operating cycles at EPU.

• Station Design Engineering working with Exelon corporate personnel shall provide a Unit 2 design change to reduce overall main steam line vibration to a level to support safe and reliable operation of the MSLs and attached components during future operating cycles at EPU.

• Project Management working with Outage Planning shall implement a Unit 1 design change to reduce overall main steam line vibrations.

• Project Management working with Outage Planning shall implement a Unit 2 design change to reduce overall main steam line vibrations.

• Perform an investigation of the organizational issues as captured in IR 448773 and issue supplemental report to IR 435858 action 07 (licensees root cause evaluation for the ERV issue.)

• Present the supplemental root cause report to the Management Review Committee/Plant Oversight and Review Committee for review and approval.

While the organization corrective action was worded somewhat open-ended, the licensee investigation documented organizational issues and went on to specify corrective actions which the licensee treated as actions to prevent recurrence.

b. Determine that the corrective actions have been prioritized with consideration of the risk significance and regulatory compliance.

The inspectors determined that the corrective actions were adequately prioritized with consideration of risk significance and regulatory compliance.

All corrective actions associated with the previous ERV failures had been completed with the exception of one action to develop a new test control program. Preliminary work required to determine process and methodology was completed and the new program is to be housed in a new corporate procedure, CC-AA-107-1002, Test Requirements Control.

That action is scheduled to be completed in March of 2007 and the information provided by the licensee supported completion with enough information included in the corrective action documentation to allow the inspector to determine the intended action will be completed as intended.

c. Determine that a schedule has been established for implementing and completing the corrective actions.

The inspectors determined that an acceptable schedule was established for the implementation and completion of corrective actions in accordance with the licensees corrective action program. With the exception of implementing the new test control program, all corrective actions have been accomplished. For the remaining actions including the identified effectiveness measures, the inspector determined that an appropriate schedule had been established.

d. Determine that quantitative or qualitative measures of success have been developed for determining the effectiveness of the corrective actions to prevent recurrence.

The inspectors determined that adequate qualitative and quantitative measures of success were developed for determining the effectiveness of corrective actions to prevent recurrence. Effectiveness measures have been established for each unit and closure dates coincide with scheduled plant outages to allow physical inspections of the valves and associated components.

03 MANAGEMENT MEETINGS

Exit Meeting Summary

The inspector presented the inspection results to Mr. Tulon and other members of licensee management and staff at the conclusion of the inspection on February 23, 2007. The licensee acknowledged the information presented.

Regulatory Performance Meeting Following the inspection exit meeting, Mark Ring, Branch Chief RIII-Branch 1, conducted a Regulatory Performance Meeting with Mr. Tulon and other members of the licensee management and staff. Mr. Ring acknowledged that the inspection concluded that the licensee understands the root causes and contributing causes of the ERV issues, that the licensee identified the extent of condition and extent of cause of the issues, and that the licensees corrective actions were determined to be sufficient to address the causes and to prevent recurrence of the issues. Per the guidance in Manual Chapter 0305, inspection findings are normally considered in the assessment program for four quarters providing the issues have been addressed and corrected. Thus, the White finding will remain active for consideration of further agency action, per the 0305 Action Matrix, through the second quarter of 2007. The licensee acknowledged the information presented with no questions.

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee

T. Tulon, Site Vice President, Exelon

R. Gideon, Plant Manager, Exelon

B. Adams, Director Engineering/QC, Exelon

R. Svaleson, Director Operations, Exelon

D. Craddick, Director Maintenance, Exelon

S. Darin, Manager Systems Engineering, Exelon

W. Beck, Manager Regulatory Assurance, Exelon

D. Moore, Manager Nuclear Oversight, Exelon

K. Moser, Manager training, Exelon

J. Trettin, Engineering, Exelon

T. Fuhs, Regulatory Assurance, Exelon

T. Wojcik, Engineering, Exelon

D. Tubbs, MidAmerican Energy

LIST OF ITEMS

Opened

None.

Closed

VIO 254/2006014-01 Failure to establish measures to Ensure that the U1 ERV actuators Remained suitable for operation at EPU

Discussed

None.

Attachment