ML062830533
| ML062830533 | |
| Person / Time | |
|---|---|
| Site: | Cooper  |
| Issue date: | 12/22/2004 |
| From: | - No Known Affiliation |
| To: | Office of Nuclear Reactor Regulation |
| References | |
| FOIA/PA-2006-0007 CR-CNS-2004-07409 | |
| Download: ML062830533 (72) | |
Text
'CR-CNS-2004-07409 Root Cause Investigation CR-CNS-2004-07409 DATE 12/22/2004 Unexpected Service Water System Pressure Drop C~ondition Reuort(s)
CR-CNS-2004-0740.9 CR-CNS-2004-07408 CR-CNS-2004-07415 CR-CNS-2004-07410 CR-CNS-2004-07286 Root Cause Team Responsible Manager Ricky Fill Root Cause Team Leader Nobel Stephan Root Cause Team Bruce Toline Kim Perry Steve Sweney Dwight Vorpahi Rhoel Tierra David Pease Bob Wulf John Wright
___________________Randy Noon Reor7Tpe FialReJo
.Date__________t
.Date______
Responsible Manager......
Responsible Sr. Manager_.
A zxt Licensing tjIAVv CARB Chairperson (sro fnA 10-T~ A Date______
Date 12-112-10:
A"
. V, I r CR-CNS-2004-07409 1 - PROBLEM STATEMENT E N................................................
.3 2 - EV/ENT SUMM 1AR'Y 3
3 - EXTENT OF CONDITION..................................
.................. 4 WHAT ARE THE POTENTIAL CONSEQUENCES OF THIS PROBLEM?'.............................................. 4 WHAT IS THE PROXIMATE CAUSE(S)?'.............................................. 4 WHERE ELSE COULD THIS FAILURE OCCUR WITH UNDESIRABLE CONSEQUENCES?7................... 4 WHAT ARE THE REASONABLE IMPLICATIONS OF THECAUSES?................................................ 4 4 - CAUSES...............................................................................................
6 ATTACHMENT A - ASSIGNED ACTIONS........................... o......................
7 A IMMEDIATE AND INTERIM/COMPENSATORY CORRECTIVE ACTIONS.....................................
7 A-2 LONG TERM CORRECTIVE ACTIONS.......................................
o............................_8 A CORRECTIVE ACTION EFFECTIVENESS REVIEW.......................................................
13 ATTACHM4ENT B - ROOT CAUSE ANLYI DETAIL............................................
13 B-i CAUSE CODES & CORRECTIVE ACTION CODES..............................................
o..........13 B-2 ROOT CAUSE DETERMINATION AND VALIDATION........-............................................
i14 B-3 ROOT CAUSE ANALYTIC TOOL "A" KEPNER-TREGEO MODEL...........
o..............................
32 B-4 ROOT CAUSE ANALYTICAL TOOL"B" - WHY STAIR CASE...
o.......................................
33 B-5 HUMAN PERFORMANCE ASPECTS OF EVENT.........o.................................................
35 ATTACHMENT C - SCOPE & RESOLUTiION...................................................
35 C-i EVENT DETAIL......o........................
....... 0... 0................
o_.............__...
o......... 35 C-2 LINES OF INQUIRY...o....................... ~o.............................................
o...0.0...0.35 ATTACHMENT D - RISK.................................................
... o................. 41 D-1 RISK ASSESSMENT
SUMMARY
o...o...o...................................
o......... o..................... 41 D-2 PROBABILISTIC SIGNIFICANCE ASSESSMENT (PSA).............
o............. 41 ATTACHMENT E - LEARNING ORGANIZATION EFFECTIVENESS..........................
42 E-1 CNS (INTERNAL) OPERATING EXPERIENCE...o.................. o..................-..o.....o...........
42 E-2 INDUSTRY (EXTERNAL) OPERATING EXPERIENCE..........
o..........................................
43 E-3 EFFECTIVENESS OF CNS SELF-ASSESSMENT ACTIVITES.........o.................
oo o...o.....49 ATTACHMENT1 F - INVESTIGATION DETAIL..........................................
..... 43 F-1 DOCUMENTS REVIEWED.....
o.........................................................
o....... 0....... 49 F-2 PERSONS CONTACTED................................................
o..... -0.......................
51 ATTACHMENT G - UNRELATED CONDITIONS DISCOVERED.............................. 51 G-1 NEw CONDITION REPORTS........._o............................0.............................-0.....
51 APPENDIX A: INTERNAL OPERATING EXPERIENCE - CONDITION REPORTS AND NOTIFICATIONS........................................................................
52 12/22/2004 Page 2 of 72
CR-CNS-2004-07409 I - Problem Statement On November 20, 2004, at 8:25:19 a.m., with "AK and "B" Service Water (SW) pumps running, Operators started the idle "D" SW pump as part of routine operations.
Operators immediately noticed an unexpected pressure drop in both SW subsystems.
During the pressure drop, Operators started the idle UC"t SW pump. Also during the event the magnitude and duration of high diferential pressures observed on both the "A" and UB" Zumn strainers were unexpected. Although the safety function of the "A" subsystem was maintained throughout the event, the condition resulted in the "B" subsystem safety function not being maintained for approximately 16 minutes.
2 - Event Summary On November 20, 2004, at 8:25:19 a.m., while SW pumps "A" and "B" were in service, operators started service water pump "D". Following the start of service water pump "D,"
discharge pressure in both the "A" and "B" divisions dropped, and a high differential pressure alarm sounded for the Zurn strainer in division "B". When discharge pressure reached 38 psi in the "A" subsystem, operators started SW pump "C". While all four SW pumps were running, the Zurn Strainer 'A" high differential pressure alarm came in, as well. Valves SW-MOV-MO-37 and 36 (SW Loop Crosstie Header Isolation Valves) closed as expected due to decreasing pressure, thus isolating water supply to the Non-Essential Load. Both Zurn Strainers automatically started their back wash cycle. SW Subsystem UAIIwas fully recovered in approximately 3 minutes, however, Subsystem "B" could not recover on its own. Operators had to bypass and isolate Strainer "B". Once operators bypassed the "B" Zurn strainer, subsystem "B" pressure increased from about 30 psig to about 70 psig.
12/22/2004 Page 3 of 72 12/22/2004 Page 3 of 72
I CR-CNS-2004-07409 3 - Extent of Condition What are the potential conseguences of this Dr-oblem?
Plugging of the SW Zurn Strainer degrades the safety function of the SW system which renders the system to be inoperable, thus leading to entry of LCO actions and possible plant shutdown. Extensive fouting of the Zurn strainers could Impede the satisfaction of the Ultimate Heat Sink function of the system, until such time when the strainers could be bypassed.
What is the proximate cause(s)?
The proximate cause for the strainer plugging was a sudden~burst of sediment injected into the partially fouled Zurn strainers when the idle SW pump was started.
Due to a lack of sedimentation monitoring in E-ba y, the accumulation rate is not certain.
However, data suggests that the sediment in E-bay accumulated since 5/10/04 (WIO 4314123). Per discussions in section B-2, Evidence Examination items 3&4, the effect of low river levels on river velocity and/or sudden changes in river levels are thought to have contributed to a higher than expected concentration of sediment in the river water entering the intake structure, including E-bay.
As described in section B-2, 1973 ONS silting study-Part Ill of FSAR Amendment 31, the report summary states that the weir wall should result in a 40 to 70 percent reduction in the amount of suspended sediment entering the CS and SW systems, for river flow conditions which prevail most of the time. Further review of the results (page 24 of the report) indicates that the amount of sediment passing through the Intake Structure is reduced significantly, except for the 10,000 cfs discharge. Table 5 of the report indicates that the reduction in the SW bay is about 65% at flows of 20,000 and 35,000 cfs but that the sediment concentration at a 10,000 cfs discharge is actually higher than without the weir wall. Therefore, there is a relatively large increase in the suspended sediment entering the SW bay as the discharge drops from 20,000 to 10,000 cfs. The report indicates that a 20,000 cfs discharge corresponds to a 877' river level.
USGS web site (on waterdata.usgs.govlnwis) provides comparative river volumetric flow rate for the period 1117M04 to 12/8/04 vs. a Median Daily Stream Flow Based on 74 Years of Record. The Median Daily Stream Flow Based on 74 Years of Record for the period 1117 to 12/8 is 35,000 cfs. The actual river flow rate was between 15,000 and 20,000 for at least 2 weeks prior to 11/20/04 (at Nebraska City). The data shows that the flow was at about 16,500 cfs during the week proceeding to the event.
Reviewed of information on several of the Sedimentation Reports at the USACE/MRD web site indicate that most of the sediment in the Missouri River is transported as suspended load. The bed load is only about 5 percent of the total load. The sediment concentration increases with depth and is greatest at the bottom of the river. These reports also indicate that the relationship between flow and suspended sediment discharge is variable. It can change With temperature, type of runoff, sediment properties, and characteristics of the river. The suspended sediment discharge has been historically greater during the spring/summer season due to higher river flows.
12/22/2004 Page 4 of 72 I22/04Pg f7
CR-CNS-2004-07409 3 - Extent of Condition However, for a given river discharge, the suspended sediment load is greater during the winter months than during the summer. Experiments have indicated that the suspended load doubles when water temperature is reduced by about 40 F.
Additionally, the USACEIMRD reports indicate'that It has been observed that, even for a given river discharge, the river level is higher in the summer and progressively decreases during the fall months, usually about 1 to 2 feet between September and November. The channel becdmes progressively smoother from the summer into fall months as the number of dunes on the river bottom decrease and become longer and shallower. This change is accompanied by a progressive increase in the average channel velocity and sedimentation transport rate (for a given river discharge).
Based on the above, it is assumed that the design of the exterior silt mitigating features has not been sufficiently adapted to the changing river conditions to ensure these design features remained effective. Additionally, as river conditions have changed, monitoring, maintenance, and operation of E-bay equipment have not been sufficiently adjusted to adapt to these changing conditions.
Where else could this failure occur with undesirable conseguences?
This is applicable to the standby Fire Protection Pump "C" that takes suction from the E-Bay. This pump is -used as a back up to supply water in case of fire. The pump has a.
duplex strainer on its discharge. This strainer can get plugged if the fire pump is suddenly started and sufficient sediment build up existed in the E-Bay.
What are-the reasonable implications of the causes?
.See potential consequences, above.
3.A - Possible Contributors to Strainer Plugging I. River/In-take Interface Water has higher than usual concentration of sediments caused by flow characteristic changes. Flow characteristics are turbulent in nature due to the higher velocity profiles over and around the weir wall caused by lower river levels. The river is approximately 876.5 feet above sea level versus the 885 feet above sea level for which the weir wall was designed. The increased turbulence and velocity causes higher concentration of sedimentation to be picked up from the river bottom and carried by the river water. Based on computational flow modeling, as the water moves across the weir wall it accelerates from 4 feet/second (ftls) to almost 10 ftls then slows down in the intake canal to 4 ft/s. Since the water crossing the weir wall is pulled into the OW and SW pumps, the intake canal acts as a concentrator for the sediment. The high velocity water slows down considerably as it enters E-Bay, thus allowing the sediment in solution to precipitate at the bottom. The sedimentation accumulates if the material influx is greater than the processing capability of the spargers and strainers.
12/22/2004 Page 5 of 72 12/22/2004 Page 5 of 72
0 I
CR-CNS-2004-07409 3 -Extent of Condition IL. Sparger Interface Sediment in E-Bay is not agitated sufficiently by Spargers due to:
" Eroded or plugged Sparger nozzles, and/or
" Spargers improperly oriented, and/or
" Sparger sequencing not efficient.
III. Zurn Strainer Interface SW Strainer is not effectively managed under current river conditions:
- Back wash or. cleaning is not frequent enough, and/or
- Differential pressure (DP) monitoring frequency is insufficient.
DIP is monitored once per shift, manually.
Data is insufficient to anticipate performance problems early, and/or
- Existing strainer condition is not adequately monitored prior to starting idle SW pumps or swapping E-bay spargers. If the pre-existing condition of the strainers is actively managed, an evolution creating an incremental fouling should have negligible impact on strainer performance.
IV. Lack of sedimentation monitoring In E-bay Historically, there has been no accumulation of material in E-bay (as recent as 5/10/04, no accumulation in E-Bay, W/O 4314123). Consequently, no formal monitoring has been initiated. Formal monitoring, including alert and action levels, provides an opportunity to take proactive action prior to an event.
4 - Causes Idenifie DecripionSupporting Idenifie DecripionBasis Velocity profile s for lower river levels, Changing River Conditions are causing higher levels of Weir Wall design, Root Cause I sediment being transported into E-Bay sndevdiente buodp See discussion in section B-2.
Monitoring. Operation, Design and Maintenance of E-bay High number of DP Root Cause 2 related equipment were not effective In mitigating sediment alarms over the intrusion Into E-bay.
past 36 months.
12/22/2004 Page 6 of 72 12/22/2004 Page 6 of 72
CR-CNS-2004-07409 Attachment A - Assigned Actions A-I - Immediate and Interim/Compensatory Corrective Actions Actions Completed Effectiveness System Engineering conducted a waikdown of SW system In No issues identified.
the intake structure immediately after the event, including E bay traveling screens.
Inspected and cleaned both Zum strainers on 11/20/04. The B After cleaning, in service differential Zum strainer was found to be approximately 90% plugged with pressure was normal L.< 0.1 psid).
small pieces of gravel. The structural integrity of the strainer Both Zurn strainers were declared basket and strainer basket support was also Inspected and no operable. (Notification 10354805) damage was noticed. The A Zurn strainer was found to be approximately 80 % plugged with small pieces of gravel.
There were also no indications of any structural damage to the strainer basket or the strainer basket supports.
Placed E-bay J2 Spargers In service after cleaning E-bay Sounding data Indicates minimal upstream and down stream.
(typically 0) accumulation outside of E screen.
Soundings in E-Bay and at the SW pumps suction were Action needed to evaluate the amount obtained. This additional monitoring continues, of sediment entering the SW pumps.
Work Order 4411383 The information is used to manage cleaning to prevent sedimentation buildup and to determine the extent of additional monitoring required.
On 11/24/04 "B* the Zurn strainer was inspected to assess the Inspection showed l ittle build up of fouling rate as sediment was transported through the system sedimentation in the strainer after the previous cleaning on 11 /20/04.
(estimated 5-10% fouled with loose material). This indicates that efforts to disallow sedimentation build up are helping in avoiding sudden in-rush of sediments Into the strainers, thus allowing strainers to automatically back wash and remain clean.
Verified no Impact to equipment cooled by SW Action ensured that no adverse impact o
TEC increased from 77 deg F to 80 deg F, well on the plant due to the event.
within limits.
o As expected, main generator end turn vibration decreased slightly with the Increase In TEC temperature.
o REC temperature increased from 75 deg F to 83 deg F, also well Within limits.
o Bypassed strainer did not adversely affect HXs.
12/22/21304 Page 7 of 72 12/22/2004 Page 7 of 72
CR-CNS-2004-07409 A-I - Immediate and Interim/Compensatory Corrective Actions Implemented a Coping strategy Action minimizes sedimentation build
" Zu Staines
-Bacwashd pior o Ebayup in E-Bay near the SW pumps and eolZumtrion. ecrd s -facwashed priofrentoi-ay also minimizes accumulation in the pevolutio.
Reciored sfoulenlasinessd.frnta Zumn strainers to prevent adverse presure.Montore forclenlinssimpact by sudden fouling.
Changed PM for Zumn strainer cleaning to six weeks. (Compensatory action) o Minimize idle SW pumps to keep material in suspension. (Compensatory action to swap idle pump daily)
A temporary configuration change was installed to provide a This action will validate the ongoing continuous readout of Zumn Strainer DP on a digital recorder.
effectiveness of interim actions and provide insight for operational improvements.
A-2 Lona Term Corrective Actions CA Type: Preventive Corrective Action
Description:
Determine the optimum Zurn Strainer and SW pump operation and maintenance under current river conditions. Consider, as a minimum:
03 0
0 0
0 0
the setpoint for Zurn Strainer automatic backwash the Zumn Strainer intermittent backwash frequency, the Zumn Strainer alarm setpolnt other operating and monitoring requirements for various E-bay evolutions Zumn Strainer Cleaning Interval (condition vs. calendar based)
Idle SW pump swapping frequency Initiate corrective actions to establish the optimum operating conditions Identified above.
Note: Completion of this action requires review and acceptance by CARB before the due date.
CA Plant Constraint Non-Outa-ge; Priority__3 Initial Due Date 3/31/05
- LT CA (YIN) N Licensing Concurrence (YIN)_N___:.... Assl!nýWM A ~GrPlant En Sys M gnL.......
Assigned Work Group Acceptance Relationship to Causes:/
This corrective action addresses root cause #2.
WV
(
Relationship to Extent of Condition: This corrective action will not Improve the reliable operation of the spare fire pump, screen wash or sparger pumps.
Expected Benefit and Suitable Effectiveness: It is expected that this corrective action will reduce the probability of recurrence of zum strainer plugging.
12/22/2004 Page 8 of 72 12/22/2004 PageS of 72
CR-CNS-2004-07409 A-2 Long Term Corrective Actions CA Type: Preventive Corrective Action
Description:
Implement Weir Wail modifications and install river turning vanes to minimize sedimentation transported into the intake structure, Including E-Bay.
Initiate additional actions to impiement the solution.
CA Piant Constraint Non-Outage
- Priority__3ý initiai Due Date 9/30105
- LT CA (YIN) Y'..
Licensing Concurrence (YIN)
N A ~t~e ~.
roup Prolects Mamt Assigned Work Group Acetn Relationship to Causes: This corrective action addresses root cause M12.
Relationship to Extent of Condition: This corrective action will improve the reliable operation of the spare fire pump.
Expected Benefit and Suitable Effectiveness: It is expected that this corrective action will reduce the transport rate of sedimentation, thus reducing the probability of recurrence of zum strainer plugging and prevent the likelihood of a similar event related to the extent of condition.
A-2 Long Term Corr~ective Actions CA Type: Preventive Corrective Action
Description:
Determine the optimum E-Bay Sparger operations under current and changing river conditions. Evaluate the condition of spargers and establish necessary maintenance to ensure reliable operation. Initiate corrective actions as necessary based upon the above evaluation.
Note: Completion of this action requires review and acceptance by CARB before the due date.
CA Plant Constraint Non-Outage
-; Priority..3____ Initial Due Date 3/31/05 ; LT CA (YIN)
N Licensing Concurrence (YIN)_N___. Assi nod WorjtGroup Pit Eng Sys Mcimt Assigned Work Group Acceptance
-&I....
/4Z4~
Relationship to Causes: This corrective action addresses root cause #2.
Relationship to Extent of Condition: This corrective action will Improve the reliable operation of the spare fire pump and sparger pumps.
Expected Benefit and Suitable Effectiveness: It is expected that this corrective action will reduce the probability of recurrence of zurn strainer plugging and prevent the likelihood of a similar event related to the extent of condition.
12/22/2004 Page 9 of 72 12/22/2004 Page 9 of 72
A-2 Long Term Corrective Actions CA Type: Preventive Corrective Action
Description:
r e 1L t4z d
Determine the optimum E-Bay monitoring under current river conditions. Establish alert and action levels for monitoring. Initiate corrective actions based upon the above evaluation.
Note: Completion of this action requires review and acceptance by CARB before the due date.
CA Plant Constraint-Non-Outage; Priority-3 Initial Due Date 3/151505; LT CA (YIN) N Licensing Concurrence (YIN)_......; Assign Wor GUp PtnaSsMmt Assigned Work Group Acceptance-......4r4 Relationship to Causes: This corrective action addresses root cause #2.
Relationship to Extent of Condition: This corrective action will improve reliabl e operation of the spare fire pump.
Expected Benefit and Suitable Effectiveness: It is expected that this corrective action will reduce the probability of recurrence of zumn strainer plugging and prevent the likelihood of a similar event related to the extent of condition.
A-2 Long Term Corrective Actions CA Type: Generic Corrective Action
Description:
Develop organizational lessons learned from this event including effective communication, sense of urgency responding to issues, and operational focus.
CA Plant Constraint No-Oua-e; Priority__A_ Initial Due Date_...j.Li0LjLT CA (YIN) N Licensing Concurrence (YIN)......... Asslg d Wor roup OS It Assigned Work Group Acceptance
{/j-2-6-w4 I z/nhy--
Relationship to Causes:
4 N/A Relationship to Extent of Condition: +b*-
or Qo"0-iciic L
4 Expected Benefit and Suitable Effectiveness: This action is designed to increase sensitivity of all CNS personnel to precursors and aggressively attack them.
12/22/2004 Page 10 of 72 12/22/2004 Page 10 of 72
CR-CNS-2004-07409 A-2 Long Term Corrective Actions CA Type: Corrective Corrective Action
Description:
Perform Computational Fluid Dynamics (CFD) analysis of E-bay sedimentation modeling the effects of a non-intentional weir being formed by the J I sparger piping (near the floor) and the accumulation of material In the rack below the piping. Compare these results to those obtained for no weir installed and perform an evaluation of the differences.
CA Plant Constraint Non-Outage; Priority 3ý_ Initial Due Date 3/1 5/05 ; LT CA (YIN)
N Licensing Concurrence (YIN)
- Assign4pok up PIt Enaq Design Mqmt Assigned Work Group Acceptance_~A~-'rz Relationship to Causes
- This corrective action addresses root cause #1&U.
Relationship to Extent of Condition: This corrective action will Improve reliable operation of the spare fire pump.
Expected Benefit and Suitable Effectiveness: This action is designed to understand the dynamics of E-Bay behavior with different design features.
A-2 Long Term Corrective Actions CA Type: Preventive Corrective Action
Description:
Perform Computational Fluid Dynamics (CFD) analysis of E-bay sedimentation without a stop log section installed. Compare these results to those obtained for a stop log section installed and perform an evaluation of the differences.,
CA Plant Constraint Non-Outage; Priority__ 3_ Initial Due Date 3115105 ;LT CA (YN) N Licensing Concurrence (YIN).... Assigned ork Gr~q p Pit Eng Design Mamt Assigned Work Group Acetne 4ý d
t1eeJ41t ts-0,Upf Relationship to Causes:/
This corrective action addresses root cause #1 &2.
Relationship to Extent of Condition: This corrective aiction will improve reliable operation of the spare fire pump.
Expected Benefit and Suitable Effectiveness: This action is designed to understand the dynamics of E-Bay behavior with different design features.
12/22/2004 Page 11 of 72 12/22/2004 Page 11 of 72
I CR-CNS-2004-07409 A-2 Long Term Corrective Actions CA Type: ENH Corrective Action
Description:
In support of reliable operation during the license renewal period through 2034, perform additional hydraulic modeling at river levels less than 875'.
CA Plant Constraint-Non-Outage; Priority__34 Initial Due Date 5/15105 ; LT CA (YIN).! N..
Licensing Concurrence (YIN)_,;s~~
Work group Plt Enaq Design Mamt Assigned Work Group Acceptance_...j
- '4..~.~?i rele, -,--I~ /ZA4el-Relationship to Causes: N/A Relationship to Extent of Condition: N/A Expected Benefit and Suitable Effectiveness: This action is designed to understand the dynamics of E-Bay behavior at the lowest river levels to support plant life extension efforts.
A-2 Long Term Corrective Actions CA Type: Generic L
,rkA-fk"W Corrective Action
Description:
Have all CNS Managers sign that they have read this SCR.
CA Plant Constraint-Non-Outage; PrIority-4__ Initial Due Date1/1505 LT CA (YIN) N Licensing Concurrence (YIN)_; Assign} Wrk roup Ops MaMt Assigned Work Group Acceptance
=l
/-
L. d,.^6a A4 e. leA. f/~
Relationship to Causes:
/
p N/A Relationship to Extent of Condition: N/A Expected Benefit and Suitable Effectiveness: This action is to be used as lessons learned for effective communication, sense of urgency responding to Issues, and operational focus.
12/22/2004 Page 12 of 72 12/22/2004 Page 12 of 72
CR-CNS-2004-07409 A Corrective Action Effectiveness Review Note: This action'shall be entered as corrective action against the Learning Organization (LO) Condition Report for Corrective Action Effectiveness Reviews during the calendar year. Do not enter as an action against the Category A Condition Report. Describe the plan to evaluate effectiveness considering the options available In the performance management module:
CA Type: WIA Corrective Action Effectiveness Review Plan: Quarterly evaluate the effectiveness of interim actions untii long-term corrective actions have been Implemented. Upon completion of long term actions evaluate their effectiveness.
CA Plant Constraint: NA;, Priority: 5; Initial Due Date 3115/05 Assigned Work Group: Corrective Action & Assessment Assigned Work Group Acceptance Describe the focus for the corrective action effectiveness review, in terms of what are we trying to prevent recurrence of (extent of conditions and or the specific significant occurrence):
Provide recommendations on available effectiveness measurements that could be used to assess overall effectiveness of corrective actions:
Attachment B - Root Cause Analysis Detail B-1 Cause Codes &!Corrective Action Codes There was no related cause code under external cause codes for Root Cause #1 Root Cause #2 is related to cause code 0-2, Inadequate attention to emerging problems. This code refers to repetitive organization crisis In repeat events. The causes are usually associated with a lack of strong self-assessment; strategic planning and root cause processes. Additionally, inadequate vertical Information flow to the decision makers in the organization and inadequate prioritization of work can contribute to a breakdown In this area.
12/22/2004 Page 13 of 72 12/22/2004 Page 13 of 72
CR-CNS-2004-07409 B-2 Root Cguse Determingtion and Validation Root cause determination logic Is based on the Kepner-Tregoe Model. As part of the model, a "Why-Stair-Case' is developed to aid in determining possible causes. These possible causes are tested against examination of available evidence.
This section will also share licensing basis related to the problem, technical rationale to aid in evidence examination, and details of evidence examinations.
USAR Section 8.1.5.1 Design Basis: The USAR documents the following on Silting:
During plant construction the presence of a large amount of silt in the SW system Intake bay indicated that silting problems may occur in the SW system. Silt buildups will occur in areas of low water velocity. Typically these areas occur where the piping undergoes large diameter changes or where the normal flow path is intersected by piping with very low normal flow. Typical examples include instrument sensing lines and passive heat removal systems.
Silt accumulation In the entrance and interior of the intake Is controlled by the following design features of the system:
'~A water jet sparging system is Installed near the bottom of the Intake Structure to agitate the silt and keep it in suspension, thus preventing its settling out The sparging system consists of three rows of jets in each circulating water bay and five rows In the SW bay, as shown in Figure X-8-4. Jet J-3 is installed within the traveling screens to keep the screen boot area free of silt buildup. The above-mentioned principal operating jet J-3 is utilized on an automatic sequential schedule for periods determined by field experiment after Installation. J-2 header is used to clear the guides for the screens when they are replaced following maintenance. Circulating water bay jets J-4 are used only in the unlikely event that an undesirable amount of silt accumulates near the circulating water pump. SW bay jets J-4A and J-4B agitate silt around the SW pumps. The jets J-4A and J-AB are double headers, either one could act as backup to the other. Supply water to the jet sparging system Is provided by two pumps, one to act as standby. Supply water for the jet sparging system, to the jets for the SW pumps (E-bay), may be supplied by SW In the event sparging pumps are not available post LOCA. A maintenance schedule for the sparging system Is set up based upon field experimentation and operating experience.
- 2.
Silt panels are provided In stop-log guides, which, can be used to block the waterway when the pumps are not in operation or -screens have been removed for maintenance. This can help prevent unnecessary silt accumulation in idle bays.
- 3. The original single speed drives of the traveling screens (10 fpm) have been replaced by two speed drives (2 1/2 fpm and 10 fpm). Continuous operation of a screen at the higher speed results In increased wear due to the silt-laden water. During Intermittent traveling screen rotation, the higher speed Is used, with the lower speed being utilized for continuous operation.
- 4. A guide wall has been installed In order to streamline the river flow in front of the intake and reduce the amount of silt and sand entering the structure.
Technical Rationale: This section provides a technical rationale (based on engineering judgment) for sedimentation transport, build up and removal In SW system to aid In understanding the problems and determining root cause.
Effectiveness of the Zumn Strainers to perform their function of removal (filtration) and disposal (back wash) of river water sedimentation relies on the following:
1.Sedimentation concentration of water entering the strainer is reasonably constant. This would allow for gradual sedimentation build up, which in turn allows the strainer to back wash at its 4 psid set-point while simultaneously removing sedimentation. This should allow for strainer self cleaning without reaching the alarm set point of 6 psid.
- 2.
If sedimentation concentration In water entering the strainer Is too high, the rate of build up of sedimentation on the strainer increases exponentially (much higher rate). As the set point of back wash Is reached, the strainer will start its back wash cycle automatically.
However, as It Is automatically back washing, the build up of sedimentation increases due to the high concentration of sediments in the water (i.e., Strainer is not able to keep uD1. This increased build UD rate increases the differential pressure (DP) on the strainer. thus reaching the alarm 12/22/2004 Page 14 of 72 12/22/2004 Page 14 of 72
CR-CNS-2004-07409 B-2 -Root Cause Determination and Validation set point of 6 psid.
Note: Strainer problem history shows when the control room received high DP alarms, operators are dispatched to investigate the cause. Operators, in most instances, report back that the affected strainer is In back wash mode and DP either decreasing or remains high. This Is evident of the above #2 rationale. The strainers in these cases are challenged with additional sedimentation removal while trying to back wash.
- 3. Sedimentation concentration in water entering Into the strainers Is a function of the amount of sedimentation transferred from the river into the E-Bay and the effectiveness of the sparger In agitating E-Bay water in order to keep sedimentation In suspension.
The E-Bay sparger system effectiveness depends on maintaining the following:
Sparger nozzles are not eroded, I.e., geometrical dimensions remain within their design limits. This ensures that nozzles provide appropriate jet force to induce sediment agitation.
Sparger nozzles are not plugged with sediment to allow for sufficient flow.
- Sparger nozzles oriented for proper agitation and mixing.
Spargers' locations have to be appropriate to avoid 'mixing-dead-zonesu. These agitation/mixing dead zones can allow sediment to come out of solution and concentrate in one area.
Proper flow to Spargers to maintain their function.
- Sparger system swaps management (which Spargers, how often and when they should be operating).
.4. Sedimentation transfer from the river to E-Bay has a direct effect on the concentration of the sedimentation in the water entering the Zumn Strainers. The more sedimentation that is transferred from the river to the E-Bay, the higher Is the sedimentation concentration In E-Bay, and the more sedimentation needs to be removed by the Zurn Strainers. Sedimentation transfer Is a function of the following:
River Level and velocity has a direct effect on river flow regime. If river flow regime changes from laminar to turbulent, more sedimentation Is carried In river water. Also, if river flow has an up-ward flow direction due to high turbulence, then It will collect sedimentation from the river floor.
" Differences in flow velocity magnitude between the Intake Canal and E-Bay have direct influence on sedimentation precipitation In E-Bay. High velocity In the canal forces sedimentation to stay in solution in the canal flow.
When flow velocity decreases In E-bay, sedimentation will precipitate and concentrate at the bottom of the E-Bay.
- 5. Doubling the flow by adding a second pump on the same strainer will immediately quadruple the value of the differential pressure across the strainer, without additional sedimentation build up.
Evidence Examination:
Several items of evidence were collected to aid the team In determining root cause.
The following is an examination of the evidence. Analysis of what each piece of evidence is revealing is documented In Italic.
I1.
Sedimentation Build-Up at the Bottom of E-Bay:
Since 11/22/2004 CNS started measuring and trending sediment build up in the E-Bay In several locations. These
,locations'are around the rack, the guide,.J-2 sparger, plant side of the traveling screen, and near J14A, J14B3, and J4C spargers. Measurements inside the traveling screen are taken every other day. Measurements outside the traveling screen are taken twice a day (every shift). The measurements taken to date have shown an Increase trend only at the areas inside the screen and near the J4A spargers.
This sedimentation build up is caused by higher concentration of sediment in the water and lower water flaw 12/22/2004 Page 15 of 72 12122/2004 Page 15 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and Validation velocity.
- 2.
SW Pump and E-Bay Spargers Operation Rotation:
On I1/211200.4 OPS issued standing order 2004-09 Rev. 0 to have pumps A&D in-service and pumps B&C rotated each shift (three pump operation). Rev. 2 of the standing order was revised on 11/23/04 to have pumps B&C In continuous operation and pumps A or D secured for planned maintenance activities (three pump operation).
On 11/29/04 the control room received a high DP alarm on Zumn Strainer B while shifting E-Bay spargers.
Operators Log for that day documents that the DID on Zurn B was 2.3 psid before shifting occurred. Operator that was dispatched to investigate reported back that strainer was found In back wash cycle and DP had been reduced to 0.5 psid.
As a result of the DP alarm, Guidance was provided via night order on 11/30/04, and then followed by standing order 2004-10 Rev. 0 on 12/3/04 which was revised on 12/7/04. Revision 1 of this standing order required the following:
" Each SW pump is operated daily to prevent sedimentation build up.
Prior to shifting pumps or spargers, as found strainer OP is recorded, both strainers are back washed for 10 minutes, back washing is maintained during the shifting, strainers are returned to intermittent operation only after DP is stable, and OPD is recorded.
This rotation strategy has been effective in avoiding high DP alarms by not allowing sedimentation build up near the SW pumps, and ensuring the strainers are clean before and after the evolution.
- 3.
Computational Fluid Dynamics (CFD) Analysis by Alio~n on Sedimentation Transport (Draft):
Figures 4.4.1 & 4.4.2 show that 1/8 inch size sedimentation can be carried over the weir wall and into E-Bay.
Figures 4.4.3 & 4.4.4 show that there is sufficient turbulence in the intake canal to carry the sedimentation into E-Bay.
Figures 4.4.5 & 4.4.6 show that 3/8 inch size sedimentation can be carried over the weir wall, through the intake canal and into E-Bay.
Fig. 4.5.11 shows flow velocity magnitude near E-Bay floor elevation. The velocity magnitude is 4 ft/s at the south end of E-Bay entrance up to the trash rack, then tapers off In the northerly direction to about 2 ft/s.
This suggests that there are sufficient water velocities to transfer material from the river, over the weir wall, into the intake canal, and then into E-Bay.
- 4.
CFD Analysis by Alion on Magnitude and Direction of Flow (Draft):
Figure 4.5.9 shows that velocity magnitude In the Intake canal is about 4 ftls, and that the velocity diminishes as flow enters E-Bay until it reaches about 0.1 ft/s at the SW pumps. It also shows that the velocity magnitude remains high closer to E-Bay floor until It reaches the traveling screen.
Figure 4.5.13 shows that there is a reverse flow at the E-Bay traveling screen where roughly half of service water flow profile is going toward the area where the SW pumps are, and the other half is exiting the traveling screen.
This suggests that sedimentation starts to precipitate as flow entering the E-Bay slows down. However, higher flow near the floor area from the entrance to E-Bay to the trash rack sweeps the precipitating sedimentation (in essence providing a concentration increasing mechanism). As flow slows down in the area between the rack and traveling screen, some sediment precipitation starts to occur. Then full precipitation is completed around the SWN pumps area as flow magnitude Is reduced to about 0. 1 Wts.
The data also suggests that there is more flow delivered to E-Bay from the intake canal than what the SW pumps remove. Assuming that the reverse flow profile at the entrance of E-Bay is similar to the profile modeled at the traveling screen, then a concentration factor of sedimentation In the E-Bay is created.
This results from precipitation caused by the lower velocity reverse flow.
12/22/2004 Page 16 of 72 12/22/2004 Page 16 of 72
CR-CNS-2004-07409 B-2 Root Cause DeterMination and Validgition
- 5. Spargers Flow Measurement:
CNS measured flow of the Spargers in order to determine their condition. Measured flow higher than calculated flow Is an Indication of possibly eroded sparger nozzles. Measured flow lower than calculated flow is an indication of possibly plugged sparger nozzles. On the other hand, measured flow that is equal to calculated flow Is not proof that the Spargers are healthy since a scenario may exist as such that some nozzles may be plugged and others eroded on the same sparging header.
Results of the measurements taken were:
Measured flow for J2 was 270 gpm @ 120 # pressure vs. expected calculated flow of 348 gpm at 120#.
(indicates possible plugging) 0 Measured flow for J3 was 500 gpm @ 125 # pressure vs. expected calculated flow of 497 gpm at 125#.
(inconclusive) 0 Measured flow for J4A2 was 600 gpm @ 130 # pressure vs. expected calculated flow of 616 gpm at 130#.
(inconclusive)
Measured flow for J4131 was 800 gpm @ 150 # pressure vs. expected calculated flow of 661 gpm at 150#.
(indicates possible nozzle erosion) 0 Measured flow for.14132 was 333 gpm @ 120 # pressure vs. expected calculated flow of 591 gpm at 120#.
(indicates possible plugging)
Measured flow for J4C was 700 gpm @ 125 # pressure vs. expected calculated flow of 497 gpm at 125#.
(indicates possible nozzle erosion)
Note:
Expected calculated flow values were computed by the system engineer based on vendor supplied information and are not formal calculations that adhere to CNS 3.4.7 procedure requirements.
This evidence indicates possible degradation of the Spargers, i. e. plugged and/or eroded.
- 6. Design Changes over Life of the Plant: DCD-3 "Service Water and Residual Heat Removal Service Water (RHRSW) Booster System", Volume I was reviewed for design modifications that may have contributed to the problem. Based on review of the titles, the list was narrowed down to modifications upstream and including the SW strainers (such as the weir wall and SW pump changes) that may have impacted the SW system. The following modifications were reviewed In detail to determine if there appeared to be a singular or overall cumulative effect that could possibly account for current problems experienced in the SW system.
The modifications reviewed were:
DOCNO:
DC 76-111 TITLE: ICE PREVENTION AT THE INTAKE STRUCTURE, MINOR DESIGN CHANGE.
RET. #: 000160843 DOCNO:
DC080-019 TITLE: INTAKE STRUCTURE, SERVICE WATER SHOCK SUPPRESSOR MOD, MINOR DESIGN CHANGE.
RET. #: 051521417 DOCNO:
DC 87-138 TITLE: SERVICE WATER STRAINER SCREEN UPDATE.
RET. #: 092580007 DOCNO:
DC 94-373 TITLE: INTAKE STRUCTURE GUIDE WALL MODIFICATION.
RET. #: 643262282 12/22/2004 Page 17 of 72 12/22/2004 Page 17 of 72
9.
CR-CNS-2004-07409 B-2 Root Cause Determination a~nd Validation DOCNO:
DC 94-373 AMEND. I TITLE: INTAKE STRUCTURE GUIDE WALL MODIFICATION AMENDMENT 1.
RET. #: 643262282 DOCNO:
CED6008700 TITLE: SERVICE WATER PUMP PERFORMANCE IMPROVEMENTS RET. #:
687340414 Additionally, all Engineering Evaluations were screened for any possible effect. No identified effect was found.
No singular or cumulative affect was identified that would contribute to the current problems associated with the SW system and Zumn Strainers. CED that delt with replacement of the trash rack and permanent removal of JI sparger was not refuted, thus an action is added to model the J1 sparger removal.
12/22/2004 Page 18 of 72 12/22/2004 Page 18 of 72
CR-CNS-2004-07409 B-2 Root Cuse Determjnationj and Validation
- 7. Time Line of operational and Maintenance Issues:
11152spD4 ZXA".rm dw~SW~ppC-pl-din -A-OwmW lV202M
'11a Wn Eve 11J17200 Zmr'KAwrn OW pla WJPpB In, sel~m V1/12c04 1IV15t2O4 ThringSavn Oce inUd pwdassanl cry~aseMa~
12/22/2004 Page 19 of 72 12/22/2004 Page 19 of 72
7 I
i CR-CNS-2004-07409 B-2 Root Cause Determination and Validation 5/2/204 High DIP Alarm on ZizT1 W. There wese. alOrm end river IevOl wa, at 89 81=72003 High DP alarms on Zurn A and broke shear pin
%/77r=0 High VP ialar on Zigin A 11/113)2003 High Do alam on Zuzn A
.ZA J~A Zumn 8 malfunction after shitting saprgar 918/200 High DP adram onZurm a 2)512OD4 High DP alarm on Zurn A 511942004 High OP alarm on Zurn B and broken &heew pin 1/24/2003 12)112002 Zum A IIDP afarm Zum "K high DIP wh~en started Idle Pop D
51712003 Zurn A high OP alarm after salrting SW Pp A 9121/200 ZuM W Hig 2
8/21/2002 Zum"KHigh OP stOP i ILAm D Zurr I11=3/002 1/21)200 Zum W hig OP Zumi 8-high DIP 4)8/003 2)8/203 Zum A high D 5/13)2003 WAhighDP mnB8ypassed ue to high DIP Frequent highj DF-am iWe-,* jk
ý'
I
-i-L.-
518/002 8/82002 7/22002 Z=m B high OP rM 2 Zum B high DIP alarm, Zum, W high OP securing SW P D
/1 Ill 12)110D200 W Zum S~r. high OPD I-1/512 Zurn W high running B&
03
/
5/12003 411912003 2)122003 Zumn A high OP Zum A high OP alarm 5/10/f2003 003 um A high OPD OP wI A&C 2)2=103 O *a ourmed Zumn A Strainer high DIP U21=)003 I-gh OPD an Zums when 3 pumps we" running.
eventhough conlineously back washing 12/22/2004 Page 20 of 72 12/22/2004 Page 20 of 72
CR-CNS-2004-07409 91-1 Dnni-r.-aaci nflgaimjinnInn snel V.-Alitlatinn 112)27200 Zimi A Mheer pin failure 0)111)20011 Zumn A high OP alarms Zum A high OP 10119/001 then pin broken 111/2002 418)201 519/2002
- 8. Sounding Results of E-Bay: The following graph shows the amount of sedimentation build up as measured every other day since the occurrence of the event.
02.76-3 02.52-2.76 02.28-2.52 0 2. 04-2.28 M 1.8-2.04 M 1.66-1.8 0 1.32-1.56 01.08-1.32 1110.84-1.08 M 0.6-0.84 00.36-0.6 El0. 12-0.36 0-0.12-0.12
~Location Date Note: E-Bay was cleaned on 1216104 12/22/2004 12/222004Page 21 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and Vplidation
- 9. Missouri River Flow Management Information from the USACE web site reveal the following on river management; System regulation of the Missouri River has been historically performed to serve navigation purposes. In years with near normal runoff, the navigation season on the Missouri River is normally supported for 8 months from April 1 to December I (at the mouth of the Missouri River). This navigation season has been extended in years with above-normal water in the system or shortened for water conservation purposes. During the navigation season, a flow of 37,000 cfs at Nebraska City is required for full-service navigation. During periods of water conservation, the flow may be reduced to a minimum navigation flow of 31,000 cfs. The navigation target flows were reduced below full service as a water conservation measure In 1989, 2000, 2001, and 2002 and were reduced to minimum service In 1990-1993, 2002 and 2003. (also 2004?) Shortening of the navigation season occurred In 1981, 1988-1992, and 2003.
(also 2004?) Releases from Gavins Point have also been modified to protect endangered species since 1986.
The release procedure has varied from year to year and can affect the level of service to navigation, in some cases even disrupting navigation.
During the non-navigation periods or periods when navigation is non supported during droughts, minimum flows are provided for water supply to thermal power plants and other intakes on the river. The current manual specifies a minimum winter release of 12,000 cfs from the Gavins Point Dam is required to meet downstream water requirements during this period. However, releases could be reduced below this for water conservation and it recommends modifying intakes which require more than 9000 cfs (minimum release required in the early 1990's) to ensure they can remain operational as the Corps continues to pursue lowering the Gavins Point Dam release in non-navigational months during drought periods to this rate. During drought years when system storage is low enough to reduce or eliminate the navigation season, a release of 18,000 cfs has been established as meeting the summer water supply requirement The flow at ONS is comprised of the release from Gavins Pt and Inflow from the tributaries. The USACE figure below shows the river flows at CNS for the past 10 years (flow converted to river level). The river flows from 1995-1999 were typically' maintained well above 50,000 cfs during the navigation season and above 20,000 in the winter. However, the flows the last 5 years are significantly lower, decreasing to below 35,000 cfs on several occasions during the navigation season and below 20,000 cfs during the winter. The river flow was aboutt 18,000 cfs on 11/20 and was 16,500 cfs to 20,000 cfs during the 2 weeks preceding.
Missouri River Levels 1994-2004 1-1994
-199
-19M
-1997
-199
-199
-2000
-2001
-2002 20M 20041
-5 4~
g 31 62 93 124 155 lea 217 248 279 310 341 Days *Mar, 12/22/2004 Page 22 of 72 12/22/2004 Page 22 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and ValIdation The water level at CNS is established primarily by the river flow. The following figure shows a correlation between the river level at CNS and the river flow at Rubo. USACE data has shown that the river level for a given flow rate is typically 1-2 lower in the winter than in the summer.
Missouri Oiver Level vs Flow (1994-2004 data) y -.0.00t? +0.2925x 872.06 900..................
899..................-~
898 4............
894.....
893...........................
1... 1..... I...
.................~
890 888...
l.
753 873.....................................................................~........
871...
e*
8
-7.
I 87.... ;, : I 4 5 l`- -;..N
..`.
.u NE Flo.
- k..
T e6 average....
river leve fro 1995-1 99 was about 886'--
while..
it.
derese about..
5I to...
881 frm20020. The.
reaon or.he.owe.rier.eve.ca.bepriariy.atriute.totheextnde.drugh.th.lat.fve.ear. (USACE).......
The rive lee.erassec. er.ntennnaiainsasndrn.hewne.oth.Ti.yiclyocr in 3 the f.s.at.fDcmerbths.enocrrn.rgrsieyeale.urn.h.as er..
Ti.ya,.h rive lee deres ocure in mi Octobe. Th rie lee wa at
.bu 877 on 11/20 an for. th 2
.wek preceding....--
Thissourag riverwtr levels have resulted i waterou aces prolemst atcreseveralou intae 8lfong t 00e04 Thererial rassoite wirthe sandbr fovrmaticn oresedimentlydeposibtion ato the intakded resricgtn cest the riv iv ears. UA TheUSAC evweb sitreastaes taha sedien in the Mo-aiaisour Rivn uin hewner itrnptepimarilyhs. suispendedallyoad.uTh bnted lodirst only aout Decepercent ofs then totaluoadindgi primaresivly composerduofrsnd. the ratelationship beatweenfo propertiees, adecharaseoctreristc ofd thetrier. The susprended wsedimenot discharge1has0bend hisorial greaterk durin thAe Mpringrummer seaonto dauel tosigerrier flows duing genoaeralte suspndedl sedimenht eroncenratonwi lowersouri avlowateflowraes which resultsd in lwaer velcitess andlem turbuen.Hwverl ittke hason been observe thiatl fsoriatgien river sadiscarge fomthen supede sediment leoadtisn graterdrn the wnaersrcintcers monh than dringeth sumer UAExperimenstsaves tndiatsedimn tha the Msuspende loadr doublenspowhen watearil tmeaturpenisdeducead. bye abelout 40 F.nIlha alsou berent obsredta the oa oaancent riationyo coarosedo prilsad The 1/1em)latinsi begeatweren thew wintertievend wihalwrwacterisdicso ha rgves.
Tesseddsdmn icag a enhsoialrae Thwer folalowing figuretaen fromh reuSGs web siter shlowiis an comprisoenoier Hwater, leve for then monthve ofat November 2004 against the 74-year flow average for the month of November. It shows that, duiring the event and the 2 weeks preceding the event, the flow is about 50% of the 74-year average.
12/22/2004 Page 23 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and Validation USGS 06807000 Missouri River at Nebraska City, NE 25000 DAE:
10/200001/0/04 01 X IIAZRD IXag Cd10.0NS0 ta0 FSARNo Amnden 31v stte thttepiayproe of the gudDali oreue th0e4 mn nutt h conentatin a a 0,00 rcsvischnarg isactuall highert tha weiitouttewinal ssae rvosyh The rmndepor summarites that the puanitay ofseimntdposite on the fudewloo is not infcnl reducedb the seietinuwotei wnall. Foructhre. SWr bayo the sedidment deoindwicths thatte wfirnall waveiry small cniuatio 35,000 efsfeAt 20,00 tfs the wercwall reduceso i theamutosspne sediment thicnessibu the deoito was stil relat fr ively lageseilldowcnsitremofnth screcen.thedpstornteS a
a eativeyno at a 10,000 cfs; discharge wit orulyhihrta without the weir wall.Astaeprvolyth Asttdpeiulteactual river discharge was between 15,000 and 20,000 for at least 2 weeks prior to 11120.
The report summaryialso sthate theqantty af significnt redutonsinthed aounth flofravelnoteriniingthantakreuedb strutue wis also expected.
However, the results of a recent CFD analysis of the Intake Structure flow indicates that 118" pebbles could be transported over the weir wall at low water levels (the analysis used a 877.5 river level) and that the flow in the 12/22/2004 Page 24 of 72 12/22/2004 Page 24 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and Validation channel front of the SW bay is turbulent enough to carry these pebbles in suspension. At the river level used in the study (877.5%) most of the flow over the weir wall is right in front of the SW pump (E) bay. The flow velocities over the weir wall and in the intake channel in front of the SW bay are relatively high (> 5 fps). Some of the flow over the weir wall actually comes from the bottom of the river and carry sediment up over the weir wall. However, the CFD study also indicates that larger 3/8" particles would not be transported over the weir wall or carried In suspension.
The CFD analysis results also Indicate that the flow entering the SW bay has a relatively high velocity at the bottom in the front part of bay but Is relatively low elsewhere in the pump bay. There is reverse flow through about half of the SW bay traveling screen. A bay silting analysis indicates that sediment quickly enters the SW bay with most being deposited at the SW bay entrance and some buildup around the SW pumps.
- 11.
CNS Historical Data
This report investigated several options for reducing the amount of sediment Intake including several weir wall configurations. River flows of 10,000 to 165,000 cfs; were evaluated using the model.
The report summary states that the weir wall should result In a 40 to 70 percent reduction In the amount of suspended sediment entering the CS and SW systems, for river flow conditions which prevail most of the time.
Further review of the results (page 24 of the report) indicate that the amount of sediment passing through the Intake Structure is reduced significantly, except for the 10,000 cfs discharge. Table 5 of the report indicates that the reduction in the SW bay is about 65% at flows of 20,000 and 35,000 cfs but that the sediment concentration at a 10,000 cfs discharge is actually higher than without the weir wall. Therefore, there Is a relatively large increase in the saspended sediment entering the SW bay as the discharge drops from 20,000 to 10,000 cfs. The report! indicates that a 20,000 cfs discharge corresponds to a 877' river level.
Note: The actual river discharge was between 15,000 and 20,000 for at least 2 weeks prior to 11/20. (from USGS data at Nebraska City from waterdata.usgs.gov/nwis)
Page 22 of the report describes unsatisfactory performance of an earlier weir wall configuration at a 10,000 cfs discharge. This wall configuration was similar to the current configuration except that It did not extend as far downstream an 'd the sloped portion of the wall was about 30' further upstream. The report states that since the water elevation Is not much higher than the elevation of the downstream portion of the weir wall, most of the water for the Intake comes from the downstream portion of the wall and from the extreme downstream end between the wall and the river bank. The velocity In the Intake In front of Bay E was much higher than the velocity out In the river for the 10,000 cfs discharge. There were large and strong eddies at the downstream end of the structure due to the change In flow direction. Hence the concentrations of sediment passing through the bays became high.
Although the current river discharge is greater than 10,000 cfs, much of what was described above has been observed at the current river level and In the recent CFD silting model. The 1973 report also states that the length of the wall extending past the downstream end of the Intake structure affects the pattern of flow over the top of the weir wall, especially In front on Bay E and downstream of the intake structure.
The thickness of sediment deposition in the intake structure was also Investigated with the model. The report summarizes that the quantity of sediment deposited on the floor Is not significantly reduced by the weir wall.
For the SW bay, the sediment deposition wifth the weir wall was very small at 35,000 cfs. At 20,000 cfs, the weir wall reduces the sediment thickness, but the deposition was still relatively large, especially downstream of the screen. The deposition in the SW bay was relatively low at a 10,000 cfs discharge with or without the weir wall.
The report summary states that a significant reduction in the amount of gravel entering the intake structure is also expected. On page 25, it further states that all model tests conducted on the final weir wall scheme Indicated that the wall elevation is sufficiently high as compared to the river bed elevations to greatly reduce the amount of gravel.
12/22/2004 Page 25 of 72 12/242004 Page 25 of 72
CR-CNS-2004-07409 B-2 -Root Cause Determination and Validation
- b. Historical CNS Documents Review:
The following information are excerpts from CNS documents archive:
Intake Structure Jet Sparaers System Description. M-28. Burns and Roe. W.O. 2978-02 (Nov. 1973)
In Section 1.1 *Function" the document specifies that "The primary function of the sparger system is to clear light particles of settled silt from the intake bay floors and keep the smaller particles in suspension. The system is not designed for river bottom load, stones and other heavier debris.
In Section 1.2 "General Description" the document specifies that:
All sparger manifolds are approximately 10 inches above the floor except the J-3s, which are approximately 12 inches off the floor. Nozzles discharge the water against the direction of intake water flow. Each nozzle capacity is 31.6 gpm at 100 psi.
Manifolds JI, J3, J4A and J4C, which are to be normally used, are provided with solenoid piloted, air operated valves to control the water supply to these manifolds. The solenoids are sequenced by a timer, or manually depending on the main selector switch position (auto or manual). Each air operated valve contains a manual bypass for use during valve repair or replacement The J4A manifolds are located in the service water bay, and are used to clear the bay floor area around the fire pumps. This manifold is on automatic timer control. Two J4A manifolds are provided, one is operational and the other redundant manifold is for emergency class IS use only.
The A4B manifolds are located in the service water bay, and are used to clear the bay floor area between the fire pumps and service water pumps. The manifold is on automatic timer control. Two manifolds are provided, one is operational and the other is a backup for emergency class IS use.
The A4C manifold is located in the service water bay, and Is used to clear the bay floor area near the service water pump inlet bell. The jests can be operated automatically or manually, however these jets should normally be operated manually.
In Section 2.2.1 "Normal overatiori" the document specifies that:
Normal operation will consist of automatic timer controlled sparging of the bays, with the sparging water supplied by screen wash pumps 1IC or 1ID. The sparging water will be drawn from the CW discharge tunnel. The main selector switch will be placed in the auto position, and the timer will be 'on".
A 14' crossover line is provided which permits the service water pumps to provide water to the suction of the screen wash pumps I C and I D, if the circulating pumps are not operative and during intake bay circulator backwash, or if Insufficient NPSH is available for the screen wash pumps. Note: During this mode of operation, the suction supply pressure available to the screen wash pumps will be approximately 40 psig. Therefore, the discharge pressure will be about 170 psig. The relief valves on the pump discharge RV2344 and RV2345 are set to open at 140 psig. In this mode of operation the relief valves will be discharging water at a rate of about 115 gpm.
CNS-31 Document, Intake Structure Guide Wall. Part I - Engineerlna Evaluation, Dec. 1973:
In Section I "General De scription" the document specifies that:
Experience both prior and subsequent to preoperational testing at the Cooper Nuclear Station Indicated the need for a means of limiting and controlling the volume of sediment transported by the fast flossing Missouri River from entering the Intake Structure and continuing Into the several systems dependent upon river water.
Initially, bed load and suspended material entered the several bays of the Intake and were deposited on the floor of the bays upon entering the stagnant areas. Major depth of deposition occurred in all bays and a program of design and installation of jets (spargers) was Initiated, first on a trial basis and later on a permanent basis with limited optimization as permitted by the physical restraints then built into the structure.
The jetting system Insured against build-up of the sediment but is incapable of limiting the amount of inflow of sediment into the Intake. In fact, It only Insures that the sediment entering the Intake will remain in suspension and be carried through to the Circulating Water and other systems within the plant. On the other hand, the jetting system does not increase the average concentration of solids In these systems since these concentrations would reach essentially equivalent values upon stabilization of sediment dropout over the floor of the Intake provided only that sediment dropout does not continue to a degree to cause complete blockage of a bay, a situation which the 12/22/2004 Page 26 of 72 12/22/2004 Page 26 of 72
'B-2 Root Cause Determination and Validation jets prevent.
The bed load increment of the total sediment Intake is the greater concern by virtue of its size and erosion capability. It was desired, therefore, to limit the entry of this coarser material, if possible. The guide wall concept was considered to be the most practical approach and an extended program of hydraulic modeling was carried out. A report entitled "Movable-Bed Hydraulic Model Study for Cooper Nuclear Station Intake System" by Yasuo Onishi and William W. Sayre of the Iowa Institute of Hydraulic Research at the University of Iowa, dated December, 1973 Is included in this submittal for your evaluation.
The guide wall, as developed in the model study, is physically attached at Its upstream terminal to the circular cell which was left in place after the remaining cofferdam structure was removed. It continues downstream starting at a slight divergent angle with the bank line along a flat arc to a point opposite the upstream corner of the Intake Structure and thence downstream past the Intake and parallel thereto at a distance 14.25' riverward therefrom.
The downstream terminus is 40 feet below the downstream corner of the Intake. No connection is made to the shore at the downstream terminal. The wall is to be constructed of steel sheet piling with the section upstream of the Intake Structure being designed to contain fill by Incorporating a tieback system. Downstream of the upstream corner of the Intake Structure the wall is designed as a cantilever. Rip rap protection at the riverward face of the wall will be provided against scour of the riverbed.
Top elevation of the upstream portion is 885' MSL which Is five feet higher than normal summer river surface elevation of 880' MSL. A transition containing a 7 1A0 slope connects the upstream portion with the submerged downstream weir section at elevation 867.5 MSL. The details of the configuration were developed from the model conducted at the University of Iowa.
The primary purpose of the guide wall is to reduce the sediment input to the Intake Structure. It will accomplish this purpose by forcing bed load and other material contained at lower elevations in the river to flow past the Intake to a point where Inflow to the Intake will not Influence river behavior. The upper elevations of the river containing relatively finer sediment will be permitted to flow over the submerged weir. The model study indicates a potential reduction of as much as 75% in the amount of sediment to be carried into the Intake Structure. It Is anticipated that actual results will be somewhat lower than this potential because of restraints placed by the Corps of Engineers on intrusion of the installation into the river and because of changes in river elevation Including periods of high flow when the effectiveness of the wall will be reduced as a result of overtopping.
Section B-2 "River Flow" specifies that:
The model studies covered the range of flows from 20,000 to 165,000 cfs, equivalent to average river surface elevations of approximately 877 and 893 respectively. The downstream weir wall submergence is 9.5 and 25.5 feet for these stages. At the upper limit, the upstream portion of the wall Is submerged under 8 feet of water while 8 feet is exposed at the lower limit.
Flow over the weir Is maintained throughout the range with greater selectivity being attained at the lower stages.
Overtopping of the structure, while permitting greater amounts of silt to approach the Intake, will not nullify the effectiveness of the wall which will still retain Its capability to guide the coarser bed material past the Intake.
The Corps of Engineers estimates that a minimum controlled flow release of 3,000 cfs at Gavin's Point Dam is possible under extreme drought conditions. A flow of 3,000 cfs at the Cooper Site Is estimated to be the equivalent of surface elevation of 872 under open water conditions. The effect of ice conditions on such a flow is unpredictable.
Even though the river surface were to be lowered to elevation 870 under icing conditions, the flow necessary to maintain full operational conditions at the Intake structures would be available In a combination of overflow of the downstream weir section and upstream flow around the end of the weir wall.
Licensing Amendment # 15 FSAR Question No. 10.18 Part of processing Amendment # 15, the NRC requested a response to the following question:
Operation of the circulating water system during construction has indicated that significant quantities of silt are suspended In the water which have deposited In Intake bays when the pumps are not In service. Since the service water system provides river water directly to the RBCCW heat exchanger, the RHR heat exchangers, the diesel 12/22/2004 Page 27 of 72 12/22/2004 Page 27 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and Validato generators, and, in the event of RBCCW system failure, to the area coolers and RHH pumps seals the effect of silt on all safety related systems should be evaluated. Provide an analysis of the effects of silt on all safety related systems and components and provide Information on design modifications, operating procedures, and surveillance requirements to assure that the operation of these systems [shall] not be impaired. Provide your plans for preoperational testing to assure reliable operation of these systems under all normal and upset conditions.
CNS response Included the following:
An analysis of the service water system was made using the piping isometric drawings. Plant inspections were made to determine areas where silt buildups may occur including such things as low velocity areas, low point traps, check valve orientation and instrument sensing lines. At the present time, none of the above areas appear to present a problem that cannot be handled operationally. However, a large amount of operating experience will be gained prior to the completion of the preoperational test program. This Information will be reviewed and analyzed along with information gathered from equipment inspections as to determine what type of silting problems exist and what operations are required to assure proper system performance. Many piping concerns were addressed with a common approach. The system would be operated for a period of time, followed by an inspection to determine the rate of silt buildup.
The response also specifies that Silt accumulation in the entrance and interior of the intake will be controlled by the following methods.
- 1. A water jet sparging system will be Installed near the bottom of the intake structure to agitate the silt and keep it in suspension, thus preventing its settling out. The sparging system will consist of four rows of jets In each circulating water bay and five rows in the service water bay, as shown In Figure 10.18-1 [similar to Drawing 2056, PLAN "C%"-C"]. Silt at the entrance of the Intake structure will be kept In suspension by jets J-1, located near the trash racks. Jets J-3 will be Installed within the traveling screens to keep the screen boot area free of silt buildup. The above mentioned principal operating jets, J-1 and J-3, will be utilized on an automatic sequential schedule for periods to be determined by field experiment after installation.
Backup for the J-1 header when it is removed for servicing, can be provided by jet header J-2. In addition, J-2 will be used to clean the guides for the screens when they are replaced following maintenance.
Circulating water bay jets J-4 will be used only in the unlikely event that an undesirable amount of silt accumulates near the circulating water pump. Service water bay jets J-4A and J-AB agitate silt around the fire pumps and service water pumps respectively. These jets, J-4A and J-AB will be double headers, either one could act as backup to the other. Supply water to the jet sparging system will be provided by two new pumps, one to act as standby, A maintenance schedule for the sparging system will be set up based upon field experimentation and operating experience.
- 2. Silt panels will be provided In all stoplog guides for blocking the waterway whenever the pumps are not in operation and/or screens have been removed for maintenance. This will prevent unnecessary silt accumulation in idle bays.
- 3. The present single speed drives of the traveling screens (10 fpm) will be replaced by two speed drives (2 Y2 fpm and 10 fpm). Continuous operation of a screen at the higher speed results in increased wear due to the silt-laden water. During intermittent traveling screen rotation, the higher speed will be used, with the lower speed being utilized for continuous operation.
- 4. Streamlining the river flow in front of the intake to reduce the amount of sand and silt entering the structure is being investigated. A review of data assembled to date has indicated that some manner of modification may be effective.
The response additionally made the point that experience gained from our present use of a temporary sparging system in the service water bay has given us confidence that the sparging system will eliminate silt buildup in the intake structure.
Licensing Amendment # 15. FSAR Question No. 10.19 In another part of processing Amendment # 15, the NRC requested a response to the following question:
t 12/22/2004 Page 28 of 72 12/22/2004 Page 28 of 72
CR-CNS-2004-07409 B-2 Root CaUse Determination and Validation The location of the service water pumps in close proximity to each other and taking suction from a common sump may permit common-mode failure or propagation of damage from one pump to another. The present installation of service water pumps Is considered to be unsatisfactory and must be revised to provide suitable isolation of service water service water system components to assure that a single failure Will not result in loss of service water.
Submit your plans and schedule for modification of the service water pump Installation together With your analysis of the revised system to substantiate that a single failure Will not result in loss of service water.
Response
The response included discussion about class IS seismic criteria, barge impact criteria, internal flooding, redundancy, and separation. There was no discussion concerning the mcommon sump* as a causal factor for common-mode failure, such as river debris.
Response to Generic Letter 89-13 On July 18, 1989, the NRC issued Generic Letter 89-13, "Service Water System Problems Affecting Safety-Related Equipment'. The letter required utilities to confirm that the safety functions of the service water system are being met. The letter included the following actions:
I.
Implement and maintain an ongoing program of surveillance and control techniques to significantly reduce the incidence of flow blockage problems as a result of blo-fouling.
- 11. Ensure by establishing a routine inspection and maintenance program for open-cycle service water system piping and components that corrosion, erosion, protective coating failure, silting, and bic-fouling cannot degrade the performance of the safety related systems supplied by service water.
Ill.
Confirm that the service water system will perform its Intended function in accordance with the licensing basis for the plant.
IV. Confirm that maintenance practices, operating and emergency procedures, and training that involve the service water system are adequate to ensure that safety related equipment cooled by the service water system Will function as Intended and that operators of this equipment will perform effectively.
In Section 2.0 "Background" of CNS response, dated 112911990, It was specified that:
CNS plant design considerations support the Intent of IOCFR50 Appendix A GDC 44, 45, 46, and Appendix B, Section X1. Supporting examples are as follows:
" Startup testing provided initial design verification and a baseline for subsequent testing.
" Intake spargers for suspension of particulate to prevent deposition and blocking.
911Weir wall was added to the river bottom In front of the intake structure as a result of a silt deposition study to reduce slt loading. This design facilitates the Intake of cleaner water neat channel surface.
" Traveling screens remove particulate In excess of 318" in diameter.
" Service Water system strainers provide for removal of particulate in excess of 1/8' in diameter.
In response to NRC Inspection Report 87-10 (SSFI) findings, the District completed actions to verify that the service water system is capable of removing safety design basis heat loads. These actions are as follows.
" Periodic Service Water system post-LOCA flow testing to verify that post-LOCA flows are being maintained.
Quantitative performance testing of REC and RHR heat exchangers each refueling outage.
MService water system Initial design basis review.
0 Service Water system casualty procedure revisions for post-LOCA equipment configuration.
CNS Actions Taken In Response to Generic Letter 89-13 were:
I.
Implement and maintain an ongoing program of s~urveillance and control techniques to significantl reduceI 12/22/2004 12/222004Page 29 of 72
CR-CNS-2004-07409 B-2 Root Caugse Determination and Validation the incidence of flow blockage problems as a result of blo-fouling.
The present intake structure inspection Includes examination of the basin for silt, debris, and deterioration (including corrosion) and frequent monitoring of silt levels. The deterioration inspection Is performed by using divers or dewatering the bay.
The present ONS surveillance program provides for flow tests once per operating cycle to ensure the service water system and the associated required safety related components meet or exceed the post-LOCA design flow requirements. In addition, pump performance is monitored and trended quarterly to detect trends adverse to system performance.
Program enhancements, as outlined below, are planned as a result of Generic Letter recommendations and guidelines.
- 1. A sampling and analyzing inspection of biological fouling organisms will be added to the intake structure basin inspection effective With the 1990 Refueling Outage.
- 11. Conduct a test program to verify the heat transfer capability of all safety related heat exchangers cooled by service water.
[The CNS response included heat exchanger performance evaluations, heat exchanger Inspections, pipes, and flow rates.]
[Note that the wording of the response to 11 is different than Action 11 above.]
11l.
Ensure by establishing a routine inspection and maintenance program for open-cycle service water system piping and components that corrosion, erosion, protective coating failure, silting and biofouling cannot degrade the performance of the safety related system supplied by service water.
[The CNS response included heat exchanger Inspection requirements, underground piping, and UT inspections of piping and components].
IV. Confirm that the service water system will perform its Intended function In accordance with the licensing basis for the plant.
The District will verify, prior to startup from the 1991 refueling outage, the following aspects of the licensing basis as defined In the CNS Updated Safety Analysis Report (USAR):
- 2. That the system designs meet the applicable Single Failure Criteria.
- 3. That the heat exchangers are capable of removing the specified heat loads.
The District considers this verification applicable only to the germane portions of the Service Water system, RHR SW system, REC system, and DGJW system.
V. Confirm that maintenance practices, operating and emergency procedures, and training that involves the service water system are adequate to ensure that safety related equipment cooled by the service water system will function as intended and that operators of this equipment will perform effectively.
[The ONS response discussed operating procedures, emergency procedures, maintenance practices, OER, and training.]
Design Change 94-373 Intake Structure Guide Wall Modification Section 1.1 "General Description" of the design change specified that:
Installs an opening in the Intake Structure guide wall with a gate and frame assembly to allow for removal and reinstallation. The opening will be utilized during non-navigational season to provide the low level Design Basis
.flow path between the Missouri River and the Forebay. This will ensure that the minimum E-bay elevation of 863 feet required for operation of a Service Water pump will be available when the river levels are between 872 feet 12/22/2004 Page 30 of 72 12/22/2004 Page 30 of 72
CR-CNS-2004-07409 B-2 Root Cause Determination and Validation MSL and the design basis low river level of 865 feet MSL.
Section 1.3 "Plant Design Description" Included:
This design change wili not affect the performance of the Service Water system and will maintain the structural Integrity of the guide wail. The addition of an opening at the north end of the guide wail will bring the design into compliance with its originai design basis by ensuring the Service Water pump operability for river water levels as low as 865 feet MSL.
This design change places restrictions on operation of the Circulating Water pumps and Fire Protection pump 1C.
These pumps may have to be shutdown during non-navigational season at river levels below 872 feet MSL if sufficient level in the E Bay cannot be maintained.
Operational and Maintenance Changes Description Operational changes will be incorporated into CNS Procedure 5.1.4 [5.2SW] to include monitoring of the E Bay water level when the river water level drops to 873 feet MSL. Additionally, provisions will be included to shutdown the Circulating Water pumps and Fire Protection pump I C If E Bay level can not be maintained sufficiently to ensure Service Water pump operability.
Maintenance changes will Include removal of the gate from the guide wail opening prior to non-navigational season.
Section 2.0 "Justification" Included:
The design change is required to resolve the condition caused by the addition of a guide wail in front of the Intake Structure, under which sufficient river flow may not be available for operation of a minimum of one Service Water pump. This design change returns the Service Water system to its design basis for low river level of 865 feet MSL.
Finally In Section 3.2.2 "Basis of Design" documented the following:
The basis of design is to ensure the ability of the Service Water system to be supplied with adequate water in order to satisfy its post-LOCA required flow of 6,243 gpm (Ref. USAR Section X-8.1.5).
'Analysis of data retrieved from the D)P recorders indicate that CNS current strategy of swaping the SW pumps on daily basis and cleaning the E-Bay is being successful In keeping the Zurn Strainers clean with low DP. The retrieved data were for the period from 1218104 to 12/16104, and were collected in one minute interval. The data show that DP one-minute average was well below 1.0 psid for that period, and in most cases were below 0.5 psid.
12/22/2004 Page 31 of 72 12/22/2004 Page 31 of 72
B-3 Root Cause Analytic Tool "A" Keoner-Trecieo Model Is Is Not (Problems that can occur but did not)
WHAT SW flow degraded after running SW flow did not degrade when pumps A and B were SW pump C (low pressure) running.
SW Strainer B high DP Alarm There was no high DID alarm on the A Strainer at the initiated after running pump D time.
SW Strainer A high DID Alarm once SW pump C was Initiated.
SW Strainer B high DPD did not SW Strainer A high DP recovered after the automatic recover from the automatic back back wash wash.
Sediment build-ups of up to 2 feet were found on the E-Bay floor River water cascading from the open river side of the weir wall into the intake canal (visual inspection)
WHERE SW System Circulating Water (OW) System did not have any flow degradation High DP in SW Zurn Stainer B, first., SW Zumn Stainer A, second.
High sedimentation build-ups in E-Bay Flow cascading (over flowing) from the open river side of the weir wall into the suction canal (between weir wall and intake structure)
WHEN 11/20/2004 8:25AM for SW flow Not Immediately before degradation EXTENT None Fire Pump was not running at the time
-'I Possible causes are determined in section B-4 "Why Stair Case".
12122/2004 Page 32 of 72 12/22/2004 Page 32 of 72
CR-CNS-2004-07409 B-4 Root Cause Analiftical Tool "B" - Why Stair Case WHY STAIR CASE (Sh.1I of 2)3 Iwash could notI clean Strianer BI High concentration of sediments Introduced wlien.SW puhip D was started build up on E-Bay I.. floor
- Se'diments 1
separating from Water In'E-B'ay is not agitated sufficiently (Spargers perf~imance)
Eroed [________
nozzles Sagr-are not oriented pro enl Current sparger sequencing Is not
.appropriate 12/22/2004 Page 33 of 72 12/22/2004 Page 33 of 72
CR-CNS-2004-07409 WH 'STIRCASE.
[
S.2-of 2)
..SWStrainer is overwhelmed by sediments (From ShK 1)
Back wash is not DP. monitoring Immediate co frequent enough.
frequency' is recognized Insufficient. DP-is additional purr monitored once per
'shift-, mannually..
Instrumentations do Operator not exist for provide continuous monitoring guida and recording.
ndition Is not rhen placing Ips In service i
are not
'dwith ince 12/22/2004 12/222004Page 34 of 72
Humin P~rfnrm~nr~ A~nAfi~ nf Ew~nt The following latent conditions or active errors were identified during this root cause investigation:
While not a latent condition or active error, precursors during the prior week (e.g. alarms, E screen carryover, SW pressure drop) were not sufficiently or aggressively addressed to prevent this event An LO was initiated to perform this evaluation (reference section A-2).
Attachment C - Scope & Resolution IC-1 Event Detail Chronology of Event The following is taken from CR-CNS-2004-07407 At 0825 the crew was starting Service water pump "D" for weekly service water pump rotation and to minimize the pressure drop effect recently seen on the service water system when E bay J-4 spargers are rotated.
The following actions took place (as recorded In RONAN) starting at 0825:19 08:25:00 Following the start of SW-P-D both the "A" and "B" SW system pressures were noted to be dropping rapidly. When pressure on the "A" subsystemn reached 38 psig "C" Service water pump was placed in service.
08:25:19 "B" SW Zurn strainer high DIP alarm was received shortly after starting the "D" SW pump.
Approximately 20 seconds after the DIP alarm was received SW-MO-37 Isolation alarm was received due to low pressure and SW-MO-37 automatically closed.
08:25:20 SW-MO-36 Isolation alarm was received shortly after the SW-MO-37 alarm and SW-MO-36 closed. "A" SW Zurn strainer high DIP alarmed - 1 second following the SW-MO-36 Isolation alarm. The Crew entered 5.2SW to mitigate the event 08:25:30 Scram actions associated with 5.2SW were assigned to scram the plant In the event SW could not be restored to the TEC system. Also TEC, REC and turbine Generator temperatures were identified as critical parameters.
08:25:50 At seconds Service Water pumps B and D low bearing water low flow alarms were received.
These alarms remained In alarm for approximately 30 seconds.
08:25:54 An attempt was made to re-open SW-MO-37 after the Isolation as "B" SW Pressure restored to above 38 psig. Upon reopening the valve the "B" SW Header pressure dropped and the Low header pressure alarm was received on the "B" side. Upon SW-MO-37 getting full open the valve stroked closed again due to low header pressure. The valve was left closed, as header did not restore to above 38 pslg upon Isolation.
08:26:27 SW B&D header low pressure alarm reset then re-alarmed and finally cleared @ -0842.
08:26:46 SW-MO-36 was reopened as the "A" Service Water header pressure was ~68 psig. As SW-MO-36 was opening the "AK Zum strainer high D/P was received again and loop pressure began to drop.
Pressure lowered until the isolation signal was received again and SW-MO-36 re-closed. As SW-MO-36 was closing the "A" Zum strainer high DIP reset. Upon reaching full closed "A" SW Loop pressure was reading -77 psig. Based on the Zumn strainer alarm resetting and the higher pressure on the "A" loop pressure SW-MO-36 was opened again. [T+1 881 SW-MO-36 remained open and '"'AK loop pressure was steady at -56 psig.
08:30 At this point the order was given to bypass and isolate the "B" SW Zurn strainer due to the high D/Ps. Upon bypassing the strainer pressure rose from - 30 psig to -70 psig.
08:45 At this point the "B" SW pump was secured and SW-MO-37 Isolation was reset.
The SW header pressure for B SW Loop was below alarm set point for - 16 min.
12/22/2004 Page 35 of 72 12/22/2004 Page 35 of 72
CR-CNS-2004-07409 C-1 Event Detail 18:00 It was recognized that both Zum strainers reaching a DIP of >1 5 psid resulted In both SW subsystems being inoperable for approximately 1 minute and 56 seconds. This condition is identified in CR-CNS-2004-07407.
Subsequent to this condition at approximately 1800 it was recognized that both Zumn strainers reaching a D/P of >1 5 psid resulted in both SW subsystems being inoperable for approximately 1iminute and 56 seconds. This condition is identified in CR-CNS-2004-07407.
C-2 Unes of Ingullrv Q: How did the following organizational factors contribute to this event:
o Strategic planning oi Tactical implementation o Prioritization Follow up, self-assessment or monitoring.
A: The following brief statement regarding Operator response to the SW Pressure Drop Event This was based on a review of documentation provided by the Operations Manager and the brief interview with him on 12/20/04.
OPS review of the event concluded that Operator response to this event is deemed to be appropriate.
Operators correctly implemented the appropriate procedures and maintained positive control of the plant during the event.
The on-watch Shift Manager provided the following comment in a submittal dated 11/22104.
The crew entered the appropriate procedures as noted in the CR logs, Individual actions, overall crew response and command and control were appropriate and positive. The crew performed appropriate actions and all had the same overall understanding of the plant, the cause and actions to be performed.
Additionally, during the Interview, the Operations Manager Indicated that he had conducted discussions with the on-watch crew shortly after the Service Water System pressure drop and had subsequently completed a review of the event. Based on these reviews, the Operations Manager concluded that the crew responded as expected and that the actions taken by the crew were conducted in accordance with the appropriate procedures.
Current training provided to operators for these types of events is symptom based. Operators are trained to monitor system performance and react to changing critical parameters. In this event the operators took action based on changing service water system parameters. Actions taken were in accordance with the symptom based procedures to restore and maintain service water system pressure. The response of the operators was consistent with training to respond to these kinds of events.
Q: Why was the differential pressure so high on the SW strainers? What was the quantity of material that was removed from the strainer? What was the trend prior to the event based on OP reading, back wash cycles, and alarms? Review any CAP items.
A: The SW Strainers were overwhelmed by the amount of sedimentation they received when idle SW pumps C and D were started. The amount of sedimentation collected from each strainer was approximately 1 'A gallons. The A Strainer was able to clean up Itself automatically. The trend of strainer dP alarms prior to the event was increasing. Alarms had been received in the near past. NotificationslCRs were written on alarms received by the operators, but actions were not sufficiently timely or aggressive enough to prevent the event This represents a missed opportunity.
12/22/2004 Page 36 of 72 12/22/2004 Page 36 of 72
CR-CNS-2004-07409 Q: What is the affect of starting a third and fourth pump on the system? How does the system and strainers respond?
A: Build up of sedimentation around the idle pumps has been occurring. Simultaneously, the strainers have been accumulating material since their last cleaning (approximately eight weeks prior to the event). The pre-existing fouling of the strainer provided a DIP of 2.8 psld. As an idle pump was started, a sedimentation burst occurred that entered into the strainer.
During the event, when idle pump D was started, the sedimentation burst combined with the pre-existing fouling level was of such a magnitude that it overwhelmed Zurn Strainer B. The strainer was not able to self clean via its automatic back wash logic and had to be manually cleaned. The high build up of sedimentation on Zumn Strainer B resulted in flow reduction in SW Subsystem B. This flow reduction was of such magnitude that pressure on the common header was reduced to about 38 psig, as seen by operators. This led to the operators' decision to start idle SW Pump C in order to recover SW header pressure.
Starting SW Pump C resulted In a second sedimentation burst that attacked Zumn Strainer. This strainer was also already fouled to 3.3 psid. The sedimentation burst was of a smaller magnitude (when compared to strainer B) such that Zumn Strainer A was able to self clean via its automatic back wash logic.
Q: Is there seasonal (spring, winter, summer, fall) affect on system operation? How does river level affect sediment suction into pumps and sediment build up on the strainers?
A: CNS events data since 1/1112001 were plotted on a time line chart and evaluated for different seasons.
The data suggest that problems with Zum Strainers high OP were occurring during all of the four seasons.
0: Determine the system operational cause for low pressure In the service water header. Consider additional loads such as the spargers and high DP on the strainers.
A: Strainer B was overwhelmed by higher build up of sediments that was caused by sedimentation burst, which In turn was caused by the starting the idle pump D. This higher build up created higher DP across Strainer B, which caused reduction in subsystem B flow rate. While this was occurring, SW system loads did not change, SW subsystem A with only pump A running was not able to meet system flow demand, thus resulting in low pressure indication on SW header. When idle pump C was brought into service to provide additional flow to recover header pressure, the sedimentation burst created high OP across Strainer A that in turn reduced subsystem A flow. With both subsystems flow reduction, SW discharge header pressure was lowered even further to the point where non-critical isolation set point were reached (non-critical supply header was automatically isolated).
Q: Did the sparger system perform effectively? Are they plugged? Worn out? Properly oriented for sediment agitation? Do they have sufficient jet force for cleaning? What is their history and PM tasks?
A: The flows to the Spargers were measured and compared to a calculated value. The calculated value is based on the physical dimensions of the Sparger and their flow pressure. The comparison shows indication of possible eroded and plugged Spargers. The Spargers nozzles were replaced in 1998, and before that in 1994. Sparger 4JAI was not repaired in 2002 and considered not needed because it was a back up, because it is bolted to the floor (not removable for maintenance).
Q: System operation history over the last 36 months: Operations with 3 and 4 pumps? Duration? CW pumps operation? Frequency of automatic and manual back wash of strainers?
A: Review of available data for SW system operation with 2, 3 & 4 pumps in service against the CW pumps in service, and the notificationslcondition-reports did not reveal a direct correlation to the sedimentation build up, strainer back washing, strainers cleaning, or strainers problems. This combination should be included in the planned mathematical model.
12/22/2004 Page 37 of 72 12/22/2004 Page 37 of 72
CR-CNS-2004-07409 Q: What input items shall be considered in ongoing CFD analysis including projected river changes (modeling, studies).
A: The overall objective of the current Weir Wall project is to develop and implement methods to minimize the amount of suspended sediment and gravel entering the OW and SW systems, thereby reducing the impact to the traveling screens and the SW System. Reducing bed load sediment deposition within the Intake Structure Will be accomplished through modifications to the existing Weir Wall (lowering) and additional sediment control devices (turning vanes in the river) will result in improved equipment reliability.
While the focus of this project is to reduce sediment in the Intake Structure, the primary objective will be to reduce larger size sediment such as gravel from entering the Intake Structure.
Based on discussions with the Project Manager and the Lead Design Engineer, the project has been modeled to provide optimum effectiveness at a river level of 875'. Current (and projected) non-navigation season river levels are expected to remain at approximately 876'. The proposed solutions are expected to provide immediate, mid-term and long-term improvement while river levels remain above 875'. Therefore, the implementation of modifications to the Weir Wall and installing river turning vanes are considered corrective actions. However, river level can only be actively managed to a certain extent. Future environmental factors cannot be controlled. In the unanticipated event that river levels continue to decline over time to levels less than 875', there is a potential that the proposed modifications will become less effective. In the Interest of designing the station for license renewal, additional modeling should be pursued for river levels less than 875'. The pursuit of additional modeling for long-term planning is considered an enhancement action.
The benefits of additional studies or modeling were also discussed with the Project Manager and the Lead Design Engineer. The following specifics were discussed.
Model river levels less than 875'-As discussed above, this activity is recommended to be performed as an enhancement action from this root cause rerport.
Model the spargers in E-bay - It was determined that the computational fluid dynamics (CFD) model would be capable of modeling the spargers in E-bay. However, there would be little, if any, compensating benefit from this course of action. Improved sediment transport from modified traveling screens (to be Installed in Spring 2005) and mitigating measures already being pursued for the Weir Wall and river turning vanes are anticipated to minimize the site's dependence on the sparger system.
Modeling the spargers in E-bay is not recommended at this time.
Perform CFO Sedimentation modeling of E-bay - This activity Is currently In progress. An analysis Is being performed in order to predict the sedimentation results from Installing a section of stop log in E-bay. Results of this analysis are expected during the week of 12/20104. It was noted, however, that a similar analysis was not being pursued to predict sedimentation for E-bay without a stop log section installed. It is recommended that this model be obtained In order to gain additional understanding of the sedimentation transport in E-bay and to perform a comparison to the analysis done for stop log installation. The comparison data will be used as Input for making the decision whether to proceed with stop log installation. An analysis should be pursued to predict sedimentation for E-bay without a stop log section installed as a corrective action.
Repeat the silting study performed In 1973 - This study was essentially redone under the current project to address the Weir Wall. The current evaluation used the original study as Input, built upon it with '
current technology, and Input actual plant data from December 2003. Consequently, there Is no need to reperform this study as a separate action.
Model J I Sparger Pipe impact - One of the operational changes that recently occurred (October 2004) prior to the event was the removal of the J1 sparger piping on the E-bay rack. The entire rack was replaced with an upgraded design. It Is possible that the location of the J1 sparger piping (near the floor) combined with the accumulation of material in the rack below the piping formed a non-intentional weir.
This weir may have been preventing the accumulation of material In E-bay. As late as 5/1 0/04, soundings of the interior of E-bay Indicated no sediment accumulation in this area. The impact of the Ji sparger piping/rack removal could not be refuted during the Investigation. It Is recommended that a CFD analysis be oerformed on this conflauration as a corrective action from this report. This information is 12/22/2004 Page 38 of 72 12/22/2004 Page 38 of 72
CR-CNS-2004-07409 intended to aid in confirminglrefuting this change in configuration as a cause.
Q: Hydrological effect of river on sediment intrusion Into E bay? Is there higher flow turbulence at river bottom, due to low river level, allowing for more Intrusion of sediments? Flow characterization that influence sedimentation build up?
A: Enercon CFD Analysis Module suggests that river flow is turbulent in nature and has an up ward direction which is causing pick up of sedimentation from the bottom of the river, at all times. Concentration of the sedimentation in river water is in state of flux, and depends on river level and frequency of rain conditions.
When CW pumps are in service, the level in the intake canal drops to a lower level than the river, due to geometrical limitation of the canal and river level. This water level difference allows for river water to cross over the weir wall Into the canal. The CFD module shows that water velocity magnitude changes from 4 ft/s in the river to 7.5 ft/s in the area over the weir wall, and the returns to 4 ft/s in the in-take canal. It is also recognized that some of the water that crosses the weir wall will flow in the direction of the river (south), but larger portion will flow into the canal in the north direction to supply the In-take pumps. This in addition to the acceleration of river water flow from 4 ft/s (on the river side of the weir wall) to 7.5 ft/s (over the weir wall),
and the deceleration to 4 ft/s (in the in-take canal) results in actual increase In the concentration of sedimentation suspended in the water entering the in-take bays.
The CFD Module also shows that that sedimentation start to precipitate as flow entering the E-Bay start to slow down. However higher flow near the floor area from the entrance to E-Bay to the trash rack sweeps the precipitating sedimentation (in essence providing additional concentration mechanism). AS flow starts to slow down in the area between the rack and traveling screen, some sediment precipitation start to occur.
Then full precipitation is completed around the SW pumps area as flow magnitude is reduced to about 0.1 ft/s. Additionally, the Module suggests that there is more flow Is delivered to E-Bay from the in-take canal than what the SW pumps are capable of pumping out. Assuming that the reverse flow at the entrance of E-Bay Is similar to the one at the traveling screen, then a third concentration factor of sedimentation In the E-Bay is created.
0: 'Review of System Precursor Events and input provided on Organizational effectiveness and response.
A:
Background:
The following identifies and summarizes the equipment issues and organizational responses that occurred during the week of Nov. 15-20, 2004 leading up to this event:
On 11/15104, CR-CNS-2004-7286 was written Identifying a high DP alarm on the A' SW Zurn strainer. This condition occurred during start of the *C' SW pump. Investigation revealed the strainer was in automatic backwash and the alarm immediately cleared.
On 11/16/04, CR-CNS-2004-7301 and 7302 were written identifying high DP alarms on the MAK and "B SW Zum strainers. This occurred during the performance of procedure 2.2.3.1 for flushing the spargers. The alarm immediately cleared.
On 11/18/04, CR-CNS-2004-7376 was written due to high DP on the OB" SW Zumn strainer. This occurred during shifting of the 'E" bay spargers. Immediate investigation found the strainer in automatic backwash, DPD of 0.6 psid and the alarm cleared.
On 11/19/04, CR-CNS-2004-7388 was written identifying that the rising silt/sediment levels in the CW bays may be attributed to the removal of the silt plates.
InterviewslAnalysls:
Various interviews were conducted of organizational members who were involved in responding to and reviewing these events. This included personnel from maintenance, operations, and engineering. These 12/22/2004 Page 39 of 72 12/22/2004 Page 39 of 72
CR-CNS-2004-07409 interviews revealed that there was knowledge and concern that these conditions were not representative of conditions normally experienced in the *E* bay. Personnel had Identified an unusual condition with sand carryover on the "E' screen which is not normally seen. Some communication occurred regarding the need to inspect and clean the Zurn strainers based on the repetitive DP alarms that were occurring. The organizational response Included scheduling the contract divers to mobilize and clean the 'E" bay outside the screen which occurred on 11/18/04. Routine trend reviews of system parameters did not identify the need to accelerate the scheduled cleaning of the Zumn strainers.
==
Conclusions:==
The organizational response to these precursor events was not aggressive enough to prevent the SW low pressure event Communications within the organization were not effective at Identifying the potential threat even though some recommendations were being made to take some additional action.
Action:
- 1. Develop organizational lessons learned from this event including effective communication, sense of urgency responding to issues, and operational focus. Communicate these lessons learned at a weekly managers meeting and a daily CRG.
0: System changes that may have contribute to the problem, e.g. setpoints, DP alarms, automatic back wash DPD, automatic timer and manual back wash frequency, etc. Compare to VM recommendations.
A: As stated in section B-3, all design changes were reviewed and found no single or cumulative affect. The VM recommendation of running the strainers in continuous back wash during expected high sedimentation period was not implemented before the event Q: Provide review of system design basis and analysis for how ONS is meeting basis with current condition.
A: Evaluation of Increased River Fouling on Service Water System Design Basis With Zurn Strainers in Service The Zumn strainer baskets have 1/8* openings in them. These strainers have not had a history of damaged baskets where particles larger than 118" particle size can get through. The components in the SW system, The DG heat exchangers, REC heat Exchangers, RHR heat exchangers, Control Building HVAC: unit, Control Room HVAC unit, and gland water supply all have openings, or passages greater than 1/8' in size and increased sediment loading will not cause blockage or prevent performance of the design function.
Also, increased loading can cause higher erosion rates, but components (pipe and heat exchanger tubes) are monitored for erosion as directed Generic Letter 89-13 program, "Service Water System Problems Affecting Safety-Related Equipment%. This monitoring is Incorporated Into the heat exchanger and erosion/corrosion program at Cooper Nuclear Station and these programs make provisions for increasing trends for erosion in the system.
With Zumn Strainers Bypassed EE 03-003, Rev 2, addresses SW system operability with the Zumn Strainers bypassed. Bypassing the Zurn strainers could Impact downstream components In the SW system by causing particles larger than 118* to accumulate in the DG heat exchangers (and DG 1 suction strainer), REC heat Exchangers, RHR heat exchangers and (and RHRSW Booster Pump suction strainers), Control Building HVAC, unit, Control Room HVAC unit, and gland water supply. When the Zurn strainers are bypassed, the traveling screens are required to be intact to remove particles greater than 3/8", the gland water supplies are crosstied so the division with the Zumn strainer In service supplies filtered gland water (results in a 7 day SW LCO). The DG's, RHRSW booster pumps, Control Building HVAC unit and Control Room HVAC unit are also not run when the Zumn strainers are bypassed (except in an emergency). Since these components are normally isolated, they do not see increased amounts of river loading. The component that normally does see flow when the Zum strainers are bypassed is the REC heat exchangers. These heat exchangers have larger 12/2-2/2004 Page 40 of 72 12/22/2004 Page 40 of 72
CR-CNS-2004-07409 tubes than the traveling screens and are periodically backwashed and the performance of the heat exchangers is monitored as required by GLB9-13. This trending of heat exchanger performance would determine if Increased fouling resulting from Zum strainer bypass operations was occurring.
In either case, the system is still operating per its current design basis with increased river fouling with the Zumr strainers in service and with the Zurn strainers bypassed.
Attachment D - Risk D-1 Risk Assessment Summar The event had considerable business risk. Loss of Ultimate Heat Sink UHS requires entrance into Technical Specifications Actions that can cause plant shut down and loss of power generation.
D-2 Probabilistic Significanlce Assessment (PSAJ Risk Management estimated the average risk of the Service Water zurn strainer plugging condition, concluding that it was not risk significant The change In the core damage probability during the condition is estimated at less than I E-06, which is less than the threshold for risk significant Increases.
The risk evaluation assumes the plugging condition affected only one Service Water subsystem and the other Service Water subsystem was maintained. Due to system and environmental conditions at the time, one Service Water subsystem was sufficient to provide the non-essential function and keep the plant on-line. Therefore, the Impact on plant trip frequency was negligible. In addition, the frequency of a total loss of Service Water was not increased for this condition.
With the manual bypass of a plugged Service Water zurn strainer, failure of the effected Subsystem is recoverable for the indicated condition. The condition was properly alarmed in the Control Room, and the operating crew had sufficient time to respond. During postulate 'd failure scenarios, the impact Is limited to the long-term plant response of equipment and containment following loss of heat sink. The initial response and short-term mitigation ability of the front-line safety systems to complete their safety function would not be affected.
However to bound the condition, the risk evaluation assumed that one SW Subsystem was rendered unavailable without recovery within the 24-hour PRA mission time. Contributions from all postulated accident sequences were Included using the average plant configuration model.
.Using PRA 01la Model, with average test and maintenance terms, the model was re-quantified with Service Water A and C pumps assumed unavailable. Note that Service Water LOOP A is the bounding Service Water subsystem as shown by comparing allowed outage times in the ORAM-Sentinel Model.
Even though the affected Service Water subsystem was restored within 16 minutes, it was conservatively assumed a degraded condition lasted 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />.
The Core Damage Frequency (CDF) with Service Water A and C pumps assumed unavailable is I.49E-4Iyr. The base CDF, with average test and maintenance, Is I.45E-5Iyr. It follows that the change in CDF is then 1.49E 1.45E-5 or I.35E-4Iyr and if an exposure period of no more than i day is assumed, then the change in core damage probability for this condition is no more than I.35E-4Iyr
- 1 day * (1 yr1365d) = 3.7E-7.
12/22/2004 Page 41 of 72 12/22/ 2004 Page 41 of 72
CR-CNS-2004-07409 Attachment E - Learning Organization Effectiveness E-1 CNS (Internal) Oneratina Exoerience Refer to the time line of operation and maintenance problems in section B-2 above, and to Appendix A for more Information on related Condition Reports and Notifications. Internal operating experience review guided the investigation into causal factors for this event We compared precursor events to design basis documents to determine system and human performance deficiencies that influenced the outcome of the event.
We searched all Notifications written between April 2001 through November 2004 affecting the Zurn strainers and associated Instrumentation. These components are the ones impacted by causal factors.
Using this initial approach ensures the capture of similar symptoms related to the physical root cause (gravel plugged the strainer elements).
Based on review of the first set of Notifications and design basis documents, we conducted a second search, using the terms "weir wall" or uguide wall" in the description field of the NAIT database. These findings are summarized In Appendix A. Three patterns emerged:
" Once the operators first Identify a problem with strainer DID, there Is typically a period of several weeks before the strainer gets cleaned. Occasionally, the strainer plugging progresses during this period to a point where the station Is forced to take prompt, emergent action. For example, between 11/19/02 and 1/3/03, eight Notifications Identified an adverse trend In Zurn Strainer 'A' DPD. It was not corrected until the situation had degraded to a point where the Service Water System safety function was significantly challenged over a two-day period (reference SCR 2003-0010).
- The root causes of this problem Include sediment transport from the river to the Zurn strainers.
The Intake structure guide wall Is the design feature Intended to reduce the rate of sediment transport. Problems with sediment Imply the guide wall function Is degraded. Notification 10212732 "REVIEW WEIR WALL EFFECTIVENESS" was the first Item In the CAP that documented this problem. This Notification was closed to RCR 2003-0023 which initiated the Intake Structure Improvement Project Plan.
" The correlation between E-Bay 3-4 sparger rotation and Zurn Strainer plugging was first Identified In Notification 10210987 on 11/30/02. In retrospect, this was an opportunity to develop compensatory procedural guidance to mitigate the consequences of rotating the spargers when sediment has collected in the SW pump bay.
12/2-2/2004 Page 42 of 72 12/22/2004 Page 42 of 72
CR-CNS-2004-07409 E-2 Industry (External) Ogeratina Experience The following INPO related documents had been dispositioned by CNS:
SER 6-03 evaluated by CR-CNS-2003-07826 SERl-02 evaluated by CR-CNS-2002-02031 O&MR 427 evaluated without a CAP order.
Of the four events discussed In SER 6-03. Cooling Water System Debris Intrusion, all of the events were caused by other than one-time events. The algae and fish intrusions can be expected to re-occur frequently. However, none of the events were ongoing and expected to last for several months.
OMR 427 addresses seaweed, and large fish Intrusion, and not a permanent change to the Heat Sink operations (lower river level).
INPO.QE database search, using key words "Ultimate Heat Sink" and "siltf. resulted in the following two related documents.
OE1 51 08 - Slilt Levels In Main Intake Structure Exceed Allowable Values addressed design issues at Beaver Valley Power Station resulting from changes In river flow or level, see corrective action 4. The LER for this event is 0M~Icrosoft Word - CR0208291 LER1 2002-00lxSclean.doc".
The Beaver Valley LER provides the following discussion:
Even though the Ohio River has not previously approached the design basis I licensing basis extreme low water level during plant operation, the BVPS Unit I safety related River Water System and the 8VPS Unit 2 safety related Service Water System must have adequate ultimate heat sink capability to support the bounding low probability extreme low river water design basis / licensing basis postulated scenario. Current BVPS design analyses show that adequate inflow to the suction of the River/Service Water System pumps may not be assured with greater than 22 Inches of solid blockage in an Intake Structure bay. Previously identified as-found silt levels in Intake Structure bays have exceeded 22 inches. This represents a potential unanalyzed condition that could significantly degrade plant safety. Therefore, this Is being reported pursuant to 10 CFR 60.73 CAUSE OF EVENT The cause of this event was inadequate/incomplete design aspects. This cause is attributed to instances where the overall quality of the design product and the supporting documents are incomplete; such as inadequate reviews and updates to associated design bases or licensing bases documents. The design bases information derived In a design calculation performed In 2000 was not translated Into the Implementing procedures' acceptance criteria. Additional design information also was not effectively Incorporated Into the implementing procedures such that the design bases/licensing bases was easily recognizable.
Beaver Valley corrective actions Included:
- 1. A multi-discipline site team was formed to address immediate actions needed for the Intake Structure silt level concerns.
- 2. An operability basis was provided for the BVPS Intake Structure to address current and future expected silt depositon levels.
12/22/20G4 Page 43 of 72 12/22/2004 Page 43 of 72
CR-CNS-2004-07409 E-2 Inoust!X (External) Ocieratinci Experience
- 3. Plant operating logs have been modified to require a condition report be entered into the BVPS corrective action program if the Ohio River water level is above a pre-determiined value for trending of potential silt loading so that the operability of the ultimate heat sink is not challenged.
- 4. The dual-unit quarterly surveillance procedure for fth Intake Structure was revised to reflect current design and operating criteria. This included taking more silt depth measurement locations and identifying as-found silt levels greater than 15 inches In an Intake bay as an Issue which should be input into the site corrective action program.
- 6. The dual-unit Abnormal Operating Procedure which would be used at either BVPS unit during a low river water level condition was revised to reflect current design and operating criteria. This included revised bay flow limits and operating instructions during an extreme low river water leve condition, new restrictions on Alternate Intake Structure pump operation, and additional criteria for contacting the dam immediately downstream from BVPS on the Ohio River.
- 6. Various long term corrective actions are being pursued as enhancements to the immediate actions identified above to ensure future operability of Intake Structure bays.
- 7. The BVPS site criteria which currently define the Emergency Action Levels during low river watel conditions as an emergency condition are being reviewed in the corrective action program.
Completion of the above and other corrective actions are being tracked through the corrective actior program.
0E16024 - (Update to 0E14740) - Excessive Buildup of River Sediment Results In Reduced Water Depth Outside Plant Intake Structure addresses Inadequate follow-up action after a design change which was to address the issue of silt buildup In front of the Intake.
The OE was issued by Duane Arnold and provides the following description of the event:
On July 18, 2002, divers were on-site to check the level of sediment and silt at the Intake Structure. The divers mapped an area directly in front of the Intake Structure to determine the extent of silt/sediment build-up. This map showed that the water depth directly in front of the intake was only about one foot. At the same time the river level Indication was normal, at approximately 732 feet. Engineering performed an Operability Evaluation. The evaluation determined, that 8 inches of water depth was sufficient to allow adequate flow to maintain River Water System (RWS) operability. The Divers pumped the silt/sediment from in front of the intake for the next several days and established a channel about 20 feet wide, to a depth of approximately 4 feet. The depth of the center of the river was approximately 4 feet during this de-silting operation.
Several years earlier the plant had installed vanes in the river near the Intake Structure to keep the silt/sediment from building up. The effectiveness of the vanes was based on directing the greatest amount of river flow along the west bank across the face of the Intake Structure. An Engineering Design Change installed these vanes, and recommended that the plant inspect the vane field to ensure that silt/sediment levels did not re-develop. However, these follow up inspections were not performed and this allowed the sediment/silt buildup to occur unnoticed. The root cause of this event is failure to perform these follow-up inspections and the timely remedial actions these results would have dictated.
Causes:
The root cause of this event is the failure to inspect the flow vane field in the river on a routine to check its performance.
The Design Change that Installed the vanes recommended this inspection, including a mapping of the river bottom to ensure the vanes were not allowing sediment build up In front of the Intake. With the failure to inspect the vane field, river silt started to backfill this area. This is the root cause because if the 12/22/2004 Page 44 of 72 12/22/2004 Page 44 of 72
CR-CNS-2004-07409 E-2 Industry (Externall Operatingi Exiperience inspections had been performed, action would have been taken to dredge the river and change the thaiweg (line of maximum flow depth and velocity) to maintain the river flow through the vane field. With the return of ordinary flow through the vanes, the sediment/silt levels would have remained low and the depth in front of the intake would have been maintained. Two causal factors were identified that contributed to the root cause:
1). The old modification procedure was inadequate. The Engineering Design Change Modification Program Procedure did not require the implementation of Preventive Maintenance upon the completion of a modification.
2). The change in river flow conditions. If the thalweg (main flow channel) of the river had not shifted over the years since the flow vane field was installed, the excessive silting condition would not have occurred In front of the Intake Structure.
Other Related Issue: A related issue that contributed to this event was the indication of normal river level with this excessive silting condition in front of the Intake Structure. The river level Indication operated as designed, however the actual water depth was only one foot. The level instrumentation is a bubbler type.
This instrument indicates the height of the river surface but does not measure the depth of the water or the height of the river bottom.
In this event, the river bottom was rising thereby causing a lower than normal depth in front of the Intake Structure. If the plant had an indication of actual water depth of the river In front of the Intake Structure, they could have taken action before the sediment/silt level increased causing low water depth.
Corrective Actions: Upon the discovery, the finding was reported to Operations and Maintenance Department Management An Operability Evaluation for the condition was performed and the Ultimate Heat Sink was determined to be operable. The area In front of the Intake Structure was de-silted. One proposed corrective action is to remove the remaining sediment in front of and upstream of the Intake Structure in the river. This will shift the main river flow back to the flow conditions that existed during the installation of the vanes and this would allow the vanes to operate as designed. The second corrective action is to inspect routinely the river and vane area for future changes to the thalweg and if necessary, to redirect the thalweg as required.
MISSOURI RIVER PLANT OPERATING EXPERIENCE Telephone interview with Fort Calhoun Station engineers, notes dated December 20, 2004:
- A similar event occurred at FCS on 11/21/04. While starting an idle SW pump, the associated strainer plugged resulting in loss of flow. The strainer rotating assembly was manually operated to clear the obstruction. Previous occurrences of this event at Ft. Calhoun were in February this year and In 2002.
- The OW system engineer has been In his current position form 1989-1 996 and 2002 to present. 2004 Is the worst year for river sediment related problem that he can recall.
- The condenser thermal performance is poor. The CW system engineer attributes this to tube sheet fouling. Back washings cause a step drop In water box DPD, but the gains are quickly lost.
- They dredged the river In front of the OWlS during the 2nd week of August, removing a 10' deep layer, the width of the OWlS, and approx 20' out. This filled back the next day.
- They dredged again in late-Oct, removing a 7' deep layer. Soundings indicated that this has also filled back in, within 3 weeks.
- On December 2, 2004, Ft. Calhoun reported a jam of a traveling water screen. The SW pumps are located in the same bays as the OW pumps, thus sharing the TWS. There were 3 screen jam events this year, include one in mid-October which resulted In 5 of 6 screen 12/22/2004 Page 45 of 72 12/22/2004 Page 45 of 72
CR-CNS-2004-07409 E-2 Industry (External) Operating Experience becoming jam med.
The following summarizes an email communication from an engineer with FCS:
Subject:
Plans for Managing the River Bottom Conditions at the FCS Intake This note is to communicate some of the discussions and thoughts I have had regarding the river bottom and how to manage the sand and rock Ingress to the Intake. Three items will be discussed that provide information pertinent to the current condition as well as a basis for some things we can do to manage it.
We have experienced similar conditions before, although not to the extreme we have now. A similar condition existed in June 1991 where the bottom was at the 976 to 977 ft elevation with a river level of 992 feet. Utility Diving Services cleaned out an area in front of the Intake. From June 3 to 8 they removed the sand to about the 968 foot elevation (two feet below the bottom of the Intake) from the grids to about 20 feet Into the river. Two weeks later bottom soundings showed the area had filled back in to its previous levels. River bottom soundings a month later on July 25 showed the bottom had dropped about 2 to three feet. However, the river level dropped the same amount so the amount of free depth was the same. The conclusion was that the elevation of the river bottom did not seem to be a factor in circulating water system operation as long as an adequate amount of free depth is maintained. The fact that the bottom changed concurrent with the top indicates that as the river elevation drops, the increase In flow velocity will scour or entrain more sand to self maintain adequate flow area. Obviously, the sanding condition noted in the summer of 1991 eventually passed and I would expect the current condition will do the same.
A second point worth mentioning Is what occurred 7/2104 when a crew was working to restore the F screen. They requested Operations to completely shut the CW-14E sluice gate and throttle the F screen to the point where they could maintain cell level. They eventually were able to throttle the gate to only 40% open with only a "couple feet" of drop in the pump cell level. This significantly increased the flow velocity and cleared out considerable amounts of sand In the cell. I would not recommend this drastic measure except for similarly drastic conditions because it does push the design limit of the screens.
However, It did achieve the desired effect and did provide good operating experience about the screens and cell performance.
Third is a discussion I had with the U.S. Corps of Engineers. Someone tipped me that the Corps had been dredging north of Blair so I called to ask if this might be contributing to our condition. They stated they dredged at three locations from mid-May until June 19, 2004. The locations were at 1-1/2, 5-1/2 and 8-112 miles north of the Blair bridge. Their modeling of dredging showed that the river had sufficient capacity to entrain what they had dredged. He said there should be no noticeable effect of their dredging within a few thousand feet down river. I then asked what his opinion was of what we are experiencing. He explained that during the last 4 -5 years there have been below normal flows. We have not experienced the rains and tributary contributions like we have had this year. He would, therefore, expect 4 -5 years of accumulation of bed material resident in the channel. His guess was that this bed material has mobilized with the increased flows and that the load is moving down river. We are on the outside bend of the river and are experiencing shoaling, a natural occurrence of load mobilization that should correct itself in time.
He mentioned that river flows have been lowering recently and that the material will move slower as a result.
The above three points are Instructive and provide a basis for the actions listed below that FCS could take to manage the current river bottom condition.
1.
Take weekly river bottom soundings to monitor the condition. We are planning to do this until the depth Is no longer a concern.
- 2. Operations should routinely throttle all of the sluice gates to approximately 50% to 60% open for several hours each day to help keep adequate free depth available In front of the grids. This should be done until the river bottom Is back to normal as determined from routine depth measurements or there Is minimal carryover on the E and F screens. CW-14E and CW-1 4F should remain throttled to 60 % open for the time being, at least until we can take the next depth 12/22/2004 Page 46 of 72 12/22/2004 Page 46 of 72
CR-CNS-2004-07409 E-2 Industry (External) Operatina Experience soundings.
- 3. We will ingest more debris if we throttle the sluice gates so we should backwash the condensers at least each shift to clear out as much debris as possible.
- 4. Lower the surface sluice and use it to blow away sand from the F screen only if the river is fairly clean. I am not certain how deep the sluice pipe can be lowered, but it probably can affect what accumulates in front of the F screen. To be honest, I am not sure how effective this would be, but I think it is worth a try.
- 5. I have asked the Environmental Affairs department to submit a renewal request for our "Nationwide No.3 Permit for Intake Maintenance Activities". This permit allows us to maintain the front of our Intake with certain restrictions. Removal of localized sand buildup in front of the grids is within the bounds of this permit. Dredging was briefly discussed with OPPD Environmental Affairs group, but my impression Is that it is not a realistic option. Dredging would require a special permit that would require environmental studies and legal due process (public hearings, etc.).
Items 2, 3 and 4 are recommendations that Operations should consider. Hopefully these recommendations (and the others) will stimulate more thoughts and discussions on the subject of managing the current condition in front of the Intake.
My intent was to communicate this email to all of FCS Operations and Supervision so that we can get as much benefit of discussion and input as possible. Please share it with others and contact me with any Input or questions.
Mid-American's Georne Neal Fossil Plant Water Intake Structures In addition to the above QEs, a search of the Internet was conducted. The search revealed a discussion of Model Investigation of Sediment and Hydraulic Concerns For Mid-American Energy Company's New Geiorge Neal North Water Intake Structures, which was prepared by Tatsuaki Nakato, Dirk Meduna, and Atsuhiro Yorozuya In December 2002.
The hydraulic model investigations were conducted for Mid-American Energy Company's George Neal North Station to improve newly designed circulating-water pump-intake structures. Both sediment and hydraulic models were built at undistorted geometrical scales of 1:30 and 1:12, respectively. The sediment model Indicated that a series of flow-tumning vanes would be needed to reduce the sediment intrusion Into the intake bays. A variety of pump-sump modifications were also developed using the intake model, including floor and back wall splitters, sidewall floor-coiner fillets, perforated plates to improve pump-approach flow distributions, and skimmer wells for free-surface vortex suppression.
This new circulating water pump intake was planned on the left bank of the Missouri River near Sioux City, Iowa for George Neal North (GNN) station by MidAmerican Energy Company. Due to past sedimentation intrusion problems experienced at other facilities along the Missouri River, sediment-related problems are anticipated at the (GNN) station as well. Potential problems related to vortices and non-uniform flow distribution inside the intake structure also exist at large capacity pump intakes. The report states that most of the newly designed intake structures, which are investigated to solve the aforementioned hydraulic problems, can be optimized without major changes. For example, scour-promoting flow turning vanes in front of intake buildings are able to prevent sedimentation. Integration of splitters, baffle columns and perforated plates into intake bays can produce a more uniform flow and suppress formation of free-surface and subsurface vortices.
Nebraska City Power Plant Per telephone conversation on 12/21/2004 with the Operations Manager of OPPD Plant at Nebraska City.
12/22/2004 Page 47 of 72 12/22/2004 Page 47 of 72
CR-CNS-2004-07409 E-2 Industry (External) Operating EXgderinc The plant's intake structure sounded very similar in design to ours, except the flow rates. They have a system that sounded very similar to our SW system, called the Raw Water System. The Raw Water pumps are rated at 8,000 gpm and typically only one is in service. The Raw Water system uses a bay that has a rotating screen system with a screen wash spray system like we use In our SW system. OPPID also uses a "grizzly' barricade similar to CNS. They have no strainers in the Pump discharge. OPPID also does not have any barriers, weir walls or "riprap" outside of the intake structure in the river. The OPPD station is located just up stream of the high velocity portion of the river bend, where as CNS is located in the high velocity portion of the river bend. OPPD cleans their heat exchangers every spring and have not noticed that there is ever a build up of river silt In their heat exchangers. However, they have had their share of sand build up in their intake lately. The Operations Manager said he was informed today for the need to clean up sand found around their rotating screen.
==
Conclusion:==
Problems of sedimentation build up are not unique to CNS and other plants are faced such problems as well, especially plants located on the Missouri River. Short term and long term actions taken by ONS are consistent with action taken by the above discussed plants.
12/22/2004 Page 48 of 72 1422/2004 Page 48 of 72
CR-CNS-2004-07409 E-3 Effectiveness of CNS Self-Assessment Activities E-3 Effectiveness of CNS Self-Assessment ctivities A review of Self-Assessments completed during the period 2001 - present resulted in the Identification of one assessment that includes Information pertinent to this investigation. In August 2003, an Interim Effectiveness Assessment (SA03054) of TIP Plan 5.3.1.2a, Service Water, was completed. The objective of the Assessment was to determine the effectiveness of the plan to Improve and sustain performance levels in relation to the plan's objectives.
The assessment cited *Decrease in Service Water Corrective Maintenance Work Orders, Corrective Action Items, and Unplanned ICO's." as a Strength. Two Areas for Improvement (AFI) were identified. The AFIs were related to inadequate documentation of motor refurbishment specifications for the Service Water Pumps and Service Water Booster Pumps. No AFIs or actions related to the Zum strainers or river quality resulted from the assessment.
The following is provided on Page 40 of SA03054 under the heading of Eauioment Reliability of Non-TIP Components.
Through a review of the NAIT database for trends related to the SW system, it was noted that the Zurn strainers were the subject of a number of notifications. The conditions documented in these notifications were the result of increased debris loading from lower than normal river levels. Since the identified conditions were not equipment related issues but rather associated with the quality of the river water, the Zum strainers were not included as part of the original TIP Action Plan for Service Water. Increased strainer monitoring and increased PM's for strai ner cleaning activities were performed to mitigate these issues and have been captured in the Corrective Action Program.
In the Assessment Conclusion Section of the report, the following is provided:
Regarding the concerns not specifically addressed within Action Plan 5.3.1.2a,....
Additionally, the Zumn strainers do not need to be a part of this Action Plan because they do not qualify as an equipment reliability issue. However, there are actions In the long-range plan of the Circulating Water (CW) system to Improve the water quality of the Incoming water to the Zumn strainers.
The comments above identify missed opportunities to address Service Water system probl ems caused by conditions of the Missouri River. The first missed opportunity occurred when the Service Water TIP Action Plan was developed without actions to address system reliability problems resulting from changes in river water quality that were documented numerous times in the Corrective Action Program. An additional opportunity was missed when no actions to address the impact of river water quality were implemented as a result of the assessment.
Attachment IF - Investigation Detail F-i Documents Reviewed USAR Section X-8 Drawings 2006, 2056, RCR 2002-2232A1 Evaluation Alion's CFD Analysis of Silting in CNS Cooling Water Intake, Doc # 261-0-2494-001, dated 11/21/03,
-CNS Operators Logs Vendor Manual 0520, "Service Water" 12122/2004 Page 49 of 72 12/22/2004 Page 49 of 72
_F-i LDocuments Reviewed ONS Procedure 2.2.3.1, "Traveling Screen, Screen Wash and Sparger System" US Army Corps of Engineers, December 10, 2004, 'Water Management Monthly News Release" US Army Corps of Engineers, March 2004, "Missouri River Main Stem Reservoir System-Master Water Control Manual" US Army Corps of Engineers, Spring 2002, *Missouri River Main Stem-System Description and Operation' US Army Corps of Engineers, April 2004, "Missouri River Main Stem-Summary of Actual Operations" US Army Corps of Engineers, March 2004, "Missouri River Main Stem-2004 Operating Plan' US Army Corps of Engineers, MRR Sediment Series Report No. 39c, January 2001, "Suspended Sediment Data Assessment Study, Missouri River at Nebraska City, Nebraska*
US Army Corps of Engineers, MRR Sediment Series Report No. 37, April 1991, "Missouri River Special Point-integrated Sediment Sampling Report" The Missouri River Division of Corps of Engineers U.S. Army, April 1954, "Terminal Report on Transportation Characteristics Missouri River Sediment" The Missouri River Division of Corps of Engineers U.S. Army, January 1958, "Experiments on the Influence of Temperature on the sediment load" The Missouri River Division of Corps of Engineers U.S. Army, February 1977, "Analysis of Temperature Effects on Stage-Discharge Relationship in a Missouri River Reach Near Omaha" The Missouri River Division of Corps of Engineers U.S. Army, December 1976, "Temperature Effects in the transition from dues to plane bed" The Missouri River Division of Corps of Engineers U.S. Army, November 1981, 'Missouri River Vertical Sediment Concentration Distribution for Different Sediment Sizes" The Missouri River Division of Corps of Engineers U.S. Army, August 1955, "Effects of Heavy Sediment Concentration Near the bed on Velocity and Sediment Distribution" The Missouri River Division of Corps of Engineers U.S. Army, 1957, "A Study of the Shape of Channels formed by Natural Streams Flowing in.Erodible Material" The Missouri River Division of Corps of Engineers U.S. Army, June 1953, "Transport of Sediment Mixtures with Large Range of Grain Sizes" The Missouri River Division of Corps of Engineers U.S. Army, June 1979, "Sediment Transport Relationships" Bums & Roe, December 8, 1972, "Cooper Nuclear Station Cooling System Silting" Amendment No. 31, Docket Number 50-298, Final Safety Analysis, "Intake Structure Guide Wall" Cooper Intake Performance of Sparging System - September - October 1973 Randall Noon, P.E., October 25, 2002, "A Brief Description of the Missouri river and Cooper Nuclear Station Operations" RCR 2002-0294, Apparent Cause "Sanding-in of Traveling Water Screens" RCR 2001-0841, Apparent Cause "Frequent Main Condenser Backwashing" RCR 2002-161 7, "Trash Rack/Rake Upgrade 12/22/2004 Page 50 of 72 12/22/2004 Page 50 of 72
0 CR-CNS-2004-07409 F-1~ Dairennc rnn1I-24w1 John Garrison, Sedimentation & Channel Stabilization Section, U.S. Core of Engineers Dr. A. Jacob Odgaard, Professor at University of Iowa Roger Grable - Omaha Public Power District Allan G. Koenig, P.E. - Fort Calhoun Nuclear Station Loyd Church, System Engineer - Fort Calhoun Nuclear Station Rick Tanner - CNS Mechanical Maintenance Supervisor Kim Su~ll~wold - CNS Mechanic Doug Hitzel - CNS Shift Manager Rod Penfield - CNS Control Room Supervisor Attachment G - Unrelated Conditions Discovered 6 -1 New Condition Remirts CR-CNS-2004-07568 (Debris found in E-Bay during sit cleaning) 12/22/2004 Page 51 of 72 12/22/2004 Page 51 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
Noti.
Date Description Component Order Notes Recent river bottom soundings do not correspond with bottom elevations stated R-09 00/4ivr bottom soundings not CR2 94-In the USAR. USAR XII.2.2.7 states "the bottom of the Intake water channel Is 1or00197n 10/09/94R 0896 approximately at elevation 855'. The bottom of the river bed at the Intake
____channel Is also at approximately elevation 855' USAR pg X-8-3, item two Is not being implemented. Statement says that, "silt Siltpanes ae no proide forallpanels are provided In all stop log guides for blocking the waterway whenever 2-25567 02/08/98 sito pnlsg ariesntpovddfo.l the pumps are not in operation or screens have been removed for stop og gudes.maintenance";
requirement not met: USAR statement Is incorrect. Closed per UCR 2000-015, as related to USARO! #98-054.
Recent PIRs associated with SW Recent evaluations Identified sand/silt contributes to Increased maintenance of 2-58 20/8Identify siltation as a contributor to SW SW equipment. Action Is required to Identify' how sand/silt can be reduced or 2-258 020998increased maintenance of removed; This was closed to CAQ 97-1550. The root cause was silt In the service
____equipment.
water system.
The procedure change to support scheduled maintenance would open the D to E bay x-tie and Install silt panels In E bay. Silt panels are needed for the work and to prevent silt buildup, (ref. USAR pg. X-8-3). The silt panels block normal flow A prcedre cang Isnecesar toand the x-tie becomes the only flow path to SW pumps. Two Items prohibit the A--51 0/19suportoedure hpange Is nheessay toprocedure change: 1. No calculation supports the x-tle capability of providing 2-251 0/3/9 spprtaonlin e
s reparofte. a adequate flow to SW pumps. 2. The x-tie valve is not seismically qualified and travlin scren.no seismic analysis could be identified for the plant conditions described above.
If flow through the x-tle was blocked or restricted In a seismic event, SW flow may also be lost. The x-tle Is non-essential, so the single failure criteria may not be satisfied In this configuration.
A & B Zurn DPs appear high even Determine If setpolnt alarms can be raised due to possibility of high SW load. DD after continuous back flush. DPs are STNR-A 2-29058 - Trends are within expectation due to multiple service water pumps 2-29058 06/06/98 1.3 to 1.5 psid with a single pump STR-B running. DPs range from 1-4 psid. Alarm setpolnt to remain as Is to ensure running and 2.5 to 3 psid with two notification of strainer build up.
pumps running.__________________________________________
12/22/2004 Page 52 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted in yellow are related to causal factors. Thiose highlighted in green are PMs.
Notif.
Date Description Component Order Notes Equipment performance problems have caused outage extensions, unplanned shutdowns, and power reductions. Other problems have resulted in Increased maintenance and the need for compensatory measures to maintain Important safety systems. Some equipment performance problems are recurring. In WANOAFI equpmet peformnceaddition, silting problems with the service water system have resulted In the 4-07111 03/09/00 WArobAle eupmnmprorac need for operators to Initiate service water flow through the RHR HXs each week problemsto prevent excessive sift buildup. RCR 2003-0362 A/C - Management systems have not consistently established and maintained high standards of performance at CNS. Another significant challenge to organizational effectiveness exists at CNS. This additional Issue can be described as ineffective site-wide prioritIzation and integrated planning.
DREDINGIN FONTOF ITAK NAI 4-Need to have the area between the grizzlies and the weir wall dredged out by 4-14487 03/01/01 DTREDGNGTNU R ON OF1448EN 7T contractors. The intake structure bays have been seeing excessive amounts of STRUCURE 4487sand buildup.
While running 2 SW pumps In A SW ioop, numerous high strainer DIP alarms 10078540 04/11/01 Possible A SW Zum strainer fouling STNR-A 41681161were received In the control room. A SW Zum strainer D/P was checked locally I
and read 4.5 psid. Cleaned strainer using WO 4171191 on 5/21/01.
10080992 04/26/01 SW A ZURN STRANER HIGH D/P STNR-A 4171191 With 2 SW pumps running, Zurn strainer d/p could not be maintained lower than high d/p alarm setpoint. Cleaned strainer on 5/21/01.
PMEXAMINED STRAINER AND ITS COMPONENTS, IT WAS IN GOOD.,
09/12401 EXAMINE SW STRAINER B3 STNR-B 417892415PJM ON 101889101901A ZURN STRAINER SHEAR PIN STNR-A 4203595 Method of Discovery: OPERATOR ROUNDS. Priority 2 CM repaired same day.
10132925 12/27/01 A SW ZURN STRAINE R SHEAR PIN STNR-A 4216776 Method of Discovery: Responding to Service Water Strainer A High D/P alarm.
FAILED Repaired 12/28/01 Remove ice deflectors in front of Intake structure Within 5 days of March 26, 10145182 03/04/02 WEIR WALL GATE CLOSED CNS-SW 4229257 2002. Activity Is normally performed in conjunction with PM to dose weir wall gate. However, weir wall gate was closed early this year due to sanding problems.
DREDE I FRNT O INAKEDredging of the channel area between the intake structure and the weir wall will 1015DRE6GE4IN3FRONTUOFNGNTA21 CNS-CW 4234794 be required In RE21. Heavy sediment and debris build up occurs in this channel, DURIN RE21particulariy during winter operations. Work completed under CM order.
12/22/2004 Page 53 of 72
CR-CNS-2004-07409 Appendix A: Internal operating Experience - Condition Reports and Notifications Note: The items highlighted in yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description Component Order Notes PM: theý top and bottomi elements are held together by. 4.brackets of which V bracket'and bolt was missing. The craft people who last'did this PM said 1 of the bolts. twisted off but the. engineer documented this and told them to put it back together. This however caused the strainer elements to push out In this particular area causing the wipers or (portseals) to wear out In this area and also 06/04/02 EXAMINE STRAINER..
STNR-A 4194443 cause it not.t6 move freely. Since th~damhping bar Is. hot stocked as a spare part, a replacement was fzibricated by machinst.. Rockwell hardneiss and bum test Identified existing mid.:section clamping bars to be 316-SS material. Because CNS does not: currently hav6-316S55materia~ln-stock~ part evaluation 4224993 evaluates the'use of 304 SS~material In this application or at anytime adjacent damping bars-ire found missing, damaged, or In need of replacement.
Strainer isn't starting until DIP is 5.5 psi. Methd of Discovery: CR Alarm.
10173405 06/24/02 A-SW-ZURN STRAINER NOT DPS-362A 4250653 7/1/02: Calibrated DPS-362A & DPIS-363A per 14.28.1. No adjustments made.
OPERATING PROPERLY Before and after calibration 363A was Indicating -'3.5 - 4.0 PSI. It appears to be controlling sat. Monitored until strainer started which was -'4.3 psi.
SW Zumn Strainer B is plugging up, receiving frequent hi DP alarms, strainer is 10177824 07/12/02 SW ZURN STRAINER B PLUGGING STNR-B 4254138 usually backwashing, DP won't come down due to backwashing, continuous UP backwashing won't lower DP, strainer Is noisy, grumbling sound. Disassembled
____and cleaned 7/19/ 02.
B ZUN STAINE CAUION IGHTB Zumn strainer caution light Is cycling on when strainer rotates In the CCW 10178616 07/16/02 BYZURNGSRIE ATO IH STNR-B 4255145 direction. Appears strainer Is operating correctly. Condition cleared after cleaning CYCLING(Order 4254138).
10178847 07/16/02'A ROCK WAS FOUND IN A ZURN STNR-A 4255716 SIZE OF THIS ROCK WAS 1 1/3-BY 1-X 3/4"
____STRAINER___________________________________
12/22/2004 Page 54 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description Component Order Notes While inspecting SW A Zumn strainer basket a rock was found wedged between two ribs on the upstream side. The rock was -' 1 1/3" by 1" x 3/4". Other smaller rocks about 1/8" diameter were found wedged in many of the 1/8" openings. The smaller rocks wedged in the 1/8" openings is normal to find In the SW-STRAINER-A Zumn strainer was Zumn strainer. Since the strainer Is bypassed during maintenance, a question was 10179059 07/17/02 70% plugged on top and bottom of STNR-A raised as to the possibility of rocks getting into the SW strainer. RCR A/C 2002-strainer baskets.
1388 - A small amount of debris >3/8" from the river bypasses the traveling screens and enters the SW system. Some small amount of this size debris is expected based on the design of hte traveling screens. The amount of this sized debris does not appear to be Increasing nor significant. The traveling screens and SW Zumn Strainers are operating as designed and no actions are required.
107979071802B WZUN TRINR YPSSD STNR-B Method of Discovery: implementing clearance order and procedural steps 101757907/8/0 B W ZUN SRAIER YPASEDannotated within the description of condition.
10179595 07/19/02 B SW ZURN STRAINER SCREEN STRBOn 7/19/02, the B ZURN strainer element was noted to have some bent ribs In
_______DAMAGED SrRBit.
The strainer was still effectively filtering debris.
10184101 08/06/02 B SW ZURN STRAINER DELTA P HlAS STNR-B 4259714 SW Pump B & D Zurn Strainer Delta P has risen to -'1.5 psid. Trend item.
RISEN Received Annunciator A-4/D-7 SERVICE WATER STRAINER A HIGH DIP. SW 10188003 08/21/02 SW STRAINER A HIGH DP IN AND STNR-A F4266060) parameters normal. Alarm was in for less than 1 second. Operator reported no CLEAR abnormal conditions. Strainer D/P was 1 psld. There have been excessive rains Within the last week. RCR 2002-1842 Apparent Cause evaluation.
101506 0/2102SW ZURN STRAINER A MOMENTARY 5ThR-A 42-59391 With 4 SWPs running, secured C.Zurn Strainer high DP alarm momentarily.
HI DP LARMTrend item.
Received A SW Strainer High D/P Alarm when securing D SW Pump. Turbine 10197630 10/02/02 SW STRAINER HIGH DI P DPIS-363A 4259391 bldg SO reported nothing abnormal, D/P was 2# and strainer was not running.
Trend Item.
10204319 10/31/02 B ZURN STRAINER BYPASSED STNR-B 4259391 B SW Zumn Strainer has been bypassed for clearance order activity.
While operating SW ZURN Strainer B In continuous, the caution light lit when the 10204533 10/31/02 "B" ZURN STRAINER CAUTION STNR-B 4277436 strainer was rotating in the CCW direction. The light lit near the end of the CCW LIGHT travel and went out when the strainer reversed direction. Corrected under
______4288274.
12/22/2004 Page 55 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The Items highlighted In yellow are related to causal factors. Those highlighted in green are PMs.
Notif.
Date Description Component Order Notes 10205966 11/07/02 "A" SERVICE WATER ZURN STNR STNR-A 4259391 "A" Service Water Zumn Strainer Is bypassed for scheduled work activities. Trend BYPASSED item.
10208829 11/19/02 SERVICE WATER LOOP 'A ZURN STNR-A 4259391 Received momentary A! Zum, Strainer high DIP alarm. Trend item.
STRAINER D/P Station Operator reported Zumn A DP was 4.8 psld and strainer was not backwashing. DP rose to 5.2 psld, HIGH DIP alarm came In and cleared, and both Zumn Strainers commenced automatically backwashing. SW Header pressures lowered approximately 6 psig. After both strainers backwashed, SW 10210987 11/30102 A SW ZURN STRAINER DP ALARM STNR-A 4259714 Header pressure restored and Strainer DPs were 0.6 psld. Strainer DP Immediately after swapped SWPs was 0.4 and 0.6 psld. Checked E Bay for Ice, Is no Ice. SW temperature is 44 degrees. 3-3 Spargers looked OK. After cycled E Bay spargers found both Zumn strainers backwashlng again. Both Zumn strainers
_______completed backwashing and DPs 0.6 psld. Trend Item.
When starting SWBP D received SW Strainer A high D/P alarm In & dear.
Reported D/P AT 4.8. The strainer Is supposed to start backwashlng at 4 psid, which did not happen. This alarm routinely comes In on A side and not the B 102100012/1/0 SW STAINR HGH DP DIS-63A 4281448 side whenever shifting SW pumps or starting SWBPs. 12/6/02 - Completed cal 102100012/1/0 SW STAINR HGH DP DIS-63Aand flush per 14.28.1. Verified both Instruments working, properly, within calibration tolerances and repeatable. Strainer started and ran as expected once the DPS was tripped. All sensing lines were back flushed per procedure - sat.
Could not determnine cause of strainer not starting per design at this time.
Notifications 10210987 and 10211000 document instances on 11/30/02 and 12/1/02 when Zum Strainer A did not backwash when DP rose above 4.4 psid.
SW-DPS-362A Is to Initiate backwash when DP Is 3.6 to 4.4 psld. In both cases the backwash initiated at a higher pressure (5.2 and 5.0 psld). Since the DPs did 10212081 12/04/02 O.5.01'S ATTACH 3 (OD) NOT DPS362A 4281274 not function as expected (outside tolerance), the switch should be considered COMPLETED "Inoperable" (this switch Is a tech spec: support component). Procedure 0.5.OPS requires Attachment 3 be completed when there is a degraded component where functionality Is called Into question. Since the switch Is not functioning correctly, strainer DP could rise to a level where required flow of the service.
water system might not be obtained. RCR 2002-2510 12/22/2004 Page 56 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted In yellow are related to causal factors. Those highlighted in green are PMs.
Noi. Date Description Component Order Notes 123 1021/10/02 SW ZURN STRAINER A HIGH DP STNR-A 4259391 Received SW Zum Strainer HI DP Alarm In and clear. Trend item.
ALARM A Zurni strainer Is blowing down and backwashlng several times a day with only 102135312/3/0 A ZRN TRANERD/P S RSIN STR-A 4284431 one SW pump operating in that loop. With the same conditions on B Zurn 1021353141/02A ZRN TRAIER /P S RSIN STR-Astrainer, DIP is barely rising. Repairs will be performed via work orders 4278129,
______4290161, and 4290162. Cancel this order.
Due to low river levels, recurring trash rack plugging (B-Bay), and the Guide Wall Gate open, an extended underwater river bottom survey Is requested by 10215857 12/20/02 REUS O XEDDRVR CNS-CW 4285677 Engineering. Need to determine current river bottom coniditions to evaluate BOTTOM SURVEY potential impact to plant operations. Survey completed. Areas of concern were the main Inlet channel south of the intake, behind the guide wail and
______downstream of the plant along the south shoreline.
This is a recurring condition. In 2001, it was evaluated under RCR 2001-0841.,
Two supporting evaluations RCR 2002-0294 & RCR 2002-1619 were subsequently performed. The primary cause Is lover river water level which 10215899 12/20/02 EXCESSIVE CONDENSER PLUGGING CNS-CW CBOAT results In the entrainment of increased quantities of larger size sediment into the AGAIN Intake Structure. The design of the traveling water screens permits material to carryover. CNS can Improve the equipment filtering the river water to minimize the impact on the condenser. An organizational weakness exists In addressing ithis recurring issue.
SHEA PINBROK ON SW URNThe shear pin on the strainer was broken and D/P was 6 psld. Within 5 minutes 10216323 12/24/02 SHEAR INEBROENASWZR STNR-A 4285913 the D/P rose to 8.2 psid. Bypassed Zumn Strainer at 10 psid. Replaced shear pin STRAINERwithin 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />.
12/2242004 Page 57 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted in yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description Component Order Notes During manual strainer blow down per step 5.19 of procedure 6.2SW.101, 102194201/310 SWSTRINER13 OSSOF OWE STR-B 4287983 strainer motion stopped and Indicating lights extinguished. Operator reported 102194201/0/03SW-SRAIER-LOS OFPOWE STR-Bthe strainer started to blow down but stopped -u2 minutes after the control switch was placed to continuous.
SW Strainer A D/P alarmed & reset numerous times after SW-MO-37 was closed per step 5.8.5. SW header pressure was originally -~46pslg after MO-37 was closed. Then Strainer DIP started to cycle and header pressure dropped as low as 40#, DIP across strainer was spiking up to 7#, Strainer was placed in 10217943 01/03/03 SW-STRAINER-A DIP PROBLEM STNR-A 4719continuous blow down and D/P remained constant. SW-MO-37 was subsequently re-opened and SW flow through B REC HX was lowered. Backed up In procedure 6.2SW. 101 to step 5.8.4. Strainer was left in blow down and D/P lowered to a cyclic band of 2-3psid. Used PM order to clean strainer on 1/17/03.
Final confirmation signed on 7/31/03.
B Zumn Strainer was 5.4 psld, not backwash ing & failed to operate with control 102193201/D/03B ZUN SRAINR FILUR STR-B 4287983 switch in continuous. Caution light and running lights were lit. Bypassed per SOP 102193201/4/0 B URNSTRINE FALUR S~R-B2.2.71 due to combination of high d/p, and failure to backwash. Modified the control panel. RCR 2003-0022.
10217933 01/04/03 B ZURN STRAINER BYPASSED STNR-B 4288835 -Administrative requirement to track out of service time. RCR 2003-0022.
SW ZRN TRAIER CAUIONYellow caution light was Intermittentiy illuminating for a few seconds and then 10217998 01/05/03 SWGH ZURNHTRINERACATO STNR-A 4288213 going out with strainer backwashing and DP 5.5 to 7.5 psid. Condition was LIGH LIGTINGcorrected by cleaning the Zumn strainer under order 4302796.
With SWPs A & C running and B & D secured (which would be alignment for SW ZRN TRANER P WN'T6.2SW.101 testing, ILe. A & C supplying system load) SW ZURN A DP is running 10218000 01/05/03 SOWZUNSRIER DPLO WONTM STNR-A 4278129 6-7 psld constantly and won't lower, this is abnormal, SWP A & C should be able to supply system load with no high DP on SW A ZURN STRAINER, we do it all the time In the past when perform SP. 6.2SW. 101.
1281010/3POTENTIAL REV NEEDED FOR 4288923 The basis for a PM should not be overly conservative (restrictive) when reviewed 10181 010/3GUIDE WALL PM against the design basis requirements.
12/22/2004 Page 58 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted -in yellow are related to causal factors. Those highlighted In green are PMs.
Notif. IDate Description Comonen Order Notes B Zurn failed to operate correctly. System aligned for 'A' Zumn maintenance, with B & D pumps running with A & C pumps secured. Header pressure 50# on CR indication. DIP on B Zurn strainer was 5#, placed the strainer In continuous and the motor did not operate. Received high dip alarm (indicated 6.2 psid). 'B' Zurn SW B ZURN STRAINER FAIL TO was not rotating. Breaker on MCC TX, had no abnormalities. Started 'A' pump 10218359 01/06/03 TUNSTNR-B 4288273 and all alarms cleared. Secured VD pump and B Zurn d/p lowered to 1.2 psid.
11.JRNFollowing Indications on B Zurn strainer control panel noted: when In continuous, amber caution light on, when in intermittent amber caution light off.
The blue run light energized during the entire evolution. Found overloads actuated for the M2 contactor and an acrid odor was observed when B Zum
________control panel was open. RCR 2003-0035 & CM 4288274.
PM: Cleaned sand out of baskets with needle gun, cleaned all debris out, 01/06/03 EXAMINE-STRAINER STNR-A 4278129 reptaced wiper blades, replaced packing. Reassembled strainer, torqued cover, and set limit switches. Inspection port has not been installed. Need to verify movement of wiper blades prior to inspection port Installation.
over the past several days, several conditions have occurred with the service 10218608 01/07/03 Service Water System Operability 4259714 water system which potentially question it~s ability to provide the required I_____
functions. RCR 2003-0063.
SW-SNR-A& BCONFGURAIONPossible lack of configuration controls as well as managing modifications. Note 10218686 01/07/03 DIFFERNRA&BCENSURTO STNR-A 4289395 that a document may exist that authorized this change, but has not been located DIFFEENCESyet.
RCR 2003-0047.
128901013REPLACE ZURN STRAINER DPS DP-6A 4911Method of Discovery: Several Notifications have been written on the problem.
SWITCH_
Implemented CED 6011321.
1289010/3REPLACE ZURN STRAINER 'A DPIS DI36A 4912Method of Discovery: Several Notifications have been written on the problem.
102863 1/8/3SWII3TCH2916 Replace SW-DPIS-363A and ensure SIL 10 retrofit is Indluded via 14.5.5.
102076 0/1703IMPLEMENT CED 6011082 CONTROL SRA 4219Zum strainer control power is not presentiy monitored for loss of power, causing 1 020621011/0 CIRCUIT SW STRAINER 429112oprao9 to perform estra tours of the SWP room. Implemented CED 6011082 12/22/2004 Page 59 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The Items higlhlighted in yellow are related to causal factors. Those highlighted in green are PMs.
Notif.
Date Description Component Order Notes B SW strainer high dip alarm received. Strainer d/p was at 12#, strainer was not backwashing, red high d/p alarm was on, green power light was on, amber light was not on. Strainer d/p went to 14.5#. Placed strainer to continuous, but it did not rotate and d/p lowered to about 7#. After a couple of minutes amber light came on and blow down was open but strainer never moved. CM orders:
10221377 01/21/03 B SW ZURN STRAINER HIGH D/P STNR-B 4290980 4290955 (contingency action to manually operate strainer), 4290981, 4290982 &
4291236 for repair. RCR 2003-0158: Apparent cause was the coupling that Is secured to the motor shaft was Installed Improperly. The coupling was not lined up correctly or the coupling screws were not tightened properly. Strainer was repaired under 4290982. Other Zumn strainer coupling was Inspected to ensure it was assembled properly.
TREN PREIOUSSTRANERB Zurn strainer was partially bypassed for 7 minutes on 1/7/03, due to strainer 10221495 01/21/03 BYPEND OPERETIOUSTIN E
STNR-B 4259714 high d/p. SW-V-194 was partially opened to control B Zumn strainer d/p less BYPAS OPERTION han 6 psid. Trend Item.
10221508 01/22/03 STRAINER BYPASSED FOR PLANNED STNR-B 4259714 B Zum Strainer bypassed at 0432 on 1/22/03, for work order 4278130. Trend MAINTENANCE.itm 10221993 01/24/03 SW-SThR-A HIGH D/P ALARM STNR-A 4293732 Wie starting SW-P-D, received a SW-STNR-A high d/p alarm for approx. 0.1
____DURING PUMP STARTseodoprtrrprestanrdpa0.
i.DD1293 IMPLEMENT RONAN TIME DELAY Generate a CM to Implement time delay for Ronan points 3449 and 3469 10222805 01/28/03 PE E302STNR-A 4292383 according to EEO3-002. SCR 2002-1332. WO IS DUPLICATE TO WO 4289089.
PER EE3-002Completed 1/31/03.
Sand continues to build up in CW bays. CW-BAY-D and CW-BAY-B had been CONTINUOUS SANDING ISSUES IN totally desilted on or around 1/21/03. Less than one week later D2 screen 10224228 02/04/03 CW BAYS CNS-CW 4288837 needed to be run on high due to what maintenance believes to be sand against screen. Divers had to return on 2/4/03 to desilt bay. RCR 2003-0023 refers to an Intake Structure Improvement Plan.
Received Strainer A High D/P alarm In and dear. D/P was 0.5# and strainer was A ZUN SRAIER UEXP~rE HIbackwashing.
No manipulations of SW system were being performed at the 10225294 02/08/03 DPAA Z RNMTANRUNXE DH STNR-A 4259391 time. We received the alarm again while performing 6.SW.202 when the division D/P~~ ALRM1W pumps were supplying all service water loads and SW-MO-37 was closed.
rend item.
12422/2004 Page 60 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
Notif. IDate Description Component Order Notes SW-SNR-AHIGHD/P LARMFORReceived SW-STNR-A high dip alarm for 20 seconds. Operator reported it was 10225975 02/12/03 20SWECONDS IHDP LR O
STNR-A 4259391 backwashlng and d/p was 1.8 psid. Notification is being written for trending 20 SEONDSpurposes.
Trend Item.
Frequent A high DP alarms. Received 7 alarms from 0954 to 1122. Reset times 102-26775 02/ 15/03 FREQUENT DIV 1 SW STNR ALARMS STNR-A 4290161 for the alarm varied from several seconds to 3 minutes. Replaced DPS 362A on 1_________
5/13/03.
10228042 02/20/03 CREATE A CM ORDER TO REPLACE DPS-3628 4306529 Closing action #3 Of RCR 2002-2489. Replaced 5/13/03.
SW-DPS-362B Received A Zumn Strainer high DP alarm at 1356. Dispatched operator who verified the strainer motor was turning but the strainer was not. Strainer DP was 10228587 02/24/03 A SW ZURN STRAINER HIGH D/P STNR-A 4296814 reading 7.6 psid. Caution light was lit. Maintenance swept wiper a few times, knocked d/p from 9.5 to 1.5. Removed sheared off part from gear box and line up holes and installed new shear pin.
Delta P trend data for A Zum strainer Indicates this strainer should be cleaned 10237365 03/29/03 CLEAN A SW ZURN STRAINER STNR-A 4302796 early to mid May. This Is being written as a CM activity because the PM to change the strainer cleaning frequency to quarterly has not been approved yet.
Cleaned 5/13/03.
Delta P trend data for B Zurn strainer Indicates that this strainer should be 102336603/9/03CLEN BSW ZRN TRANER STNR-B 4302797 cleaned early to mid May. This is being written as a CM activity because the PM 102336603/9/03CLEN BSW ZRN TRANERto change the strainer cleaning frequency to quarterly has not been approved
_________yet.
Cleaned 5/14/03.
High DP WA Zumn strainer reported to control room. Initial report 5 psid, highest reported 7 psid with blow down In progress. DP rose about 2 psld In "45 min.
102386304/0/03A ZRN SRAIER D HIH SNR-A 430796Plant conditions: A & B SW headers split with SW-MO-37 closed for B SW IST 102386304/0/03A ZRN SRAIER O HIH SNR-A 430796SP, A&C pumps running, SWBP A running @ 3800 gpm, screen wash and sparger systems In service, A REC HX IN servilce with "'350 gpm flow, SW flow
______to TEC H)(S Isolated. Cleaned 5/13/03.
Zumn strainer A DP 4.2 psid. Ran Strainer in continuous for 3 hrs and OP only 10242506 04/19/03 A' SW ZURN STRAINER OP HIGH.
STNR-A 4302796 lowered to 3.5 psid, returned strainer to Intermittent. This was with A, C & D
______________________________________pumps running and SW load low due to plant startup._Cleaned_5/13/03.
12/22/2004 Page 61 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The Items highlighted in yellow are related to causal factors. Those highlighted In green are PMs.
Motif.
Date Description Component Order Notes A Zurn strainer has high d/p with 2 pumps running In A loop. Pumps were swapped last shift to 2 In A loop and 1 In B. High d/p alarmed numerous times between 0930 and 1030. D/P was -5 to 6 psld; strainer was placed in NUMEOUS SWZURNSTRANERcontinuous backwash and dip remained -5 psid. Swapped pumps back to 1 In A 10244648 04/30/03 NUMEROURS A WZR TANR STNR-A 4302796 loop and 2 in B. Both strainers were placed in continuous backwash and d/p on D/P AARMSboth eventually stabilized at -'1.5 psid white backwashing. Suspect river conditions may have contributed to this condition even though conditions were reported as normal, river Is typically dirtier this time of year. See notification 110226775 from 2/15/03 for a related condition. Cleaned 5/13/03. DD 10242606 When started RHR SWBP A for 6.1SWBP.101 received momentary Zurn strainer 10246050 05/07/03 SW ZURN STRAINER A HIGH DP STNR-A 4259391 high DP alarm, found Zurn A strainer DP at 4.0 psid and strainer not ALARM backwashing, subsequently strainer automatically backwashed and lowered DP
____to 2.5 psld. Trend Item.
Received multiple strainer A high d/p alarms during SWBP ERFOM runs. Strainer DP remained >6 psld during evolution. Condition needs resolved In the very near 10246524 05/10/03 SW-STNR-A HIGH D/P CONDITION STNR-A 4302796 future because with summer heat loads approaching rapidly, SW system demand will be significantly greater than that which Is required today to perform SWBP ERFOM runs. Cleaned 5/13/03.
10246839 05/13/03 A ZURN STRAINER BYPASSED STN-R-A 4259391 Trend Item.
10247087 05/13/03 B ZURN STRAINER BYPASSED STNR-B 4259391 Trend item.
PM: Disassembled Zumn strainer. Found sat. Did find gravel in holes of basket strainer. Cleaned and reassembled. Strainer element plugged -"50% with small*
08/05/03 EXMINE STRAINER STNR-B 4321325 pebbles and som e small sticks and the black rock material. Nothing -abnormal noted. Examined and reassembled strainer found everything sat. Did dean basket and had engineer check after dleaning.
PM: Disassembled SW-STNR-A. Found strainer to be sat. Cleaned baskets, and 0811103EXAINESTRINE STR-A 4321324 hd engineer inspect before start of reassemble. Also set strainer for complete 08/1/03 XAMNE SRAIER ShRAwipe.
Strainer element was -.60% plugged-with small pebbles. Some small
____sticks and "black rocks" were also found.
12/22/2004 Page 62 of 72
CR-CNS-2004-07409 AppCntinenc wor ordersa needtin toeiec be generated forrt andTR-for disassembly 1026685 08/6/0 CONINGmC WOhlghe FOR yWelloRw are-4328924 into andsa totos repoac thelghe shea pine ase neddospprhepsiblt A! ~ ~
~
~
~
~
o aUR STossE ofEE thiRcivd sA Z strainer duingthe pealrfoSrmaincer fof n the strotuptransfoutmertork 10266792 08/26/03__BROKE________4328851_was.
R eplaced shear pin bon en 5/19/0 &
taie 5/29/04
.Relaedsha SW-STNR-A has a higher than normal DP, and there are low river conditions.
Request the cover be removed, and the screen be inspected to verify the cause of the higher than normal DP. The strainer DP is not higher than normal; there Is resistance in the wiper motion at a specific location.
Clarification Comment: the resistance In wiper movement does not appear to be 102606008/7103INSECTSW-SNR-STR-A 4328815 excessive; however, the shear pin sheared twice in the last two days. Strainer 102606008/7/03INSECTSW-SNR-STR-Ashould be disassembled and inspected to ensure no abnormalities exist. As found inspections did not show any signs of where the binding was occurring. The wiper rotated smoothly and no Interferences.
Inspection showed the basket "-40% fouled with debris. Mostly small pieces of gravel lodged In the basket openings and some sticks and black rocks matted up
______against the basket In the recessed areas.
12/22/2004 Page 63 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The Items highlighted in yellow are related to causal factors. T'hose highlighted in green are PMs.
Notif.
Date Description Component Order Notes Received SW strainer A high DP. Operator reported the shear pin had broken, and DP was 8.4#. SW pressure dropped to 48#. Entered 5.2SW & secured sparger system, SW pressure restored to 54#. Maintenance replaced the pin, 102710 0/2703A SW ZURN STNR BROKEN SHEE.R SINR-A 4328851 operator restarted strainer and DP dropped to 1#. Restarted sparger system, PIN (.2SW)SW pressure Is 54#. Replaced pin within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />. RCR 2003-1617. Apparent cause of pin failures was a stick of other type of river debris caught In recessed area of the element creating a large force on the pin. Replaced shear pin under WO 4328851. Cleaned and Inspected strainer under 4328815.
B' Zum Strainer continuously backwashlng with D/P >4 psid. Cycles through a backwash, stops for -30 seconds, and backwashes; again. (5/30/04) Strainer basket heavily fouled with grass (-75%) and other fibrous material. Some pebbles were plugging the holes but percentage plugged by pebbles is very small. Strainer shear pin failed on 5/29/04 and DP reached -"15 psid. No apparent damage to basket internals from this. Returned to service and SW-BZUR STRINE HI /Pbackwashed and DP went down to 0.8 psld. Based on this low DP and the type 10269173 09/08/03 CO-BINURNUSTRIE BI/WP STNR-B3 4328928 of plugging of the basket decided to return the strainer to service asap. Heavy CONTNUOU B/Wrains caused Increased river loading and it is not recommended to leave the strainer bypassed for an extended period of time. Since the DP came down from 15 psid to 0.8 psid a day before, it is believed that DP will lower because the fibrous material Is easily backwashed. Opened Inspection hatch prior to removing top and the wiper arms were found damaged. No sticks or other obstructions.
Removed top and the rubber on both halves of the arm was pulled away from 1the metal backing (one '-50% loose rubber and the other -15%).
SW-STNR-A high DP alarmed momentarily. Strainer was backwashing and DP was 4 to 4.5 psid. When the backwash finished DP reached 4 psid In -1 minute and started the backwash cycle again. Currently there are two Division 1 SW 10272768 09/27/03 SW-STNR-A HIGH D/P CONDMTON STNR-A 4334782 pumps. Placed SW-STNR-A In continuous to lower the DP and prevent the backwash from cycling too frequently. Inspection on 9/28/03 showed strainer -
60-70% plugged with small pebbles, some sticks and some of the "black rocks".
Strainer condition was as expected as Indicated by the DP prior to cleaning.
________Nothing abnormal noted.
12/22/2004 Page 64 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted in yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description Component Order Notes Due to low river water levels, sediment is accumulating in significant amounts in 10276195 10/15/03 DREDGE IN FRONT OF INTAKE CNS-C 4341141 the channel between the Guide wall and the Intake structure. This condition will DURING 2004 MOO continue to degrade. This area needs to be dredged dear In preparation for
________summer 2004 operations.
PM: Disassembled found both baskets unsatisfactory. Engineer resolved discrepancies. Installed one new basket, cleaned other one, finstalled new scrapers, and reassembled. Engineering examined 'B' strainer as-found condition. Top basket fouled -0'0% with sand and other dlebris indluding sticks and rocks. The lower basket fouled "'80% with the same type of debris. A -'1/4 10/2103EXAMNE TRAIER,(REVFORInch hole was Identified on the top basket, and Will be replaced. A cracked weld 102403EXMENE STAIERL(EVFO STNR-B 4323350 was Identified on the lower basket. This crack would not allow material to pass through, however the lower basket Is being replaced as well. After further Inspection of the lower basket, it was found that It was not a cracked weld, but~a section of screen that had pulled away from the tab on the brace. These sections had not been welded prior to installation. The old basket will be re-used after tack welding to preclude further separation of the screen. Therefore; only the top screen element will be replaced under this pm work order.
I I
frmto raig HOLE IN 'B' ZURN STRAINER During the quarterly PM on the 'B' Zumn strainer (4323350), a hole -'1/4" In 10281416 11/13/03 BASKET STNR-B 4323350 diameter Was found In the top section of the strainer basket. Revised pm work order.
12/22/2004 Page 65 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description' Component Order Notes With 2 pumps running on'A' loop,'A' Zumn strainer DP continuously bounces at the alarm setpoint, with local reading between 5.5 & 6.5psid. WA header pressure was 52 pslg on PI-2715A. Requirement not met: Zurn strainer supporting 2 pump operation. Immediate actions taken: placed Zum strainer In continuous A' ZRN SRAIER HS HID/Pmode with no success, shifted pumps to a 3rd pump on 'B' side, placed 'A' Zum 10281468 11/13/03 A'T Z URNSRIEP HSH I
STNR-A 4323349 strainer back In Intermittent. With 1 pump operating 'A Zurn strainer -'1.5 psid.
WiTH PUMP12/16/03:
Strainer Inspected and found to be approximately 75% fouled with the typical types of debris normally found. Most of the blockage was caused by small rocks/ pebbles caught In the basket openings and several sticks and "black rocks" found In the recessed areas of the basket. The baskets are In average
_____condition with some rolling of the high points and are being replaced.
BATHMETIC URVE ARUNDDredging activities in RE21 removed rip-rap from around the Guide Wall and 10282515 11/20/03 BNATHYETISUVYAON CNS-CW 4346940 south shoreline of the Intake Structure, potentially adversely Impacting INTAKEsedimentation rate In the bays.
102612 1/1603SW-STNR-A BYPASSED FOR Sh-353 rn tm 102612 1/1603MAINTENANCE SN-353 rn tm Following maintenance to examine Zum strainer 'A', the inspection cover was LEAKNG ASKT ON"A!SW URNleaking a pencil lead sized stream. After maintenance retightened bolts the 10286354 12/16/03 STRANERAK N""
WZR STNR-A gasket wept. The gasket appeared to be damaged and needs replacement.
STRAINERRebypassed strainer due to unable to complete required post work testing.
Maintenance replaced strainer gasket.
PM: Inspected and assembled strainer. Inspection sat. Replaced baskets and port seals at engineering request. Strainer inspected and found to be approximately 75% fouled with the typical types of debris normally found. Most 12/16/03 EXAMINE STRAINER STNR-A 4323349 of the blockage was caused by small rocks & pebbles caught in the basket openings and several sticks and "black rocks" found In the recessed areas of the basket. The baskets are in average condition With some rolling of the high points
__________and a're being replaced with new.
B SW Zum strainer does not reverse direction while backwashlng. Actuating cam on the strainer drive shaft Is not contacting the reversing limit switch, which 10289072 01/06/04 B SW ZURN STNR DOESN'T STNR-B 4355164 appears to have been temporarily modified In the past due to failure of the limit REVERSE switch roller. The switch seems to have worn to the point that It no longer contacts the switch sufficiently to cause it to move. 1/7/04: replac~ed limit switch
____lever and adjusted to work properly 12/2212004 Page 66 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note:- The items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description Component Order Notes PM: Disassembled & cleaned for examination of strainer. Engineer Inspected prior to cleaning. Significant debris In the baskets ('-90% blockage). Majority of debris was larger sticks and other debris (1/4" to 3/4"l diameter). There was still gravel In the basket as well. Probably 50% of the strainer holes were plugged 01/21/04 EXAMINE STRAINER SThR-B 4324147 with gravel and the rest were blocked with the larger debris. Baskets looked to be In good shape and will not need to be replaced. Maintenance will clean the strainer and collect the debris. No other discrepancies Identified other than the blockage. Installed new wipers. Had bps rotate to make sure wipers covered screens.
On 2/4/03 received SW-STNR-A high dip alarm. Operator reported DP was 2
-STN-A IGHD/P LAR INpsid when backwashing and when backwash was Complete DP returned to 4 10294092 02/05/04 SW AHGHDPA~1I STNR-A psid. Ran strainer in continuous for '-4 hrs. DP lowered to 1.6 psid and steady.
AND CLEAR Recommendations: Contacted ~WWD to clean strainer next week. CAP screen: WI only. Work order screening: trend.
PM: Examine & reassembled. The strainer elements were full of gravel. Engineer Inspected. prior to dleaning. Found elements were '-95% plugged with small 02111/04 EXAMINE STRAINER STNR-A 4342773 gravel (large sand). There weren't too many pieces of large debris In the strainer (only '-5 pieces). This may be due to the fact that the strainer was in continuoul backwash prior to the disassembly. Pictures taken of the as-found Inspection and
_____will be kept by the system engineer.
During design/document review for Guide Wall and Intake computer modeling to 10297252 02/24/04 SILT PLATES INSTALLED A2, B1 AND 4370692 support Traveling Screen Project, determined there are 3 silt plates Installed In B2 BAYSfront of A2, B1 and B2 screens. The plates are single silt plates with legs welded to the bottoms so they stay '-10-12 ft. above the bay floor.
AS OUN FO S-DPS-33B UTAs founds for SW-DPIS-363B were out of calibration tolerance during O
031/4OF CAL TOL DPIS-363B 4328300. Adjusted switch In tolerance. CAP screening: WI trend. Work Order
_________screening:
trend.
12/22/2004 Page 67 of 72
CR-CNS-2004-07409 Appendix-A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted in yellow are related to causal factors. Those highlighted in green are PMs.
Notif. IDate Description Component Order Notes PM: Disassembled Zumn B and found In good condition. Cleaned baskets and had 04/14/04 EXAMINE STRAI NER STNR-B 4334895 engineer examine before and after cleaning. Strainer was found approximately 85-90% plugged with small pebbles, the "black rocks" and sticks. Not much
______grass/algae noted. Strainer condition comparable to previous Inspections.
SW-DIS-63AFOUN OU OFWhile performing routine PM 4335251, SW-DPIS-363A was found out of 10308993 04/19/04 SWPS33 ON U
F DPIS-363A 4375836 calibration tolerance with no limit given. Adjusted & notified CRS, and I
TOLERANCE supervisor. Trend Item.
103183105/1/0 SWZUR STRINE B HEA PI STR-B 4377548 B Zum strainer found not rotatlng. Station operator reported shear pin broke, 103183105/1/0 SWZUR STRINE B HEA PI STR-Bcurrent d/p Is 4#. New pin installed within 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> and strainer rotates freely.
B Zurn strainer shear pin broke. Method of discovery: CR alarm for strainer high 10315819 05/19/04 SW-SThR-B SHEAR PIN BROKE STNR-B 4328924 d/p, operator reported d/p was 5.6 psld and shear pin broken. 5/19/04: replaced shear pin rotated freeiy with nut wrench. 5/29/04: replaced pin verified movement manually then installed pin. RCR 2004-0402.
PM: Examined and reassembled. Condition of wipers, packing and all moving parts In good condition and were reused per engineering direction;. Engineering did as found. Inspection of baskets prior to cleaning. Cleaned debris from baskets 05/19/04 EXAMINE STRAINER STNR-A 4343101 and saved evidence. Watched strainer make full sweep. Strainer wasfound to be approximately 80-90% plugged with the usual amount of small pebbles lodged In the basket openings and some black rocks and some small~sticks and
______grass/algae.I 10315975 05/20/04 A' SW ZURN STRANER BYPASSED, STNR-A 4375836 Zumn strainer'A' Is bypassed for planned maintenance activities to clean the
__________PLANNED
_____strainer and repair the wiring, which will close an open 01D. Trend item.
12/22/2004 Page 68 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The Items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
1Notw. I Date I Description I componentj Iorder I Notes 05/29104 10317883 "B" ZIRN STRAINER SHEAR PIN BROKE STNR-B 4328924 B Zum shear pin broke. Strainer DP was 15# and rising due to high system load.
Shear pin broke on 5/ 19/04 also. Work order 4328924 prepared previously as a contingency. Replaced pin within 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. The wiper turned normally & shows no Internal damage evident. The highest DP seen was 15.2 psid. Per 2.2.71 the operability limit is 15 psid. Per the system engineer, 15 psid is also the design pressure, based on the operating characteristics flow model of the strainer during DBA conditions. 15 psid was conservatively adopted as the design limiit because it was used in the flow model. Maintenance noted "play" between the pin and the slot, which may create higher shear force on the pin due to the motor turning prior to engaging the pin. These higher shear forces may have caused recent failures. RCR 2004-0402 12/22/2004 Page 69 or 72 12/22/2004 Page 69 of 72
CR-CNS-2004-07409 D.endix A: Internal Operating Experience - Condition Reports and Notifications
-The items highlighted In yellow are related to causal factors. Those highlighted in green are PMS.
-r I
T r
I Descaiptlon Componentl order Notes PM: Examined and reassembled strainer. Cleaned baskets and replaced packing.
System engineer examined strainer prior to dleaning. Top basket of the 'B' strainer was -.98% plugged with mud. It appeared that this mud was over a layer of cottonwood seed or something. The bottom basket was about 60%
EXAMINE STRAINER STNR-B 4337281 plugged with small pebbles. There was no large debris on either basket. There did not appear to be any damage to either basket so they will Just be cleaned In place and not replaced. An as-left examination will also be done. The wipers were also examined and were In good condition. The rubber wipers will not need
____to be replaced at this time.
Maintenance Plan resulting from PM Optimization under the Equipment Reliability Project. The basis Is In the PM Basis document for Filters, Screens, Ion Exchangers, and Strainers, line Item 11.1. JUSTIFICATION: Strainers are to be inspected and cleaned where necessary. These strainers were determined to have a criticality of 1 during the FEG and CCD process. A criticality 1 item should normally have a preventive maintenance task. These strainers are not routinely INMTATE NEW MAINTENANCE PLAN STNR-A monitored, and therefore should have a periodic inspection and dleaning PM.
Frequency has been Initially set at 1R. This frequency Is based on the strainers needing an Initial Inspection and cleaning. Provision to be made In the PM for the strainers to be Inspected by the system engineer for evaluation of condition for adjustment of frequency of the PM. After Initial examination a determination of a new frequency should be made based on the results of the Inspections.
IFREQUENCY: 1 Refueling PM: Disassembled cleaned and Inspected SW-STNR-ZURN-A and retrieved the debris for engineering, then reassembled. The strairier baskets were approximately 50% fouled. This Is relatively clean considering the time in service and previous trends. There were a few larger sticks sightly' larger than 3/8";
EXAMNE SRAIER SNR-434189however, the size and amount were expected & acceptable. No 'holes or tears in the straining elements. Wiper assembly was in'acceptable-condltion.*The rubber was not gouged and was tightly adhered to the metal backing strips. Packing was well lubricated and not deteriorated. The straining elements and wiper assembly are acceptable for reuse following cleaning. Requested the'shear pin
______be replaced.
12/22/2004 Page 70 of 72 12/22/2004 Page 70 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The Items highlighted In yellow are related to causal factors. Those highlighted in green are PMs.
Notif.
Date Description Component Order Notes During the as-found inspection of 'B' Zurn strainer, the rubber on one of the wiper blades was found pulled away from the metal backing strip due to a stick that had wedged between the rubber wiper and the strainer basket. The stick and portion of the rubber that had been peeled away from the backing strip were wedged In the gap between the two wiper blade assemblies. The wiper blade was not defective based on the following: the wiper blades were 10343252 09/27/04 B'SW ZURN STRAINER WIPER STNR-B Inspected prior to installation to ensure the rubber was adequate adhered to the DAMAGED metal backing strip. No deficiencies were noted. Once the damaged wiper assembly was removed, attempts were made to remove the remainder of the rubber wiper from the metal backing strip. These attempts were unsuccessful.
Also, the ribs of the strainer basket were slightly bent at approximately the same area as the damaged rubber on the wiper, These ribs were only bent and no additional holes In the straining element were observed. Installed new wipers
____under WO 4345190.
PM: Removed shaft and oscillator from strainer. Found wipers pulled away from metal. Baskets full of gravel & sand. Every thing else was sat. Reassembled 09/2/04 XAMIE 91AINR STR-B 4345190 strainer using new wipers. Cleaned strainer elements & had engineer look when 09/2/04 XAMIE SRAINR STR-Bcleaning was done. Had ops rc tate* wipers to make sure limit switches operate properly. Strainer was '80"-5% fouled by smaller sand/gravel particles typical
______of the last few inspections. Some smaller sticks were present.
SW Zum Strainer is bypassed for Cleaning and Inspection of Strainer Internals.
103415610/6/0 SW URNSTRINE 'AýBYPSSE STR-ACondition Report is being written for tracking purposes per 2.2.71 Section 13 103415610/6/0 SW URNSTRINE "A"BYPSSE STR-AStep 13.1.3. CAP review: WI trend. Work Order Screening meeting: trend to PED.
PM: Disassembled cleaned, examined & reassembled. Engineer performed *as found inspection. Strainer was In good condition and only needs to be cleaned.
10/0/04 XAMNE SRAIER SNR-436.98No parts need to be replaced as the wipers and baskets were all in good 10/0/04 XAMNE SRAIER ShR-436598condition. The basket elements were about 80% plugged with small gravel and there were some larger sticks caught in the strainer folds. All materials appeared normal for an as-found Inspection.
12/22/2004 Page 71 of 72
CR-CNS-2004-07409 Appendix A: Internal Operating Experience - Condition Reports and Notifications Note: The items highlighted In yellow are related to causal factors. Those highlighted In green are PMs.
Notif.
Date Description Component Order INotes During start of D SW pump, B Zum strainer high DP alarmed and cleared.
SW ZUN STAINERB DPREQUIREMENT NOT MET, Zumn strainer DP Is trending higher, alarm should not SW472 ZURN/ STRENIN AINHER.BD STNR-B come In. METHOD OF DISCOVERY: starting 4th pump. SW Zumn strainer DP was 1.6 psid and lowering in backwash. CAP review: WI trend - PED. Work Order Screening meeting: PED for trending.
PM: The strainer was found to be "'80 % plugged with small pieces of *torpedo.
gravel". This gravel Is slightly Irregular In shape and wedges itself Into the 1/8" 11/20/04 EXAMINE STRA.INER STNR-A 4399413 openings of the strainer basket. This type of debris Is expected and normal In the strainer basket. There were several of the small "black rocks" and there were several sticks in the basket. There We~re also no indications of any structural
_____damage to the strainer basket or the strainer bas ket supports.
PM: Cleaned and Inspected strainer. Strainer was In good shape just a lot of sand. Cleaned sand reassembled. The strainer was found to be "'90 % plugged with small pieces of "torpedo 'gravel". Gravel Is slightiy Irregular in shape and wedges into the openings (1/8") of the strainer basket. This type of debris is 11/20/04 EXAMINE STRAINER STNR-B 4368226 expected and normal in the strainer basket. There were a couple of the small black rocks and there was a pile of approximately 1" of sand and "torpedo gravel" in the bottom of the'basket. The structural Integrity of the strainer basket and strainer basket support was also Inspected and no damage was noticed.
PM: Performed the examination and dleaned the SW-Sm R-B per 7.2.30 and 11/24/04 EXAMINE STRAINER STNR-B STNR-B 4411390 reassembled per same. Inspected 'B3' SW Zurn strainer. The strainer was very dean. It was approximately 5-10% fouled. This fouling was not tightly adhered and was easily removable.
12/22/2004 Page 72 of 72