ML20217N377

Insp Rept 50-277/98-03 on 980212-0303.Violations Noted.Major Areas Inspected:Reviewed Events,Significance,Reportability & CAs for 2 a RFP Turbine High Level Trip Events Involving 3 1997 Failures & 1 Trip on 980206

ML20217N377
Person / Time
Site:	Peach Bottom
Issue date:	03/30/1998
From:	NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I)
To:
Shared Package
ML20217N342	List: ML20217N336 ML20217N365 ML20217N377
References
50-277-98-03, 50-277-98-3, NUDOCS 9804090089
Download: ML20217N377 (13)

See also: IR 05000277/1998003

Text

, - -

..

.

U.S. NUCLEAR REGULATORY COMMISSION

REGION l

Docket Nos.

50-277

License Nos.

DPR-44

Report No.

98-03

Licensee:

PECO Energy Company

Facility: .

' Peach Bottom Atomic Power Station Unit 2

Dates:

February 12 through March 3,1998

Inspectors:

' A. McMurtray, Sensor Resident inspector

M. - Buckley, Resident inspector

.s

Y

'

y

, , -

.-)

,

'

9904090089 990330

PDR 'ADOCK 05000277

G

PDR.

'

..

r-

.(,

,

,

_

i

.

. , -

\\

TABLE OF CONTENTS

EX EC UTIVE S UMM A RY , ; . . . . . . . . . . . . . . . . . . . . . . . , , , , , , , , , , , , , , , , , , , , , ji .

o

111. Eregineering - . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , , , , , , , , , , , , , , , , , , , , , 3

'

E2 Engineering Support of Facilities and Equipment

.......................1

.

E8. , Miscellaneous Engineering issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

E8.1

Unresolved item No. '50-277/97-07 04:2 'A' RFPT Trip Mechanism (Closed)

....................................................7

E8.2 ~ Unit 2 Reactor Scram Resulting from a Generator Lockout Condition &

Subsequent Turbine Trip Failure of the 2 'A' Reactor Feedpump Turbine to

Trip on Demand during Recovery _ Activities. LER 50-277/97-009(Closed) and

,

Failure of the 2 'A' Reactor Feedpump Turbine to Trip LER 50-277/97-010

(Closed)

..............................................7

V. ' M anage ment Meetings ; . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

X 1 Exit Meeting Sum m ary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

.

X2 Review of Updated Final Safety Analysis Report (UFSAR) Commitments . . . . . . . . . 8

1

.

'

H

.,-

t

. <

'

s

t

,I"'"

'

)

t.

.

..:

,

.

..

EXECUTIVE SUMMARY

Peach Bottom Atomic Power Station, Unit 2

NRC Inspection Report 50-277/98-03

This special inspection was conducted to review events, significance, reportability, and

corrective actions for the 2 fA' Reactor Feedwater Pump Turbine high level trip events

. involving three 1997 failures and one slow trip on February 6,1998.

~

Corrective actions from previous events failed to prevent noncompliance with Technical

- Specification 3.3.2.2 requirements to maintain operability of the Digital Feedwater Control

System high water level trip capability for the 2.'A' Reactor Feedwater pump. This -

condition existed for an indeterminate period for up to 9 months. This noncompliance

resulted in an apparent violation of technical specification 3.3.2.2. Of particular concern

-was the dec sion not to perform test ng o tf his trip function at power following the second

i

i

. event in November 1997.

The failure to submit a Licensee Event Report (LER) for the noncompliance with Technical

- Specifications due to the 2'A' Reactor Feedwater Pump Turbine high water level trip

function event on April 1,1997, was considered a violation of 10 CFR 50.73 -

requirements.

The delay to change the status of the 'A' feedwater train to (a) 1 until February 1998, by

the Maintenance Rule Expert Panel was not proactive given the technical specification

- significance of these events. However, the change in status met the maintenance rule

procedural and regulatory requirements.

-

__

ii

,

,,

1

'

s

9

q

, .

,

m<

k

Report Details

i

111. . Engineering

E2'

' Engineering Support of Facilities and Equipment

)

- E2.1

Inonerable Reactor Feedwater Pumo Hiah Water Level Turbine Trio

a.

Inspection Scone (37551. 62707. & 92701)

The insl.ectors reviewed events, significance, reportsbility, and corrective actions -

for the ? 'A' Reactor Feedwater Pump Turbino (RFPH Ngh level trip events

involving three 1997 failures and one slow trip on aw my 6,'1998. The

c

feedwate r system performance was also reviewed for compliance with the

Mainten 4nce Rule criteria.

b.

Observ stions and Findinas

The Eeactor Feedwater system for each unit consists of three 50% flow capacity

RFP'/s. The Digital Feedwater Control High Water Level Trip System (DFCS) and

mr.n turbine high water level trip instrumentation is designed to detect a potential

folure of the Feed Level Control System that causes excessive feedwater flow.

'With excessive feedwater flow, the water level in the reactor vessel rises toward .

the high water level set point, causing the trip of the three feedwater pump turbines

and the main turbine. A trip of the feedwater pump turbines limits further increase -

in reactor vessel water level by limiting further addition of feedwater to the reactor

vessel. This prevents moisture carryover into the main steam lines. A trip of the

main turbine and closure of the stop valves protects the turbine from damage due to

water entering the turbine. As described in the Peach Bottom UFSAR, the reactor

high water level condition would trip the main turbine causing a reactor scram thus

avoiding unacceptable safety results and ensure that the Minimum Critical Power

Ratio (MCPR) safety limit is not exceeded. MCPR limits are imposed to avoid fuel

damage due to severe overheating of the fuel cladding.

During the normal shutdown and testing evolutions in 1997 and early 1998 several

failures were noted during testing of the 2'A' Reactor Feedwater Pump Turbine high

water level trip function. A description of these events is provided below and in

. inspection report 97-07.

Aoril 1.1997: During a response to an EHC problem, operators reduced power to

40% and attempted to trip the 2 'A' RFPT from the control room. The RFPT failed

to_ trip. An attempt to_ trip locally using the manual trip also failed to trip the RFPT.

The RFPT finally tripped after some mechanical agitation of the trip dump valve. In

_

October 1996, the licensee had successfully functionally tested all three unit 2

-

RFPTs using digital computer trip inputs signals.

' The licensee's troubleshooting found debris in the oil with a sludge consistency in L

and around the trip ' ump valve. The manufacturer verified that this condition could

d

cause a failure to trip and was consistent with failures described by other industry

_

_.

1

.

..

2

]

operating experience. - Maintenance removed the sludge, reassembled the trip

mechanism, and verified the RFPT would trip both from the control room and :

locally. The Ikensee chose to bring the 2'A' RFPT back into service with only static

s

testing as pmof of trip capability.' Static testing provided only. verification of the.

l

control oil system trip and no steam was admitted to the turbine. During this

testing, operations personnel performed two trips with the push-button in the

control room and two local trips at the turbine. All trips occurred within one second .

as described in section 14 of the UFSAR.

November 9,1997: Following a reactor scram, the Unit 2 reactor operator

attempted to trip the 2'A' RFPT from the control room. The RFPT failed to trip and

a subsequent local attempt failed. Again, light physical force directed to the dump -

-

valve resulted in a trip.

During troubleshooting, maintenance personnel disassembled the trip mechanism

and identified intermittent binding of the manual trip rod spring cup against the trip

device housing due to deformation of the spring cup. A slight misalignment was

also identified between the trip lever, trip rod and reset piston above the trip dump

housing. This misalignment was scheduled to be corrected during the next refueling

outage. The manual spring cup, spring, and trip rod were replaced and the trip

mechanism was reassembled and reinstalled.

The inspectors observed the technicians in the shop duplicating the failure of the

trip mechanism to dump oil. The mechanism would hang up so that the trip dump.

valve would not move. The inspectors observed that the spring cup was deformed

and the spring rod was scored, gouged, and out of round. The inspectors also

observed that these parts had damage that was dull in appearance in some

locations indicating that the damage had occurred over an extended period of time.

Other locations were brighter indicating that the damage was ongoing. The

inspectors determined that these deficiencies could have caused the RFPT failure to

trip when required and that this condition had existed for some time.

n

Following the November 9 event, the licensee returned the 2'A' RFPT back to

service with static testing consisting of three trips with the push-button in the

control room and three local trips. All trips occurred within one second as described

. in the UFSAR. No subsequent testing during operation occurred that would have

. assured the cause had been identified and corrected.

The inspectors noted similar symptoms for both the April and November events.

Following this event, the inspectors questioned the lack of at-power verification -

testing and opened Unresolved item (URI) 50-277/97-07-04.

December 29I1997: During a normal shutdown evolution for EHC system repairs,

the Unit 2 reactor operator. attempted to trip the 2 'A' RFPT from the control room.

~

.

' The RFPT failed to trip at this time. Subsequently, the 2 'A' RFPT tripped during a

transient that occurred when all the turbine bypass valves inadvertently opened.

' The licensee initiated troubleshooting and repair activities for both the electrical and

mechanical components to the trip function.

[

,-

._,

'*b

.

,

3

' Following the third trip failure, maintenance and engineering personnel rigorously

investigated possible root causes, both mechanical and electrical. The main focus

of these investigations was on the electrical portion of the trip mechanism, since

the turbine had mechanically tripped during the December 29 transient.

Initial resistance checks during troubleshooting activities on the trip solenoid did not

reveal any unusual readings. Further troubleshooting identified an intermittent short

by vibrating the trip solenoid. The licensee's investigations indicated that the '

probable cause of failure to trip on December 29 was the result of an intermittent

short in the trip solenoid coil.

An analysis by Valley Forge laboratories confirmed an intermittent short existed

when the trip dump valve solenoid was vibrated. Laboratory testing and analysis

determined that the short likely was caused by long term vibration. Laboratory.

- results revealed there was no change in pull force even with the short present. A

contributing factor to the intermittent short was that the trip coil mounting -

assembly / bracket was missing two hold down bolts. Maintenance replaced the trip

dump valve trip solenoid coil.

Other out-of-specification or anomalous conditions found during the inspections

ws,re repaired or replaced. Maintenance and engineering personnel found and

resolved the following conditions that could have contributed to the trip failure:

Trip dump piston scored on lands.

Many holes drilled in trip valve piston in piston travel set area.

Trip lever out of alignment.

Trip dump valve anti-rotation pin galled.

SV-12 trip coil plug out of alignment, and undersized.

SV-12 trip coil indicated intermittent shorts under vibration and missing 2

hold down bolts.

Lockout valve binding in the bore and land scoring.

1.ength of service for springs on the trip dump, trip valve, and trip rod.

• '

Control Room manual trip push-button reliability verification.

The licensee performed a visual verification on the trip coils for the Unit 3 RFPTs

and tested the coils for the other Unit 2 RFPTs. This testing and verification

indicated no shorts or similar mounting conditions.

Upon completion of repairs, the post-maintenance testing included 2 trips with tho'

control rcom push-button and 2 trips locally in the static condition. All trips

occurred within one second as described in the UFSAR. The licensee decided after

. an engineering review to initiate a periodic testing program. Testing included

tripping the turbine wh% running norma?ly but isolated from the reactor vessel and

. recirculating to the condenser.1his test was performed just before restoring 2'A'

RFPT from standby to service, durieg a c own power approximately ten days later,

and every subsequent thirty days. The ',icensee previously chose not to perform at-

" -

power testing as recommended in Technical Information Letter (TIL) 83-4 " Lockout =

' Suppressed Overspeed Trip Test Frequency-Utility Turbines."

.

,

.

..

4

The inspector noted that periodically at power testing with the turbine in operation

,

provided better verification that the repairs resolved the problem. This addressed

' the concerns identified in URI 50-277/97-04regarding testing of the 2'A' RFPT

during power operations to ensure that all failure mechanisms from the previous

events had been identified and corrected.

February 6.1998: During the performance of an at-power test, the Unit 2 reactor

operator attempted to trip 2'A' RFPT from the control room. The operator had to

hold the push button for three to five seconds, while the expected trip should have

occurred immediately ( l.a., less than one second).

- After the slow response of the RFPT trip system, the licensee instrumented the

control oil system and performed testing both from the control room and locally.

The slow response could not be duplicated during the testing. Engineering change

request (ECR) 98-00329 and 10 CFR 50.59 evaluation indicated that no thermal

limits would be exceeded for a feedwater pump trip delay of less than ten seconds

,

provided the other RFPTs and the main turbino trip within one second. Further, the

.

10 CFR 50.59 evaluation concluded that there were no safety impact and no

unreviewed safety question for this trip delay. The inspectors reviewed the NCR

p

- and the 10 CFR 50.59 and identified no concerns with the analysis presented.

The licensee restored the 2'A' RFPT back to service on February 11 and performed

a satisfactory at power test on February 21,1998. The licenses planned to perform

an at-power trip test of the 2'A' RFPT around March 21,1998.

Following the February 1998 slow trip, the inspectors reviewed the unit 2 'A'

feedwater train Maintenance Rule classification. During the monthly Maintenance

Rule Expert Panel (MREP) meeting for February 1998, the system manager

presented a corrective action plan for the Unit 2 'A' feedwater train with the

recommendation to change the status to an (all system. The expert panel agreed

'

with this recommendation and accepted the action plan which included inspection

of the 2 'B' and 'C' RFPT durir.g the Fall 1998 outage. Although a MREP meeting

" had been held in January 1998, the 'A' feedwater train remained an (a)2 system.

The performance criteria that would initiate consideration of the feedwater trains

into (all status were:

more than one maintenance preventable function failure (MPFF)

%

unavailability greater than 2%.

'Although the unavailability of the 'A' feedwater train had been between 5% and

6% since January 1996, the train was not changed to (all status because of

previous modification and operational activities. The inspectors determined that this

met the licensee's administrative requirements for the Maintenance Rule. However,

the inspectors questioned the timeliness of the status change of the 2'A' feedwater

train to (a)1. After review of the MREP activities with respect to the 2'A' RFPT

events in April, November and December, the inspectors had no overall concerns.

,.

,

1

.

-

J

L __

v

.

3-

5

1

i

'

At the end of this inspection the licensee was still reviewing this issue and was

performing additional analysis. The licensee had still not determined the root cause

and was re-evaluating the operability determinations for the prior failures to trip.

The inspectors noted that, based on the UFSAR section 14.5.2.2 and figure 14.5.5,

all RFPT trips are expected within one second of a trip demand. Following the three

to five second delay of the 2'A' RFPT tripping on February 6,1998, the licensee -

evaluated the impact of a 10 second trip delay.

Throughout these events, the main turbine trip function and the trip functions on

the two other reactor feed pumps remained operable. The licensee's analysis

showed that the fuel thermal and MCPR limits were protected. However, moisture

carryover into the main steam piping would increase with the feedwater puinp trip

delay time. The licensee identified that even for the cases where the main steam-

_

pipe filled with water; the design criteria for the piping was maintained and the

safety relief valves remained operable. Also, the reactor trip and reactor high level

procedures require operator action to limit and prevent increasing reactor water

level. The inspectors reviewed the documentation for the trip failures and the

safety evaluation, ECR, and supporting analysis from the February 6, slow trip and

independently concluded that these events were of limited safety significance.

However, these events constitute a departure from the regulatory requirements.

Peach Bottom Atomic Power Station, Unit 2, Technical Specification 3.3.2.2,

"Feedwater and Main Turbine High Water Level Trip instrumentation," requires that

the Digital Feedwater Control System high water level trip capability be maintained

operable with reactor power 225%. If the Digital Feedwater Control System high

water trip capability is inoperable, reactor power shall be reduced to < 25% within

'

six hours. The basis information for technical specification 3.3.2.2 describes the

trip of the RFPTs as part of the logic functional testing for the Digital Feedwater

Control System.

.

Based on the following, the inspectors determined that the requirement to maintain

the RFPT trip function or restore the function within two hours had not been met for

an indeterminate period for up to nine months:

4*-

the failures to trip on April 1, November 9, and December 29,1997.

the long term mechanical binding indications observed by the inspectors.

k

,o

lack. of preservation of original parts for future evaluation.

• -

.;.

• I

the detailed analysis required for the trip' failures after February 6,1998,

for operability justification.

-

2

,

3

'

l?

,

s

-] $ l

a.

$

.:

..

H

6

Therefore, from April to December the 2 'A' RFPT high water trip function was not

maintained operable. The failure to maintain the Digital Feedwater Control System

high' water level trip capability operable for the 2 'A' reactor feedwater pump when -

-

reactor power was equal to or greater than 25% is an apparent violation of TS 3.3.2.2. (eel 50-277/98-03-01)

During the review'of these events, the inspectors noted that no Licensee Event

Report (LER) had been submitted for the failure to trip of the 2'A' RFPT on April 1,

.

1997.10 CFR 50.73 requires that an LER be submitted within 30 days for any

operation or condition prohibited by Technical Specifications. ' NUREG-1022, Rev.1,

" Event Reporting Guidelines 10 CFR 50.72 and 50.73" discussed the need for firm

evidence that the condition existed prior to the time period prohibited by technical

specifications. However, the inspectors determined based on the suspected cause

of the failure and history of failure that the condition existed for some time.' After

discussion with the system manager and manufacture representative, the inspectors

"

determined that the cause of the failure to trip on April 1 had existed for longer than

the two hours limit specified in section 3.3.2.2 of Technical Specifications, prior to

the attempt by the operator to trip the 2 'A' RFPT. Therefore, not reporting the

April 1,1997 failure to trip of the 2 'A' RFPT within 30 days is contrary.to '.0 CFR

50.73. (VIO 50-277/98-03-02)

The inspectors noted that LERs were submitted for the failures that occurred on

November 9 and December 29,1997. The LER for the event on November 9 also

discussed the April event and the corrective actions for both events. Based on the

fact that a full discussion of this issue was presented in this LER, no additional

submittal of an LER for the April event is required.

c.

Conclusions

This special inspection was conducted to review events, significance, and corrective

actions for the 2 'A'. Reactor Feedwater Pump Turbine high level trip events

'

involving three 1997 failures and one slow trip on February 6,1998.

Corrective actions from previous events failed to prevent noncompliance with

technical specification requirements to maintain operability of the Digital Feedwater

Control System high water level trip capability for the 2 'A' Reactor Feedwater

pump. This noncompliance from April to December 1997 resulted in an apparent

violation of technical specification 3.3.2.2. Of particular concern was the decision

not to perform testing of this trip function at-power following the second event in

November 1997. _ The failure to submit a Licensee Event Report (LER) for the -

noncompliance with Technical Specifications due to the 2'A' Reactor Feedwater

Pump Turbine high water level trip function event on April 1,1997, was considered

~ a violation of 10 CFR 50.73 requirements. The delay to change the status of the

"A" feedwater train to (a)1 until February 1998, by the Maintenance Rule Expert

Panel was slow and not proactive to the problem.

s'

y

[

,

.,

7

E8

Miscellaneous Engineering lasues -

' E8.1

Unresolved item No. ' 50-277/97-07-04:2 'A' RFPT Trio Mechanism (Closed)

This unresolved item questioned the decision by th' licensee, following the

e

November 9,1997 failure to trip,'not to perform at-power testing of the 2 'A' RFPT

trip mechanism to verify that the cause of the failure was identified and corrected.

.The safety significance of this event was also unresolved.

As discussed above in Section E2.1, the inspectors determined that the licensee'can

perform at-power testing to identify further failures or deficiencies. The February 6,

1998 slow trip of the' RFPT was identified due to at-power testing initiated after the

December 29,1997, failure to trip. The inspectors also documented the safety

significance of these events in Section E2.1 Licensee' action on this issue will be

tracked by the apparent violation No. eel 50-277/98-03-01.

l E8.2 Unit 2 Reactor Scram Resultina from a Generator Lockout Condition & Subseauent

Turbine Trio Failure of the 2 'A' Reactor Feodoumo Turbine to Trio on Demand

durino Recovery Activities. LER 50-277/97-OO9(Closed) and Failure of the 2 'A'

Reactor Feodoumn Turbine to Trio LER 50-277/97-010(Closedl

The inspectors reviewed PECO's rcot cause and corrective actions for the reactor

scram' documented in LER 50-277/97-009. This scram occurred when an

equipment operator failed to follow procedure while swapping a .125 volt DC system

battery charger. This event was documented in inspection Report 50-277(278)/97-

' 07 and resulted in a violation for failure to properly implement procedures. This

issue will be tracked under VIO 50-277(278)/97-07-02.

The inspectors reviewed LERs 50-277/97-009and 50-277/97-010for

completeness and corrective actions taken in response to the 2'A' reactor feed

,

"

pump turbine high water level trip failure. Further licensee action on this issue will

be tracked via apparent violation No. eel 50-277/98-03-01.

V. Management Meetings

~X1-

Exit Meeting Summary

The inspectors presented the inspection results to members of licensee management at the

conclusion of tha inspection on March 4,1998. The licensee acknowledged the findings

presented, but disagreed with the inspectors conclusions regarding previous operability of

the feedwater trip function.

The inspectors asked the licensee whether any materials examined during the inspection

Lshould be considered proprietary. No proprietary information was identified.

.

i

- - - _ -- -

sprA

e

,

.

8'

X2

Review of Updated Final Safety Analysis Report (UFSAR) Corrnntments

- A discovery of a licensee operatire their facility in a manner contrary to the Updated Final

'

Safety Analysis Report (UFSAR) description highlighted the need for a special focused

review that compares plant practices, procedures and/or parameters to the UFSAR

descriptions. While performing the inspections discussed in this report, the inspectors

.' reviewed the applicable portions of the UFSAR that related to the areas inspected. The

inspectors verified that the UFSAR wording was consistent with the' observed plant

practices, procedures and/or parameters.

-,

i

,

b

.

-

e

i

'

.

, *

9

LIST OF ACRONYMS USED

digital feedwater control high water level trip system (DFCS)

'

electro-hydraulic' control (EHC)

,

escabAed enforcement item (EEI)

licensee event report (LER)

maintenance rule expert panel (MREP)

minimum critical power ration (MCPR)

' Peco Energy (PECO)

public document room (PDR) _

reactor feed pump turbine (RFPT)

safety relief valve (SRV)

surveillance requirement (SR)

technical information letter (T!LL)

' t techr ical specification (TS)

unresolved item (URI)

. updated final safety analysis report (UFSAR)

,

i

-

i

i

.

i

. .

4. . ; '.

10

INSPECTION PROCEDURES USED

IP 37551:

Onsite Engineering Observations

IP 62707:

Maintenance Observation

IP 92700:

Onsite Follow-up of Written Reports of Nonroutine Events at power Reactor

Facilities

ITEMS OPENED, CLOSED, AND DISCUSSED

Ooened

50-277/98-03-01

eel

2 'A' RFPT High Water Level Trip Failure per TS 3.3.2.2

50-277/98-03-02

VIO

Failure to Submit LER for TS Non-compliance

Closed

50-277/97-07-04

URI

'A' RFPT Trip Mechanism

.50-277/97-009

LER

Reactor Scram and 2 'A' RFPT Trip Failure

50-277/97-010

LER

2 'A' RFPT Trip Failure

<

+

n-

jf /

i

_.