ML080600164

IR 05000445-07-008, on 12/4/07-1/24/08, Comanche Peak, Final Significance Determination for a White Finding and Notice of Violation, Special Inspection

ML080600164
Person / Time
Site:	Comanche Peak
Issue date:	02/29/2008
From:	Collins E Region 4 Administrator
To:	Blevins M Luminant Generation Co
References
EA-08-028 IR-07-008
Download: ML080600164 (42)

See also: IR 05000445/2007008

Text

February 29, 2008

EA-08-028

Mike Blevins, Senior Vice President

and Chief Nuclear Officer

Luminant Generation Company, LLC

ATTN: Regulatory Affairs

Comanche Peak Steam Electric Station

P.O. Box 1002

Glen Rose, TX 76043

SUBJECT:

FINAL SIGNIFICANCE DETERMINATION FOR A WHITE FINDING AND

NOTICE OF VIOLATION - COMANCHE PEAK STEAM ELECTRIC STATION -

NRC SPECIAL INSPECTION REPORT 05000445/2007008

Dear Mr. Blevins:

On January 24, 2008, the U.S. Nuclear Regulatory Commission (NRC) completed its reviews

related to a Special Inspection at your Comanche Peak Steam Electric Station, Unit 1, facility.

This Special Inspection Team was chartered to review the circumstances related to the failure

of Emergency Diesel Generator (EDG) 1-02 to start on November 21, 2007, and to evaluate the

actions taken in response to the problem. The NRC's initial evaluation satisfied the criteria in

NRC Management Directive 8.3, NRC Incident Investigation Program, for conducting a special

inspection. The possibility that adverse generic implications were associated with the EDG

failure mechanism was the deterministic criterion met. Additionally, the result of the NRCs

initial conditional risk assessment associated with this degraded condition indicated that a

special inspection was warranted. The determination that the inspection would be conducted

was made by the NRC on November 30, 2007, and the inspection started on December 4,

2007.

The inspection examined activities conducted under your license as they relate to safety and

compliance with the Commissions rules and regulations and with the conditions of your license.

The inspectors reviewed selected procedures and records, observed activities, and interviewed

personnel.

The enclosed inspection report documents the inspection results, which were discussed on

December 7, 2007, and again on January 10, 2008, with Mr. R. Flores and Mr. T. Hope,

respectively, and other members of your staff. On January 24, 2008, an exit meeting was held

with Mr. F. Madden, Director, Regulatory Affairs, and other members of your staff to convey the

Luminant Generating Company, LLC

-2-

final disposition of the inspection findings. Following a discussion of the preliminary safety

significance of this finding during the exit briefing, Mr. Madden indicated that Luminant Power

does not contest the characterization of the risk significance of this finding, and that you have

declined to further discuss this issue at a Regulatory Conference or provide a written response.

Accordingly, the NRC is issuing the final significance determination for the inspection finding as

discussed below. On February 25, 2008, an additional exit meeting was held with Mr. T. Hope,

and other members of your staff to convey a revision to one of the inspection findings.

This report documents one finding concerning a failure to satisfy Technical Specification (TS)

Limiting Condition for Operation (LCO) 3.8.1 due to EDG 1-02 being in an inoperable condition

following maintenance. Following the discovery of this condition, the TS required actions were

satisfied however, the time period between the occurrence of the condition and the discovery of

the condition exceeded the TS allowed outage time for the EDG. This finding has been

determined to be of low to moderate safety significance (White). This finding does not

represent an immediate safety concern because of the corrective actions you have taken.

These actions included restoring EDG 1-02 to an operable status, ensuring that all other EDGs

were not in a similar degraded condition, and curtailing painting activities pending the

implementation of suitable measures to prevent the recurrence of a similar condition.

You have 30 calendar days from the date of this letter to appeal the NRCs determination of

significance for the identified White finding. Such appeals will be considered to have merit only

if they meet the criteria given in NRC Inspection Manual Chapter 0609, Attachment 2. In

accordance with the NRC Enforcement Policy, the Notice of Violation is considered an

escalated enforcement action because it is associated with a White finding.

You are required to respond to this letter and should follow the instructions specified in the

enclosed Notice when preparing your response.

In addition, we will use the NRC Action Matrix to determine the most appropriate NRC response

to this issue, and we will notify you by separate correspondence of that determination.

The report also documents one NRC-identified finding of very low safety significance (Green).

This finding was determined to involve a violation of NRC requirements. However, because of

the very low safety significance and because it is entered into your corrective action program,

the NRC is treating the finding as a noncited violation (NCV) consistent with Section VI.A.1 of

the NRC Enforcement Policy. If you contest this NCV, you should provide a response within

30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear

Regulatory Commission, ATTN.: Document Control Desk, Washington, DC 20555-0001; with

copies to the Regional Administrator, U.S. Nuclear Regulatory Commission, Region IV, 611

Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the Director, Office of Enforcement,

U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident

Inspector at the Comanche Peak Steam Electric Station.

In accordance with 10 CFR 2.390 of the NRC's Rules of Practice, a copy of this letter, its

enclosure, and your response (if any) will be available electronically for public inspection in the

Luminant Generating Company, LLC

-3-

NRC Public Document Room or from the Publicly Available Records (PARS) component of

NRC's document system (ADAMS). ADAMS is accessible from the NRC Web site at

http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room). To the

extent possible, your response should not include any personal privacy, proprietary, or

safeguards information so that it can be made available to the public without redaction.

Sincerely,

/RA/

Elmo E. Collins

Regional Administrator

Dockets:

50-445

Licenses: NPF-87

Enclosures:

1. Notice of Violation

2. NRC Inspection Report 05000445/2007008

w/Attachments

Attachment 1: Supplemental Information

Attachment 2: Special Inspection Charter

Attachment 3: Significance Determination Evaluation

cc w/enclosures:

Fred W. Madden, Director

Regulatory Affairs

Luminant Generation Company LLC

P.O. Box 1002

Glen Rose, TX 76043

Timothy P. Matthews, Esq.

Morgan Lewis

1111 Pennsylvania Avenue, NW

Washington, DC 20004

Anthony Jones, Chief Boiler Inspector

Texas Department of Licensing

and Regulation

Boiler Program

P.O. Box 12157

Austin, TX 78711

Somervell County Judge

P.O. Box 851

Glen Rose, TX 76043

Luminant Generating Company, LLC

-4-

Richard A. Ratliff, Chief

Bureau of Radiation Control

Texas Department of Health

1100 West 49th Street

Austin, TX 78756-3189

Environmental and Natural

Resources Policy Director

Office of the Governor

P.O. Box 12428

Austin, TX 78711-3189

Brian Almon

Public Utility Commission

William B. Travis Building

P.O. Box 13326

Austin, TX 78711-3326

Susan M. Jablonski

Office of Permitting, Remediation

and Registration

Texas Commission on

Environmental Quality

MC-122

P.O. Box 13087

Austin, TX 78711-3087

Environmental and Natural

Resources Policy Director

Office of the Governor

P.O. Box 12428

Austin, TX 78711-3189

Lisa R. Hammond, Chief

Technological Hazards Branch

National Preparedness Division

FEMA Region VI

800 N. Loop 288

Denton, TX 76209

Luminant Generating Company, LLC

-5-

Electronic distribution:

ROPreports

RIDSSECYMAILCENTER

RIDSOCAMAILCENTER

RIDSEDOMAILCENTER

RIDSOEMAILCENTER

RIDSOGCMAILCENTER

RIDSNRROD

RIDSNRRADIP

RIDSOPAMAIL

RIDSOIMAILCENTER

RIDSOIGMAILCENTER

RIDSOCFOMAILCENTER

RIDSRGN1MAILCENTER

RIDSRGN2MAILCENTER

RIDSRGN3MAILCENTER

RIDSNRRDIPMIIPB

OEWEB

ECollins

AHowell

Fuller - KSF

C Maier

Vasquez - GMV

D Furst, NSIR

Vegel - AXV

Chamberlain - DDC

N Hilton, OE

Caniano - RJC1

Powers - DAP

June Cai, OE

ACampbell - ACC

CJohnson - CEJ1

John Wray, OE

DLoveless - DPL

DAllen - DBA

ASanchez - AAS1

Herrera - MSH3

SSanner - ESS

Starkey, OE - DRS

Mary Ann Ashley, NRR

Paulk -CJP

M Burrell, OE

Dricks - VLD

WMaier - WAM

R Barnes, OE

DPowers - DAP

DPelton - DLP1

SUNSI Review Completed: CEJ ADAMS: / Yes G No Initials: CEJ

/ Publicly Available G Non-Publicly Available G Sensitive

/ Non-Sensitive

R:\\_REACTORS\\_CPSES\\2007\\CP2007-08 CHY.wpd

ADAMS ML080600164

RIV:RI:DRP/E

SRI:DRP/E

SRA

C:DRP/A

SES/ACES

CHYoung:vlh;mjs AASanchez

DPLoveless

CEJohnson

GMVasquez

E-CEJ

E-CEJ

/RA/

/RA/

/RA/

1/31/08

2/08/08

2/08/08

2/12/08

2/11/08

D:DRS

D:DRP

C:ACES

DRA

RA

RJCaniano

DDChamberlain

KSFuller

ATHowell

EECollins

/RA/

/RA/

/RA/

/RA/

/RA/

2/11/08

2/21/08

2/14/08

2/22/08

2/28/08

OFFICIAL RECORD COPY

T=Telephone E=E-mail F=Fax

Enclosure 1

NOTICE OF VIOLATION

Luminant Generation Company, LLC

Docket No. 50-445

Comanche Peak Steam Electric Station

License No. NPF-87

EA-08-028

During an NRC inspection completed on January 24, 2008, a violation of NRC requirements

was identified. In accordance with the NRC Enforcement Policy, the violation is listed below:

Unit 1 Technical Specification (TS) 3.8.1, AC Sources - Operating, requires that while

the plant is in Modes 1, 2, 3, or 4, two diesel generators (DGs) capable of supplying the

onsite Class 1E power distribution subsystem(s) shall be operable. For the condition of

one DG being inoperable, the required action is to restore the DG to an operable status

within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> and within 6 days from the discovery of the failure to meet the Limiting

Condition for Operation (LCO), or be in Mode 3 within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and Mode 5 within 36

hours.

Contrary to the above, from November 1, 2007, through November 21, 2007, while the

plant was in Mode 1, one of the two DGs capable of supplying the onsite Class 1E

power distribution subsystem(s) was inoperable, and action was not taken to either

restore the DG to an operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or be in Mode 3 within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and

Mode 5 within 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br />. Specifically, Emergency Diesel Generator (EDG) 1-02 was

made inoperable as a result of painting activities due to paint having been deposited and

remaining on at least one fuel rack in a location that prevented motion required to

support the operation of the EDG. This condition caused EDG 1-02 to fail to start during

a surveillance test on November 21, 2007.

This violation is associated with a White significance determination process finding.

Pursuant to the provisions of 10 CFR 2.201, Luminant Generation Company, LLC is hereby

required to submit a written statement or explanation to the U.S. Nuclear Regulatory

Commission, ATTN.: Document Control Desk, Washington, DC 20555-0001 with a copy to the

Regional Administrator, Region IV, and a copy to the NRC Resident Inspector at the facility that

is the subject of this Notice, within 30 days of the date of the letter transmitting this Notice of

Violation (Notice). This reply should be clearly marked as a Reply to a Notice of Violation;

EA-08-028, and should include for each violation: (1) the reason for the violation, or, if

contested, the basis for disputing the violation or severity level; (2) the corrective steps that

have been taken and the results achieved; (3) the corrective steps that will be taken to avoid

further violations and (4) the date when full compliance will be achieved. Your response may

reference or include previous docketed correspondence, if the correspondence adequately

addresses the required response. If an adequate reply is not received within the time specified

in this Notice, an order or a Demand for Information may be issued as to why the license should

Enclosure 1

-2-

not be modified, suspended, or revoked, or why such other action as may be proper should not

be taken. Where good cause is shown, consideration will be given to extending the response

time.

If you contest this enforcement action, you should also provide a copy of your response, with

the basis for your denial, to the Director, Office of Enforcement, United States Nuclear

Regulatory Commission, Washington, DC 20555-0001.

Because your response will be made available electronically for public inspection in the NRC

Public Document Room or from the NRCs document system (ADAMS), accessible from the

NRC Web site at http://www.nrc.gov/reading-rm/adams.html, to the extent possible, it should

not include any personal privacy, proprietary, or safeguards information so that it can be made

available to the public without redaction. If personal privacy or proprietary information is

necessary to provide an acceptable response, then please provide a bracketed copy of your

response that identifies the information that should be protected and a redacted copy of your

response that deletes such information. If you request withholding of such material, you must

specifically identify the portions of your response that you seek to have withheld and provide in

detail the bases for your claim of withholding (e.g., explain why the disclosure of information will

create an unwarranted invasion of personal privacy or provide the information required by

10 CFR 2.390(b) to support a request for withholding confidential commercial or financial

information). If safeguards information is necessary to provide an acceptable response, please

provide the level of protection described in 10 CFR 73.21.

Dated this 29th day of February 2008.

U.S. NUCLEAR REGULATORY COMMISSION

REGION IV

Dockets:

50-445

Licenses:

NPF-87

Report:

05000445/2007008

Licensee:

Luminant Generation Company, LLC

Facility:

Comanche Peak Steam Electric Station, Unit 1

Location:

FM-56, Glen Rose, Texas

Dates:

December 4, 2007, through January 24, 2008

Team Leader:

C. Young, P.E., Resident Inspector, Arkansas Nuclear One

Inspectors:

A. Sanchez, Resident Inspector, Comanche Peak Steam Electric Station

D. Loveless, Senior Reactor Analyst

Branch Chief:

C. Johnson, Chief, Project Branch A

Division of Reactor Projects

Approved By:

D. Chamberlain, Director

Division of Reactor Projects

Enclosure 2

-2-

SUMMARY OF FINDINGS

IR 05000445/2007008; 12/04/07 - 01/24/08; Comanche Peak Steam Electric Station (CPSES),

Unit 1; Special Inspection in response to the failure of the Train B Emergency Diesel Generator

to start on demand on November 21, 2007.

The report covered a 6-day period (December 4-7, 2007) of onsite inspection, with inoffice

review through January 24, 2008, by a special inspection team consisting of two resident

inspectors and one senior reactor analyst. Two findings were identified, including one Green

noncited violation, and one White violation. The significance of most findings is indicated by its

color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, Significance

Determination Process. Findings for which the significance determination process does not

apply may be Green or be assigned a severity level after NRC management review. The

NRC's program for overseeing the safe operation of commercial nuclear power reactors is

described in NUREG-1649, Reactor Oversight Process, Revision 4, dated July 2000.

A.

NRC-Identified and Self-Revealing Findings

Cornerstone: Mitigating Systems

White. A violation of Unit 1 Technical Specification 3.8.1, AC Sources -

Operating, was identified for the licensees failure to satisfy Limiting Condition

for Operation 3.8.1 in that painting activities conducted on the Unit 1 Train B

EDG 1-02 resulted in paint being deposited and left in a location that caused the

EDG to become inoperable. As a result, EDG 1-02 failed to start on demand

during the subsequent monthly surveillance test. Following the discovery of the

condition, the required actions were satisfied; however, the time period between

the occurrence of the condition and the discovery of the condition exceeded the

allowed outage time. This issue was entered into the licensees corrective action

program as SMF-2007-03253.

The finding was greater than minor because it was associated with the human

performance attribute of the mitigating systems cornerstone, and it affected the

cornerstone objective to ensure the availability, reliability, and capability of

systems that respond to initiating events to prevent undesirable consequences.

The Phase 1 Worksheets in Manual Chapter 0609, Significance Determination

Process, were used to conclude that a Phase 2 analysis was required because

the performance deficiency affected the emergency power supply system that is

a support system for both mitigating and containment barrier systems. Based on

the results of the Phase 2 analysis, the finding was determined to have low to

moderate safety significance (White). The senior reactor analyst determined

that a more detailed Phase 3 analysis was needed to fully assess the safety

significance. Based on the results of the Phase 3 analysis, the finding was

determined to have low to moderate safety significance (White). The Phase 1,

2, and 3 Significance Determination Process analyses associated with this

finding, including assumptions and limiting core damage sequences, is included

as Attachment 3 to this report. The cause of this finding was determined to have

a crosscutting aspect in the area of human performance associated with work

Enclosure 2

-3-

practices in that the licensee failed to provide adequate supervisory and

management oversight of work activities, including contractors, such that nuclear

safety is supported H.4(c). Specifically, the actions planned and taken to

assess and control the operational impact of the painting activities on the

functionality of the emergency diesel generator were not reflective of adequate

supervisory and management oversight of the activities (Section 2.1).

Green. The inspectors identified a noncited violation of Unit 1 Technical Specification 5.4.1.a, Procedures, for an inadequate alarm response

procedure. The inspectors determined that Procedure ALM-1302A, Diesel

Generator 1-02 Panel, Revision 5, was inadequate in that it was ambiguous and

did not cause the responders to verify that the fuel racks were free as part of the

response actions to investigate the cause of the unit failing to start.

Consequently, the licensee failed to identify that the Unit 1 Train B Emergency

Diesel Generator 1-02 fuel racks were not free to move, which led to an

extended period of inoperability and a significant delay in diagnosing the cause

of the emergency diesel generator failure to start. This issue was entered into

the licensees corrective action program as SMF-2007-03426.

The finding was determined to be more than minor because it was associated

with the procedure quality attribute of the mitigating systems cornerstone, and it

affected the cornerstone objective to ensure the availability, reliability, and

capability of systems that respond to initiating events to prevent undesirable

consequences. Using Manual Chapter 0609, Significance Determination

Process, Phase 1 Worksheet, the finding was determined to have very low

safety significance (Green) because it was not a design or qualification

deficiency, did not represent a loss of safety function, did not represent an actual

loss of a single train for greater than its Technical Specification allowed outage

time, did not represent a loss of a non-Technical Specification Train of

equipment for greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, and did not screen as potentially risk

significant due to a seismic, flooding, or severe weather initiating event

(Section 2.2).

B.

Licensee-Identified Violations

None.

Enclosure 2

-4-

REPORT DETAILS

1.0

SPECIAL INSPECTION SCOPE

The NRC conducted a special inspection at Comanche Peak Steam Electric Station to

better understand the circumstances surrounding the failure of the Unit 1 Train B

Emergency Diesel Generator (EDG) 1-02 to start on demand during a monthly

surveillance test on November 21, 2007. Following approximately 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> of

troubleshooting, EDG 1-02 was restored to an operable status. In accordance with NRC

Management Directive 8.3, it was determined that the period of inoperability of the EDG,

both prior to and during the failure to start event, had sufficient risk significance to

warrant a special inspection. The initial incremental conditional core damage probability

associated with the assumed period of EDG inoperability was estimated to be

1.76 x 10-5. The possibility that adverse generic implications were associated with the

EDG failure mechanism was the deterministic criterion met to warrant a special

inspection.

The team conducted the inspection in accordance with Inspection Procedure 93812,

Special Inspection, and the inspection charter, which is included in this report as

Attachment 2. The special inspection team reviewed procedures, corrective action

documents, operator logs, and maintenance records for the EDG system. The team

interviewed various licensee personnel regarding the events that led up to and response

actions that followed the EDG failure, as well as design and operational characteristics

of the EDG and its support systems. The team reviewed the licensees root cause

analysis report, past failure records, extent of condition evaluation, immediate and long

term corrective actions, and industry operating experience. A list of specific documents

reviewed is provided in Attachment 1.

1.1

Event Summary

On November 21, 2007, at 10:20 a.m., EDG 1-02 failed to start on demand during a

monthly slow start surveillance test. Prior to this, the last successful surveillance test on

EDG 1-02 was on October 24, 2007. The licensees response to the failure to start is

described in Section 1.2 below. Troubleshooting efforts were ultimately successful, and

EDG 1-02 was restored to an operable status at 3:08 a.m. on November 22, 2007. The

failure was determined to be the result of fuel racks being stuck in the closed positions,

and not responding to a full open governor demand, thereby preventing sufficient fuel

from reaching the engine. The failure mechanism is described in Section 1.4 below.

The cause of the fuel rack binding was ultimately determined to be a drop of paint on a

fuel rack which prevented the rack from being able to move through the fuel pump

housing. The root and contributing causes of this failure are discussed on Section 1.3

below.

Prior to the failed surveillance test on November 21, 2007, painting was conducted on

and around EDG 1-02 and EDG 2-02 (Unit 2 Train B EDG). The painting activities

associated with EDG 1-02 began on October 15, 2007, and continued through

November 8, 2007.

Enclosure 2

-5-

Included below is a timeline that includes significant elements pertaining to this event.

Date/Time

Event

October 15, 2007

Painting begins on EDG 1-02 on top of engine and around

heads.

October 15, 2007 -

November 1, 2007

Painting activities continue on a daily basis.

October 24, 2007

Successful monthly slow start surveillance test of EDG 1-02.

No painting is done on this day.

October 29, 2007 -

November 1, 2007

Painting occurs around the 6L fuel pump.

November 1, 2007

Painting in locations that could have reasonably resulted in

stray paint/drops on fuel rack(s) is completed.

November 21, 2007

3:28 a.m.

EDG 1-02 declared inoperable due to bar over in preparation

for monthly surveillance test.

4:09 a.m.

Bar over completed. Successful water roll via the air start

system was completed. EDG 1-02 declared operable.

10:17 a.m.

EDG 1-02 declared inoperable for monthly surveillance test.

10:20 a.m.

EDG 1-02 failed to start on demand for monthly slow start

surveillance test. Operations personnel believed the EDG did

not roll. Troubleshooting commences.

4:49 p.m.

Slow start attempt of EDG 1-02 resulted in EDG rolling up to

90-100 rpm and failing to start. Troubleshooting continues.

6:02 p.m.

Fast start attempt of EDG 1-02 resulted in a failure to start.

Fuel racks were observed not to move in response to a

governor demand.

7:39 p.m.

Fuel racks were manually stroked. Rack 6L was found to be

stuck. Rack 2L moved approximately half of its travel range,

then became bound. Both racks were freed and stroked until

normal free range of motion was restored.

9:25 p.m.

Walkdown inspections revealed residual paint on 6L, 4L, and

4R fuel racks. Residue of paint on 6L was wiped away.

9:32 p.m.

Successful start and run of EDG 1-02.

November 22, 2007

3:08 a.m.

EDG 1-02 was declared operable.

Enclosure 2

-6-

1.2

Licensee Response to the Failure of the EDG to Start

The inspectors evaluated the licensees implementation of procedures (abnormal, alarm,

troubleshooting, and normal operations) and Technical Specifications, reviewed plant

managements control and decision making actions, and reviewed the troubleshooting

and investigating activities that occurred following the Unit 1 Train B EDG failure to start

during the monthly surveillance test on November 21, 2007. The inspectors reviewed

corrective action documents, procedures, Technical Specifications, and operations logs.

The team performed system walkdowns and interviewed engineering, maintenance, and

operations personnel.

The inspectors determined that, in general, the licensee responded to the event properly

and in accordance with plant procedures. Nuclear equipment operators (NEO) quickly

identified that the EDG 1-02 failed to start and immediately responded to the local EDG

alarm panel. The operations field support supervisor performed an inspection to look for

any obvious problems that could have caused the EDG to fail. NEOs noted that it did

not sound like a normal start, and assumed that a possible issue associated with the

starting air system had something to do with the failure to start. The licensee had

already declared the EDG inoperable prior to the attempted start in conjunction with the

surveillance test.

In response to the Unit Failure To Start alarm on the local EDG alarm panel, NEOs

performed the steps of the local alarm panel procedure, Alarm Procedure

Manual ALM-1302A, Diesel Generator 1-02 Panel, Revision 5, which instructed the

operations personnel to investigate the cause of the failure. This included checking for

proper operation and issues associated with the fuel racks, day tank, and the starting air

system. One of the applicable steps was to check fuel racks free. This was

accomplished in accordance with the expectations of senior operations personnel by

visually verifying that there were no apparent conditions that would obstruct the motion

of the fuel racks. No abnormalities were identified at this time.

The operations staff reviewed drawings and diagrams, interviewed the NEOs, and

consulted with meter and relay representatives, system engineering department

personnel, and the mechanical services department to develop a troubleshooting plan.

Due in large part to the testimony of the NEOs that the EDG did not even roll in

response to the start attempt, the troubleshooting plan focused on the starting air

system as the suspected cause of the failed start. The plan called for meter and relay

personnel to monitor various solenoids and relays during a subsequent slow start

attempt of the EDG. This attempt resulted in the EDG rolling up to 90-100 rpm, and

again failing to start. Indications now suggested that a fuel-related problem must exist,

and focus was shifted accordingly. A third attempt was performed with the EDG in a

fast start configuration. Again, the EDG failed to start. Observers noted that the fuel

racks did not move from their closed positions in response to the mechanical governors

attempt to drive the fuel racks to the full open position. The licensee then attempted to

exercise the fuel racks and metering rods individually and discovered that two metering

rods (2L and 6L) were partially and fully bound, respectively. Licensee personnel

physically exercised the metering rods until they were free to move, and removed

evidence of paint that was found to be on the 6L metering rod by the fuel pump housing

interface. The licensee then performed a fourth attempt to start EDG 1-02, which was

Enclosure 2

-7-

successful. The EDG was fully loaded, and operations personnel completed the

surveillance testing. The EDG 1-02 was subsequently declared operable on the

morning of November 22, 2007 at 3:08 a.m.

The inspectors determined that the initial troubleshooting plan was too narrowly focused

on finding an EDG starting air problem (despite a successful water roll via the air start

system that occurred earlier that morning), as opposed to pursuing all likely causes of a

failed start. If the focus of the response were broader, it is likely that the stuck metering

rod would have been discovered earlier, and the duration of EDG inoperability following

the failed start would have been reduced.

Subsequently during the troubleshooting efforts, the joint engineering team developed a

confirm and refute matrix to process the results from troubleshooting. Possible causes

that were analyzed over the course of troubleshooting included:

Starting air receiver discharge valves mispositioned

Manual stop button mispositioned

Tachometers operational

Malfunctioning of air start solenoid valves

Mechanical governor bound

Fuel supply to the engine

Main control board handswitch

Electronic governor not operating

Fuel racks not functioning*

• Determined to be the cause of the failure

As described in Section 1.3 below, the Unit 2 Train B EDG was also in the process of

being painted. Once the cause of EDG 1-02 inoperability was determined to be stuck

metering fuel rods, the operations staff inspected the Unit 2 Train B EDG and

determined that the same issue did not exist. Operations also inspected the Units 1

and 2 Train A EDGs and determined that the stuck metering rods issue did not exist.

The Unit 1 Train B EDG was the only EDG affected.

1.3

Root Cause and Corrective Action Assessment

.1

Root Cause Analysis

The inspectors reviewed and assessed the licensees root cause analysis for technique,

accuracy, thoroughness, and corrective actions proposed and taken. The inspectors

reviewed the scope and processes used by licensee personnel to identify the root cause

for the failure of the Unit 1 Train B EDG to start during a monthly surveillance test. The

inspectors compared information gained through inspection to the event information and

assumptions made in the root cause reports. The inspectors interviewed licensee

personnel, reviewed logs, reviewed personal statements, and observed root cause team

meetings. The inspectors evaluated the licensees extent of condition review and

common cause evaluation.

Enclosure 2

-8-

The licensee captured the EDG 1-02 failure to start problem in the corrective action

program as SMF-2007-03253, and performed a root cause analysis in response to

determine the cause of the failure. Evaluation techniques utilized by the licensee

included an Events and Causal Factors Chart and a Barrier Analysis. The result of

these efforts identified the most probable root cause of the failure to be a drop of paint

that was deposited and adhered to the 6L fuel rack in a location that prevented the rack

(along with all other fuel racks) from moving in the open direction in response to the

governor demand associated with an EDG start signal. This failure mechanism is

further discussed in Section 1.4 below. Although there was no documented evidence of

the actual paint drop, there was paint residue observed which remained in the subject

location following the manual manipulation and freeing of the stuck fuel rack during

troubleshooting. This residue was wiped off upon discovery.

Additionally, the following four contributing causes to the failure were identified in the

final root cause analysis:

Work practices of painters and other groups who performed daily inspections

failed to identify paint spatter and drops that should have been cleaned off

sensitive engine components.

The tools and techniques used by painters were not completely effective in

preventing paint spatter and drips.

• Because the directions in alarm response procedure ALM-1302A were not

specific, the time period following the failure until the discovery of the cause of

the problem was extended.

The fuel control shaft break away force may have increased over time due to

wear and aging effects. This may have added to the force required to overcome

the adhesion of the paint drop.

• This issue was also identified early in the inspection process by the inspectors and is

further discussed in Section 2.2 below.

The root cause team assessed that the engineering confirm/refute evaluation performed

during troubleshooting, along with the subsequent investigative actions outlined below,

were effective in considering and ruling out all other potential causes of the failure:

Electrical and control circuitry problems were investigated and ruled out. Due to

the initial reports that the field operator did not believe that the EDG even rolled

over, the root cause team investigated other possibilities that could have caused

the EDG not to have rolled, and still brought in the alarms that were received.

One viable possibility considered was a possible fault associated with the EDG

Start/Stop hand switch in the control room. The hand switch in question was a

piece of original equipment. One of the corrective actions was to replace the

hand switch when the 6L fuel pump and metering rod was replaced after the

event. The switch was bench tested, disassembled, and inspected, and it was

determined that the switch not only functioned properly without signs of

degradation, but it would not be physically possible to have the switch

Enclosure 2

-9-

manipulated to send a stop signal to the diesel while an operator takes the

switch to the start position. The inspectors performed a visual inspection of the

switch internals and reviewed the testing methods and results. The inspectors

concluded that the EDG start/stop control switch would not have caused the

EDG failure to start on November 21, 2007.

The starting air system was examined and proven to be functional. The

inspectors confirmed this by performing system walkdowns. A water roll check

was performed satisfactorily.

The fuel day tank was inspected to ensure proper alignment and fuel quality.

Inspections of the joints that connect the fuel pump control shaft levers to the

fuel racks were performed, and determined that none were exhibiting mechanical

binding. The inspectors confirmed this by performing a system walkdown.

The 6L fuel pump was replaced and sent to the vendor for testing, disassembly,

and inspection. No abnormalities were identified, and internal binding of the

pump was determined not to be a cause of the event.

The capability of single paint drop to counter the force applied and prevent the

motion of the fuel control shafts was assessed. A spare fuel pump was

subjected to a series of field tests to determine the force required to overcome

the adhesion of a drop of paint in the location that had been identified. The

results were consistent with the hypothesis that the force applied from the

mechanical governor could have been overcome by the presence of the paint

drop becoming wedged in the minimal clearance between the fuel rack and the

pump housing. Another pull test was done to confirm that a fuel rack exposed to

various combinations of dirt and grit would not require appreciably more pull

tension to operate.

Aspects of organizational and programmatic effectiveness were also evaluated by the

root cause team, and confirmed by the inspectors. These included inadequate

supervisory and management involvement with the painting activities, work practices

employed during the job, and the less than comprehensive development of the

procedures and work packages associated with the activity.

The extent of the condition that was determined to be the cause of the EDG 1-02 failure

was assessed by the root cause team. All other EDGs were thoroughly inspected to

verify that the same condition did not exist, particularly with the Unit 2 Train B EDG 2-02,

which had been similarly painted in September and October. All other EDGs were

verified to be free of the subject degraded condition. Emergency Diesel Generator 2-02

successfully passed its monthly surveillance test on November 28, 2007. The

inspectors reviewed the licensees actions and concluded that the licensees extent of

condition evaluation was adequate.

The inspectors concluded, following interviews as well as a review of personal

statements made by the maintenance personnel, that the work practices of painters and

other work groups who performed daily paint clean-up inspections to identify paint

Enclosure 2

-10-

spatter and drops that needed to be cleaned off of sensitive engine components was a

valid contributor to the event. The inspectors also determined that neither

documentation nor feedback from the inspections to the painters or operations

management regarding the results of those inspections was performed. The

communication of those results, to the right individuals, could have identified the need to

reinforce expectations, alter paint methods or barriers, or institute a stand down that

may have led to the prevention of the event. At a minimum, communication between

organizations (maintenance, inspection, operations, and management) was not as

strong as it could have been for this work on highly risk significant, safety-related

equipment.

Along with the discussion above, the inspections that were performed as part of the

postpainting activities were agreed upon between operations and maintenance. Neither

the inspections nor any other applicable postmaintenance testing was specified by the

work order for performing the painting activities. Also, there was no discussion

concerning foreign materials control exclusion (FME) controls. FME has been a

significant issue with the licensee in the recent past, but no mention of this sensitivity

was made. The inspections that were performed were not documented anywhere as

having been done nor were any of the findings stemming from the inspections.

The inspectors found that the licensee assembled an effective root cause team. The

root cause team investigated every lead that was available to determine exactly why the

Unit 1 Train B EDG failed to start on November 21, 2007. The inspectors determined

that the scope, methods, and rigor associated with the root cause analysis were

appropriate and consistent with the safety significance of the problem, and that the

evaluation was successful in determining and addressing the most probable root and

contributing causes of this issue.

.2

Corrective Action Assessment

The inspectors evaluated the scope, adequacy, and timeliness of the licensees

corrective measures that were both planned and implemented in response to the cause

of the EDG 1-02 failure. The inspectors concluded that the actions planned and taken

by the licensee were appropriate to address the degraded condition, to result in the

prevention of a future similar failure, and were consistent with the safety significance of

the event. Corrective actions to be taken prior to resuming painting activities include:

Revise Procedure MSM-G0-0220 used for painting to require a shiftly

manipulation of the fuel racks in addition to a visual inspection of components to

be free of paint spatter/drops

Verify the information contained in the painting pre-job briefing book to ensure it

contains all sensitive areas on the EDG that should not be painted

Revise Procedure MSM-G0-0220 to include pictures and other information

contained in the painters prejob briefing book used during EDG painting

Revise Procedure MSM-G0-0220 to provide for as you go inspections and

cleaning when painting is done around sensitive components

Enclosure 2

-11-

Include this event in prejob briefings for future activities to heighten sensitivity to

the potential effects of paint spatter/drops in areas that can bind mechanical

components or block air pathways

Improve tools and techniques used by painters to minimize drops and spatter.

Also research available FME barriers that could be used to shield sensitive areas

Additional planned corrective actions include:

Develop a preventive maintenance activity to perform a fuel control shaft break

away force test to monitor for potential degradation in the shaft linkage or

bearings

Revise alarm response Procedure ALM-1302A to remove ambiguity regarding

checking components for freedom of movement by providing specific instruction

to include a manual manipulation of the components

1.4

Scope of the Failure Mechanism

The inspectors, through inspection and investigation, interviews of system engineers,

reviews of EDG design documentation, and assessment of the licensees root cause

analysis, developed a scope of the mechanism that was determined to be the root

cause of the EDG 1-02 failure. The fuel pump control racks (fuel metering rods) were

prevented from moving from their normal standby (closed) positions in response to a

governor demand by the presence of a drop of paint that had adhered to the fuel rack in

a location where the rack enters the housing of the fuel pump (with very minimal

clearance) when moving in the open direction. Since all fuel racks are mechanically

linked by the common fuel control shafts and cross shaft linkages, the motion of the

entire system in the open direction (back to the extensible link from the mechanical

governor) was inhibited by one fuel rack that was stuck in the standby (closed) position.

A torsion spring on the control shaft associated with each fuel pump control shaft lever

functions to allow continued motion of the system in the closed direction if one or more

individual fuel racks become bound. However, the feature does not provide this function

for system motion in the open direction, as in the response to an EDG start signal.

1.5

Event Precursors

The root cause of the EDG failure to start was determined to be paint that was

inadvertently dropped onto a fuel pump metering rod. The inspectors reviewed

corrective action documents and interviewed system engineers in order to identify any

previous related issues that may have been precursors to the Unit 1 Train B EDG failure

to start. The inspectors reviewed all available documented issues dating back to 1999

that fell into each of the following two categories: (1) Previous similar or related EDG

failures, and (2) Previous issues involving equipment failures related to painting. The

inspectors determined that there had been no previous EDG or painting related issues

that may have been precursors to this event.

Enclosure 2

-12-

1.6

EDG Maintenance and Testing

The inspectors reviewed the licensees EDG Maintenance and testing programs. The

inspectors reviewed maintenance and testing records as well as the licensees plans

and schedules related to preventive maintenance and testing of the EDGs. The

inspectors also interviewed several system engineers to gain an understanding of the

licensees approaches and programs involving EDG maintenance and testing. The

inspectors determined that the licensees EDG routine maintenance and testing

programs are adequate and that the licensee is following the program provisions.

However, the inspectors determined that these maintenance and testing practices for

painting activities were not adequate as discussed in Section 2.0.

1.7

Industry Operating Experience (OE)

The inspectors reviewed the industry operating experience (OE) the licensee gained

through their normal review, as well as that which was referenced in the licensees root

cause evaluation. The inspectors conducted interviews of licensee personnel, reviews

of pertinent OE materials discovered independently as well as with the assistance of the

NRCs Operating Experience Section, and an evaluation of actions taken by the licensee

in response to relevant OE. The specific documents reviewed during this review is listed

in Attachment 1 of this report.

The inspectors determined that the licensee had appropriately reviewed and

incorporated OE associated with the circumstances of the EDG failure, and that a failure

to incorporate applicable OE into station practices was not a contributing cause to the

EDG failure. The inspectors reviewed several items of OE, inspection reports, and

licensee event reports (LERs). The inspectors reviewed the licensees responses to the

applicable cases. The licensee did have all of the OE in their OE review system, with

the exception of LERs. The licensee reviews industry OE that comes from INPO and

not specifically the LER database. It appeared that the licensee had accounted for all

available OE at the time that could have reasonably been obtained and reviewed.

All of the OE pertaining to notification events of inoperable diesels due to painting

described gross painting errors that resulted in inoperable diesel generators (e.g.,

inappropriate/movable components being painted). The licensee did take those events

into consideration when developing the work plan for painting of the EDGs in the

associated rooms. The licensee held meetings well in advance of the scheduled

painting window, ensured that operations and maintenance personnel were

communicating, and developed a painters handbook that presented precautions as well

as clear photographs of the areas and components not to paint. The preparation was

adequate for the knowledge that the plant had on site at the time. The sensitivity that

one paint drop in a specific, unintended location could render the EDG inoperable was

not considered by the licensee in their preparation and conduct of the EDG painting

activities, but this was not a subject of previous OE.

One item that was not specifically incorporated into the procedures for painting the EDG

was a specific postmaintenance test to be performed to prove operability. The

licensees procedure described and recommended any of several postmaintenance

Enclosure 2

-13-

options, including visual inspections and equipment functionality tests. This procedure

and its weaknesses were discussed as part of the root cause evaluation in Section 1.3.

The licensee sent two of its employees (a system engineer and a painting supervisor)

on a benchmarking trip prior to cleaning up, painting, and relamping the EDG Rooms.

The licensee employees were aware of the potential to make the EDG inoperable by

painting activities, but did not get enough information to be as sensitive as necessary for

their painting activities. After the failed EDG start, the licensee called the plants visited

during the benchmarking trip to ask more questions, and then discovered that one plant

had knowledge that very little paint or other foreign materials on the metering rods could

render the EDG inoperable. The licensee could have possibly obtained this information

if their staff were to have asked more probing questions, given the work that was

planned at the site. The inspectors concluded that the licensee was not fully effective in

addressing operating experience associated with painting impacts on emergency diesel

generator operability.

1.8

Potential Generic Issues

The inspection team evaluated the circumstances surrounding the event and assessed

the root cause of the Unit 1Train B EDG failure to start. The team interviewed

numerous licensee personnel and reviewed industry operating experience as well as

NRC generic communications with the goal of identifying any potential generic issues

that should be addressed as a result of the event.

The inspection team concluded that, while painting activities occur at all plants, there are

no specific generic concerns associated with this instance of procedural compliance.

The licensee has also issued an action in the corrective action program to issue an OE

report to INPO for future reference.

2.0

SPECIAL INSPECTION FINDINGS

2.1

Painting Activities Result in Inoperability of EDG

Introduction: A White self-revealing violation of Unit 1 Technical Specification (TS) 3.8.1, AC Sources - Operating, was identified for the licensees failure to satisfy TS

LCO 3.8.1 in that painting activities conducted on the Unit 1 Train B EDG 1-02 resulted

in paint being deposited and left in a location that caused the EDG to become

inoperable. As a result, EDG 1-02 failed to start on demand during the subsequent

monthly surveillance test. Following the discovery of the condition, the TS required

actions were satisfied; however, the time period between the occurrence of the condition

and the discovery of the condition exceeded the TS allowed outage time.

Description: On October 15, 2007, the licensee commenced painting activities that

occurred on and around EDG 1-02. A successful monthly slow start surveillance test

was performed on October 24, 2007. Painting activities continued through November 1,

2007. The inspectors reviewed Work Order (WO) 4-07-175968-00, which implemented

the painting activities on and around EDG 1-02 and specified that painting was to be

performed per the requirements of Procedure MSM-G0-0220, General Plant Painting,

Revision 2. The inspectors noted that the WO did not contain requirements for

Enclosure 2

-14-

postmaintenance testing of the EDG, and that Procedure MSM-G0-0220, General Plant

Painting, Revision 2, contained the following steps:

NOTE: System engineer, operations, maintenance services or other

departments may provide useful guidance in determining appropriate protection

of equipment and post-painting functional testing.

5.1.1.2 Painting conducted on equipment should be done in such a manner as

to ensure paint does not bind components required to move. Prejob briefings,

visual verification of postpainting operation, equipment functional testing or other

similar activities are recommended practices that should be employed when

painting equipment.

Through interviews, the inspectors determined that representatives from the

maintenance services, system engineering, maintenance, and operations departments

discussed plans for verifying at the end of each day that the EDG remained operable.

The above requirement and guidance of the general plant painting procedure was not

referenced in this discussion. It was decided that a senior operations department

personnel would perform a visual inspection at the end of each day to verify that

painting had not been done so as to affect the operability of the EDG. This plan was

understood and executed, but was not documented, nor were any inspection results

documented. Prejob briefs and postpainting inspections were focused on avoiding the

painting of components that were not supposed to be painted and were appropriate and

effective in that regard. However, appropriate sensitivity to the potential functional

impact of stray drop(s) of paint in sensitive location(s) was not emphasized.

On November 21, 2007, EDG 1-02 failed to start on demand during its next monthly

surveillance test. Following approximately 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> of troubleshooting, EDG 1-02 was

successfully started. This issue was entered into the licensees corrective action

program as SMF-2007-003253-00. The licensee performed a root cause analysis to

determine the cause of the failure. The most likely cause of the failure was determined

to be a paint drop that had been deposited on the 6L fuel rack that caused the rack to

become stuck. This prevented motion of all 16 fuel racks, thereby preventing the EDG

from receiving sufficient fuel to run. Corrective actions planned and taken by the

licensee are discussed in Section 1.3 of this enclosure.

Analysis: The performance deficiency associated with this finding involved the

licensees failure to ensure that the assumed operability of safety-related equipment was

not affected by the performance of scheduled maintenance activities. Specifically,

painting was conducted on and around EDG 1-02 in such a manner that paint was

deposited and remained in a location that caused the EDG to become inoperable and

fail to start on demand during a subsequent surveillance test. Postpainting verification

of equipment functionality was inadequate. Consequently, the requirements of TS LCO 3.8.1.b and the associated required TS Actions B.4 and G.1 and 2 were not met. The

finding was greater than minor because it was associated with the human performance

attribute of the mitigating systems cornerstone, and it affected the cornerstone objective

to ensure the availability, reliability, and capability of systems that respond to initiating

events to prevent undesirable consequences. The Phase 1 Worksheets in Manual

Chapter 0609, Significance Determination Process, were used to conclude that a

Enclosure 2

-15-

Phase 2 analysis was required because the performance deficiency affected the

emergency power supply system that is a support system for both mitigating and

containment barrier systems. Based on the results of the Phase 2 analysis, the finding

was determined to have low to moderate safety significance (White). The senior reactor

analyst determined that a more detailed Phase 3 analysis was needed to fully assess

the safety significance. Based on the results of the Phase 3 analysis, the finding was

determined to have low to moderate safety significance (White). The Phase 1, 2, and 3

significance determination process analyses associated with this finding, including

assumptions and limiting core damage sequences, is included as Attachment 3 to this

report. The cause of this finding was determined to have a crosscutting aspect in the

area of human performance associated with work practices in that the licensee failed to

provide adequate supervisory and management oversight of work activities, including

contractors, such that nuclear safety is supported H.4(c). Specifically, the actions

planned and taken to assess and control the operational impact of the painting activities

on the functionality of the EDG were not reflective of adequate supervisory and

management oversight of the activities.

Enforcement: Unit 1 Technical Specification (TS) 3.8.1, AC Sources - Operating,

requires that while the plant is in Modes 1, 2, 3, or 4, two diesel generators (DGs)

capable of supplying the onsite Class 1E power distribution subsystem(s) shall be

operable. For the condition of one DG being inoperable, the required action is to restore

the DG to an operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> and within 6 days from the discovery of the

failure to meet the Limiting Condition for Operation (LCO), or be in Mode 3 within 6

hours and Mode 5 within 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br />. Contrary to the above, from November 1, 2007,

through November 21, 2007, while the plant was in Mode 1, one of the two DGs capable

of supplying the onsite Class 1E power distribution subsystem(s) was inoperable, and

action was not taken to either restore the DG to an operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or be

in Mode 3 within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and Mode 5 within 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br />. Specifically, Emergency Diesel

Generator (EDG) 1-02 was made inoperable as a result of painting activities due to paint

having been deposited and remaining on at least one fuel rack in a location that

prevented motion required to support the operation of the EDG. This condition caused

EDG 1-02 to fail to start during a surveillance test on November 21, 2007. Following the

discovery of the condition on November 21, 2007, the licensee satisfied the TS required

actions by restoring the EDG to an operable status on November 22, 2007. This

violation is the subject of the enclosed Notice of Violation: VIO 05000445/2007008-01,

Painting Activities Result in Inoperability of Emergency Diesel Generator.

2.2

Inadequate Alarm Response Procedure for EDG Failure to Start

Introduction: The inspectors identified a Green noncited violation of Unit 1 Technical Specification 5.4.1.a, Procedures, for an inadequate alarm response procedure. The

inspectors determined that Procedure ALM-1302A, Diesel Generator 1-02 Panel,

Revision 5, was inadequate in that it was ambiguous and did not cause the responders

to verify that the fuel racks were free as part of the response actions to investigate the

cause of the unit failing to start. Consequently, the licensee failed to identify that the

Unit 1 Train B EDG 1-02 fuel racks were not free to move, which led to an extended

period of inoperability and a significant delay in diagnosing the cause of the EDG failure

to start.

Enclosure 2

-16-

Description: On November 21, 2007, at 10:20 a.m., EDG 1-02 failed to start during a

slow start monthly surveillance test. Field operators responded to the EDG local alarm

panel. Operators referenced the alarm response Procedure ALM-1302A, Diesel

Generator 1-02 Panel, Revision 5, and reviewed the section for Alarm Window 6.6 Unit

Failure To Start. A limited number of system malfunctions that could have caused the

failure to start were indicated. These included fuel rack or fuel oil day tank issues,

improper starting air alignment, failed timing chain, and a Governor malfunction.

Operators implemented the Operator Actions section of the procedure, which included

actions to determine the cause of the unit failing to start. The first action indicated was

to Check fuel racks free and in max fuel position. The field support supervisor (senior

reactor operator) believed that the appropriate action was to perform a visual inspection

of the fuel racks. The fuel racks were not in the "max fuel" position. The inspectors

later determined that, following the majority of postulated failed start scenarios, the fuel

racks would not be expected to remain in the "max fuel" position, even if they had

initially moved. In accordance with the operators' training, the expectation for

performing this step was to visually inspect the racks. However, the inspectors

determined that without observing them being in a position other than their normal

standby (closed) position, this visual check would not be sufficient to meet the intent of

the procedure step (i.e., to ensure that the racks were not stuck in the "no fuel" position,

which was a probable failure cause that was indicated earlier in the procedure). The

operator completed this procedure step, as well as subsequent steps for starting air

alignment, EDG day tank alignment, and fuel quality with no abnormalities identified.

Field operator actions were completed at 11:05 a.m.

The licensee developed a troubleshooting plan and attempted two more starts of the

EDG (both unsuccessful) before determining that the fuel racks and metering rods were

not responding to the Governor demand to open. At 7:39 p.m. the licensee exercised

the fuel racks and discovered that two of the metering rods were stuck, with one fully

stuck in the closed position and one which became partially stuck following some motion

in the open direction. Operations and maintenance performed followup inspections and

successfully started the EDG at 9:32 p.m. The diesel was declared operable following

the surveillance run and post run inspections on November 22, 2007 at 3:08 a.m.

The inspectors concluded that the field operators performed the actions of the alarm

response Procedure ALM-1302A, in accordance with station procedures and training,

and operations managements expectations. The inspectors further concluded that the

inadequacy of the alarm response procedure to give clear instruction and guidance to

ensure that the EDG fuel racks were verified to be free and not binding resulted in

missing an opportunity to identify the cause of the EDG failure to start in a timely

manner. This missed diagnosis not only led to a narrowly focused troubleshooting effort

by the licensee, but also allowed the EDG to remain unnecessarily inoperable for

approximately an additional 8.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />.

Analysis: The performance deficiency associated with this finding involved the

licensees failure to adequately establish clear procedure guidelines to implement alarm

response Procedure ALM-1302A. This resulted in the licensees failure to identify the

binding of the Unit 1 Train B EDG fuel racks and metering rods in a timely manner

following a failure to start. The finding was determined to be more than minor because

Enclosure 2

-17-

it was associated with the procedure quality attribute of the mitigating systems

cornerstone, and it affected the cornerstone objective to ensure the availability,

reliability, and capability of systems that respond to initiating events to prevent

undesirable consequences. Using Manual Chapter 0609, Significance Determination

Process, Phase 1 Worksheet, the finding was determined to have very low safety

significance (Green) because it was not a design or qualification deficiency, did not

represent a loss of safety function, did not represent an actual loss of a single train for

greater than its Technical Specification allowed outage time, did not represent a loss of

a non-Technical Specification train of equipment for greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, and did not

screen as potentially risk significant due to a seismic, flooding, or severe weather

initiating event.

Enforcement: Unit 1 Technical Specification 5.4.1.a requires that written procedures be

established, implemented, and maintained covering the procedures listed in Regulatory

Guide 1.33, Quality Assurance Program Requirements, Revision 2, Appendix A,

Section 5, for Abnormal, Off-Normal, or Alarm Conditions. Contrary to the above, on

November 21, 2007, the licensee failed to adequately establish, implement, and

maintain a procedure for an alarm condition. Specifically, alarm response

Procedure ALM-1302A, Diesel Generator 1-02 Panel, Revision 5, was not adequately

established and maintained, which resulted in the licensees failure to recognize that the

EDG 1-02 fuel racks and metering rods were bound and caused the failure of the EDG

to start on November 21, 2007. Consequently, the EDG remained inoperable for

approximately 8.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> longer than necessary. Because the finding was determined to

be of very low safety significance and has been entered in the licensees correction

action program as SMF-2007-003426, this violation is being treated as an NCV

consistent with Section VI.A of the Enforcement Policy: NCV 05000445/2007008-02,

Inadequate Alarm Response Procedure for EDG Failure to Start.

4OA6 Meetings, Including Exit

On December 7, 2007, and January 10, 2008, the results of this inspection were

presented to Mr. R. Flores, Site Vice President, and Mr. T. Hope, Regulatory

Performance Manager, respectively, and other licensee personnel who acknowledged

the findings. Additionally on January 24, 2008, the final results of this inspection were

presented to Mr. F. Madden, Director, Regulatory Affairs, and other members of the

licensee staff who acknowledged the findings. On February 25, 2008, an additional exit

meeting was conducted with Mr. T. Hope and other licensee personnel who

acknowledged the findings. The inspectors confirmed that no proprietary material was

retained during the inspection.

ATTACHMENT 1: Supplemental Information

ATTACHMENT 2: Special Inspection Charter

ATTACHMENT 3: Significance Determination Evaluation

Attachment 1

A1-1

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

J. Bain, System Engineer

J. Bales, Maintenance Services

T. Bennette, Operations

M. Blevins, Senior Vice President and Chief Nuclear Officer

H. Davenport, System Engineer

D. Davis, Performance Improvement Director

R. Flores, Site Vice President

D. Goodwin, Manager, Shift Operations

T. Hope, Manager, Regulatory Performance

M. Kanavos, Plant Manager

D. Kross, Director, Operations

S. Lakdawala, Corrective Action Program Manager

F. Madden, Director, Regulatory Affairs

D. McGaughey, Manager, Shift Operations

G. Merka, Regulatory Affairs

J. Meyer, Technical Support Manager

W. Morrison, Maintenance Smart Team Manager

J. OQuinn, Maintenance

W. Reppa, System Engineering Manager

D. Scott, Root Cause Analyst

S. Smith, Director, System Engineering

R. Sorrell, System Engineer

T. Terryah, System Engineering Manager

T. Tigner, Programs Supervisor

B. Wagner, PROMPT Team

W. Williams, Maintenance Services

M. Wisdom, System Engineering

NRC

D. Allen, Senior Resident Inspector

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

05000445/2007008-01

VIO

Painting Activities Result in Inoperability of Emergency

Diesel Generator (Section 2.1)

Attachment 1

A1-2

Opened and Closed

05000445/2007008-02

NCV

Inadequate Alarm Response Procedure for EDG Failure to

Start (Section 2.2)

Closed

None

Discussed

None

LIST OF DOCUMENTS REVIEWED

Procedures

NUMBER

TITLE

REVISION

ALM-1302A

Diesel Generator 1-02 Panel

5

MSM-G0-0002

General Plant Painting

2

MSM-P0-3374

Emergency Diesel Generator Monthly Run Related

Inspections

3

OPT-215-1

Offsite Transmission Network Operability Data Sheet

14

OWI-104-28

Plant Equipment Operator Diesel Generator 1-02

Operating Log

13

OWI-104-26

Control Room Diesel Generator 1-02 Operating Log

11

MSM-G0-0216

Protective Coatings

23

MSM-G0-0217

Maintenance Protective Coatings-Concrete

0

MSM-G0-0218

Maintenance Protective Coatings-Steel

0

CMP-CV-1009

Application of Protective Coatings to Carbon Steel

Surfaces in the Containment and Radiation Areas

Outside of Containment

0

ODA-102

Conduct of Operations

24

ODA-401

Control of Annunciators, Instruments, and Protective

Relays

9

ODA-407

Guidelines on Use of Procedures

12

Attachment 1

A1-3

OPT-214A

Emergency Diesel Operability Test

19

SOP-609

Diesel Generator System

17

STA-426

Industry Operating Experience Program

1

STA-692

Protective Coatings Program

0

TSP-503

Emergency Diesel Generator Reliability Program

3

Smart Forms

SMF-2007-03253

SMF-2007-03426

SMF-2007-03302

SMF-2007-02319

WOs

4-07-176522

4-07-176543

5-05-501230-AA

5-07-502391-AK

4-07-175968

4-07-176544

4-07-176582

4-07-175492

4-07-176545

4-95-091357-00

4-94-078722-00

Miscellaneous Information

Evaluation EVAL-2007-003253-02-00, Root Cause Analysis

Post-Work Test Guide, Revision 12

LER 07-004-00, Emergency Diesel Generator Failed Surveillance Test Due to Paint on Fuel

Injector Control Linkage

TUElectric Office Memorandum, CPSES-9125952, October 10, 1991

TUElectric Office Memorandum, CPSES-9108929, April 3, 1991

TUElectric Office Mamorandum, CPSES-91000861, January 11, 1991

Technical Evaluation TE# SE-90-1814

Cooper-Enterprise Clearinghouse R4/RV4 Preventative maintenance Program (PMP) for

Nuclear Standby Applications, Revision 0

Operations Guideline 3, Attachment 4, Operations Department Alarm Response Expectations,

August 2006

Attachment 1

A1-4

CPNPP Operations Logs, November 21-22, 2007

Amercoat 220, Waterborne Acrylic Topcoat Product Datasheet, circa 1999

Information Notices

IN 93-76, Inadequate Control of Paint and Cleaners for Safety-Related Equipment

IN 91-46, Degradation of Emergency Diesel Generator Fuel Oil Delivery Systems

NRC Inspection Documents

Inspection Procedure 93812, Special Inspection, 7/18/2007

Special Inspection Charter to Evaluate the Comanche Peak Steam Electric Station Diesel

Generator Failure to Start Event, November 30, 2007

NRC Inspection Reports

ML073060511 (RBS)

ML072040388 (DC Cook IR 05000316/2007004)

LIST OF ACRONYMS

ADAMS

agency document and management system

CFR

Code of Federal Regulations

CPSES

Comanche Peak Steam Electric Station

EDG

emergency diesel generator

FME

foreign material exclusion

INPO

Institute of Nuclear Power Operations

LER

licensee event report

NRC

Nuclear Regulatory Commission

OE

operating experience

PARS

publicly available records system

NEO

nuclear equipment operator

SDP

significance determination process

SMF

smart form

WO

work order

A2-1

Attachment 2

November 30, 2007

MEMORANDUM TO: Cale Young, Resident Inspector, ANO

Alfred Sanchez, Resident Inspector, CPSES

FROM:

Arthur T. Howell III, Director, Division of Reactor Projects AVegel for/RA/

SUBJECT:

SPECIAL INSPECTION CHARTER TO EVALUATE THE COMANCHE

PEAK STEAM ELECTRIC STATION DIESEL GENERATOR FAILURE

TO START EVENT

A Special Inspection Team is being chartered in response to the Comanche Peak Steam

Electric Station emergency diesel generator (EDG) failure to start event on November 21, 2007.

You are hereby designated as the Special Inspection Team members. Mr. Cale Young,

Resident Inspector, ANO, is designated as the team leader. The assigned SRA to support the

team is David Loveless.

A.

Basis

On November 21, 2007, Comanche Peak Unit 1 diesel generator, DG-102, failed to start

during the monthly surveillance test. After several failed attempts to start the diesel,

licensee engineers developed a trouble shooting plan to determine the cause of the

diesel failing to start. During the trouble shooting efforts, licensee personnel identified

that two fuel rack linkage/metering rods (L2 and L6) on DG-102 appeared to be binding.

Additional inspections indicated that there were very small signs of paint on the metering

rods for the L2 and L6 fuel pumps, but not enough to prevent movement. Painting

activities in all EDG rooms were suspended until further measures were taken to prevent

reoccurrence of this issue. During the trouble shooting activities, each individual fuel

pump was manually operated by maintenance personnel and all but two moved freely.

Maintenance personnel were able to manually move, and subsequently free, the L2 and

L6 metering rods. Operations personnel then performed the surveillance test

satisfactorily. Maintenance personnel verified that the metering rods on the remaining

EDGs had free movement of all fuel rack linkage/metering rods.

During further investigation into when painting had occurred inside the EDG room, it was

discovered that the painters continued to paint in the diesel room after the last

successful surveillance test. This brings into question whether DG-102 would have

been able to perform its intended function if called upon from October 24 to

November 21, 2007.

A2-2

Attachment 2

This Special Inspection Team is chartered to review the circumstances related to the

failure of DG-102 to start, and to assess the effectiveness of the licensees actions for

resolving these problems.

B.

Scope

The team is expected to address the following:

1.

Develop a chronology (time-line) that includes significant event elements.

2.

Evaluate the licensees response to the failure of the EDG to start. Ensure that

plant personnel responded in accordance with plant procedures and Technical

Specifications.

3.

Assess the licensees root cause determination for the EDG failure, the extent of

condition review, the common cause evaluation and corrective measures.

Evaluate whether the timeliness of the corrective measures are consistent with

the safety significance of the problem.

4.

Develop a complete scope of the failure mechanism identified by the licensees

root cause determination.

5.

Identify previous EDG issues that may have been precursors to the November 1,

2007, event. Evaluate the licensees corrective measures and extent of

condition reviews for those problems.

6.

Evaluate the licensees EDG maintenance and testing programs. Verify that the

programs are adequate and that the licensee is following the program provisions.

7.

Evaluate pertinent industry operating experience that represents potential

precursors to the November 21, 2007, event, including the effectiveness of

licensee actions taken in response to the operating experience.

8.

Determine if there are any potential generic issues related to the EDG failure at

Comanche Peak Unit 1. Promptly communicate any potential generic issues to

Region IV management.

9.

Collect data as necessary to support a risk analysis. Work closely with the

Senior Reactor Analyst during this inspection.

A2-3

Attachment 2

C.

Guidance

Inspection Procedure 93812, Special Inspection, provides additional guidance to be

used by the Special Inspection Team. Your duties will be as described in Inspection

Procedure 93812. The inspection should emphasize fact-finding in its review of the

circumstances surrounding the event. It is not the responsibility of the team to examine

the regulatory process. Safety concerns identified that are not directly related to the

event should be reported to the Region IV office for appropriate action.

The Team will report to the site, conduct an entrance, and begin inspection no later than

December 4, 2007. While on site, you will provide daily status briefings to Region IV

management, who will coordinate with the Office of Nuclear Reactor Regulation, to

ensure that all other parties are kept informed. If information is discovered that shows a

more significant risk was associated with this issue, immediately contact Region IV

management for discussion of appropriate actions. A report documenting the results of

the inspection should be issued within 30 days of the completion of the inspection.

This Charter may be modified should the team develop significant new information that

warrants review. Should you have any questions concerning this Charter, contact me at

(817) 860-8148.

A3-1

Attachment 3

ATTACHMENT 3

SIGNIFICANCE DETERMINATION EVALUATION

Comanche Peak Steam Electric Station

EDG Inoperability Caused By Painting Activities

Significance Determination Basis

1.

Phase 1 Screening Logic, Results, and Assumptions

In accordance with NRC Inspection Manual Chapter 0612, Appendix B, Issue

Screening, the team determined that this finding represented a licensee performance

deficiency. The team then determined that the issue was more than minor because it

was associated with the equipment performance attribute and affected the mitigating

systems cornerstone objective to ensure the availability, reliability, or function of a

system or train in a mitigating system in that Emergency Diesel Generator DG-102

would not have started upon demand.

The team evaluated this finding using the SDP Phase 1 Screening Worksheet for the

Initiating Events, Mitigating Systems, and Barriers Cornerstones, provided in Manual

Chapter 0609, Appendix A, Determining the Significance of Reactor Inspection

Findings for At-Power Situations. For this finding, a Phase 2 estimation was required

because the performance deficiency affected the emergency power supply system that

is a support system for both mitigating and containment barrier systems.

2.

Phase 2 Risk Estimation

In accordance with Manual Chapter 0609, Appendix A, Attachment 1, User Guidance

for Phase 2 and Phase 3 Significance Determination of Reactor Inspection Findings for

At-Power Situations, the senior reactor analyst evaluated the subject finding using the

Risk-Informed Inspection Notebook for Comanche Peak Steam Electric Station, Units 1

and 2, Revision 2.01a. The following assumptions were made:

a.

The identified performance deficiency occurred some time between the last

successful test on October 24, 2007, and the test failure that occurred on

November 21, 2007.

b.

In accordance with Manual Chapter 0609, Appendix A, Attachment 2, Site

Specific Risk-Informed Inspection Notebook Usage Rules, Rule 1.1, Exposure

Time, the analyst determined the time frame over which the finding impacted

the risk of plant operations. Because the exact time of failure was unknown, an

exposure time of t/2 from the last valid test was used. This was 1/2 of the 28 days

between tests, or 14 days. Therefore, for the phase 2 analysis, the exposure

time used to represent the time that the performance deficiency affected plant

risk was between 3 and 30 days.

A3-2

Attachment 3

c.

Table 2 of the risk-informed notebook requires that when a performance

deficiency affects the diesel generators, the following initiating event scenarios

are applicable: LOOP and LEAC. Therefore, the analyst utilized these

worksheets from the risk-informed notebook.

d.

According to the risk-informed notebook, Table 1, for a 3-30 day exposure, the

initiating event likelihood should be 3 for a loss of offsite power and 5 for a loss

of offsite power with loss of one vital 6.9kV bus.

e.

The analyst gave no operator action credit as discussed in Manual

Chapter 0609, Appendix A, Attachment 1, Table 4, Remaining Mitigation

Capability Credit. The requirements to have procedures in place and to have

trained the operators in recovery under similar conditions for such credit were not

met.

The dominant sequences from the notebook were documented below:

TABLE C.b

Failure of Emergency Diesel Generator 102 to Start

Phase 2 Sequences

Initiating Event

Sequence

Mitigating Functions

Results

Loss of Offsite Power

2

LOOP-AFW-FB

8

4

LOOP-EAC-REC5

6

7

LOOP-EAC-TDAFW

6

Loss of Offsite Power with

Loss of One Vital 6.9 kV Bus

1

LEAC-PORV-HPR-MKRWST

8

3

LEAC-PORV-HPI

7

Using the counting rule worksheet, the result from this estimation indicated that

the finding was of low to moderate safety significance (WHITE). However, the

analyst determined that this estimate did not include a full coverage of the risk

related to the failure identified and that a better evaluation of the internal risk

would be necessary for fully assessing the risk related to external initiators.

3.

Phase 3 Analysis

In accordance with Manual Chapter 0609, Appendix A, the analyst performed a Phase 3

analysis using the Standardized Plant Analysis Risk (SPAR) Model for Comanche Peak,

Revision 3.31, dated August 2006, to simulate the failed Diesel Generator 1-02.

Additionally, the analyst conducted an assessment of the risk contributions from external

initiators using insights and/or values provided by the licensees probabilistic risk

assessment model, in the licensees recent submittal for extension of completion times

for diesel generators (Reference 1), and simplified fire probabilistic risk assessment.

Reference 1: Letter dated November 15, 2007, Blevins to U.S. NRC, Subject: Comanche Peak Steam Electric Station (CPSES)

Docket Nos. 50-445 and 50-446, Response to Request for Additional Information Related to Licence Amendment Request (LAR)06-009,

Revision to Technical Specification (TS) 3.8.1, AC Sources - Operating; Extension of Completion Times for Diesel Generators.

A3-3

Attachment 3

Assumptions

To evaluate the change in risk caused by this performance deficiency, the analyst made

the following assumptions:

A. The vital batteries at Comanche Peak will deplete after approximately 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> of full

postaccident loads without an operating battery charger, assuming that operators do

not take actions to shed unnecessary loads from the vital dc buses. This is the

value used in the licensees probabilistic risk assessment.

B. The Comanche Peak SPAR model, Revision 3.31, represents an appropriate tool for

evaluation of the subject finding.

C. The failure of Emergency Diesel Generator 1-02 was the result of binding of the fuel

rack on at least one injection pump that was caused by painting activities on and

around the diesel.

D. Emergency Diesel Generator 1-02 successfully started and loaded during a

surveillance performed on October 24, 2007. The diesel failed to start during a

surveillance on November 21, 2007, because the fuel rack on at least one injection

pump was bound to the extent that the entire fuel rack assembly was unable to leave

the no fuel position.

E. Painting activities in and around the Emergency Diesel Generator 1-02 engine

ended on November 1, 2007. Therefore, the conditions that caused the engine to

fail had to have been in place at that time for the root cause to be valid (See

Assumption C).

F. The exposure time used for evaluating this finding should be determined in

accordance with Inspection Manual Chapter 0609, Appendix A, Attachment 2, Site

Specific Risk-Informed Inspection Notebook Usage Rules. Attachment 2 discusses

the approach to establishing the exposure time that should be used for the

significance determination process. Step 1.1 states:

The exposure time used in determining the initiating event likelihood should

correspond to the time period that the condition being assessed is reasonably

known to have existed. If the inception of the condition is unknown, then an

exposure time of one half of the time period since the last successful

demonstration of the component or function (t/2) should be used.

G. The appropriate exposure time (EXP), representing the time that Emergency Diesel

Generator 1-02 was not functional, for use in this evaluation is 24 days.

The exact time at which the residual paint that caused the binding of the fuel

racks occurred is unknown. However, it is reasonable to assume that the

condition existed after the completion of painting activities on November 1, 2007.

A3-4

Attachment 3

Therefore, in accordance with Assumption F, Emergency Diesel Generator 1-02

would not have started upon demand for the 20 days November 1 through

November 21, 2007.

Additionally, the inception of the condition could have occurred any time between

the last successful run of the machine on October 24 and the completion of the

painting activities on November 1. Therefore, in accordance with Assumption F,

Emergency Diesel Generator 1-02 would not have started upon demand for one

half of the period from October 24 through November 1, or for an additional 4-

day period.

Based on these two arguments, the analyst determined that the appropriate

exposure time was the sum of the 20 days that the machine was reasonably

assumed to have failed and one half the 8 day period that could have resulted in

the failure condition.

H. Given the condition of the fuel rack and the interpretation by licensed operators of

annunciator response procedures, operators would not have been able to recover

Emergency Diesel Generator 1-02 prior to postulated core damage for sequences

less than 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. Licensed operators stated that the annunciator response

procedure would not have directed operators to manipulate the fuel racks by hand

nor does it require operators to request maintenance personnel perform such a task.

I.

The appropriate nonrecovery probability for sequences longer than 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> is 0.29.

The analyst conducted a human reliability analysis using the SPAR-H method to

determine an appropriate nonrecovery probability. To calculate this value, the

analyst used the following assumptions:

a. The analyst assumed that nominal time was available for recovery diagnosis and

action. The licensee recovered the diesel in 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> 11 minutes during

nonemergency conditions. Therefore, the analyst assumed that, if required,

recovery could have been reasonably performed within the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> coping period

plus extended boil down times.

b. The analyst assumed that emergency response personnel would be under high

stress during the diagnosis and recovery. This is based primarily on the belief

that recovery personnel would know that the consequences of the task would

represent a direct threat to plant safety.

c.

The complexity of this task was considered to be nominal. There is some

ambiguity in the diagnosis. However, there are only two fundamental paths in

diagnosis. The probability that the wrong path would be initially investigated is

taken into account in the performance shaping factor for procedure quality.

d. Procedures for the diagnosis were incomplete. Solely following the procedures

available would not have led to recovery. The basic items to consider were

available in the annunciator response procedure, although it is not clear that this

procedure would have been governing and/or utilized by the recovery personnel.

A3-5

Attachment 3

e. All other performance shaping factors were considered nominal for obvious

reasons.

J.

Emergency Diesel Generator 1-01 would not have failed from the same cause as

Emergency Diesel Generator 1-02 because painting activities had not been

conducted on that diesel. Therefore, the analyst left the common cause failure

probability at its nominal value.

K. The nominal nonrecovery values used by the SPAR model are for the average

nonrecovery for either of two diesel generators. Therefore, given that recovery of

Emergency Diesel Generator 1-02 would be handled separately, the analyst

adjusted the generic nonrecovery value to account for only Emergency Diesel

Generator 1-01 being the only machine available for random failure recovery.

L. The nominal likelihood for a loss of offsite power was unaffected by the subject

finding.

M. Evaluating the risk contribution of this finding related to seismic events is

appropriately conducted by utilizing the licensees assessment found in Reference 1.

The conditional core damage frequency (CCDFSEISMIC) given by the licensee was

2.1 x 10-6/year.

N. The licensees fire risk model is an appropriate tool for evaluation of the subject

finding. The CCDF for fire (CCDFFIRE) provided by the licensee in Reference 1 was

7.8 x 10-6/year.

The analyst independently evaluated the risk change related to internal fires.

These insights were then used to challenge and evaluate the results of the

licensees model. In all cases, the licensees model covered the scenarios

posed by the analyst and included a larger scope of fires than was feasible for

the analyst to evaluate.

O. Traditionally, the initiation of most high wind events, including those that cause a

loss of offsite power, are included in the licensees PRA and/or the SPAR model.

However, the licensees assessment in their individual plant evaluation for external

events did not include events that damage other pieces of equipment that may affect

risk. As stated in Reference 1, the licensee estimated the CCDF for tornados

(CCDFWIND) given the failure of a diesel generator to be 2.3 x 10-5/year.

P. The best estimate of the risk contribution from the subject finding related to internal

flooding is best evaluated using a ratio from the licensees PRA as was discussed in

Reference 1. In their evaluation, the licensee stated that the risk from internal

flooding derived from their internal events PRA was approximately 1 percent (PFLOOD)

of the total plant core damage frequency.

Q. The ratio of sequences going to core damage in the first 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> to those going

through battery depletion is the same for internal and external initiators. This

A3-6

Attachment 3

assumption permits the analyst to use the ratio from the internal events SPAR in

applying recovery to external initiators.

R. The differences between the SPAR and the licensees models were inconsequential.

The analyst, in reviewing the differences between the models, determined that there

were several global differences including: the lack of random failure recovery for

diesel generators in the licensees model and the lack of convolution integrals in the

SPAR model. However, the analyst determined that these differences were not of

consequence to this evaluation because the final results were within the same color

band.

Internal Initiating Events

The senior reactor analyst used the SPAR model for CPSES to estimate the change in

risk associated with internal initiators that was caused by the finding. Average test and

maintenance of modeled equipment was assumed and a cutset truncation of 1.0E-12

was used.

Consistent with guidance in the RASP Handbook, including NRC document,

Common-Cause Failure Analysis in Event Assessment (June 2007), and Assumptions

3.C, 3.G, and 3.K, the SRA modeled the condition by adjusting the following basic

events in the SPAR model:

Basic Event

Original Value

Conditional Value

EPS-DGN-FS-1EG1

5.0 X 10-3

TRUE

EPS-XHE-XL-NR01H

7.72 X 10-1

8.79 X 10-1

EPS-XHE-XL-NR02H

6.48 X 10-1

8.05 X 10-1

EPS-XHE-XL-NR03H

5.56 X 10-1

7.46 X 10-1

EPS-XHE-XL-NR04H

4.84 X 10-1

6.95 X 10-1

The SPAR baseline core damage frequency (CDFBASE) was 1.80 x 10-5/year. The

evaluation case for the above change set resulted in a conditional core damage

frequency (CCDFSPAR) of 3.78 x 10-4/year. The dominant core damage sequences were

documented in the table below:

A3-7

Attachment 3

Initiating Event

Sequence

Preponderant Failures

Frequency

Loss of Offsite

Power

20-03

Failure of EDG 1-01 with

Battery Depletion at 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />

2.62 x 10-4/year

20-06

Failure of EDG 1-01 with

Battery Depletion at 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />

combined with RCS Pump

Seal Failure .

6.56 x 10-5/year

20-45

Failure of EDG 1-01 and the

Turbine-Driven Auxiliary

Feedwater Pump with Core

Damage at 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />.

2.37 x 10-5/year

19

Failure of Motor and Turbine-

Driven Auxiliary Feedwater

Pumps and/or Operator Fails

to Control.

1.07 x 10-5/year

The change in incremental conditional core damage frequency (ICCDP) was calculated

as follows:

ICCDF

= CCDFSPAR - CDFbase

= 3.78 x 10-4/year - 1.80 x 10-5/year

= 3.60 x 10-4/year

Given Assumptions 3.C through 3.G, the exposure time, representing the time that the

performance deficiency impacted the plant, for this analysis was 24 days. Therefore,

the change in core damage frequency (CDFIntNR) caused by this finding, without

applying any recovery to the subject condition, and related to internal initiators was

calculated as follows:

CDFIntNR

= ICCDF * EXP

= 3.60 x 10-4/year * (24 days ÷ 365 days/year)

= 2.37 x 10-5

Given Assumption 3.H, the analyst determined that recovery credit for Emergency

Diesel Generator 1-01 would not be provided for any sequence that led to core damage

in less than 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. Using utilities in the SAPHIRE software to slice cutsets by basic

event, the analyst determined that 7.6 percent of all internal cutsets went to core

damage in less than 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> (PSHORT).

Given Assumption 3.I, the analyst applied a nonrecovery factor (PNR) of 0.29 to all

A3-8

Attachment 3

remaining cutsets. Therefore, the change in core damage frequency (CDFInternal)

caused by this finding and related to internal initiators was calculated as follows:

CDFInternal

= [CDFIntNR * PSHORT] + [CDFIntNR * (1 - PSHORT) * PNR]

= [2.37 x 10-5

• 0.076] + [2.37 x 10-5

• (1 - 0.076) * 0.29]

= 8.15 x 10-6

External Initiating Events

In accordance with Manual Chapter 0609, Appendix A, Attachment 1, Step 2.2.5,

Screen for the Potential Risk Contribution Due to External Initiating Events, the analyst

assessed the impact of external initiators on each of the findings, because the Phase 2

SDP result provided a Risk Significance Estimation of 7 or greater. The analyst

determined that, for the risk of an external initiator to be impacted by this performance

deficiency, the external event would have to cause a loss of offsite power that was not

accounted for in the internal events model. Using the licensees individual plant

evaluation for external events and Reference 1, the analyst determined that the

dominant sequences affected by the subject performance deficiency were from seismic

events, high winds, fire, and internal flooding events.

A. Seismic Event Initiators

As discussed in Assumption 3.M, the analyst utilized the licensees value

for the affects on the risk of seismic events associated with a failed diesel

generator. The incremental risk without recovery (ICCDPSeisNR) was

calculated as follows:

ICCDPSeisNR

= CCDFSEISMIC * EXP

= 2.1 x 10-6/year * (24 days ÷ 365 days/year)

= 1.38 x 10-7

Given Assumption 3.H and 3.Q, the analyst applied PSHORT in quantifying

the change in risk from seismic events.

Given Assumption 3.I, the analyst applied a nonrecovery factor (PNR) of

0.29 to the remaining portion of the risk from seismic events. Therefore,

the change in core damage frequency (CDFSEISMIC) caused by this

finding and related to seismic events was calculated as follows:

CDFSEISMIC

= [ICCDPSeisNR * PSHORT] + [ICCDPSeisNR * (1 - PSHORT) * PNR]

= [1.38 x 10-7

• 0.076] + [1.38 x 10-7

• (1 - 0.076) * 0.29]

= 4.75 x 10-8

A3-9

Attachment 3

B. Internal Fire Initiators

As discussed in Assumption 3.N, the analyst utilized the licensees value

for the affects on the risk of internal fires associated with a failed diesel

generator. The incremental risk without recovery (ICCDPFireNR) was

calculated as follows:

ICCDPFireNR

= CCDFFIRE * EXP

= 7.8 x 10-6/year * (24 days ÷ 365 days/year)

= 5.14 x 10-7

Given Assumption 3.H and 3.Q, the analyst applied PSHORT in quantifying

the change in risk from internal fires.

Given Assumption 3.I, the analyst applied a nonrecovery factor (PNR) of

0.29 to the remaining portion of the risk from seismic events. Therefore,

the change in core damage frequency (CDFFIRE) caused by this finding

and related to seismic events was calculated as follows:

CDFFIRE

= [ICCDPFireNR * PSHORT] + [ICCDPFireNR * (1 - PSHORT) * PNR]

= [5.14 x 10-7

• 0.076] + [5.14 x 10-7

• (1 - 0.076) * 0.29]

= 1.77 x 10-7

C. Tornados and High Wind Initiators

As discussed in Assumption 3.O, the analyst utilized the licensees value

for the affects on the risk of high wind events associated with a failed

diesel generator. The incremental risk without recovery (ICCDFWindNR)

was calculated as follows:

ICCDPWindNR

= CCDFWIND * EXP

= 2.1 x 10-5/year * (24 days ÷ 365 days/year)

= 1.38 x 10-6

Given Assumption 3.H and 3.Q, the analyst applied PSHORT in quantifying

the change in risk from high wind events.

Given Assumption 3.I, the analyst applied a nonrecovery factor (PNR) of

0.29 to the remaining portion of the risk from high wind events.

Therefore, the change in core damage frequency (CDFWIND) caused by

this finding and related to seismic events was calculated as follows:

A3-10

Attachment 3

CDFWIND

= [ICCDPWindNR * PSHORT] + [ICCDPWindNR * (1 - PSHORT) * PNR]

= [1.38 x 10-6

• 0.076] + [1.38 x 10-6

• (1 - 0.076) * 0.29]

= 4.75 x 10-7

D. Internal Flooding Initiators

As discussed in Assumption 3.O, the analyst utilized the ratio determined

by the licensees PRA for internal flooding to other initiators. Given a

value of 1 percent, the change in core damage frequency (CDFFLOOD)

caused by this finding and related to internal flooding was calculated as

follows:

CDFFLOOD

= CDFInternal * PFLOOD

= 8.15 x 10-6 * 0.01

= 8.15 x 10-8

Total Change in Core Damage Frequency

Given that each of the initiators in this analysis were treated to ensure that the final

probabilities were independent of each other, the analyst determined that he total

change in core damage frequency (CDF) could be calculated by taking the sum of

each independent change. Therefore, the final Phase 3 result was calculated as

follows:

CDF = CDFInternal + CDFExternal

= CDFInternal + [CDFSEISMIC + CDFFIRE + CDFWIND + CDFFLOOD]

= 8.15 x 10-6 + [4.75 x 10-8 + 1.77 x 10-7 + 4.75 x 10-7 + 8.15 x 10-8]

= 8.93 x 10-6

This result indicated that the finding was of low to moderate significance to the risk of

internal initiating events.

Large Early Release Frequency Contribution

In accordance with Manual Chapter 0609, Appendix A, Attachment 1, Step 2.2.6,

Screen for the Potential Risk Contribution Due to LERF, the analyst assessed the

impact on the large early release frequency because the Phase 2 SDP result provided a

risk significance estimation of greater than 7.

Using NRC Inspection Manual Chapter 0609 Appendix H, Containment Integrity

Significance Determination Process, the senior reactor analyst determined that this was

a Type A finding (i.e., a finding that can influence the likelihood of accidents leading to

A3-11

Attachment 3

core damage that is also a LERF contributor). For a pressurized water reactor with a

large, dry containment, like Comanche Peak Steam Electric Station, findings related to

inter-system loss-of-coolant accidents and steam generator tube ruptures have the

potential to impact LERF.

Appendix H, Table 5.1, "Phase 1 Screening - Type A Findings at Full Power," provides

that station blackout scenarios and all other transients, including loss of offsite power

initiators, screen out from further evaluation. These accident sequences are not

considered to be significant to LERF. Therefore, the estimated LERF was calculated

to be less than 6.8 x 10-7. Because the LERF was less than the 1 x 10-6 White/Yellow

threshold, the finding remains characterized as of low to moderate safety significance

(White).