IR 05000348/2006009

IR 05000348-06-009 and IR 05000364-06-009 on 04/25/2006 - 05/16/2006 for Joseph M, Farley, Unit 1, Special Inspection

ML061840350
Person / Time
Site:	Farley
Issue date:	06/30/2006
From:	Shaeffer S M NRC/RGN-II/DRP/RPB2
To:	Sumner H L Southern Nuclear Operating Co
References
IR-06-009
Download: ML061840350 (32)

Text

June 30, 2006

Southern Nuclear Operating Company, Inc.ATTN: Mr. H. Vice President - Farley Project P. O. Box 1295 Birmingham, AL 35201-1295

SUBJECT: JOSEPH M. FARLEY NUCLEAR PLANT - NRC SPECIAL INSPECTIONREPORT 05000348/2006009 AND 05000364/2006009

Dear Mr. Sumner:

On May 16, 2006, the US Nuclear Regulatory Commission (NRC) completed a specialinspection at your Farley Nuclear Plant, Units 1 and 2. This inspection reviewed the circumstances surrounding the failure of all three Unit 1 downstream safety-related main steam isolation valves (MSIVs) to close on April 8, 2006. In accordance with Management Directive 8.3, a special inspection was warranted because the event involved multiple failures in a systemused to mitigate an actual event, involved possible adverse generic implications, and involved repetitive failures of safety-related equipment.The enclosed inspection report documents the inspection findings, which were discussed onMay 16, 2006, with Mr. Randy Johnson and other members of your staff. The determination that the inspection would be conducted was made by the NRC on April 25, 2006, and theinspection started on April 25, 2006.This inspection examined activities conducted under your license as they relate to safety andcompliance with the Commission's rules and regulations and with the conditions of your license.

The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel. Based on the results of this inspection, no findings of significance were identified. In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, itsenclosures, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of theNRC's document system (ADAMS).

SNC2ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (thePublic Electronic Reading Room).

Sincerely,/RA/Scott M. Shaeffer, ChiefReactor Projects Branch 2 Division of Reactor ProjectsDocket Nos. 50-348, 50-364License Nos. NPF-2, NPF-8

Enclosure:

Inspection Report 05000348/2006009 and 05000364/2006009 w/Attachment1. Supplemental Information 2. Farley MSIV Event Timelinecc w/encl: (See page 3)

_________________________OFFICERII:DRPRII:DRSRII:DRPRII:DRPRII:DRPRII:DRPRII:DRPSIGNATURE//RA////RA////RA//NAMEJBaptist:awsMScottCRappDATE6/29/20066/30/20066/30/2006 E-MAIL COPY? YESNO YESNO YESNO YESNO YESNO YESNO YESNO SNC3cc w/encl: B. D. McKinney, Licensing Services Manager, B-031 Southern Nuclear Operating Company, Inc.

Electronic Mail DistributionJ. R. JohnsonGeneral Manager, Farley Plant Southern Nuclear Operating Company, Inc.

Electronic Mail DistributionJ. T. GasserExecutive Vice President Southern Nuclear Operating Company, Inc.

Electronic Mail DistributionBentina C. TerrySouthern Nuclear Operating Company, Inc.

Bin B-022 P. O. Box 1295 Birmingham, AL 35201-1295State Health OfficerAlabama Department of Public Health RSA Tower - Administration 201 Monroe St., Suite 700 P. O. Box 303017 Montgomery, AL 36130-3017M. Stanford BlantonBalch and Bingham Law Firm

P. O. Box 306 1710 Sixth Avenue North Birmingham, AL 35201William D. OldfieldQuality Assurance Supervisor Southern Nuclear Operating Company Electronic Mail DistributionDistribution w/encl: (See page 4)

SNC4Report to from Scott M. Shaeffer dated June 30, 2006.

SUBJECT: JOSEPH M. FARLEY NUCLEAR PLANT - NRC SPECIAL INSPECTIONREPORT 05000348/2006009 AND 05000364/2006009Distribution w/encl:R. Martin, NRRC. Evans (Part 72 Only)

L. Slack, RII EICS RIDSNRRDIPMLIPB PUBLIC EnclosureU. S. NUCLEAR REGULATORY COMMISSIONREGION IIDocket Nos.:50-348, 50-364License Nos.:NPF-2, NPF-8 Report Nos.05000348/2006009 and 05000364/2006009 Licensee:Southern Nuclear Operating Company, Inc.

Facility:Joseph M. Farley Nuclear PlantLocation:Columbia, AL 36319 Dates:April 25 - May 16, 2006 Inspectors:J. Baptist, Resident Inspector - Farley Nuclear Plant (Team Leader)M. Scott, Senior Reactor InspectorApproved by:Scott M. Shaeffer, ChiefReactor Projects Branch 2 Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000348/2006-009 and 05000364/2006-009; 04/25/2006-05/16/2006; Joseph M. FarleyNuclear Plant, Unit 1, Special Inspection.The report documents special inspection activities conducted by a resident inspector and asenior reactor inspector to investigate the failure of three Unit 1 Main Steam Isolation Valves (MSIVs) to close.A.

NRC-Identified and Self-Revealing Findings

No findings of significance were identified.

B. Licensee-Identified Violations

None.

Enclosure

REPORT DETAILS

Farley MSIV Design OverviewEach unit of the Farley plant has three lines of main steam flow in which each line has twoMSIVs installed in series downstream from the safety relief valves outside containment. The upstream and downstream MSIVs, manufactured by Atwood & Morrill (now WEIR), are identicalin construction and their bodies are separated by approximately six inches of piping. The MSIVs are 32-in., 600-lb disk, full-flow, swing-check, nonreturn-type valves with pneumatic actuators. During normal plant operation, the valves are kept open against a spring force by airpressure in the actuator. In the normal open position, the disc is held well out of the steam flowby the air operator. In case of high pressure in the containment, or high steam line flow in coincidence with low-low Tavg or low steam line pressure, the air pressure in the cylinder isrelieved and the valve is closed within seven seconds by action of the spring to prevent the forward flow of steam through the valve. In the event of a steam line break, the MSIVs prevent continuous uncontrolled steam release from more than one steam generator, regardless of whether the break is inside or outside the containment, even when it is assumed that there is afailure of one of the isolation valves. The valve design also incorporates a means to perform an approximate 5% movement partial closure test. This test can be performed with or without steam flow present.

4EnclosureEvent DescriptionOn April 8, 2006, Unit 1 was reducing power in preparation for a scheduled refueling outage. At2:38 a.m., plant operators performed , and observed that downstream MSIVs Q1N11HV3370B(3370B) and Q1N11HV3370C (3370C) did not move from of their open position while MSIV Q1N11HV3370A (3370A) closed approximately 70 percent. The MSIVs were tested at normal operating temperature and pressure (NOT/NOP) with no steam flow as was typical for this test.

Due to all three downstream MSIVs failing to reach the fully closed position within seven seconds, the licensee entered Technical Specification (TS) 3.7.2. Per the applicable LimitingCondition for Operation (LCO), the plant was required to restore the downstream MSIVs to operable status or verify at least one valve was closed in each line. Consistent with plant conditions to support a refueling outage, all upstream MSIVs were already closed which met the LCO. At 7:00 a.m., mechanics were requested to assist in closing the downstream MSIVs with each valve requiring varying degrees of assistance for closure. Condition report (CR)2006103043 was written to document the valve closure problem and the plant continued withthe shutdown. On April 9, the licensee demonstrated the ability of the downstream MSIVs to close using the normal valve actuator.On April 17, the downstream MSIVs were disassembled, but did not reveal an immediate rootcause for the failures. The licensee formalized a root cause investigation team on April 21, and began investigating each MSIV failure. Following substantial root cause investigation, the downstream MSIVs were reassembled and successfully stroke tested on May 17. All MSIVs were opened on May 23, to facilitate plant startup and then fully stroke tested under normaloperating conditions to verify proper reassembly and operation.Following a period of operation at 100% power, on June 28, during partial stroke testing of theUnit 1 MSIVs, MSIV 3370B became stuck in the test position. The licensee locally verified the disk was at the 5% test position and entered a 72-hour LCO for an inoperable MSIV. The licensee attempted various methods including lubrication, packing adjustment, and mechanical assistance to free the disk, but there was no change in position. On June 30, the licensee decided to shutdown Unit 1 to perform troubleshooting and inspection of the MSIVs.Special Inspection Charter Inspection ObjectivesBased on the event and the criteria specified in Management Directive (MD) 8.3, NRC Inci dentInvestigation Program, a Special Inspection was initiated by the NRC in accordance withInspection Procedure 93812, Special Inspection. The inspection charter objectives are listed below and addressed in the identified report sections.1.Develop a sequence of events, including applicable management decision points prior tothe time the MSIVs failed through troubleshooting and repair activities. (4OA3.1)2.Review licensee documents to assess if the licensee had previous failures of MSIVs toclose and also review current and previous maintenance practices for the MSIVs.

(4OA3.2)5Enclosure3.Assess any corrective action the licensee took prior to the event to address MSIVclosure problems and determine if the actions were appropriate and timely. (4OA3.2 and

4OA3.3)4.Assess operating procedures and operator training concerning this scenario anddetermine if the procedures and training were adequate for operators to compensate for the lack of MSIV closure. (4OA3.4)5.As requested by the Regional Senior Reactor Analyst, assist in the collection of datanecessary to support completion of the significance determination process. (4OA3.1)6.Review the licensee's MSIV operability determination for Unit 2. (4OA3.5)7.Review this event for generic safety implications. (4OA3.6)

Summary of Licensee Root CausesBased on the information available at the end of the inspection period, the licensee identifiedthe following could have contributed to the recent Unit 1 MSIV failures.Maintenance Procedures and PracticesNo preventative maintenance replacement schedule nor defined replacement criteriafor MSIV bushings (shaft load bearing components).The orientation of the thrust bearing on the shaft was not specified nor was there anacceptance criteria for inspection of components subject to wear, notably shaft load bearing components. Techniques or tools used for removing and replacing packing. Key/keyway installation was skill-of-the-craft.The maintenance procedure lacks detail and depends on knowledgeable mechanicsto implement successfully.Lubrication of shaft and packing conflict with vendor recommendations for packinginstallation and may have contributed to increased packing drag.DesignThe square keyway design creates a stress riser that makes the shafts susceptible tofatigue cracking under the service conditions of the downstream valves. This was exacerbated by use of keys that were less than the full length of the keyway.Severe service conditions in the downstream valves, due to the turbulence inducedby the upstream MSIVs, was a known condition that affects fatigue and wear factors 6Enclosureidentified in the above causes. Substantial design changes could reduce or eliminatethe severe service environment of the downstream valves.Missed OpportunitiesInvestigations prior to 2002 typically stopped at the first plausible or most visiblepossible cause.In 2002, an investigation into partial stroke failure (CR 2002001026) may havemissed the significance of sticking on the open stop and possibly misdiagnosed the cause of the failure.In 2000 (CR 2000005687) a similar failure occurred, and was corrected by lubricatingthe yoke bushing and shaft. The potential significance of this event was missed although its occurrence was referenced in the 2002 CR.In 2002, the partial stroke test was discontinued based on industry operatingexperience and further opportunity to identify degrading conditions in the shaft support components was lost until the hot as-found surveillance specification wasadded in 2004.The surveillance requirements for these valves consist of the ASME Code requiredCold Shutdown/Refueling (CSD/RF) timed stroke test. Conditions under which this test should be performed were not clearly specified to match as closely as possible, the actual operating conditions under which the valve could be called upon. The test procedure did not establish consistent conditions for testing all valves, precluding good trending.Other than startup and shutdown, these valves are not normally operated. Theprevious partial stroke test requirement was deleted based on industry experience in that MSIV partial stroke testing posed a plant trip risk believed to outweigh the benefits of quarterly partial stroke testing. It is believed that partial stroke testing mitigated part of the drag increase, notably "sticking" packing that develops over time.Due to the uniqueness of the design, and perception that an aggressive PM programwas in place to deal with the then known issues, these valves were not placed in the AOV program for "Flowscan" monitoring.

As a consequence of the above lack of test specification, surveillances that may haveprovided detection of trends, degradation, or incipient failure were not in place.The surveillance test procedure does not contain instructions for equalizing pressureacross a closed MSIV to allow reopening the valve which results in testing the upstream and downstream valves under different conditions.

7Enclosure4.

4OA3 Event FollowupUnit 1 Downstream MSIV Failures.1Inspection Charter Objectives 1 and 5a.

Inspection ScopeThe inspectors reviewed in detail CR 2006103043, documentation of the troubleshootingefforts, and operator logs covering relevant time periods. Cognizant licensee personnel were interviewed and the NRC Resident Inspectors, who had observed the sequence ofevents and the licensee's initial response, were consulted. Additionally, debriefs were held with the licensee to aid in the understanding of key events leading up to the MSIV failures. Relevant risk information was also collected and provided to the NRC seniorrisk analyst to support an evaluation of the risk significance associated with the MSIV failures.b. ObservationsMSIV closure which did not facilitate amethodical approach to understanding the MSIV failures. The inspectors concluded this resulted in a more challenging root cause determination process. For example, thermal expansion appeared to be a contributor as 3370A and 3370B were found in a more closed position on April 8, when the mechanics arrived to close the MSIVs. In hindsight, a quarantine of the failed MSIVs for careful disassembly, inspection, and review of valve ambient conditions could have provided additional opportunities to further develop the root cause of the MSIV failures. The inspectors also noted that a thorough evaluation ofthe MSIV issues had not commenced until April 21, when licensee management identified the CR describing the event had been assigned to the Maintenancedepartment and dispositioned as work orders (WO). The CR was then reassigned to Engineering Support.The root cause team proceeded with troubleshooting efforts and formulated the datainto table format to assist in failure mechanism classification and corrective action completion. Table 1 identifies the MSIV failure mechanisms attributed to the Unit 1 MSIV failures. Based on the results of the root cause, the downstream MSIVs were reassembled on May 15 with numerous new components and cold stroked successfully in the required time. The downstream MSIVs were later hot-torqued and fully stroked at normal operating conditions during plant startup on May 23 with successful results.

8EnclosureTable 1: MSIV Failure MechanismsMechanismValve(s) ActionBearing backwardsBProcedure guidanceKeys wrong sizeA, B, CNew keys and controlsStress riser in keywaysA, B, CNew shafts with radius keywaysTight key on B shaftBKey/keyway control, new keysPacking rubs on shaftA, B, CNew packing configuration toreduce friction; partial stroke testingSevere rubs from yokebushingA, B, CNew bushings all valvesEnd plate clearanceA, B, C (and upstreamvalves)Clearances opened per vendorrecommendations, opened on Unit 2Lubrication on packingA, B, CRevised procedure and briefedmaintenance crewGouges from foreignmaterial entry into valveAReplaced shaft, revised procedureand briefedRubs from spacers andbushingsA, B, CReplaced all spacers and bushingsSevere ServiceConditions identified fordownstream valvesA, B, CEngineering review for potential futuremodificationsThe data in Table 1 demonstrates the complexity of the downstream MSIV failures andthe corrective actions to prevent recurrence. The licensee provided timeline information which was used to develop Attachment 2 for a detailed sequence of events concerning discovery and correction of CR 2006103043. This information, along with licensee MSIV failure data provided by the licensee on June 19, 2006, was provided to the Regional Senior Reactor Analyst to assist in the completion of the significance determination process.Based on observations of the post-failure component inspections, the inspectorsconcluded that the licensee collected detailed information to develop effective future corrective actions. Initial action to preserve valve failure information could have been improved..2Inspection Charter Objectives 2 and 3a. Inspection ScopeThe inspectors performed reviews of selected component's preventive-maintenanceprocedures, vendor documents, completed work orders, and CRs generated for non-conformances to verify that the MSIV maintenance was based on vendor recommendations and appropriate industry operating experience. During these reviews, 9Enclosurethe inspectors focused on potential common mode failure vulnerabilities that could beintroduced by maintenance activities. The team reviewed procedures used to assemble and disassemble the MSIVs as well as procedures used to verify MSIV operability duringnormal and test conditions. A keyword search utilized industry operating experiencerecords to look for cases similar to this event. The inspectors interviewed responsible engineers, supervisors, and other cognizant personnel. The inspectors reviewed maintenance training procedures pertaining to MSIV maintenance to verify that training was consistent with the procedures. Previous corrective actions taken were also evaluated. b. ObservationsThe licensee has a history of operational and maintenance issues regarding thedownstream MSIVs. As early as 1980, the effects of a postulated severe environment in which the downstream MSIVs operate, appears to have resulted in increased internal component failures and repairs. The environment for the downstream MSIVs was believed to differ from that of the upstream MSIVs. The downstream MSIVs experience more turbulence because of their proximity to the upstream MSIV. The MSIVs were installed with six inches of straight piping between the upstream and downstream MSIV bodies and this arrangement does not allow steam flow to settle prior to entering the downstream MSIV. This added turbulence appears to have resulted in several failures primarily as disc-to-disc arm fastener failures, shaft cracking, shaft bending, and shaft binding precluding MSIV closure. Table 2 has been included as a brief history of similar mechanical issues concerning the downstream MSIVs. In contrast, the inspectors identified very few mechanical problems related ot the upstream MSIVs.

Table 2: Similar MSIV Failures at FNPUnit/ValveFailureDateSymptomsCause As Stated In CR2/ 3369B&C09/83Would not movefrom full open.Excessive Packing Friction.2/ 3370A04/86Would not movefrom full open.Excessive friction from shaft seal"O" rings.2/ 3370A06/86Would not moveto test position.Packing Friction and Weak TestActuator.2/ 3370A07/86Would not movefrom full open.Packing Friction and Weak TestActuator.1/ 3370B01/87Closed 60% and then bound.Packing Friction and Zero SteamFlow.1/ 3370C12/92Would not movefrom full open.Improper Assembly of Valve and Main Actuator.

10Enclosure2/ 3370A10/95Would not movefrom full open.Excessive Packing Friction.1/ 3370B06/97Would not gocompletely closed.Packing and Yoke BearingFriction.2/ 3370A10/99Closed 80% and then bound.Worn Thrust Bearing and HardPacking. ( Listed as "Not Uncommon")2/ 3370A11/00Would not moveto test position.Friction From Dirty Main ActuatorStem.2/ 3370A05/01Would not fullystroke.Friction due to Misalignment ofIndicator Plate.1/ 3370B&C05/02Would not moveto test positionValve Friction and Variances inAir Actuator Pressure.1/ 3369C05/03Would not gocompletely closed.Improper Assembly of Valve and Main Actuator.The inspectors noted that several downstream MSIV failures were attributed to packingfriction. The licensee has modified the packing program several times to incorporate improved industry packing arrangements and methods. However, two key areas identified during the current MSIV root cause reviews were 1) the licensee historically was implementing generic MSIV and packing practices including lubricating the shaft and packing during MSIV assembly. The packing vendor, ARGO, identified that this practice would actually increase friction between the ARGO packing and the MSIV shaft; 2) the licensee did not coordinate between ARGO and the MSIV vendor, WEIR, such that packing specifications for this type of MSIV were not communicated and implemented. Since this recent event, the licensee has worked with the packing and MSIV vendors to develop a packing arrangement best suited for the MSIVs.The root cause team also identified that the MSIVs have not been in a testing programthat would identify failure types observed during this incident. The initial testing program to ensure operability was a quarterly test (partial) stroke of all MSIVs. This test verifiedthat the MSIVs would break away from the open position when the main actuator forcewas removed. This test stroke was discontinued in 2002 to minimize the potential of a spurious reactor trip from accidental MSIV closure. Historically, the only other testing of the MSIVs was a full stroke test, in accordance with the Inservice Plan for Pump and Valve Testing, performed as each unit was starting up from an outage. The current failures were identified during the licensee's first Unit 1 implementation of an as-found full stroke test performed at NOT/NOP with no steam flow as the unit was being shut down. This test had been suggested by site personnel to potentially reveal degraded "as-found" conditions of the MSIVs. This test was first performed on Unit 2 during unit shutdown for the October 2005 refueling outage. No Unit 2 MSIV failures were identified.

11EnclosureThe root cause team also identified that maintenance procedure FNP-0-MP-39.0, MainSteam Isolation Valve Disassembly and Reassembly, had deficiencies. With respect to MSIV disassembly, there was no acceptance criteria for inspecting MSIV components as the MSIV was disassembled. This did not provide an avenue for data transmittal from mechanic to engineer and, therefore, did not facilitate a detailed evaluation ofcritical dimensions within the MSIV. Aged and worn components no longer maintaining their dimensional tolerances were identified to have been a portion of the frictional sum of forces which prevented the MSIVs from closing. Additionally, the reassembly instructions lacked sufficient detailed to ensure that the MSIV components were installed in the correct arrangement with proper tolerances. This had a direct effect on the current failure of MSIV 3370B, as the thrust bearing was installed backwards and was identified as the leading contributor to significant shaft cracking and the inability ofthe MSIV to close on demand. At the end of the inspection period, FNP-0-MP-39.0 wasbeing modified to include specific assembly, disassembly, and inspection criteria and willfocus on mechanical tolerances, cleanliness, and guidance when unexpected adverse conditions are encountered..3Inspection Charter Objective 3a. Inspection ScopeThe inspectors reviewed the history of MSIV failures which exhibited similar symptomsto the failures on April 8. The review included pertinent corrective action program documents, maintenance work requests, and interviews with cognizant engineers.b. ObservationsReview of the maintenance history for the MSIVs identified several performanceproblems with the downstream MSIVs. While the licensee did not identify a single predominate root cause for the failures on April 8, it appears that many factors haveresulted in the repetitive MSIV issues. Specifically, the inspector identified that CRs 2000005687, 2001001155, and 2002001026 had overall inadequate problem identification and resolution. The licensee's current Root Cause Team also identified weaknesses in the resolution of previous CRs. For example, the root cause for CR 2006103043 identified that "Investigations prior to 2002 typically stopped at the first plausible or most visible possible cause. In 2002, an investigation into a partial stroke failure may have missed the significance of sticking on the open stop and possibly misdiagnosed the cause of the failure." The licensee's current root cause evaluation team also noted that, due to extensive programmatic changes in the FNP Corrective Action Program since 2002, no programmatic corrective action was recommended for the resolution of issues prior to 2002.Based on historical failures and the licensee's initial root cause conclusions, theinspectors determined the licensee had experienced three MSIV failures since 2000 which exhibited symptoms similar to the failures in April 2006. The inspectors identified a number of missed opportunities the which licensee had to properly identify and 12Enclosurecorrect the failure mechanisms which led to the most recent failures. Table 3summarizes the similar MSIV failures experienced between 1999 and 2006.Table 3: Recent Similar MSIV Failures at FNP 2/ 3370A11/00Would not moveto test position.Friction From Dirty Main Actuator Stem.2/ 3370A05/01Would not fullystroke.Friction due to Misalignment of IndicatorPlate.1/ 3370B&C05/02Would not moveto test positionValve Friction and Variances in AirActuator Pressure.This failure data illustrates three notable examples of missed opportunities for thelicensee to have identified and corrected the actual failure mechanism. The specifics of these examples are discussed below.

!On November 3, 2000, CR 2000005687 was written for Unit 2 due to a failure ofMSIV 3370A to stroke during maintenance test FNP-1/2-STP-21.1. Initial indications of the MSIV failure were that the test actuator would not move the MSIV from the OPEN position to the TEST position and the main actuator was unable to return the MSIV to OPEN. Work Order (WO) 20009003 was written and the licensee suspected binding friction between the MSIV yoke bushing and shaft. The WO indicated that lubrication was applied between these two surfaces and the test stroke was re-performed. An evaluation performed on January 5, 2001, stated that the mainactuator was stuck due to the accumulation of dirt on the actuator stem and the cleaning and lubricating of the stem removed the additional friction and allowed the MSIV to stroke. A second evaluation performed on January 19, 2001, supported the earlier evaluation by stating that the "dirt on the stem at the main actuator bushingswas the root cause." In the CR, the licensee stated that the main actuator couldovercome the opposing forces of the MSIV weight, spring tension, and the added friction at the actuator bushing. It was also stated the MSIV would still perform its design function as the main actuator would be "released" and the MSIV weight and spring tension would be sufficient to overcome the friction from the dirty stem. These statements are inconsistent with the indications present at the time of the failure andno documentation could be identified that would indicate that a formal root cause wasperformed. While it is not possible to recreate the situations which existed at the timeof the MSIV failure, the information identified in the root cause of CR 2006103043 indicated that there was sufficient margin to both open (main actuator) and close(disc weight and spring tension) the MSIV even in the presence of dirty actuatorstems. The inspectors concluded that the November 2000 CR was a key opportunity to investigate and resolve the potential problem.

!On May 8, 2001, CR 2001001155 was written for Unit 2 due to a failure of MSIV3370A to fully stroke during full stroke surveillance FNP-2-STP-45.7. TheMaintenance department was requested to investigate and the licensee concluded 13Enclosurethat the MSIV indicator plate was providing additional friction due to thermalexpansion and preventing the MSIV to fully close. This failure was listed as a maintenance rework item as the MSIV successfully closed during "cold" conditions.

WO 1003978 stated that the indicator plate was adjusted, packing was adjusted, andlubrication was applied to the yoke bushing and main actuator assemblies.

Subsequently, the test was performed satisfactorily and the plant continued with startup activities. CR 20001001155 was subsequently closed due to actions taken.

Discoveries from the current root cause of CR 2006103043 indicated that the frictionavailable from the surface area that the MSIV shaft would come in contact with the MSIV indicator plate, would not be individually significant enough to prevent MSIV closure. Based on the above, the inspector concluded the original evaluation for CR2001001155 was inadequate, in that, an unlikely root cause was determined and further investigations were not made. During the licensee's current evaluation the indicator plate expansion theory was not included in the Table 1 corrective actions.

!On May 4, 2002, CR 2002001026 was written for Unit 1 due to the failure of MSIVs3370B and 3370C to stroke during maintenance test FNP-1/2-STP-21.1. WOs 2002625 and 2002626 were implemented and, contrary to previously identified guidance, maintenance personnel lubricated MSIV stems using WD-40, which resulted in successful MSIV strokes. In 2000, the licensee had identified that "WD-40 provides no lubrication per CR 2000005198 due to the evaporation of WD-40 at the temperatures of the stem in operation. Based upon the associated Action Item (AI)2000201180, the lube PM for the MSIV's that utilized WD-40 was deleted. It was alsounderstood at that time, that any other lubricant would act as a dust and debriscollector and would counteract the intended purpose of lubricating the stem." An apparent cause was performed for CR 2002001026 and it discounted the frictional forces existing from main actuator stem dust/debris as a credible cause preventing MSIV operation. CR 2002001026 did state that the apparent causes could benumerous and consist of actuator air pressure consistencies, friction, or lateral misalignments. However, the CR stated that most likely the cause of the event wasthe performance of the partial stroke itself. The licensee was concerned that the test was causing unnecessary stress and wear on MSIV internal components resulting in MSIV stroke failures.Later in 2002, the conclusion of the apparent cause of CR 2002001026 appeared toshift focus from problem resolution and became supporting information needed to stop performing the FNP-1/2-STP-21.1 MSIV test procedure. The licensee incorporated guidance that the "NRC recommends plants not perform a partial stroketest at power." Section 4.2.4 (Section on Supplemental Guidance on Inservice Testing of Valves, Power Operated Valves, Main Steam Isolation Valves) of NUREG-1482, published April 1995, stated "NOTE: Related to MSIVs, a number ofplants perform a partial-stroke exercise quarterly during power operations." The revised standard technical specifications bases for MSIV surveillance requirementsstated that "MSIVs should not be tested at power, since even a part-stroke exercise increases the risk of a MSIV closure when the unit is generating power." Based on this information, MSIV test procedure FNP-1/2-STP-21.1 was subsequently suspended from use and resulted in the MSIVs only being stroked when 14EnclosureFNP-1-STP-45.7 was required. The current Root Cause Team of CR 2006103043stated, in part, that the MSIV test stroke procedure could have helped mitigate the drag increase from sticking packing which develops over time. The inspectors considered that the removal of this partial test stroke procedure may have had a detrimental effect on the reduction of cumulative friction forces which accumulate over time. The licensee has since temporarily reinstated this partial stroke on both operating units MSIVs at a quarterly periodicity to gather data and aid in determining long term operability throughout the current operating cycles.By the end of the inspection, the licensee's final root cause evaluation for CR2006103043 was completed. Based on the root cause, the inspectors concluded thatthe licensee's corrective actions focused on short term mechanical refurbishment of the failed MSIVs and establishment of baseline data for development of long term corrective actions. An accurate root cause determination for the failed downstream MSIVs is critical not only to fully evaluate the corrective actions for the failures but also to more accurately define potential common mode failure mechanisms for the upstream MSIVs. This can also greatly affect the risk determination for these failures. Therefore, pending final review of the root cause, contributing factors, long term enhancements for these failures, and review of the most recent June 28 th MSIVfailure, this issue will be identified as Unresolved Item (URI)05000348/2006009-001,Repetitive MSIV Closure Failures.

.4 Inspection Charter Objective 4a.

Inspection ScopeThe inspectors reviewed the operating procedures and training provided by the licenseeand also discussed the scenario with plant operators. The inspectors also evaluated whether the licensee's use of night orders was sufficient to inform operators about abnormal MSIV behavior during post-accident conditions. Particular attention was focused on Unit 2 operations during the initial phases of the Unit 1 root cause evaluation.b. ObservationsThe inspectors determined that the licensed operator's overall procedural knowledge ofabnormal MSIV operations was sufficient. Key operators were aware of the history the FNP has had with MSIV problems and the actions required by them if the MSIVs did not operate properly. The training department had been responsive to past MSIV failures and had implemented four separate simulator scenarios in 2003 and 2005 requiring the operators to compensate for inoperable MSIVs. Following the current MSIV failures, the licensee issued a night order on April 22, which identified the MSIV issues on Unit 1 and provided guidance to operators should such an event occur on the operating unit. The night order referenced the proper emergency operating procedures, as well as management expectations, and was understood by the control room operators.

15Enclosure.5Inspection Charter Objective 6a. Inspection ScopeThe inspectors reviewed the operability determination (OD) provided by the licensee todetermine if the conditions which caused the Unit 1 MSIV failure were also present on Unit 2.b. ObservationsThe inspectors reviewed the OD to confirm that the licensee successfully performed the"as found" full stroke test prior to entering the latest Unit 2 refueling outage. During Unit 2 restart from the refueling outage, the licensee successfully performed full strokes of the MSIVs under Hot Standby conditions. When it was identified that the Unit 1 MSIVs had possible common mode failure mechanisms, based on the current failures, the licensee added Appendix 4 to procedure FNP-1-SOP-17.0, Main and Reheat Steam, to test for partial movement of the Unit 2 MSIVs. On April 24, the licensee performed procedure FNP-1-SOP-17.0, Appendix 4, to test (partial) stroke all the Unit 2 MSIVs.

The test stroke was observed by NRC inspectors. The test strokes were successful on all six Unit 2 MSIVs and an operability determination was completed based on theseresults and data collected from the previous Unit 2 refueling outage which ended in November 2005. The licensee concluded that the tests supported operability of the Unit2 MSIV's based on the known conditions and limited root cause evaluations to date.The licensee planned to perform quarterly test strokes of both units' MSIVs until thelicensee gains confidence in the corrective actions taken to date and further evaluation of long-term corrective actions to prevent recurrence. The licensee concluded that the Unit 2 MSIVs would remain operable until the next opportunity to replace key components identified for replacement by the root cause team. This information was supported by the most current inspection of Unit 2 MSIVs, the fact that Unit 2 components are less aged that Unit 1 components, and the incorporation of thequarterly partial test stroke.

.6 Inspection Charter Objective 7a.

Inspection ScopeThe inspectors evaluated this event for potential industry-wide generic implications.

b. ObservationsOne key attribute of the MSIV failure mechanism is that the two MSIVs are in series,oriented in the same direction, and in close proximity. This configuration results in turbulent flow through the downstream MSIVs. The turbulent flow induced severe vibration which may have resulted in the observed failure mechanisms. Therefore, the severity of service conditions of the downstream MSIVs is a condition which may be of generic industry-wide interest.

16Enclosure4OA6Meetings, Including ExitOn May 16, 2006, the inspectors presented the inspection results to Mr. Randy Johnsonand the other members of his staff who acknowledged the unresolved issues. The inspector confirmed that proprietary information was not provided or examined during the inspection.Following the exit meeting, on June 28 during partial stroke testing of the Unit 1 MSIVs,MSIV 3370B became stuck in the test position. The licensee locally verified the disk was at the 5% test position and entered a 72-hour LCO for an inoperable MSIV. The licensee attempted various methods including lubrication, packing adjustment, and mechanical assistance to free the disk, but there was no change in position. On June 30, the licensee decided to shutdown Unit 1 to perform troubleshooting and inspection of the MSIVs.ATTACHMENTS:1.

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee personnel

W. L. Bargeron, Assistant General Manager - Operations

W. R. Bayne, Performance Analysis Supervisor

S. H. Chestnut, Engineering Support Manager

P. Harlos, Health Physics Manager

L. Hogg, Security Manager

J. Horn, Training and Emergency Preparedness Manager

J. R. Johnson, Plant General Manager

T. Livingston, Chemistry Manager

B. L. Moore, Maintenance Manager

W. D. Oldfield, Quality Assurance Supervisor

J. Swartzwelder, Work Control Superintendent

R. J. Vanderbye, Emergency Preparedness Coordinator

R. Wells, Operations Manager

T. L. Youngblood, Assistant General Manager - Plant SupportNRC pers onnel

C. Patterson, Senior Resident Inspector

W. Rogers, Senior Reactor Analyst

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

05000348/2006009-01URIRepetitive MSIV Closure Failures (Section 4OA3.3)

LIST OF DOCUMENTS REVIEWED

DrawingsD-177867, Elementary Diagram Solenoid ValvesD-175033, Main and Auxiliary SteamD-177863, Elementary Diagram MSIV

U-265175, Unit 1 32" 600# WE Steam Swing Disc Trip Valve Air Cylinder Operated

U-265176, Unit 1 32" 600# WE Steam Swing Disc Trip Valve Air Cylinder Operated

U-264996, Unit 1 Disc Arm Assembly - 32" 600# Swing Trip Valve

A-2Attachment 1ProceduresFNP-1-STP-45.7, MSIV and Bypass Valves Cold Shutdown Valves Inservice TestFNP-0-MP-39.0, Main Steam Isolation Valve Disassembly and Reassembly

FNP-0-IMP-213.3, MSIV Actuator maintenance

FNP-2-SOP-17.0, Main and Reheat Steam

FNP-1-STP-21.1, Main Steam Isolation Valve Inservice Test

FNP-1-EEP-0, Reactor Trip Or Safety Injection

FNP-1-EEP-2, Faulted Steam Generator Isolation

U-448660, Valve Packing Manual

FNP-0-87, Maintenance Rule Scoping Manual Design Basis DocumentsFSAR 5.5.5 Main Steam Line Isolation System FSAR 10.3.9 Main Steam Isolation Valves Completed Work Orders (WOs) and Work Requests (WRs)

2052705201,

98006279,

96000217,

533642,

99007782,

99001608,

1003978,

20009003,

20009008,

2002625,

2002626,

98006279,

98006282,

98005037, 504662Problem Evaluation Reports

IR 86101.2, MSIVs on A&C Steam Lines did not go closed

IR 86147.2, MSIV 3370B Failed to close

IR 86224.2, Failure os MSIV 3370A to test stroke

IR 86287.2, MSIV 3370A would not close without mechanical assistance

IR 87011.1, MSIV 3370 B would not fully close

IR 92432.1, During performance of

STP-45.7 MSIV 3370C did not close from MCB

IR 95240.1, MSIV 3369A and 3370A would not go closed from the MCB

IR 96374.2, MSIV 3370B failed STP-21.2

IR 97215.1, MSIV 3370B failed to fully close during STP-45.7

IR 98381.1, MSIV Q1N11V002A has a crack in keyway area

IR 99681.2, MSIV 3370A failed to close

IR 99772.2, Excessive wear on MSIV's actuator stems

CR 2002002130, MSIV 3370C would not move

CR 2003000777, MSIV 3370B shaft cracking

CR 2003001092, MSIV 3369C would not completely closeVendor ManualU-258570, Instruction Manual, Unit 1 Main Steam Swing Disc Trip ValveU-418291, Part Numbers for Disc Arm 32" 600# Swing Trip ValveOther DocumentsFarley Nuclear Plant System Monitoring PlanLER 83-039, Failure of three MSIVs to close.

LER 84-004-00, MSIV Shaft Indications(FNP)

LER 95-008-00, Loop A Main Steam Line Isolation Valves Fail To Close.(FNP)

LER 92-006-01, Main Steam Isolation Valve 1MS-2018 To Fully Shut(PB1)

LER 84-002-00, MSIV Excessive Closure Time(IP2)

LER 84-005-00, Two MSIVs Failed To Close Within Required Time Limit(IP2)

LER 87-002-00, Main Steam Isolation Valves Failure To Close(IP2)

A-3Attachment 1LER 89-002-01, Main Steam Isolation Valve Fails To Close(IP3)NRC Information Notice 80-16 and Licensee ResponseNRC Information Notice 83-54 and Licensee ResponseMSIV Valve Packing Data Sheets Main and Reheat Steam Training Text, OPS-52104A

Main Steam System-System Health Reports 2003-Present Letter: WEIR Valves to Alabama Power Company dated 4/1/2002

2Farley MSIV Event TimelineApril 29, 1980Information Notice 80-16 published identifying binding of Main Steam swing disc check andisolation valves. Causal factor was excessively tight shaft packing preventing freedom of disc movement.May 6, 1980FNP response to

IN 80-16 stated that FNP has had no problems with packing binding. FNPstated that the valves are tested quarterly and that the absence of problems could be due to theuse of Grafoil packing which requires a minimal amount of packing gland adjustment. October 19, 1983LER 83-39 Issued, Farley Unit 2 during a unit shutdown (Mode 3) had two valves that failed toclose due to packing binding; corrective action was to repack the valves on both units each refueling with the latest vendor recommendations.

The Unit 1 valves were stroked at a chronologically close shut down (one month later) to prove their functionality.March 21,

198484-04, Farley Unit 1 found cracks ranging from 1 to 13 inches in length on three 410 stainlessMSIV shafts during an outage; corrective action was to replace all Unit 1 shafts with 17-4 PH

material (A564 GR630).

In 1983, the Unit 2 shafts had been inspected and two shafts were replaced due to being bent (17-4 PH material) and the remaining population of shafts had no obvious problems.

The remaining Unit 2 shafts were replaced the next outage in 1985.April 5, 1986Unit 2 MSIVs 3369C and 3370A failed open during demand in Mode 3 during downpower. Cause: Excessive packing friction

Resolution: Valves packing modified.June 8, 1986Unit 2 MSIV 3370A failed open during test stroke post plant trip.

Cause: Packing friction &degraded test actuator.

Resolution: Ordered new test actuator and increased test frequency to monthly for 3370A.July 19, 1986Unit 2 MSIV 3370A failed open in Mode 4 during downpower.

Cause: Degraded test cylinderand packing friction.

Resolution: Valve repacked.

1987Original packing arrangement consisting of 20 packing rings is replaced with 5 Chestertonpacking rings and a spacer. January 18, 1987Unit 1 MSIVs 3370B failed 60% closed in Mode 3 during full stroke request. Cause: Degradedtest cylinder and packing. Resolution: Modified packing arrangement.

A-2Attachment 2

1989FNP ordered double hardened MSIV shafts (ASTM A564 Grade 630

HTC 1150M) Moremalleable material that should not crack.

No upstream MSIV shafts have been replaced with the new material.December 1, 1992Unit 1 MSIV 3370C failed open. Cause: Valve reassembly issue regarding bushing alignmentwith actuator arm. Resolution: Procedure

FNP-0-MP-39.0, Main Steam Isolation ValveDisassembly and Reassembly, changed and mechanical maintenance training. October 1993Unit 2 downstream MSIV 3370B and 3370C shafts replaced

1993-1995The licensee changed MSIV packing type to current configuration.

June 1995Unit 1 MSIV 3370C shaft replaced.

September 16, 1995Unit 1 MSIV 3369A and 3370A failed open in Mode 3 during test stroke request .

Cause: Airwould not bleed off of valve actuator.

Resolution: Improved procedural control of MSIV testing.

LER 95-008-00 written.June 3, 1997Unit 1 MSIV 3370B failed mostly closed in Mode 2 during full strike request.

Cause: Excessivefriction on valve.

Resolution: Modified packing arrangement and lubed Yoke bushing.October 1998Unit 1 MSIV 3370A shaft replaced.

October 23, 1998Unit 1 3370A found crack in shaft actuator keyway in Mode 6 during routine valve inspections. Cause: Fatigue failure.

Resolution: Replaced shaft. October 1999Unit 2 MSIV 3370A shaft replaced.

October 16, 1999Unit 2 MSIV 3370A failed 80% closed in Mode 3 during testing.

Cause: Packing friction due tohard packing and thrust bearing worn. Identified as "not uncommon".

Resolution: Worn parts replaced.March 11, 2000Unit 2 MSIV 3370A failed to move in Mode 1 on test stroke request.

Cause: Dirty main actuatorstem causing friction.

Resolution: Cleaned main actuator stem and lubricated

A-3Attachment 2August 5, 2001Unit 2 MSIV 3370A failed to move in Mode 2 on full stroke request.

Cause: Indicator platebinding due to thermal expansion.

Resolution: Plate repositioned. May 4, 2002Unit 1 MSIVs 3370B and 3370C failed to move in Mode 3 during test stroke.

Cause:considered the partial stroke test itself.

Resolution: Requested deletion of procedural requirements to perform partial-stroke testing. March 2003Main Steam system health report identified divergence between both units upstream (23 of 48)and downstream (41 of 48) disk assembly and shaft ( 6 of 6 downstream) replacements.

The apparent cause mentions the flow disturbances of steam as it passes the upstream MSIV as felt on the downstream valves.

However, as identified in t he system health report, the licenseedetermined that increasing the distance of the downstream MSIV to the upstream MSIV may be cost prohibitive.

As such, this action was not recommended at this time. April 2003Unit 1 MSIV 3370B shaft identified cracked. Replaced shaft.

May 1, 2003Unit 1 MSIV 3369C failed to completely close in Mode 2 during full stroke testing. Cause: Valvereassembly issue similar to that in 1992 causing misalignment between actuator arm linkage and air actuator. Resolution: Assembly corrected, training of mechanics, and increased mechanical supervisor oversight.Farley Unit 1 Timeline of Current Events (Post Failures)April 8, 2006 02:38 (test completed)Operations performed

FNP-1-STP-4.7, MSIV AND BYPASS VALVES COLD SHUTDOWNVALVES INSERVICE TEST.

CR 2006103043 was written to document failure. 3370 A - would not fully stroke closed (70% closed and would not reopen)

3370 B - would not close (remained fully open)

3370 C - would not close (remained fully open)April 8, 2006 07:00Maintenance is called to assist in closure of downstream MSIVs. Found "C" valve full open, "A"90% closed, and "B" 75% closed.

Lubricated actuator stem with

WD-40 (removed dirt and grime off stem).

Below is a list of events during this evolution:3370C - would not stroke from MCB. "Bumped" clevis and valve went 100% closed.

3370B - "Bumped" clevis; did not move as easily as "C".

Incrementally bumped completely shut.

3370A - Found 90% closed.

Would not move open or closed.

Likely drifted closed from as found 70% due to cooling.

A-4Attachment 2April 9, 2006 12:00

Maintenance Team Leader and OPS Shift Manager observed stroke of downstream MSIVs.

All

MSIVs stroked SAT after cooled down and no dP on steam lines.

Also, system engineerinspected actuators and clevises in the field.April 11, 2006

System Engineer and Component Engineering discuss possible problems with valves, possible causes, and root cause status.April 12, 2006Licensee contacted Kaman Industrial Technology to discuss actuators supplied to FNP.

April 13, 2006Licensee continued conversations with Kaman to determine information about lubrication, seals,and springs.

Licensee contacted WEIR Valve about previous shipments of actuators.April 17, 2006

Licensee started with disassembly of downstream valves.

Inspected upstream MSIVs and determined no serious hardware issues. See Following Results.

3369A, B, and CNo deficiencies found

3370A2 upper nuts loose2 lower nuts SAT

Arm centered Stroke SAT

Disc Seat SAT (Feeler gauge - 0.003 would not fit)

3370BAll nuts SAT (no looseness)Arm centered Stroke SAT (squeal - dry packing)

Disc Seat UNSAT (Feeler gauge - 0.008 would fit)

3370CUpper and lower left nuts - SATUpper and lower right nuts - loose Arm centered Stroke SAT (squeal - dry packing)

Disc Seat SAT (Feeler gauge - 0.004 would not fit 0.005 would not fit)Licensee secured used MSIV actuators for investigation.Licensee continued investigation of the actuator (phone conversations) by contacting MillerFluid Power.

Discussed lubrication of the actuator and Viton versus Buna-N seals.

Licensee found

CR 2006103043 dispositioned to WOs by Maintenance. Requested CR be backed up and assigned to ES.

A-5Attachment 2April 18, 2006: Licensee Component Engineering reported to site.Disassembly continued on downstream as well as upstream valves.April 19, 2006: WEIR Valve Representative reported to site.MSIV disassembly continued.

Licensee requested Work Order to determine actuator leak by I&C.April 20, 2006Licensee disassembled 3370B actuator. Cylinder showed no signs of binding. It appears thespring is boxed at 7.125 on the open position, which is only 1.125 from the solid boxed height of6 inches.

WEIR vendor began Faro arm measurements of upstream valves.April 21, 2006WEIR vendor continued Faro arm measurements and provided shim measurements for cutting.Component reviews began in the MSVR. The outboard shaft arm bearing in the support bracket showed signs of scoring that correlated with the three blows that the maintenance had given thevalves.

Licensee found letter WEIR had provided to the site in 2002 that allowed the bearing ID to be opened 0.030 to provide for an isolated failure experienced that shutdown. NRC Briefing with Region II regarding operability of Unit 2. Licensee stopped work on Unit 1 MSIV's pending a full root cause investigation.

Licensee requested WEIR resources for help in the investigation.

Licensee looked for any discontinuities in the three upstream valves. Retrieved pictures of components in the field.April 22, 2006Licensee reviewed operation of valves, emails from the control room on events, and determinedwhat possible areas might bind the shaft.

Maintenance assembled in shop the new 3369B disc with the cut shims.

Engineering for WEIR Valves and controls arrived. April 23, 2006 The seat gap of 3369B disc assembly was set. Licensee visually inspected the air cylinders that came off of all six unit one valves.

Licensee gathered and reviewed all of the data with WEIR.

Developed a list of all the possible areas that might have contributed to the valve not fullystroking. April 24, 2006Licensee interviewed the mechanics that were present during valve closure assistance.Licensee orchestrated total disassembly of downstream valves.

Licensee found 3370B shaft cracked.

Partial stroke test completed SAT on all Unit 2 valves. Resident inspectors locally observed all Unit 2 MSIVs performed test stroke.

A-6Attachment 2April 25, 2006 Licensee assembled the 3369A and 3369C modified disc assemblies.Licensee performed visual inspections of all the downstream valves and took photos of all the components.

Licensee witnessed the packing assembly for C upstream valve.

Licensee discovered that the gland for 3370A valve had gouges on the head that may havebeen a source for the material that was stuck to the shaft in the housing bushing.

Licensee reviewed shaft replacement

WOs.April 26, 2006Licensee started reviewing and assembling root cause information.Inspectors investigated previous and present shaft cracking on downstream MSIVs.

Licensee prepared cracked shafts to send to Birmingham to investigate stress fractures. Also prepared 3370C MSIV actuator to send to WEIR for investigation.

Tested MSIV actuators to determine spring constant.

Wrote MSR for MSIV shaftsApril 27, 2006Licensee continued root cause analysis*Metallurgical stress analysis

• Additional inspections on 3370B components (disc arm interfaces and bushings)

• Reverify dimensions

• Validation of packing programAfternoon discovered thrust bearing backwards in 3370B

Wrote MSR for gland and gland plateApril 28, 2006Licensee continued root cause analysis*Metallurgical stress analysis continued in Birmingham

• Investigated effects of reverse bearing (question in to vendor)

• Investigated packing (AOV packing expert and vendor)Change order request issued for chamfer keyways on shaft (Minor Mods group to perform ED)

Conference call with Metallurgical team in Birmingham about shafts. Determined to be high cycle fatigue (torsional) April 29, 2006Licensee reviewed operating history, surveillances, procedures, and original design calculation(1971)

Initiated question list to address keys/keyways, design changes.

Licensee drew out picture of shaft and components to incorporate measurementsApril 30, 2006Licensee took measurements of key/keyways, shaft, and bearingsReviewed

MDC 03-0-9886 on MSIV shaftsMay 1, 2006Licensee discussion with WEIR valve, requested original calc from 1971 to be revised.Requested assistance with responses to

NRC questions.Discussion with Metallurgical staff in Birmingham regarding keys, bearings, etc.

A-7Attachment 2Walk through

FNP-0-MP-39 with mechanics to ensure issues coming out of root cause arebeing addressed. Worksheet written to assist mechanics during reassembly.

Verified bushing and spacer information for MSR.

Reviewed existing packing procedure (U447660).May 2, 2006Reviewed existing packing software program.Licensee discussed NRC questions with WEIR valve, including actuator calculations WEIR

willsupply complete updated calc today.

Brief discussion with licensee regarding actuator calculation.May 3, 2006Licensee continued interviews with personnel familiar with the packing program history. Determined possible packing data.

Licensee reviewed maintenance disassembly/reassembly procedure

FNP-0-MP-39 step by step with mechanics. Documented numerous comments.

Licensee initiated WO for repairs on downstream MSIVs and staged parts.May 4, 2006Licensee started incorporating changes into the maintenance procedures.

Discussed changeswith I&C tech.

Licensee continued to work on packing configuration issues.

Licensee reexamined pictures to determine all wear on the components (upstream versus downstream) was identified.May 8, 2006Licensee continued Root Cause report construction.Licensee held conference call with metallurgist to answer questions and finalize shaft report.May 9, 2006Licensee continued Root Cause report construction.Held conference call with WEIR, Farley ES, and Component Engineering to discuss parts ordered and packing.May 10, 2006Licensee continued Root Cause report construction.Licensee discussed potential testing (UT, PT, etc.) of MSIV shafts during upcoming outages with Corporate Materials and Inspection Services group to determine best course of action for early detection of cracks.May 12, 2006Continued reassembly of B and C valves and started assembly of A valve.Licensee continued Root Cause report construction.

Root Cause Team presented Preliminary Root Cause to management.May 13, 2006Licensee continued reassembly of A, B, and C MSIVs.

Finished installing all internal parts and bonnet.

A-8Attachment 2May 14, 2006Licensee worked on installing support plates and maintenance stroked valves several timeswith temporary air.May 15, 2006Stroked downstream MSIVs from MCB with the following stroke times:*3370A - 3.37 sec

• 3370B - 2.49 sec

• 3370C - 2.41 sec

MCR Meeting to discuss root cause and corrective actions and assign due dates.May 17, 2006FNP-1-STP-45.7 Performed. All MSIVs successfully stroked.

May 23, 200610:25am - Opened all MSIVs as part of plant startup.10:39am -

FNP-1-STP-45.7 Performed. All MSIVs successfully stroked.

Unit 1 restartedMay 24, 2006FNP-1-SOP-17.0 Appendix 4 completed, Test stroke of Unit 1 MSIVs completed satisfactorily.