ML17229A061
| ML17229A061 | |
| Person / Time | |
|---|---|
| Site: | Saint Lucie  |
| Issue date: | 09/30/1996 |
| From: | Culpepper D, Heffelfinger L, Scarola J FLORIDA POWER & LIGHT CO. |
| To: | |
| Shared Package | |
| ML17229A060 | List: |
| References | |
| NUDOCS 9610080068 | |
| Download: ML17229A061 (33) | |
Text
FPL PSL SelfAssessment September 30, 1996 Jim Scarola, Plant General Manager (Team Leader)
David Culpepper, Chief, Engineering Assurance Eim Heffelfinger, Protection Services Supervisor Robert De La Espriella, QA Supervisor (Facilitator) 9610080068 961003 PDR ADOCK 05000335 P
~ ~
~
PSL SelfAssessment Acknowledgment Signatures This a'ssessment was completed withthe fullsupport ofplant personnel and station supervisors and
- managers, and was led by a team ofPSL plant stafF, as acknowledged below.
PSL PLANTGENERALMANAGER:
SELF ASSESSMENT TEAM:
J. Searola, Team Leader LX lf
.C r
R J. De Esprielia, Team Facilitator NOTE This report focuses on identifying the fundamental weaknesses at St. Lucie, and was not intended to be a balanced account ofstrengths and weaknesses.
While some strengths and many positive initiatives were documented, they were provided for historical and contextual perspective.
To provide a balanced view ofoverall station performance would require additional efForts to document other strengths and positive aspects ofPSL performance that were not identified as part ofthis self-assessment's self-critical mission.
~ ~
TABLEOF CONTENTS EXECUTIVESUMII~Y INTRODUCTION METHODOLOGY IDEN'HFIEDWEAKNESSES I.
Complacency..
II.
Training III.
Self-assessment IV.
Corrective Actions V.
Accountability..
VI.
Programs/Proces sesfProcedu res VO.
Change Management VIII.
Communications 10 10 12 15 16 18 21 25 27 PSL MANAGEMENTINITIATIVES.
29
PSL SelfAssessment EXECUTIVE
SUMMARY
The performance and operating history ofthe St. Lucie nuclear station Rom 1984 to 1994 earned it one ofthe best reputations among nuclear plants world wide. During that decade, St. Lucie set world records for continuous plant operations, receiving the highest marks Rom regulators and praise and recognition fiomthe nuclear industry. For example, in 1985, St. Lucie Unit 1 (PSL-1) was the first reactor ever to record an average annual load factor ofover 100%.
From 1989 to 1992, PSL-2 successfully ran for 427 consecutive days, and following a successful outage, ran for another 502 consecutive days which, at that time, set the world record for continuous operation.
The NRC graded St. Lucie a perfect 1.00 forthe SALP evaluation periods ending in 1992 and 1994, noting that "St. Lucie continued its'history of exceptional performance, attaining superior ratings in all SALP functional areas forthe second consecutive SALP period." INPO also rated St. Lucie's performance as a "1" forthe three consecutive grading periods ending in June 1990, April 1992, and August 1995, "inrecognition ofthe achievement ofexcellence in nuclear plant performance."
Beginning in August 1995, a series ofsignificant problems and events indicated that St. Lucie plant performance had declined.
On August 8, 1995, during shutdown conditions, an inadvertent main steam isolation signal (MSIS) was generated and reactor coolant pump seals were damaged, because operators did not meet procedure requirements.
Atthis time, it was also discovered that the PSL-1 power operated reliefvalves (PORVs) had been improperly assembled during the 1994 outage, and were inoperable throughout the operating cycle.
On August 17, 1995, 10,000 gallons ofborated water was sprayed into the PSL-1 containment through the containment spray (CS) system, when that system was inadvertently cross connected with the shut down cooling system.
EfFective corrective actions were not taken to preclude system reliefvalves fiomliftinginappropriately on three occasions in early 1995, and in August 1995, the shutdown cooling system relieflifted again, releasing 4000 gaHons ofreactor coolant into the Unit 1 pipe tunnel. Later in the year, the PSL-2 reactor pressure vessel 0-rings failed to seal properly for approximately the fifthtime in the history ofthe Nuclear Division, causing significant delays in the refueling outage.
The latest in this series ofevents occurred on January 22, 1996, when an operator error resulted in an inadvertent dilution ofthe reactor coolant system at PSL-1.
PSL plant management determined that a critical self-assessment ofthe operation ofPSL was needed, to determine the cause for this decline in performance, so that PSL could regain its status as one of the finest plants in the nuclear industry. On April26, 1996, the PSL Site Vice President directed the Plant General Manager to conduct a broad self-assessment ofthe operation at PSL, to identify the fundamental (most limiting) weaknesses which have caused a decline in PSL performance.
This assessment covered the areas of safety assessment/corrective
- actions, operations, engineering, maintenance, plant support, and management policies. It was conducted from May to October 1996, and required approximately 4000 man-hours to complete.
The assessment period covered approximately two and a halfyears ofoperation (document reviews &omJanuary 1994 through April 1996, and observations from May through June 1996).
The assessment team identified eight fundamental (limiting)weaknesses that have caused a decline in performance at St. Lucie.
Com lacen: Lack ofmana ement reinforcement ofcontinuous im rovement establishment ofchallen in pals and inco oration ofbest indust ractices.
During its many years as a leader in the nuclear industry, the PSL management team became comfortable with its performance.
Near the end ofthe 1980s, PSL had efFectively isolated itself&om its peers, and focused on developing its own processes.
PSL plant management did not readily embrace the improvements or good practices developed elsewhere in the industry, and was not very receptive to suggestions for improvement &omoversight groups, regulators and the industry.
The PSL culture graduaHy developed into one ofcomplacency.
Trainin; Lack ofsufficient ownershi offormal trainin b
the line or anization and lack of ownershi ofline erformance b trainin
. Formal training at PSL historically focused on those programs specifically emphasized by the regulator and accredited by INPO, and those training-programs continued to be adequate.
However, the training programs lacked sufficient ownership by the plant departments, and the training department did not take sufBcient ownership in the'performance of its trainees on-the-job. Additionally, the Training Department did not receive formal feedback &omthe plant line organization to allow timely identification ofweaknesses and new training needs, and to identify those training elements and techniques which were particularly useful.
Self-assessment:
Lack ofem hasis on identi n
and correctin roblems at the worker and su ervisor levels first and second levels ofdefense of uali
. Self-assessment was not well defined in the Nuclear Division, and PSL plant management did not set clear expectations for the performance ofroutine self-assessment or emphasized the value offormal self-assessment.
AtPSL, implementing procedures and training on self-assessment were not established for all departments.
Self-assessments were mostly conducted to address significant events, but self-assessment did not become a part ofthe normal plant routine to identify and correct problems at PSL.
Corrective Action: Less than effective im lementation ofcorrective action rocesses and lack ofcorrective action followthrou
. PSL corrective action processes (e.g., the site wide process for documenting problems, root cause and other processes/procedures to analyze problems, and corrective action tracking methods) provided a solid foundation which was generally used to effectively investigate and correct consequential (high visibility)problems and events.
However, the same processes were not always implemented efFectively to address problems that were less visible or oflesser consequence, and the analysis ofthose problems did not consistently identify underlying causal factors.
As a consequence, the resulting corrective actions were sometimes narrowly focused, symptom
- based, inefFective in preventing recurrence, and were often not applied to other programs, processes, procedures, or equipment that suffered from the same weaknesses.
Corrective action tracking and
~ ~
PSL, SelfAssessment trending processes did not typically monitor the effectiveness of corrective actions, or effectively identify adverse trends.
Lastly, corrective actions were seldom provided to correct self-assessment deficiencies and corrective action weaknesses when self-disclosing problems and repeat events occurred.
Accountabili:
Lack of consistent assi ent and/or enforcement of individual accountabilities.
PSL management did not clearly state expectations and hold individuals accountable for meeting those expectations.
Implementing procedures were less comprehensive and less specific than that required for effective establishment of accountability.
Plant personnel were not held accountable for their performance by supervisors and managers, and as a result, weaknesses developed in the areas ofattention to detail, procedural adherence, configuration control, and other areas ofresponsibility.
Pro rams/Processes/Procedures:
Lack of em ha is on and accountabili for im ortant ro s
rocesses or rocedures.
The PSL events of 1995 and 1996 revealed weaknesses in a variety of the programs, processes and procedures used at PSL.
Knowledge based procedures were in use for many years, relying on experienced, trained personnel to properly perform tasks despite limited procedure detail. Personnel performance and job completion were valued more than having correct procedures.
Inadequate procedures and processes were "worked around" and were not seen as a significant problem for the plant. Program performance has suffered due to a lack ofprogram accountability.
Chan e Mana ement:
Less than ade uate im lementation f chan es.
Significant organizational and administrative changes were implemented at PSL in 1995 and 1996.
However, after years ofstable operations, the PSL team was not accustomed to change, and had not developed the "change management" skills needed to assure organizational and process changes were successfully implemented.
As a result, the significant changes were not carefully evaluated and managed to ensure that the organizations transitioned smoothly into their new areas ofresponsibility. In some cases, continuity was not maintained for critical functions ofthe organization.
Communications: Less than ad uate communications contributed to a lack ofcommon focus and cohesiveness in the or anization.
Communication at PSL suffered because the importance ofclear written and verbal communications was not emphasized, and methods for communicating issues and expectations up and down the management chain were not well defined.
As a result, a number of station performance problems 'developed due to communications weaknesses.
At the time of this report, it is recognized that St. Lucie has undertaken numerous changes in personnel, processes and procedures over the past year. Between August 1995 and September
- 1996, most key managers were replaced at St. Lucie, including the Site Vice President, the Plant General Manager, the Operations Manager, the Work Control Manager, the Engineering Manager, the
PSL Seg Asscssmees Services Manager, the Licensing Manager, the Business Systems Manager, the Materials Manager, and the Training Manager. Oftwelve key managers replaced since 1994, all but two have worked the majority oftheir careers outside ofSt. Lucie.
The St. Lucie experience has provided valuable lessons for FPL as well as the nuclear industry. Self-assessment, combined with a robust corrective action program, are essential to achieving and maintaining good perfoimance.
PSL must not rely solely on regulators or oversight groups to grade its performance.
Satisfying minimum regulatory requirements assures that public health and safety are maintained.
However, meeting minimum regulatory requirements does not guarantee that the facility will perform well as a business entity.
NRC and INPO ratings provide for high level comparisons between facilities, but they are not good enough for early identification of declining trends. It is up to PSL to develop those essential process indicators to monitor performance, to perform critical self-assessments that identify limiting fundamental weaknesses, and to provide effective corrective actions such that those weaknesses do not manifest themselves in events.
To protect public health and safety, and to succeed as a business organization, PSL must have sound pro~ and procedures, maintain aMgh state ofequipment reliability and availability, and maintain a highly trained and accountable workforce.
~ I
~
INTRODVCTION PSL SdfAssessment DETAILS The performance and operating history ofthe St. Lucie nuclear station &om 1984 to 1994 earned it one ofthe best reputations among nuclear plants world wide. During that decade, St. Lucie set world records for continuous plant operations, receiving the highest marks &om regulators and praise and recognition &om the nuclear industry.
Representatives
&om the US and international nuclear industry &equently visited St. Lucie to benchmark their performance against St. Lucie nuclear plant operations and processes.
St. Lucie Unit 2 (PSL-2) began commercial operation in late 1983, and FPL received industry recognition for completing its construction within six years, on time and under budget. During its first cycle ofoperation, PSL-2 achieved a 92.5 % capacity factor and attained a 203 day continuous run at power. The NRC ranked PSL-2 first in initialpower operation among new US nuclear plants during 1983-1985. This ranking was based on statistical analyses ofthe first 12 months ofoperating experience using reactor protection system actuations, engineered safety feature actuations, safety system performance, radiological releases, and external events. In 1984, the first refueling outage for PSL-2 was completed in less than 38 days, which was in itselfan achievement that was recognized with a cover stop on Nuclear News, a national publication. In 1985, St. Lucie Unit 1 (PSL-1), which started commercial operation in December 1976, was the first reactor ever to record an average annual load. factor of over 100%, which received national recognition &om Nuclear Engineering International. From 1984 to 1986, the average NRC Systematic Assessment ofLicensee Performance (SALP) score for St. Lucie was 1:40, compared to an industry average of 1.70 (SALP scores range
&om 1 to 3, with 1 being the highest).
In 1986 and 1987, PSL-'2 achieved a capacity factor of98.3% with the unit on-line for all but 8 days ofa 485 day cycle. In 1988, PSL-2 never went off lin, and was recognized in Nucleonics Weekly for achieving a 100% capacity factor. The single and multiple unit rankings ofSt. Lucie placed these units at the top ofUS nuclear plants for capacity factor in the late 1980s. In 1989, FPL was the first company in the world outside ofJapan to be awarded the Deming Prize by the Japanese Union of Scientists and Engineers forits achievements in Total Quality Management.
The pinnacle ofSt. Lucie performance occurred from 1989 to 1992, when PSL-2 successfully ran for 427 consecutive days, and (followinga successful outage) ran for another 502 consecutive days which, at the time, set the world record for continuous operation.
1 The average NRC SALP scores for St. Lucie improved from 1.14 in 1989, to a perfect 1.00 for the SALP evaluation period ending in 1992.
The NRC again assessed St. Lucie with a perfect 1.00 SALP rating for the period ending on January 1, 1994, noting that "St. Lucie continued its history ofexceptional performance, attaining superior ratings in all SALP functional areas for the second consecutive SALP period. This continued high level ofperformance resulted from a dedication to excellence and teamwork by those associated with the facility. It was fostered by proactive
PSL SelfAss~~
ement setting high standards ofsafety performance, and also providing the resources necessary to attain those standards."
INPO also rated St. Lucie's performance as a "1" (INPO ratings range
&om 1 to 5, with 1 being the highest) for the three consecutive grading periods ending in June 1990, April 1992, and August 1995, "in recognition ofthe achievement of excellence in nuclear plant performance."
For the three year period of1993-1995, the St. Lucie capacity factors declined slightly below that of the industry average.
However, in 1994, PSL-2 completed a successful 35 day refueling outage, shortest to date among CE plants.
During that three year period, St. Lucie was cite or approximately 8 NRC violations and filed with the NRC approximately 16 Licensee Ev'ent Reports
~~'
both relatively low numbers for the industry. In May 1995, the NRC wrote to the s) per year, President ofFPL's Nuclear Division, forwarding the results ofa review ofrecent performance a e
St. Lucie facility, stating that "at St. Lucie, we noted that there were no indications of declining performance."
B Au t 1995 a series ofsignificant problems and events indicated that St. Lucie plant performance had declined.
On August 8, 1995, during shutdown conditions, an inadvertent main steam isolation signal (MSIS) was generated and reactor coolant pump seals were damaged, because operators did not followprocedure requirements.
At this time, personnel also discovered that the PSL-1 power operated reliefvalves (PORVs) had been improperly assembled during the 1994 outage, and had been inoperable throughout the operating cycle.
On August 17, 1996, 10,000 gallons of borated water was sprayed into the PSL-1 containment through the containment spray (CS) system, when that system was inadvertently cross connected with shut down cooling. EfFective corrective actions were not taken to preclude system reliefvalves &omliftinginappropriately on three occasions in earl 1995, and in August 1995, the shutdown cooling system relieflifted again, releasing 4000 gallons ofreactor coolant into the Unit 1 pipe tunnel. Later in the year, the PSL-2 reactor pressure vessel 0-rings failed to seal properly for approximately the fifthtime in the history ofthe Nuc ear Division, causing significant delays in the refueling outage.
The latest in the series ofevents occurred on January 22, 1996, where an operator error resulted in an inadvertent dilution ofthe reactor coolant system at PSL-1. Following an extensive evaluation ofthe event, and given the trend of performance problems since August 1995, PSL plant management determined that a criti se-assessment of the operation ofPSL was warranted, to determined the cause for the decline in performance, such that PSL could regain its place among the best plants in the nuclear industry.
On A ril 26, 1996, the PSL Site Vice President directed the Plant General Manager to formally conduct a broad self-assessment ofthe operation ofPSL, to identify the fundamental (most limiting) weaknesses that have caused a decline in PSL performance., The assessment covered the areas o safety assessment/corrective
- actions, operations, engineering,, maintenance, plant support, an gement policies. FPL management provided substantial resources and support to the Integrated Performance Assessment, dedicating a team of3 FPL employees to completmg the assessment, an supplementing the team with additional personnel from Turkey Point, FPL's Juno Beach of5ces, an the nuclear industry.
The assessment was conducted from May to October 1996, and required
PSL SelfAssessment approximately 4000 man hours to complete.
The assessment period covered approximately two and one halfyears ofoperation (document reviews from January 1994 to April 1996, and observations Rom May to June 1996).
METHODOLOGY Phase I: Initial Assessment The purpose of the first phase of the assessment was to develop an initial assessment of the performance ofSt. Lucie operations.
From May to July 1996, the Self Assessment Team performed document reviews, conducted personnel interviews, and observed various aspects ofplant operations.
During this phase, the team was supplemented by approximately 15 knowledgeable individuals fiomFPL and the nuclear industry, which allowed the team to gain independent insights of performance at PSL The following functional areas were assessed.
I.
Operations II.
Engineering III.
Maintenance IV.
Plant Support - Health Physics V.
Plant Support - Security VI.
Plant Support - Emergency Preparedness VII.
Site Management Policies and Expectations VIH.
Safety Assessment/Corrective Actions Examples ofgood or weak performance were grouped together within each functional area, and assessments were developed to characterize the general performance during that time.
The assessments and their supporting examples formed the basis for the overall Phase I initial assessment.
The following references/sources were used during the document reviews, covering the period of January 1994 to April 1996.
B.
C.
D.
E.
F.
G.
I.
NRC Inspection and SALP Reports.
PSL 1 and 2 Licensee Event Reports.
NP-700 Problem Reports.
PSL In-house Event Reports.
PSL Condition Reports.
PSL Plant Manager Action Items.
Quality Assurance audits.
Independent Technical Reviews.
Department self-assessments.
PSL SdfAssessmcnt K.
L.
M.
N.
O.
Nuclear plant work orders.
Radiological deficiency reports.
Skin and personnel contamination reports.
Nuclear Division monthly indicators.
PSL procedures.
Operations Oversight Team (OOT).
Note:
INPO evaluations ofPSL were not used in order to avoid any infiuence &om past assessment conclusions.
The initial assessments for each functional area are documented in Appendix Ato this report.
Phase II:Validation and Anal sis The purpose ofthis phase was to validate the assessments developed during Phase I, and to further analyze the problem areas using experienced plant personnel, supervisors, department heads and managers for each functional area.
The results ofPhase I were presented to representatives
&om each functional area, and all affected departments were provided with copies ofthe assessments for their respective department.
The examples used in the assessment were verified as correct, and the conclusions drawn &om those examples were validated to be supported by the examples.
The problems identified in the Phase I assessments were evaluated using cause and effect (C&E) analysis.
The Self Assessment Team met with the Department Head and key personnel from each affected department to perform the analyses.
The analysis process relied on the knowledge, expertise, opinions and perceptions ofthose key individuals to establish the underlying causal factors for each problem assessment.
Those underlying causal factors which appeared repeatedly throughout many departments were identified as potential limiting(fundamental) weaknesses, and further evaluated by the team.
The results ofthe Phase IIanalyses are documented in Appendix B to this report.
Phase III: PSL Limitin Weaknesses The purpose ofthis phase was to identify those fundamental problems that are at the root ofmany issues or events at St. Lucie.
The underlying causal factors identified by plant personnel inPhase IIwere evaluated by the PSL Self Assessment Team, to determine which ofthese weaknesses was at the root ofproblems in numerous functional areas.
Those weaknesses which appeared across many functional areas were determined to be the underlying fundamental problems at the site, and were labeled limitingweaknesses.
Due
to their inherent impact on many aspects of PSL operation, these limiting weaknesses must be addressed before the overall performance ofSt. Lucie can improve.
The limitingweaknesses are discussed in detail in the main body ofthis report.
Phase IV: Corrective Actions Following the completion ofPhase III,management initiatives were developed to address the PSL limitingweaknesses identified in this report. Additional corrective actions willbe implemented by each Department Head for problems identified during Phase I and II that were not previously addressed.
In many cases, corrective actions have already been taken in response to condition reports (CRs), audit findings, NRC violations, etc.
Management initiatives to address the issues identified during the PSL Self-assessment are documented in Appendix C to this report.
Phase V: Im lementation and Follow u Following completion ofPhase IV, the management initiatives willbe implemented by the assigned Managers and Department Heads. Action plans for broad problem areas (e.g., limitingweaknesses) willbe screened by PSL plant management for inclusion into the PSL 1997 business plan.
To monitor the efFectiveness ofinitiatives and action plans, appropriate indicators willbe developed and maintained to monitor the performance ofidentified problem areas.
IDENTIFlED%'EAKNESSES PSL SelfAssessment PSL was an industry leader because ofits inherent strengths, and many ofthose strengths remain.
However, the focus ofthis self-assessment was the identification and analysis ofPSL problems, such that the fundamental weaknesses that are causing PSL performance to decline could be identified and corrected.
PSL personnel, supervisors, Department Heads, and Managers (including the Plant
.Geheral Manager) participated in the overall self-assessment, which required over 4000 man hours to complete.
The fundamental weaknesses a6ecting PSL performance are discussed below.
L Complacency:
Lack of management reinforcement of continuous improvement, establishment ofchallenging goals, and incorporation ofbest industry practices.
The construction of PSL Unit 2 on schedule and within budget was considered a major accomplishment in an industry plagued by construction schedule delays and cost overruns.
When the unit was declared commercial in 1983, the PSL management team received industry-wide recognition for its aggressive e6orts and commitment to excellence. During its first cycle of operation and refueling outage, Unit 2 was widely recognized as a top performer. In subsequent years, industry and business observers noted St. Lucie's world record runs, high capacity factor, few cases of enforcement action, and its stable, experienced workforce, as key indicators of PSL's success.
Other utilities began using PSL as a benchmark, and looked to PSL to set the pace for the industry. The PSL management team became comfortable withthe plant's overall performance, and focused on defending its position; near the end ofthe 1980s, PSL had effectively isolated itself&om its peers.
The PSL plant management team did not readily embrace the improvements or good practices developed elsewhere in the industry, and was not receptive to suggestions for improvement Rom oversight groups, regulators and the industry. The culture at PSL gradually developed into one of complacency.
Complacency had a negative impact on many PSL programs and processes.
In the management
- area, several examples were noteworthy.
First was the issue of goal-setting; during this period of complacency, plant management did not challenge the organization to benchmark the best performers in the industry.
Instead, plant management became reactionary, lost its vision for leading the industry, and failed to embrace goals and objectives which would have kept the PSL organization focused on continuous improvement.
PSL relied on its oversight organizations and regulators to identify problems.
The fact that external agencies publicly praised PSL for its high quality performance contributed to a sense that improvement was not needed.
Secondly, the plant management team did not recognize the importance ofself-assessment to the process ofcontinuous improvement.
Self-assessment was not incorporated into the PSL culture, and PSL plant management did not insist that individuals flnd and correct their own problems.
The absence of strong self-assessment expectations and the lack ofemphasis on basic trending and tracking processes in work controls, resulted in a shift away from indicators which tracked daily work process 10
PSL SelfAssessment effectiveness, to high level indicators monitored by regulators.
As a result, plant management did not utilize all the information available to fullyevaluate performance ofthe entire organization.'
third effect ofcomplacency at PSL was an overall resistance to change.
Site personnel and plant management were reluctant to accept evidence ofperformance weaknesses unless that evidence was presented by the regulators.
The plant team actively worked to miiiimize attention to problems identified by internal and external oversight activities. There was an emphasis on keeping problems in-house, and problems were not typically viewed as opportunities for improvement.
An adversarial relationship between PSL Quality Assurance (QA) and plant management reduced the effectiveness ofthe QA organization, and QA executive management's performance standards for QA did not suf6ciently empower oversight personnel.
Finally, plant management's isolationist approach stifled interaction with other successful Power Reactor licensees.
Nuclear Division policies and recommendations &om outsiders were not well received or accepted at PSL The PSL management team was not eager to share its problems with others and did not recognize the importance of organizational learning as a contributor to continuous performance improvement.
Complacency affected key plant departments in a variety ofways. In the Operations Department, procedural and performance weaknesses were identified but not effectively resolved.
Poor procedure quality was tolerated in some cases, and individuals who operated successfully in spite ofprocedure weaknesses were recognized as good performers.
Procedure upgrades were recognized as being manpower intensive but sufficient resources were not allocated to resolving all known procedure deficiencies.
Operator performance expectations did not foster continuous improvement and management attention was not focused on continuous improvement. Reports issued by NRC and QA frequently praised operator response to real-time plant events and failed to identify broad based operator weaknesses; operations management did not utilize lower level indicators to reveal the need for performance improvements.
In the Engineering Department, formal problem identification and evaluation tools and techniques were not routinely utilized, because departmental management did not set high expectations for effective problem identification, evaluation and correction, and problem solving skills became a low priority. As a result, there was a lack ofrigor in problem analysis, and problem evaluations were not effective in identifying underlying causal factors.
Underlying causes went unidentified, corrective actions often addressed symptoms, and problems recurred.
The engineering processes resulted in costly and frequently ineffective solutions.
The Maintenance Department suffered &om its inabilityto evaluate and resolve repetitive equipment failures and prevent repeat events.
Maintenance accepted poor equipment performance, and lacked the ownership and resolve to correct long standing equipment problems.
The Outage Management Department was also hampered by complacency, and the continuation ofpoor outage scope control practices prevented PSL &om consistently meeting outage schedule, performance and budget goals.
Plant departments lacked discipline in adhering to pre-outage milestones and most outages were initiated without preparations being fullycompleted; crisis management became a &equent outage
practice due to failure to implement the outage schedule.
Improvement opportunities identified during outage ciitiques were often not addressed and weak programs were tolerated. For example, weak foreign material exclusion and inventory management practices resulted in job delays and inappropriate material installation in the plant, but significant improvements were not made in those processes.
Similarly, plant management was satisfied with the Emergency Preparedness program because it was highly rated by outside evaluators during annual evaluations.
As a result, plant management accepted those ratings as justification for maintaining the status quo rather than pursuing improvement initiatives that would provide greater emergency readiness.
The majority ofexperienced Emergency Response Organization (ERO) members who gained expertise through years ofpractice masked the need for implementing procedure upgrades and training enhancements.
Likewise, plant management was satisfied withthe plant's ALARAprogram and did not establish challenging goals for exposure at PSL because high ratings (SALP1/INPO1) indicated that performance continued to be good.
Consequently, exposure at PSL remained above industry averages, and longstanding problems, such as the high source term on Unit 1, were not effectively addressed, even though some plants had developed good practices which could have been implemented at PSL.
Itis evident that the PSL plant management team allowed its record ofgood performance and lack ofsignificant criticism to lead itto a culture ofcomplacency where continuous improvement was not emphasized, and the industry's most'beneficial and cost-effective initiatives were not embraced.
IL Training:
Lack ofsufficient ownership offormal training by the line organization, and lack ofownership ofline performance by training.
Formal training at PSL historically focused on those programs specifically emphasized by the regula'tor and accredited by INPO.
The PSL Training Department effectively used a systematic approach to training (SAT) to support the training requirements for most key job positions. INPO accreditation ofthe twelve specified training programs and routine NRC inspection ofthe key training programs has documented continued compliance with regulatory requirements and overall training program effectiveness.
Nonetheless, opportunities for improvement have been noted.
In the Operations Training area, high quality training is provided to licensed operators, non-licensed operators and Shik Technical Advisors. NRC inspectors who observe operator training sessions have commented that the program supports management expectations and includes timely and important topics [NRC IR 96-04]. In addition, NRC inspectors who performed the SALP review in 1996 indicated that observations ofoperator performance during numerous plant startups and reduced inventory conditions revealed operator competence [2/8/96 SALP report]. These comments, as well as numerous specific documented observations, support the conclusion that the Operations Training program effectively accomplished its mission of preparing operators to respond properly to 12
~ ~
~
0 PSL SelfAssessmenf emergency and off-normal operating events.
However, there is some evidence that the operator training program did not keep pace with industry standards for excellence during routine operations.
Successful operator training depends on routine constructive communication between Training Department management and Operations Department management.
Evidence ofweakness at this level ofcommunication was found in the comments ofon-shift operators who expressed a need for increased training in areas ofpersonnel m inagement techniques, Technical Specification interpretation and Emergency Plan duties. The marginal effectiveness ofthe STOP self-checking program indicated that training emphasis was needed to ensure operators recognized the value of self-checking and reinforced the management expectations for operating crews.
Some administrative duties, such as Post Maintenance Testing coordination, and implementation of Equipment Clearances, have periodically caused problems on-shift which indicate a need for increased training emphasis.
These needs have not resulted in timely curriculum modification.
Weak communications Rom Training Department management to Operations management were indicated by the fact that inadequate operating procedures were not consistently identified by the operations training group. Evaluation ofrecent operator performance issues led to the conclusion that operator simulator training was results oriented and instructors sometimes accepted the trainees'bility to "get the job done" rather than insisting on procedure compliance during simulator training.
In general, observers commented that training has sometimes accepted the standards used for day-to-day operation in the plant rather than maintaining performance standards consistent with all management expectations.
Finally, several weaknesses in Emergency Plan classification and notification noted during drills and NRC inspections indicate a need for added training emphasis in these areas.
Supervisors in the Operations Department have commented that formal managerial skills development training would help NPSs more effective in managing their shifts, setting expectations and holding individuals accountable.
Although the Maintenance Training programs continue to meet NRC requirements and INPO accreditation criteria, specific curriculum weaknesses were mentioned by maintenance representatives.
Supervisory skilltraining for first line supervisors was found lacking and the necessity and importance of pre-job tailboard meetings was not emphasized.
Supervisors commented that industrial safety issues did not receive the same training attention as maintenance production processes, which may have contributed to the large number ofinjuries reported during the first halfof 1996. Maintenance personnel interviewed stated that less than adequate training existed for topics such as Foreign Material Exclusion, Crane Operator qualification, and materials management using the Passport system.
In the Engineering organization, most training was handled informallyon-the-job, and Engineering Department supervisors concluded that their personnel selection and development processes were weakened by a lack oftraining resources.
Supervisors indicated that the emphasis on on-the-job training (OJT) rather than formal professional development was detrimental to the fulldevelopment oftheir personnel.
Engineering supervisors stated that some instances ofless than adequate plant 13
modification packages and document updates have resulted &om the lack of a formal'raining program with a continuing training element.
This weakness has also contributed to inadequate updates ofplant drawings, procedures, and licensing documents.
The turnover ofthe drawing update process &omEbasco to FPL resulted in a lack ofquality in as-built documents due to the insufficient training ofFPL engineers assigned to perform drawing updates. Since a large portion ofEngineering resources were assigned to Condition Report evaluation, supervisors noted the need for procedures and training for problem identification and resolution.
Root cause analysis training has not been provided for all staff engineers, and the training that was provided lacked management reinforcement ofthe processes and techniques taught in these courses.
Asignificant amount oftraining ofplant personnel was conducted outside the Training Department, and a systematic approach to training was not used for most ofthese programs.
The lack of a systematic approach resulted in less than effective training for some plant positions and functions.
For example, plant personnel received little training on problem identification and correction processes.
Problem evaluation tools and techniques, such as those included in the Problem Identification and Correction (PIC) course, were not well understood.
On-site training in formal problem analysis skills was not emphasized after FPL was awarded the Deming Prize in 1989. The lack ofemphasis on continuing training contributed to the inconsistent identification ofunderlying causal factors, and the development of less than effective corrective actions.
One example ofa successful training program outside the Training Department was Security Force training, which was developed and implemented withinthe Security Department.
Security training employed qualified and experienced instructors who provided specialized instruction.
NRC inspectors have cited the training and qualification programs for security personnel as a plant strength
[NRCIR 96-05] and the 1996 SALP report characterized the Security Force training as "effective".
Several training needs were identified which cross departmental boundaries.
In some ofthose cases, programs were under development as a result ofprevious Corrective Action efforts. For example, ERO training was being upgraded to ensure that ERO personnel received adequate instruction in the performance of their duties.
Similarly, Facility Review Group (FRG) member training is being developed to ensure that personnel with varied backgrounds receive adequate initial and continuing training to remain well prepared to evaluate the nuclear safety significance ofplant processes and products under their purview. In the past, these two programs utilized the Training Bulletin process to document annual training completion, but the need for more formal and direct training was identified.
Significant changes were made to the plant's Corrective Action Program when the Turkey Point Condition Reporting (CR) process was implemented at PSL in April 1996.
Early CR process implementation problems at PSL indicated that the process was not well understood by the staff, and that formal training was needed.
However, training for personnel who perform the trending ofCRs at PSL was not provided.
Weaknesses in the CR closeout process also indicated that specialized training for personnel doing close-out evaluations was needed in order to ensure consistency.
14
PSI, SelfAssessmenl Similarly, management expectations for self-assessment were stated in late 1995, but self-assessment training was virtuallynon-existent at PSL.
Akey element oftraining success is formal feedback Rom the plant line organization to the training department to allow timely identification ofweaknesses and new training needs, and to identify those training elements and techniques which are particularly useful. In early 1996, itwas recognized that an inadequate amount offormal feedback was being received from Managers and Department Heads concerning training effectiveness for their key programs.
Corrective action was implemented to ensure routine feedback was provided on each ofthe accredited programs.
IIL Self-assessment:
Lack of emphasis on identifying and correcting problems at the worker and supervisor levels (first and second levels of defense of quality).
Asound self-assessment program is one that promotes the identification and resolution ofproblems at the worker and supervisor levels, before problems become more significant'or manifest theinselves during events.
There are numerous methods by which self-assessment can be performed, and different times when each method should be utilized.
For example: routine self-checking is appropriate when performing plant evolutions; periodic self-assessments should be conducted to evaluate the performance of a group on a continuing basis; reactive self-assessments should be performed following events or significant problems; and pre-emptive self-assessments should be conducted followingsignificant changes, to determine whether new vulnerabilities were created.
Nuclear Division Policy NP-805, Revision 0, dated 3/15/94 states that "each direct report to the President, Nuclear Division shall perform appropriate self-assessments."
However, self-assessment was not well defined within the Nuclear Division, and PSL plant management did not set clear expectations for the performance ofroutine self-assessment or emphasize the value offormal self-assessment.
AtPSL, implementing procedures and training on self-assessment were not established (withthe exception ofthe Security Department), because the need for continuous improvement was not recognized by plant management and personnel.
For many years, PSL station performance was rated among the best in the country by regulatory standards, and PSL management sought to maintain the status quo. As a result, self-assessments were mostly conducted to address significant events, and self-assessment was not routinely used to identify and correct problems at PSL.
The lack ofeffective self-assessment was evident in many departments at PSL. In the Operations Department, self-assessments were superficial and conducted mostly by outsiders.
As a result, operator weaknesses inthe areas ofattention to detail, procedural adherence, configuration control, and watch standing/log keeping practices were self-revealing, or identified by internal and external oversight organizations.
Operations department supervisors and training instructors did not recognize the value ofself-checking and did not adequately reinforce its use during training and on shift. The operator self-checking process (STOP) was marginally effective in preventing inappropriate operator 15
actions.
The lack ofeffective self-assessment also obscured the significance ofPost Maintenance Testing (PMT) process problems, which resulted in some safety related components being returned to service without verification ofa satisfactory PMT.
In engineering, the quality ofmodification packages declined because drawing discrepancies were not actively resolved.
The identification oferrors invendor drawings, particularly ofelectrical equipment,
,was not sufBciently emphasized.
As a result, this type ofdiscrepancy was not always identified, and the troubleshooting ofelectrical equipment was complicated by mismatch between some components and associated wiring diagrams.
In the Maintenance Department, self-assessment weaknesses were evident.
For example: the progressive loss of institutional equipment knowledge was not recognized and compensated for; repetitive equipment problems were not always adequately evaluated and resolved; rework occurrences were not consistently identified, analyzed, and resolved; and injury occurrences among workers were not always thoroughly analyzed.
In Emergency Preparedness, emergency response exercises (drills) were opportunities to self-assess the performance ofthe ERO. However, the drills were only performed for the purpose ofpreparing for the annual evaluated exercise.
The methodology for identifying and correcting performance problems was not formally defined, therefore the data collected during drills did not result in effective problem identification and resolution.
Also, personnel assigned to evaluate drillperformance and develop/resolve critique items were not trained in problem solving techniques.
In January and February 1996, critical self-assessments determined that overall skill level ofthe ERO did not meet PSL management expectations.
The assessments identified longstanding problems with less than effective training of the ERO, and less than adequate implementing procedures.
Additionally, repetitive drillcritique issues were indicative ofless than effective problem resolution.
When problems were identified, the lack ofownership ofthe results hindered the development of effective corrective action. Results ofassessments were often treated as isolated problems, rather than opportunities to explore the underlying causes ofperformance weaknesses.
These weaknesses persisted and sometimes manifested themselves in significant events.
IV.
Corrective Actions:
Less than effective implementation ofcorrective action processes and lack ofcorrective action followthrough.
PSL corrective action processes (e.g., the site wide processes for documenting, resolving and tracking problems) provided a solid foundation which was generally used to effectively investigate and correct consequential (high visibility)problems and events.
However, the same processes were not always implemented effectively to address problems that were less visible or oflesser consequence.
The analysis ofthose problems did not consistently identify underlying causal factors, and the resulting corrective actions were sometimes narrowly focused, symptom based, ineffective in preventing 16
PSL, SelfAssessment recurrence, and were often not applied to other programs, processes, procedures, or equipment that suffered from the same weaknesses.
Corrective action tracking and trending processes did not typicallymonitor the e6ectiveness ofcorrective actions, or effectively identify adverse trends. Lastly, corrective actions were seldom provided to identify self-assessment deficiencies and corrective action weaknesses when self-disclosing problems and repeat events occurred.
After receiving the Deming Prize in 1989, Nuclear Division management expectations to reduce adininistrative burden resulted in an emphasis on less formal approaches to problem solving, withthe assumption that the Nuclear Division would continue to identify and correct problems in a high quality manner. PSL management reduced training on the fundamentals ofproblem solving and root cause evaluations, did not set clear expectations for e6ective problem resolution at the worker and supervisor level, and did not emphasize tracking in-process performance indicators or trending the long term effectiveness ofcorrective actions.
PSL focused on prompt action item closure, which resulted in briefproblem evaluations and less than effective corrective actions.
Procedure changes were often accepted as a corrective action because they were the most expedient.
Although processes were in place to evaluate and correct significant problems and events, PSL did not develop its own site wide process for documenting and addressing gow threshold) problems until 1994. In the absence ofa mechanism to capture low level problems, some departments developed their own problem documentation and correction processes.
However, the processes developed by departments were largely ineffective in preventing problems &om recurring, as they were not as visible to PSL upper management, and lacked critical attributes such as tracking the completion of corrective actions, and trending the effectiveness ofcorrective actions.
Site-wide processes for documenting and correcting problems similar to the PSL Condition Report process, are highly effective methods for capturing problems ofall levels ofsignificance.
The PSL CR process was implemented in April 1996.
The threshold ofthe'R process was low, which provided management with a source oflow level indicators ofthe health ofplant programs, processes and equipment. However, plant personnel were not well trained on the CR process, which diminished its efFectiveness.
For example, problems were not always properly characterized, CR evaluations did not always identify the appropriate causal factors, and generic implications were in&equently considered.
PSL's implementation ofthe CR process did not provide an effective prioritization method.
AH CRs were required to be dispositioned (with corrective actions identified) within 30 days of CR issuance.
The short turnaround time and high volume resulted in the need to prioritize problems according to significance.
This was not initially done, and the quality of the problem evaluations su6ered.
In addition, the CR closeout reviews were sometimes ine6ective in identifying deficiencies.
As a result, corrective actions were less than effective in many cases.
The CR process also trends causal factors, which is instrumental in the early identification of adverse trends.
However, littleguidance or training was provided to personnel performing the trending.
In the Operations Department, corrective actions were provided to reduce temporary system alterations and other hardware deficiencies that caused operators to work around problems.
17
However, repetitive operator errors caiised by weaknesses in attention to detail, procedure adherence, and a lack of familiarity with requirements persisted.
Although many such occurrences were documented in internal/external reports and operations department data sheets, those weaknesses were largely uncorrected.
In August 1995, a series of significant events. indicated that operator performance had declined.
In Engineering and Maintenance, equipment problems or failures have recurred, due to a focus on short term objectives and schedules, and a lack ofemphasis on long term equipment trending and monitoring.
Equipment problems were sometimes not thoroughly evaluated such that lasting corrective actions could be identified. In many cases, repetitive equipment problems were unnoticed until a self-disclosing event or interest Rom oversight groups highlighted the issue.
For example, effective corrective actions were not taken to repair various system relief valves that lifted inappropriately on three occasions in early 1995, and one of them lifted again in August 1995, releasing 4000 gallons ofreactor coolant into the Unit 1 pipe tunnel. Emergency Diesel Generator performance did not improve until several failures and significant questioning by the NRC prompted thorough corrective actions.
Additionally, the reactor vessel 0-rings repeatedly failed to seal properly, after thorough evaluations and corrective actions were completed.
In the area ofradiological controls, PSL successfully addressed significant challenges.
For example, permanent modifications were implemented inthe containment cooling system, to help reduce outage dose and contamination events.
- However, less significant radiological deficiencies were not adequately addressed by corrective actions.
Since 1994, numerous Radiological Deficiency Reports (RDRs), NRC reports, and QA audit reports documented the improper storage and labeling of nuclear materials and contaminated tools. However, corrective actions focused largely on individual events, and were not effective in preventing the radiological deficiencies &om recurring. Finally, in February 1996, the NRC issued a non-cited violation to address the improper control ofcontaminated tools. In addition, after successfully drivingthe number ofpersonnel contaminations to less than 100 per year, personnel contamination cases have been on the rise since 1994. And during the 1996 Unit 1 refueling outage, workers were repeatedly contaminated while working in clean areas ofthe laundry trailer.
The performance ofthe ERO was consistently evaluated as strong by the regulators.
- However, following the 1996 EP practice and evaluated exercises, the Emergency Preparedness (EP)
Department and PSL plant management determined that the ERO's performance did not meet management expectations.
The exercises uncovered weaknesses in the skill level ofERO members, the EP training program, EP implementing procedures, and the EP drillprocess. Weaknesses in EP training and implementing procedures were identified as early as 1989, but earlier corrective actions were ineffective.
V.
Accountability:
Lack of consistent assignment and/or enforcement of individual accountabilities.
18
Effective assignment of accountability is accomplished through policies, procedures and quality instructions which clearly establish responsibilities.
High level policies are progressively explained in increasing detail down through the hierarchy ofprocedures.
Nuclear Division Policies clearly defined the expectations, performance standards, and implementation responsibilities for key Division processes.
As these policies were further defined forthe PSL site organization through implementing procedures, the communication ofresponsibilities was less comprehensive and less specific than that required for effective establishment ofaccountability.
Several programs and processes which apply to all plant departments were affected by management's reluctance to clearly state expectations and hold individuals accountable for meeting those expectations.
Corrective actions, the Facilities Review Group (FRG), Emergency Preparedness, radiation protection, Industrial Safety, design control, overtime control, and training aH suffered due to a lack ofclear assignment and/or enforcement of accountabilities.
Operators were not held accountable for their performance by supervisors and managers and as a result, weaknesses developed in the areas ofattention to detail, procedural adherence, configuration control, and watch standing/log keeping practices.
Procedure quality deteriorated because the expectafion and accountability for maintaining high quality operating procedures was not properly emphasized and strict procedure adherence was not consistently enforced on-shift and in training.
Department Heads and Managers were not held accountable for the quality of the analysis and corrective actions provided in CR closures, as evidenced by many problem evaluations which were not effective in identifying underlying causal factors.
Management expectations for communications &om acting FRG Chairmen and the Plant General Manager were not consistently understood by acting FRG Chairmen. FRG subcommittee reviews were not consistently performed as defined by the Quality Instructions (QIs) due to a lack of enforcement ofthis accountability.
PSL plant management did not emphasize individual responsibility for continuous emergency readiness, and ownership of all EP problems was perceived to reside with the EP Department.
Interdepartmental issues relating to ERO assignments, performance, and program support, were usually assigned to EP, and not to those with the authority to resolve the issues.
Emergency preparedness procedures did not describe all ERO positions and their responsibilities.
As a result, the performance of the ERO during the 1996 evaluated exercise did not meet management expectations.
Individual accountabilities for programs such as radiation protection and hot to'ol control were not adequately reinforced. Many repeat problems in the HP area were due to a lack ofaccountability for personnel radiation exposure at the worker and supervisor level. HP was ultimately held accountable for radiological problems, even when created by other departments.
Foreign material exclusion program requirements were not taken seriously by plant personnel because individual accountability was not enforced.
19
PSI. SdfAsscament Management expectations offirst line supervisors (foremen, chiefs and field supervisors) were not clearly established and standardized among departments.
First line supervisors were not held accountable for personnel issues and were less than effective in reinforcing production and personnel safety. As a result, poor production performance and an unacceptable number ofinjuries and unsafe conditions developed.
The lack of individual ownership of safe work practices within plant departments and an over-reliance on the safety department, rather than a clear assignment of accountability, also contributed to injuries and unsafe conditions. Management did not reward good personnel safety performance to the degree that it rewarded getting the job done on time, and as a result, the workers'nd supervisors'ccountability for personnel safety were not sufficiently re-enforced.
Individuals were not held accountable for conducting effective tailboard meetings which, when conducted properly, have the potential ofreducing injuries to plant personnel.
The accountability for plant equipment performance was not clearly established.
As a result, the focus required to address dif6cult equipment performance issues was lacking.
The lack of clear accountability for equipment performance also led to a lack ofmanagement commitment to fund long-term solutions of.equipment related problems. Plant personnel were reluctant to go through the effort ofproviding the necessary justification forimprovements, because it was not clear who should be the one providing such justification.
Equipment performance also suffered due to a perceived management expectation that CR close-out was more important than the effectiveness ofthe CR analytical work A clear establishment ofaccountability for the effectiveness ofCR analytical work was Iilissiilg.
In Engineering, the lack of clearly defined and enforced accountabilities resulted in process weaknesses.
Modificationpackages were consistently identified late, and had to be developed under time pressures.
As a result, less than adequate modification packages were developed in haste
- because, although a requirement existed to identify needed modification packages early enough to facilitate their development, the organization was not held accountable for this requirement.
Out-of-date drawings became dioicult to reconcile during the design and as-building processes because of a lack ofmanagement emphasis on getting modification packages closed out. The accountabilities for the actions required to close packages out were not clearly assigned and enforced.
The accountability for supporting the up-&ont design review (UDR) and implementation review (IR) meetings by plant departments was not enforced, and this led to inadequate modification package review by plant departments.
The failure of Engineering supervision and oversight to hold Engineering personnel accountable for process quality contributed to modification package problems.
The need to revise the FSAR was not always identified because ofa lack ofemphasis on the FSAR as an active design document.
The accountability forthe maintenance ofthe FSAR was not clearly established and enforced. The evaluation of past operability was not consistently performed by Engineering because ofa lack ofclear assignment ofaccountability for this function. Past operability is usually associated with reportability, and the determination ofreportability was the responsibility of Licensing.
Engineering, therefore, was not fully participating in this determination of past operability.
20
PSL, SelfAsscssrneel PSL plant management did not strictly enforce overtime policies because overtime had become a normal way of getting the job done, and it was not felt to have had a negative impact on plant personnel.
Programs/Processes/Procedures:
Lack ofemphasis on, and accountability for, important programs, processes or procedures.
At PSL, plant management designed many programs, processes and procedures to,establish clear roles, responsibilities and performance standards, monitor performance and initiate action when needed, to ensure the plant is operated and maintained in a manner which protects the health and safety ofthe public and plant workers. During the evaluation period, a variety ofplant programs, processes and procedures were evaluated by regulators and inspectors and a number of strengths were reported in these areas.
For example, the NRC SALP report issued on February 8, 1996, commented that the overall performance ofthe Engineering group was superior with specific strength demonstrated in the area of design and installation support, the inspectors commented that the department had produced a number of well engineered and implemented plant modifications.
Furthermore, the department received praise for its demonstrated commitment to safety and compliance with regulations as well as its support ofvarious maintenance programs such as Non-Destructive Examinations.
Engineering emphasis on the monitoring requirements for alloy 600/690 applications in plant systems was perceived as a strength, as was the maintenance specification program. In addition, the NRC identified (NRC Inspection Report 96-08) a strength in the Engineering Department's Steam Generator eddy current testing program and Safety Evaluations, which demonstrate a commitment to safety and compliance with regulations.
The Maintenance organization received positive feedback on improvements made to the preventive maintenance program, through reviews and evaluations of basis, scope and frequency of tasks..
Maintenance efforts to improve unit reliability and reduce outage work scope by using effective on-line maintenance processes were highly regarded by INPO during a June 1995 evaluation.
Additionally,the INPO report identified that the PSL multi-disciplinary team working to improve air operated valve performance, and the diesel fuel filtration skid for new fuel, were maintenance program strengths.
In 1995, INPO cited the Operation Department's use of a computer model for predicting core reactivity changes during power maneuvers as a strength in reactivity management.
In addition, the use ofexperienced Operations personnel in the support ofdaily maintenance planning was recognized as a strength.
Operations management was recognized for cautious and deliberate decisions concerning plant operations, and on-shift operating crews were commended for their professional response to plant off-normal events.
Operating and maintenance events at PSL beginning in August 1995 resulted in a level ofscrutiny by internal and external reviewers which effectively revealed weaknesses in a variety of programs, 21
PSL, SeIfAssessment processes and procedures.
Programs exist and are assigned to responsible groups and individuals for implementation and maintenance, but program performance has suffered as a result of a lack of accountability. Each ofthe followingplant programs and processes were afFected.
Foreign Material Exclusion Program (FME): Although the FME program received management attention, foreign material exclusion practices did not reflect accepted industry st'andards.
2.
Operating Experience Feedback Program (OEF): The organization and administration ofthe FPL OEF Program generally met INPO guidelines.
- However, weaknesses in OEF implementing procedures, less than effective transfer oflessons learned from in-house events, and poor assessments ofthe OEF Program reflect a lack ofPSL plant management attention to this program.
Operator Work Around Program (OWA): Even though the total number of OWAs was reduced in the past 2 years, the OWAProgram was weak in that it did not include provisions for prioritizing corrective actions for OWAs based on their impact on operators and equipment.
Self-Assessment Program: Self-assessment was infrequently used to identify and correct problems. The fact that no formal plant-wide requirement for departmental self-assessment existed was indicative ofthe low prioritythe program had received. Most departments had not implemented a specific departmental self-assessment action plan.
ALARAProgram: The ALARAprogram has been effectively implemented.
However, the program had not kept pace with industry standards, and had not achieved first quartile performance.
Emergency Preparedness Program (EP): The EP drillexercise program had inappropriately stressed preparation forthe NRC evaluated exercise, rather than maintenance ofa high state ofemergency preparedness.
The drillcritique process was informal and was not effective in identifying and resolving repetitive performance problems.
7.
8.
Outage Work Control Process:
The Outage Management milestones for pre-outage deliverables resulted in such demanding schedules that the quality ofengineering packages was sometimes affected.
Some Unit 1 1996 outage schedule conflicts were not effectively resolved, and some problems Rom previous outages hampered productivity.
Material Management Process:
The process for obtaining (and returning) parts &om stores required detailed knowledge ofthe Passport program but such knowledge was possessed by relatively few individuals. The fact that the system was complex and difBcultto operate led to instances ofinstallation ofincorrect material in plant systems and significant job delays.
22
PSL SelfAsseamcnf Post Maintenance Testing Process (PMT):
The process for completing PMT prior to returning equipment to service was weak, and resulted in some safety related components being returned to service without verification ofa satisfactory PMT.
Facility Review Group (FRG)
Process:
FRG processes were inefficient and the subcommittees were not efFective in reducing the FRG's work load. Managers identified that personnel assigned to perform FRG subcommittee reviews lacked accountability for that task, and in general, FRG members did not feel ownership for their FRG duties.
PSI. plant management expectations for communications between acting Facility Review Group (FRG)
Chairmen and the Plant General Manager were not consistently understood by acting FRG Chairmen.
Management Review Board (MRB) Process:
The MRB was tasked with evaluating and approving plant changes and modifications.
As a result, engineering work was largely determined by MRB approval.
The MRB process was less than efFective because it lacked prioritization and integration" elements for addressing improvement and cost avoidance projects, and MRB cost benefit considerations did not sufficiently consider benefit.
Condition Report (CR) Process:
The CR procedure provided littleguidance with respect to what constituted effective problem solving. The CR process lacked an adequate prioritization scheme, CR corrective actions were sometimes unnecessarily delayed by deferring action to the PMAIprocess, and corrective actions resulting &om CRs were not clearly linked to the MRB process.
Equipment Performance Monitoring Process: Procedures and processes for the maintenance ofequipment lacked sufficient detail to adequately monitor and trend equipment performance problems.
Document Update Process: The process ofidentifying and updating plant procedures was inadequate to keep up withthe volume ofplant modification packages.
Similarly, the revision ofplant drawings to reQect as-built conditions were not completed in a timely manner, and discrepancies between drawings and field conditions existed.
FSAR reviews and FSAR updates were not routinely considered as part ofthe modification package process.
This resulted in numerous discrepancies between operating practices and those described in the FSAR.
Identification and Analysis ofRework: Procedural guidance for this program was not well defined and, as a result, the importance ofcapturing rework data was not well understood.
In addition, the difference between rework and repetitive equipment problems was not clearly defined.
23
PSL SelfAssessment 16.
Work Control: The processes for work package planning and job scheduling have not su8iciently incorporated safe work practice requirements, parts procurement and maintenance support, and resulted in production delays which could have been avoided ifproperly considered during PWO planning, job scheduling and crew tailboard briefings.
In the past, due to a sense ofcomfort with the status quo, personal performance and job completion was valued more than having correct procedures.
Inadequate procedures and processes were worked around and inadequate procedures were not seen as a significant problem for the plant. The plant events of 1995 led to a site policy of verbatim procedure compliance (Plant Policy 502) which resulted in a large number ofprocedure changes in order to comply with that policy.
Procedure problems have a6ected aH departments at PSL. In some cases, problems have resulted fiomthe absence ofspecific procedures.
Significant plant processes such as Root Cause Evaluations, Nuclear Problem Reports, FaciTity Review Group activities, Condition Reports and Self-Assessments have been hindered by the lack ofclear management expectations communicated in the form ofplant procedures.
In other instances, problems occurred because procedure compliance was not enforced.
NRC inspectors identified numerous instances ofMure to comply with procedures in 1995 and 1996.
Examples ofthis are: failure to apply appropriate PMT acceptance criteria; failure to sign field copies ofWork Orders; inadvertent removal ofthe wrong incore detector f'rom the reactor due to procedure non-compliance; and failure to followthe procedural requirements for installation ofelectrical leads to E/P 2110Q.
In the Operations Department, knowledge based procedures had been in use for many years based on the assumption that operator experience and training would result in proper task performance in spite oflimited procedure detail.
Some operating procedure weaknesses were not corrected when detected and less than adequate procedures were accepted because Operations and plant management valued and rewarded getting the job done on time over stopping to correct procedures.
Some procedures did not adequately incorporate all technical specification requirements electively and verbatim procedure compliance requirements, initiated in the fourth quarter of 1995, resulted in a large number procedures which required revision in order to support verbatim compliance.
Some procedural weaknesses had a negative impact on the Engineering organization. For example, less than adequate guidance for maintaining equipment led to plant modifications when only maintenance was required.
In addition, a weak PCM close-out procedure resulted in some modification packages which were not closed in a timely way due to the lack of standardized acceptance criteria forPCM package format.
Other procedural weaknesses involve the adequacy and clarity ofinstructions.
The ALARAProgram procedure, the Condition Report Procedure, as well as numerous Operating, Emergency Plan Implementing and Maintenance Procedures have been recognized as weak.
Most were initially knowledge-based and not well suited to verbatim compliance.
The cumbersome and time consuming procedure change process presented a hurdle to procedure revision and departments with large 24
numbers ofprocedures, like Operations, have embarked on a major procedure revision project. The upgraded procedures, along with those strong programs, processes and procedures which remain in place, are expected to formthe basis for continued improvement at PSL and a solid foundation Rom which to build future success.
VIE Change Management:
Less than adequate implementation ofchanges.
A strength ofthe PSL organization has always been the stable, experienced workforce, and their abilityto work as a team. Many employees have been at PSL since the startup ofthe nuclear units.
The result has been many years ofconsistently safe operations, and industry recognition for its record setting performance.
Over a period oftime, those successes led to a reluctance on the part ofPSL plant management and site personnel to accept evidence ofperformance weaknesses brought forward by regulators and internal and external oversight organizations, as well as a reluctance to implement costly modifications that could further improve plant operations.
Formal feedback &om regulators and oversight organizations helped reinforce that the plants were among the best in the nuclear industry, and that PSL plant management had a winning team.
The combined eFect was an overall resistance to change.
In August 1995, a series ofevents caused PSL plant management to recognize the need for significant changes in PSL management and organizations.
But the PSL team had not had to make many changes after years of stable operations, and had not developed the "change management" skills needed to assure organizational and process changes were successfully implemented.
As a result, significant organizational and administrative changes implemented at PSL in 1995 and 1996 were not carefuHy evaluated and managed to ensure that the organizations transitioned smoothly into their new areas ofresponsibility. In some cases, continuity was not maintained for critical functions ofthe organization.
Since August 1995, extensive managerial and organizational changes were implemented at St. Lucie, including a new Site Vice President, 10 new managers (5 from outside FPL), and over 12 riew department heads/key supervisors.
Extensive organizational realignments have also been made. By September 1995, a new Plant General Manager and Operations Manager were in place. The Technical StafF Department was reorganized into the new System Component Engineering (SCE) Department, and a new SCE Manager was assigned.
Additionally, an Operations Support Department and an Operations procedure group were established.
By February 1996, the PSL Site Vice President was re-assigned to Juno Beach, and new Licensing and SCE Managers were appointed.
A major reorganization of the Maintenance Department was implemented to establish functional groups (Valves, Welding, Rotating Equipment, Stationary Equipment, and Outage &Projects), with new supervisors assigned to each group. Maintenance Managers were also established for each operating shift. ByMay of1996, the President ofFPL Nuclear Division had retired, and was succeeded by the Turkey Point Site Vice President.
The Operations Manager &omTurkey Point was reassigned to St.
Lucie, and a new Site VP was hired Rom outside PSL.
Between June and August 1996, new 25
Services and Training Managers were hired &om outside PSL, and a 'new Materials Manager was reassigned &omthe FPL Juno Beach once.
Also, many new department supervisors were assigned, including Instrumentation and Control, Electrical Maintenance, Chemistry, and Maintenance Projects 8'c Services Departments.
The on-site engineering groups (e.g., Reactor Engineering, SCE, Shift Technical Advisors, In-service Testing, In-service Inspections) were consolidated under the Site Engineering Manager.
Maintenance planners and schedulers and Outage Management personnel were reassigned to the newly formed Work Control Department, and a new Work Control Manager was assigned.
Anew Operations Support Department was also established, and a new Operations Support Supervisor was hired &om outside FPL. In summary, 10 of 12 key managers at PSL were replaced with personnel who were not from withinPSL.
The reorganization ofthe Maintenance Department in the first quarter of 1996 had positive and negative effects.
The new organization was established to provide more focused ownership and accountability for equipment problems and programs.
For example, the welding program was consolidated under a single umbrella, and a single point ofaccountability was established for the maintenance ofvalves and all rotating equipment, with the expected outcome that expertise in these areas would be better focused and highly developed.
However, the transition to the new organizations were not carefully managed, and continuity was not maintained for all areas of responsibility. Maintenance Department component engineers were not able to work as closely with the mechanics when they were transferred to engineering.
PSL management did not establish clearly defined lines of accountability and responsibility for maintaining equipment and equipment performance, causing confusion between the Maintenance and Engineering Departments.
In addition, some positions in the Maintenance Department were eliminated altogether, without assuring that the critical functions performed by those positions were seamlessly transferred.
For example, the responsibility forthe m'aintenance reliefvalve in-service testing program was dropped followingthe reorganization, and maintenance process and organizational changes just prior to the 1996 Unit 1 refueling outage decreased productivity, and resulted in inaccurate resource estimates for the outage.
In the PSL operating experience feedback (OEF) program, a series of changes were made which challenged an already marginal program.
First, a new PSL OEF Coordinator was appointed in October 1995, but the turnover was less than adequate, because insuf6cient time was devoted to the turnover process. InNovember 1995, the responsibility for providing operating experience feedback on PSL in-house events was administratively shifted &omthe Shift Technical Advisors (STAs) to the new OEF Coordinator.
OEF screenings performed by the STAs were considered more effective than those performed by an OEF Coordinator, because ofthe STA's extensive training on site-specific systems and operating procedures.
Secondly, the OEF Program Administrator position in Juno Nuclear Licensing was eliminated in May 1996, without contingencies to assure a seamless transfer of accountabilities to the two nuclear sites.
The position had overall responsibility for the OEF program, and performed many administrative functions. However, there was confusion at PSL as to who was supposed to continue to perform those functions, ifany.
26
PSL SelfAssmment In Emergency Preparedness (EP), site wide organizational changes resulted in a loss ofclear ERO accountabilities in some areas.
The Fall 1995 reorganization at PSL did not consider the impact on EP, as some responsibilities ofERO members were not maintained.
Previously, the technical support center (TSC) was manned entirely by the Technical Staff When the Technical Staffwas reorganized, those ERO responsibilities were dispersed, but personnel did not recognize that they maintained their ERO responsibilities associated with the previous department.
Additionally, gaps in the ERO organization were not identified when personnel were terminated.
As a result, the EP exercises conducted in January and February 1996 did not meet management expectations.
In the Engineering Department, the elimination ofthe configuration management group affected the configuration management function. There was an uncompensated loss ofknowledge, experience, and focus, and as a result, drawings, procedures and licensing documents have not always been updated to refiect modifications. Additionally, multiple organizational changes in a relatively short period oftime were not carefully managed (such as the elimination ofthe Technical Staffgroup and subsequent formation of the Systems and Component Engineering group, the formation and subsequent elimination of the Operations Support Testing group, and the formation of a Configuration Group in the Information Systems Department),
and have subsequently caused instability, unclear accountabilities, and in some cases, areas ofresponsibility were not seamlessly transferred.
In April 1996, the PSL site wide process for documenting problems was changed fi'om the St. Lucie Action Request (STAR) process, to the Condition Report (CR) process used at Turkey Point, to provide uniformitybetween the two sites. However, the impact ofthe CR process on PSL was not carefully evaluated and managed prior to its implementation. Littletraining was provided on the new CR process, and the process was not well understood by the staff, limitingthe effectiveness ofthe process.
The CR administrative staff was quickly overwhelmed by the high volume of issues generated by the Unit 1 refueling outage, due to a lack ofresources to adequately run the program.
In addition, the mission ofpersonnel performing CR closure reviews was changed from processing, tracking and trending CRs, to include a verification ofthe adequacy ofthe corrective actions listed in the CRs, without consideration for the required level of training and resources required to successfully manage those additional tasks.
The same individual performing the close-out reviews was also administering the In-House Event (IHE) reporting system, was the sole individual performing HPES (human performance evaluation system) evaluations at PSL, and was the chairman ofthe CR oversight group (CROG). As a result, CR close-out reviews have been less than effective in identifying weaknesses in the CR closure documents, such as the failure to identify all underlying causal factors, the lack ofconsideration ofgeneric implications, and corrective actions that do not address all causal factors.
VIIL Communications:
Less than adequate communications contributed to a lack ofcommon focus and cohesiveness in the organization.
'27
Effective communications up, down and throughout the organization are essential to the accomplishment ofimportant Division goals.
PSL had been known for many years for its teamwork and interdepartmental communication skills. However, as rapid changes in the organization and management expectatioris took place, existing communications methods could not keep pace with the changing environment.
Communication at PSL suffered because the importance ofclear written and verbal communications was not emphasized, and methods for communicating issues and
,expectations up and down the management chain were not defined.
Although some policies and expectations were clearly communicated, efFective communications depended on individual manager styles, and these varied greatly. As a result, a number ofstation performance problems developed due to communication weaknesses.
Weaknesses in operator performance, emergency drills, QA audits, the FRG, parts availability,HP/AL~ and outage planning and were all related in some way to communications.
Operations management communication and enforcement ofexpectations was not consistent or effective in all cases.
Engineering response to apparent Unit 1 hot leg temperature stratification was slow because inter-departmental communications failed to provide sufficient information and direction.
2.
Many emergency preparedness weaknesses were the result of poor and untimely communications within the ERO.
QA audits were poor in communicating problems to line management in a manner that solicited the appropriate response.
The use ofstandard boilerplate in the audit cover letters and executive summaries did not facilitate clear communications and ended up masking important issues, rather than highlighting them.
Plant management made hard hitting assessments painful for QA to issue because the communication ofinformation contrary to PSL's image was discouraged.
4.
Expectations for communicating the results ofFRG reviews by acting FRG Chairmen to the Plant General Manager were not clearly communicated.
5.
Maintenance, Engineering, and Materials organizations were working toward competing goals, and not working together (inventory reduction, dedication vs PC-l, stockpiling ofparts to a comfort level; etc.).
6.
PSL plant management communication ofHP/ALARAprogram expectations was largely informal, and many of these program expectations were not translated into procedural guidance.
7.
PSL plant management expectations to resolve long standing equipment issues during the 1996 outage were not adequately evaluated for their impact on meeting the original outage scope and duration.
28
PSL SelfAssessmenl PSL MANAGEMENTINlTIATIVES The challenge before PSL management is to preserve the excellent parts ofthe PSL culture; those programs and behaviors that contributed to long, uneventful runs, short outages, the excellent handling ofplant transients, and world class performance.
As early as 1994, FPL corporate management recognized the need to infuse the plant stafF with new management personnel that brought withthem a &esh and difFerent perspective to PSL. Historically, all ofthe key mamigement positions were filled strictly&omwithinthe plant. Corporate management desire for a difFerent perspective at PSL resulted in the appointment ofnew Quality and Maintenance Managers, both &om Turkey Point. Following the events in August 1995, to the present, most key managers have been replaced at PSL, including the Site Vice President, the Plant General Manager, the Operations Manager, the Work Control Manager, the Engineering Manager, the Services Manager, the Licensing Manager, the Business Systems Manager, the Materials Manager, and the Training Manager. Ofthe twelve key managers replaced since 1994, all but two ofthese individuals have worked the majority oftheir careers outside ofPSL.
In addition to significant management
- changes, extensive organizational changes were also implemented at PSL since August 1995. The Technical StafF Department was reorganized into the new System Component Engineering (SCE) Department, and a new SCE Manager was assigned.
An Operations Support Department and an Operations procedure group were established.
Amajor reorganization of the Maintenance Department was implemented to establish functional groups (Valves, Welding, Rotating Equipment, Stationary Equipment, and Outage 8c Projects), with new supervisors assigned to each group. Maintenance Managers were also established for each operating shift. The on-site engineering groups (e.g., Reactor Engineering, SCE, Shift Technical Advisors, In-service Testing, In-service Inspections) were consolidated under the Site Engineering Manager.
Maintenance planners and schedulers and Outage Management personnel were reassigned to the newly formed Work Control Department, and a new Work Control Manager was assigned. Anew Operations Support Department was also established, and a new Operations Support Supervisor was hired &om outside FPL.
These changes in personnel and organizational adjustments provide PSL the talent and experience needed to regain PSL's place among the best performers in the nuclear industry. PSL management is committed to forging a new era of success at PSL by building on past successes, capturing the lessons to be learned &om recent events, and correcting the weaknesses identified during this self-assessment.
Management initiatives willbe implemented to address opportunities for improvement identified during this self-assessment.
The initiatives are detailed in Appendix C to this report.
444 END 444 29