ML20099D889
| ML20099D889 | |
| Person / Time | |
|---|---|
| Issue date: | 06/30/1992 |
| From: | Dingman S, Rasmuson D NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD), SANDIA NATIONAL LABORATORIES |
| To: | |
| References | |
| NUREG-CP-0124, NUREG-CP-124, NUDOCS 9208070150 | |
| Download: ML20099D889 (133) | |
Text
<
NUREG/CP-0124 Proceedings of the Wor::cs:aoa on t:ae Use of PRA Metaoco_ogy for :ae
~
Ana:ysis of Reactor Everr:s anc Coera~iona Da:a Held at Loew's Hotel Annapolis, Maryland January 29-30,1992 Sponsored by U.S. Nuclear Regulatory Commission
,.s * "%
e'
+.,+
9208070150 920630 cNo1$h PDR
4 NOTICE l
These proceedings have been authored by a contractor of the United States Government. Neither the United States Government not any agency thereof, or any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party's use, or the results of such use, of any information, apparatus, product or process disclosed in these proceedings, or represents that its use by such third ri hts. The party would not infringe privately owned 0
views expiessed in these proceedings are not necessarily those of the U.S. Nuclear Regulatory Commission.
Available from Superintendent of Documents U S. Government Printing Office P.O. Cox 37082 Washington D.C. 20013-7082 and National Technical Information Service Springfield, VA 22161
i a
NUREO/CP-0124 Proceedings of the Workshop on the4Use of PRA Methodology for the TAnalysistof Reactor Events andLOperational Data e
Heki at Loew's Hotei Annapolis, Maryland January;29-30,_1992
- Manuscript Completed: July 1992 Date Published: June 1992 i
I
'4
- Sponsored by'
- Division of Safety Programs :
Omc'e for Analysis and Evaluation of Operational Data
. U.S. Nuclear Regulatory Commission Washington / DC 20555.
x
}
4 I s
' c',
. - +..-
f.#7 q.._= 4,-~,-.
,,-e
- 4.....
. _ _ ~
.,m.,-..
.sm_.,,-
..__ ac
,,,mu, 3p
.p.
L e i k,li
,. :o, y
>< t c' :
') ) $.4
-g d
I 5t o-j 7
f.
q M
_A
}
g T
4 a
r d
O A
1 9
9 i',
I'
.,..f.'.
?
- .. {:.'.
a
~
1 s
M 4
?
s
,,3..
I
-\\
f
' t W
h
\\
h
^
,b f
=
r I
1 M
y
- t 3
el, t _
. 3
?
I f
Ea j*.
s 3
1 S'
i f
i 1
i. (
')
u m
t
t
'h.
p.
5 f
;17 'j v
1
=
v we-+-+
.,e
>~.~w
=t-..gw
..-..., _w e
y
..-g.,,,.-
2 p
w h
EXECUTIVE
SUMMARY
A workshop entided 'The Use of PRA Methodology for the Analysis of Reactor Events and -
Operational Data" was held on January 29-30, 1992 in Annapolis, Maryland. Over 50 -
participants from the NRC,its contractors, and others participated in the meetings. During the first day, presentations were made by invited speakers to discuss issues in relevant topics, On; the second day, discussion groups were_ held to focus on three areas:
(1) risk i
significence of operational events, (2) industry risk profile and generic concerns, and (3) risk monitoring and risk-based performance indicators.
Important considerations identified from the workshop are the following:
._ Improve the Accident Sequence Precursor models and data.
Improve the SCSS and NPRDS (e.g., by adding detailed performance information on selected components, by improving narratives on failure causes).
Develop risk based performance indicators.
'Use risk insights to help fccus trending and performance analyses of components.
systems, initiators, and sequences.
Improve the statistical quality of trending and performance analyses.
Flag implications of special conditions (e.g., external events, containment performance) during data studies.
,. Trend common cause and human performance using appropriate models to obtain a better understanding of the impact and causes of failure.
. Develop a method for producing an industry risk profile.
i l
NUREG/CP-0124 iii i.
CONTEh"I'S EXECUTIV E S U M M ARY................................
iii ACKNOWLEDGMENTS viii LIST O F A CRON YM S.............................................
ix
- 1. INTR ODU CTI O N...............................................
1
- 2. S U M MAR Y O F PR ES ENTATION S.................................. 3 2.1 Accident Sequence Precursor Program Methods. Joseph W.
M i n ari c k............................................... 3 2.2 Methods for Identifying Risk Significant Trends - Gareth W. Parry....
5 2.3 Approaches for Analyzing Data to Address Generic Issues Related to Common Cause Failures, Human Factors, and Systems Interactions - Ali Mosleh...................................
6 2.4 Industry Risk Profiles: Do We Need More Modeling? - George Apo s t ol a kis.............................................
8 2.5 Use of PRAs and IPEs for Event Risk Analysis - Arthur C. Payne, Jr.....................................................
9 2.6 Living PRA Concept - Dennis Bley..........................
11 2.7 Trending Plant Performance: Thoughts on Risk-Based Performance Indicators - Joseph R. Fragola.............................
12
- 3. DISCUSSION GROUP 1 - RISK SIGNIFICANCE OF OPERATIONAL EVENTS 15 3.1 Summary of Discussion Group 1............................
15 3.2 Details of Discussion Group 1..............................
16 NUREG/CP-0124 y
I.
--_m
4; DISCUSSION GROUP 2 -INDUSTRY RISK PROFILE AND GENERIC 21 CO N C E R N S...................................................
21
- 4.1 Summary.of Discussion Group 2............................
22-
_4.2 Details'of Discussion Group 2....
Sc DISCUSSION GROUP 3 - RISK MONITORING AND RISK-BASED-PERFORMANCE INDICATORS...................................
27.
5.1 Summary of Discussion Group 3............................
27-5.2 -
Details of Discussio_n Group 3.............................,
2fs
. 6. O VE RALL I N S I G HTS...........................................
31 33
-7.. REFERENCES................
35
- Appendix A - List of Workshop Attendees...............................
41
- Appendix B - Sammary Paper
- Appendix C - Discussion Group.1 Questions and Participants...........
55 Appendix-D - Discussion Group 2 Questions and Participants.................
59
- Appendix E - Discussion Group 3 Questions and Participants.... _.............
63 Appendix F - View Graphs for " Accident Sequence Precursor Program 67 Methods". Joseph W. Minarick......................................
Appendix G:- View Graphs for " Methods for Identifying Risk Significant 79 Tre nds" G are th _ W. Parry..........................................
. Appendix H - View Graphs for " Approaches for Analyzing Data to Address Generic Issues Related to Common Cause Failures, Human Factors and Systems Interactions - All Mosleh....................................
87
- Appendix I - View Graphs for " Industry Risk Profiles: Do We Need More-95
- ModelingT - G eorge Apostolakis.....................................
Appendix J - View Graphs for "Use of PRAs and IPEs for Event Risk
~
Analysis - Arthur C. Payne, Jr......................................
105 1
77,
~ Appendix K - View Graphs for "Living PRA Concept" - Dennis Bley..........
113 Appendix L - View Graphs for " Trending Plant Performance: Thoughts on Risk-Bated Performance Indicators" - Joseph R. Fragola................
123 i
_ yj;
ACKNOWLEDGMENTS
- Patrick Baranowsky, NRC, Dale _ Rasmuson, NRC, and Susan Dingman, Sandia National Laboratories, organized.the workshop. Gratitude is extended to the speakers and the
. discussion group moderators (Allen Camp, Gareth Parry, Ali Mosleh, and Joe Fragola) for their time and effort in making the discussion sessions so productive. Special thanks are also extended to those individuals who took notes of the discussion sessions. Their efforts helped make the preparation of this report easier.
Susan Dingman prepared a draft of this report from the notes and tapes of the discussion sessions. _ Dale Rasmuson and Susan prepared the final report. Speakers provided written summaries of their presentations.
e NUREG/CP 0124 viii
LIST OF ACRONYMS ACRS Advisory Committee on Reactor Safeguards AEOD Office for the Analysis and Evaluation of Operational Data ASEP Accident Sequence Evaluation Program ASP Accident Sequence Precursor BNL Brookhaven National Laboratory BWR Boiling Water Reactor CCF Common Cause Failure CUSUM Cumulative Summation Method 50.72 Immediate notification of reportable events to the NRC by licensees FRA Future Resources Associates, Inc.
11RA Human Reliability Analysis-INEL Idaho National Engineering Laboratory i
INPO Institute for Nuclear Power Operations IPE Individual Plant Examination IPEEE Individual Plant Examination for External Events JBFA JBF Associates, Inc, LANL Los Alamos National Laboratory LER Licensee Event Report NPRDS Nuclear Plant Reliability Data System NRR Office of Nuclear Reactor Regulation NUS IIalliburton NUS Embanmental Corporation ORNL Oak Ridge National Laboratory PI Performance Indicator PLG PLO, Inc.
PRA Probabilistic Risk Assessment RES Office of Nuclear Regulatory Research SAIC Science-Applications International Cogoration SCSS Sequence Coding and Search System SNL Sandia National Laboratories e
'.h
}
A n
z i -
- f..f c -
fr -d r '
r -
)
A f
2
(_.y
\\;--
T.
g
. j 1
i l
' I J
i
.."Is--.
1 i
i e
k A
I f
a T
'_'{
I b
t p
d' M' F a.
b 1
t 4
a'-, -
).
iI y 3., -
i y
(
> vye g-vr e - MT-g+g--
yw.
wy y---'hi TT,ym%
rShTp
'TW
'W-7+M.a
_e j
- 1. INTRODUCflON y
Operational data from'a variety of sources are reviewed and evaluated by NRC to identify
- 1) sign!ficant events and any associated safety concerns and root causes,2) the trends and patterns displayed by these events,3) the adequacy of the corrective actions taken to address these concerns, and 4) the generie applicability of events and concerns to other plants.
Although programs are in place to perform these evaluations, it is recognized that there are shortcomings in the processes currendy used.
A workshop _was held in Annapolis, Maryland,~on January 29 and 30,1992, to discuss the
- current methods and the potential for their improvement. The objective of the workshop
- was to exchange technicalinformation on enhancing the use of PRA methods, information, and insights in NRCs analyses and evaluations of operating reactor experience. Over 50 participants attended the workshop. ' A list of attendees is included in Appendix A. The workshop was directed at the following topics:
(1) Use of existing PRAs and IPE models and results in performing routine event evaluations (rapid / quick-look assessments and enhancements to ASP modcls and methods),
(2) Methods and approaches for evaluating industry risk profiles (trends).
(3) Innovative uses of existing /available data sources (LERs,50.72s, NPRDS, other untapped data sources) to identify risk significant trends and safety
' issues.
(4) Potential new plant-specific risk-based performance intlicators (direct and indirect (surrogates), data needs and availability).
(5)- Risk-based approaches to se!ecting and analyzing plant-specific and " generic" trends and patterns, including common cause failure, systems interactions, and human performance concerns.
(6)~ Analytic methods, software, and/or procedures which could be used, adapted.
modified, improved, or developed to enhance operational data analysis.
The workshop was conducted to explore methods and approaches _that could provide the ability to focus on risk relevant concerns more quickly and provide a more thorough culling
- of operating reactor experience data than techniques currently available and in routine use at the.NRC.
A' paper summarizing the NRCs recent and current programs and activities related to-operating reactor events / data _ analysis was provided to each participant prior to the workshop. That paper is found in Appendix B.
Seven invited speakers were asked to
- NUREG/CP-0124 -
1
~
address selected issues related to the workshop objective. Summaries of their presentations are found in Section 2.
The first day was dedicated to introduction and presentations by the initial speakers. On the second day, the participants divided into three smaller groups to exenange ideas. Each group was assigned a general topic for major consideration. The discussion groups focused on 1) the risk significance of operational events,2) evaluating an industry risk profile end the consideration of the generic implications of events, and 3) methods for monitoring risk -
and developing risk based performance indicators. Each discussion group was furnished a set of questions for consideration (listed in Appendicer C, D, and E). The groups were also encouraged to deviate from the list if they felt it to be appropriate. Following the discussion sessions, a summary and wrap-up session was held where the discussion gr..up moderators presented highlights of their sessions.
j The workshop discussions are summarized in Sections 3,4, and 5 of this report. Each section consists of two subsections. The first is a summary of the discussion group findings, and the second contains more detailed information from the group discussions. The ideas summarized in this rcport were not necessarily shared by all participants. Meaningful ideas expressed by individuals are contained in the report, even though others in the discussion groups might not have agreed with the concept or its perceived importance. Sesion 6 contains a summary of the important insights obtained from the workshop.
1 NUREG/CP-0124 2
- 2.
SUMMARY
OF PitESENTATIONS Seven speakers were invited to address selected issues related to the workshop objective.
The titles of the presentations and the speakers are:
(1) Accident Sequence Precursor Program Methods - Joseph W. Minarick, (2) Methods for Identifying Risk Significant Trends - Gareth W. Parry, (3) Approaches for Analyzing Data to Address Generic Issues Related to Common Cause Failures, Human Factors, ano Systems Interactions - Ali
- Mosleh, (4) Industry Risk Profiles: Do We Need Mere Modeling? - George Apostolakis, (5) Use of PRAs and IPEs for Event Risk Analysis - Arthur C. Payne, Jr.,
e (6) Living PRA Concept - Dennis Bley, and (7) Trending Plant Performance: Thoughts on Risk Based Performance Indicators - Joseph R. Fragola.
Summaries of these presentations, based on summaries furnished by the speakers, are given in this section.
2.1 Accident Sgquenec Precursor Program Methods Joseph W. Minaricki Accident sequence precursors are operational events that are important elements in severe core damage accident sequences. Such precursors can be infrequent initiating events or equipment failures that, when coupled with one or more postulated events, could result in a reactor plant condition leading to severe core damage. The NRC's Accident Sequence Precursor (ASP) program searches operational evets for such precursors, analyzes and ranks them as to their likelihood of proceeding to core damage, and identifies important sequences that, more likely than others, could lead to severe core damage.
Events are currently selected and documented as accident sequence precursors if + hey include a core damage initiator requiring safety system response, or the failure of a system or degradation in more than one system required to mitigate the consequences of a core damage initiator, and if the conditional p obability of proceeding to core damage is 4
estimated to be at least 10.
Events not addressed due to low significance and 2Appendix F contains the view graphs for this presentation.
programmatic constraints include uncomplicated reactor trips, losses of feedwater without additional failures, single failures in mitigating systems, and design errors discovered by reanalysis. With the exception of initiating events, precursors typically involve event 3 not considered when applying the single failure criterion used in the design of safety-related systems.
Precursors are quantified primarily for ranking purposes - to identify events which may deserve additional scrutiny.
Quantification involves determination of a conditional probability of subsequent severe core damage given the failures observed during an operational event.This is estimated by mapping observed failures onto event trees depicting potential paths to severe core damage, and calculating a conditional probability of the event through the use of branch probability estimates modified to reflect the event.
The conditional probability estimated for each precursor is useful in ranking events because it permits estimation of the measure of protection against core damage remaining once the observed failures have occurred.
The event sequence models used to rank precursors as to significance consist of plant-class specif:c event trees and simplified plant-specific system models. These models describe mitigation sequences for three initiating events: a nonspecific reactor trip (which includes loss of feedwater within the model), loss of offsite power and small-break LOCA. The event sequence models are system-based and include a model applicable to seven plant classes -
three for BWRs and four for PWRs.
The potential for recovery is addressed in the precursor analyses by assigning a recovery step to each failure and initiating event. This assignment is based on engineering judgment, which considers the specifics of each cperational event and the likelihood of not recovering from the observed failure in a moderate to high-stress situation following an initiating event.
For analysis purposes, consistent probabilities of failing to recover an observed failure are assigned to each event in a particular recovery class. Four recovery classes, based primarily on the location where recovery actions would be required and the extent that such actions e
are proceduralized, are currently used to describe the different types of recovery that could be involved.
The quantification process for each event involves a determination on initiators which must be modeled and their probability, plus any modifications to system probabilities necessitated by failures observed in an operational event. Once the branch probabilities that reflect the conditions of the precursor are established, the sequences leading to the modeled end states (core damage and ATWS) are calculated and summed to produce an estimate of the conditional probability of each end state for the precursor. So that only the additional contribution to risk (incremental risk) associated with a precursor is calculated, conditional probabilities for precursors associated with equipment unavailabilities (during which no mitiating event occurred) are calculated a second time using the same initiating event probability, but with all branches assigned normal failure probabilities (no failed or degraded states), and subtracted from the initially calculated values. This eliminates the NUREG/CF-0124 4
l' contribution for sequences not impacted by the precursor, plus the normal risk contribution for impacted sequences during the unavailability.
In the quantification, it is assumed that the failure probabilities for systems observed failed during an event are equal to the likelihood of not recovering from the failure or fault that actually occurred. Failure probabilities for systems observed degraded during an operational event are assumed equal to the conditional probability that the system would fail (given that it was observed degraded) and the probability that it would not be recovered within the required time period. The failure probabilities associated with observed successes and with systems unchallenged during the actual occurrence are assumed equal to a failure probability estimated from either system failure data (when available) or by the use of system success criteria and typical train and common mode failure probabilities.
Operational events which satisfy the precursor selection criteria are documented annually (NUREG/CR-4674 series reports [1]). While the selection of precursors has remained relatively consistent over the 1984-90 observation period, some differences do exist in those which have been documented. These differences relate to the types of events selected, the accident sequence models utilized, and the application of a minimum conditional probability 4
(10 ) before an operational event is documented (for 1987 and later). These inconsistencies must be considered when comparing the numbers and types of events in different time periods.
Improved event tree models are being developed for use in evaluation of daily events by NRR. These models are based on the ASP models, but reflect NUREG-1150 insights to a greater degree than the current ASP models, include additional initiating events, and address alternate long-term cooling strategies. These models will be usable in the ASP program once they are completed.
2 2.2 Methods for Identifying Risk Significant Trends - Gareth W. Parry r
The topics discussed during this presentation included: (1) methods to screen event data for risk significance, (2) analysis of the reduced data for trends, and (3) data needs for meaningful analysis.
The talk addressed the use of PRAs to help identify trends in data provided by event data bases such as the LERs, and component data bases such as NPRDS. One important role of a PRA was identified as being a filter to screen for risk important events using the tools ofimportance analysis. However, PRAs have limitations. They generally do not model non-full power states, they do not include all componen*s, and they do not model failure causes.
In addition, there may be important assumptions which impact the structure of the model, 2Appendix G contains the view graphs for this presentation.
1
and thereby the assessment of importance of certain events. Nevertheless, with care, a PRA can be a very useful tool.
An important issue is how to perform screening. Should events be assessed against all plants or just at the plant in which they occurred? The reason for assessing events against all plants is that events, which do not cause significant problems at the plant where they occurred, may indicate potential problems at others. Also, degraded states, as opposed to failures, when analyzed for their causes, may indicate trends or underlying problems.
Analyzing events in this fashion would increase the work considerably, and also means an analyst must understand a lot about the events. This is probably not very practical, although this is probably where the biggest value may lie. Essentially one would have to have a "model" of the event that addresses the whole chain of sub. events that lecd to the event.
This was illustrated by reference to the cause-defense approach to the analysis of CCFs
[ Reference 2]. (This would be in the nature of an ideal case.)
Given a set of screened data has been obtained, there are many methods to analyze for trends. The methods themselves were not discussed in any detail. However,it was pointed out that this cannot be done in a random fashion. The analyst has to have some idea of what he is looking for, as this will tell him how to partition the data. Therefore, it is important to establish a cause-effect hypothesis before analyzing the data. Some examples were given of how this impacts grouping of data. For example, in exploring aging, the time origin is start of life, whereas in exploring the impact of regulatory changes, the origin should be taken as the date ofimplementation. Since the models are to be used to identify trends, they do not need to be mathematically structured, a qualitative understanoing of the effect of changing the independent variable may be adequate.
Establishing these cause-effect models also helps understand what, if anything, is missing from the data as it is currently collected that prevents the hypothesis /models being tested correctly. An example was given of the data needs that have been established for common cause failure analysis [2].
2.3 Approach _ts for Analyzing Data to Address Generic Issues Related to Common 3
Cause Failures. Human Factors and Systems Interactions - Ali Mosleh To improve the quality of the accuracy of PRA models, operational data must be used both qualitatively and quantitatively. Equally important, but much less acknowledged,is the need for an underlying model to guide data collection and analysis. These two processes ought to be interactive and iterative, leading to an evolutionary improvement in models and data.
A common cause failure (CCF) event can be decomposed into two key elements. The failure depends on the occurrence of a trigger event (such as a flood in a particular room) 3Appendix H contains the view graphs for this presentation.
l 1
and then on a coupling mechanism which results in multiple failures (such as two pumps being located in the same room with motors susceptible to moisture). CCF events can be classified into two categories, with Type I indicating failures almost immediately after the trigger events and Type II indicating delayed failures. Each of these classes can be further divided into classes in which the coupling factor couples components in either a random or a dependent fashion.
Data needs for analysis depend on the level of the CCF model. For effective use of current models, the following are needed:
More accurate description of the events is needed in terms of cause., and impact of the event, Level of redundancy, and Success data.
Improved models will need (as a minimum) information on:
Coupling factor (s),
Barriers and defenses both against the cause and the coupling, and Failure times.
Future models will need,in addition to the above,information on the physical nature of the root cause and coupling facter of the events.
CCF events are rare. For example, a two-unit plant with more than 22 years of operational data has only experienced six CCFs, which is about 5% of all failures experienced. Out of more than 4000 LERs reviewed, only about 150 were CCFs of the type modeled in PRAs (power operation only). Review of the CCF data indic:ites common characteristics with generic implications, particularly with respect to coupling factors and defense strmegies.
Plant-specific PRAs must consider industry (generic) experience for completeness of CCF modeling and realistic assessment of probabilities. Data from various plants need to be analyzed according to a cornprehensive classification system in order to gain generic insights into the underlying causes of CCFs. Current data reporting systems (including LER and NPRDS) lack adequate recording and reporting guidelines for CCF events.
Human reliability estimates as applied to nuclear power plant PRAs are almost completely based on judgement. Even in those cases where data collection has been attempted, models which are neither validated nor supported by a theoretical or empirical foundation dominate NUREG/CP-0124 7
the resuhs. With the exception of a recently instigated AEOD program, there has been no systematic effort 'to compile and analyze actual operating experience from a human performance point of view! Generally speaking, current models do not reflect actual operating experience. Even qualitative insights from the limited operational data have not
~
been used systematically in the models.
. Complete quantitative data required for calculation of error probability estimates are sparse a
(at least for direct estimation) since success data are very difficult to obtain. Nevertheless, some consideration should be given to identifying possible approaches for collecting success data. This might be easier in the case of operator response to initiating events. Efforts in the area of-collecting, analyzing and classifying human performance data should be expanded, The direct benefit will be in gaining insights into causes and modes of human errors. Such insights can be used to improve plant safety, sometimes with minor changes in plant or operating practices and procedures. They can also provide much needed "real life" mput to the human reliability model building activities.
4 2.4 Jndustry Risk Profiles: Do We Need More Modeling? - George Apostolakis The principal thesis of this presentation is that operational experience is of limited value, unless it is interpreted through validated models, llowever, it is recognized that develop ng i
n such mcdels may require the expenditure of significant resources. This thesis is supported by three classes of problems that can serve as illustrative examples.
The first class of problems deals with failure rates of components.
Neglecting the plant to plant or environment-to-environment variability of the failure rate may lead to distortions of its distribution. For example, the high tail of the distribution normally reflects severe accident environments.- Since most of our operational experience is from routine tests, any updating of the distribution should not affect this tail. This important point is not
. ahvays appreciated and, as a result, unreasonably narrow distributions may be produced.
Relevant references are [3-4]. Furthermore, the evidence itself may require interpretation
- and this can only happen through the appropriate models [5-6].
The second class of problems emphasizes the usefulness of reliability physics modelt These require that individual models be developed for the various physical mechanisms that may
-lead to. component or system deterioration and failure. By going down to this detail, we can
- include the evidence in the place where it belongs. For example, such physical models are used in _the analysis of " external"' events (earthquakes, fires, etc.). Thus, strengthening a structure or improving the fire resistance of cables can be accommoda:ed at the light place in the overall methodology. Aging effects can also be considered [7].
dAppendix 1 contains the view graphs for this presentation.
-NUREG/CP-0124 8
i
The third class of examples deals with human actions and the factors that may influence them. A very $vell known in';ident is the ont that occurred at Davis 13 esse on June 9,1985
[NUREG-1154). Part of the squerice of events was the hesitation of the shift supervisor to initiate bleed-and. feed coohng, in spite of the recommendations of the secondary-side operator and of the operations superintendent. This hesitation has been discessed widely, and people have specuHed regarding ;ts causes. For example,it is stated in NUREG-1154 that 'the shif sener-isor appreciated the economic consequences of initiating MU/IIPI 1
f
[ bleed and-feed] cooling."
What is very interesting is that the precursor report NUREG-,674 treats this heatation as a non event. All kinds of recommendations are made l
based on events that are clearly perceived as failures (e.g., the iacorrect actuation of the Steam and Feedwater Rupture Control System),
llowever, there are no specific tecommenf ations stemming from this observed hesitation. One could, in fact, argue that this hesitation may be an indication of senior management's priorities, in which case many different accidents may be affected. The reason for this omission is that the writers of NUREG 4674 had no model of operator behavior which allowed them to place the incident in perspective and to investigate its possible causes and/or consequences in a systematic way. The problem is, of course, that, even if they had been willing to use such a model, they would have found very quickly that it did not exist. Some preliminary thoughts are given in [8], but they are, indeed, very preliminary. This is a striking example in which the lack of a model leads us to essentially ignore an important piece of operating experience.
2.5
_Use of PRA, and IPEs for Event Risk Analysis - Arthur C, Payne. Jr 5
= '
Current estimates of risk from nuclear power operation are based on either PRAs or operational event analyses. Through PRAs, we identify combinations of equipment failures and human actions that lead to risk-significant events. These assessments are limited by our current understanding of aspects such as system performance, phenornena, human performance, etc.13ecause we are concerned that we might have missed some important aspects of plant response, we turn to operational data for potential insights. Ilowever, operating data can not indicate all potential problems because the events rarely occur. An approach is suggested here that would gain insights by comparing PRAs and operational data assessments. These insights could then be fed back into both types of analyses to yield better results.
The basic idea of the proposed system is to use expert systems to continually contrast PRAs and operational data analyses to get constant improvement in our identification and understanding of potential accident precursors. A two-fold approach is used. First, new events are examined to help identify any weaknesses in the analysis methods. Secend, comparisons of operational events with known risk-significant events 1re made to identify those events that are important enough to merit furtner investigation. The following method is proposed:
5Appendix J contains the view graphs for this presentation.
' Event Identification and Classification P
- 1. - A model of the characteristics of both PRA events and operational events is constructed.
Because the approaches each have unique aspects, the characteristics would be expected to be different.
- 2. - As new PRAs are performed and operational events are observed, the model describing the characteristics of events is reviewed for completeness, and updated if necessary. An expert system would be developed for this process.
The expert system and analyst would compare the new events to the model of event characteristics and determine whether or not the current model A adequate. If not, the additbnal characteristic. encountered with the new eve'nt are added to the model of event characteristics.
3.
The system modeling and data analysis techniques are compared to the theoretical model to develop an understanding of the limitations of each technique and possibly generate improved models.
4.-
The expert system reviews the data. base of PRA and operational events to reclassify events using any new ch" acteristics. The expert system would identify events that might have the new characteristics for which the current information is insufficient to conclude this for sure.
Event Imp.gitance to Safety 1.
A data base is constructed that consists of PRA results, data bases such as LERs, NPRDS, etc.
2.
An expert system is devised that will allow the analyst to enter a description of an event'( cither observed through operation or identified in a PRA) and then compare the event to the characteristics of events that have previously been determined to be significant to risk. If the event characteristics do not 4
match _ theJcharacteristics of any risk-significant event, the-system would identify those additional characteristics that the event would require in order to be important to risk. The system should also be able to compare the characteristics of different plant designs and be able to identify those plan:s at which an event might be important.
-3.
For the evaluation of the importance of new events, an interface with a set of PRA analysis codes such as IRRAS would be developed so that the analyst could: modify the models as appropriate, input the effect of an event into the
- affected models, and then evaluate the results for affected plants. The models could be surrogates at various levels of detail (current ASP, ASEP, IPE, or full PRA) or could be individual plant models, also at various levels of detail.
NUREG/CP-0124 10 1
-<-e m
L The level of detail should probably be at least to the level that support systems are explicitly in the model.
4.
A data analysis capability would be developed so that the analyst could keep track of all identified precursor events anr1 hen an up-to-date quantification of those events which have occurred.
2.6 livi_ng PRA Concept - Dennis Blev6 The most important characteristics of a living PRA were discussed. The models and data need to reflect current plant conditions so that tl'e PRA model will provide an accurate representation of the current status of the plant. Ti.e models need to be constructed such that the physics, system interactions and dependencies, and human interactions properly reflect the real plant. Also, to provide an accurate representation, the essential systems all need to be included in the model. To be useful, the PRA must be adaptable so that new questions can be answered. The model must be easily modifiable so that proposed plant changes (hardware, procedures, technical specifications) can be tested.
There are several uses for living PRA. Living PRA can be used to identify weaknesses in the plant, as a risk management tool, and to support the higher level overall risk / decision analysis. As a risk management tool, it can be used to set priorities for maintenance, training, and plant changes, thus optimizing fixed budget / time frame efforts. It can provide the basis for planning and developing accident management procedures and has the potential for providing real-time projections and decision support during some accidents.
Living PR/ can also be used as an educational tool for developing an improved risk awareness by operators and management. It can also be used to evaluate the significance of operating experience.
To be fully effective,living PRA must be understandable so that the risk implications can be communicated outside the PRA community. Examples were shown of some approaches for more effectively presenting the PRA.
There are some additional requirements for living PRA beyond those present in traditional PRAs. Configuration management of the PRA model will be a concern because the models and dc.ta will be continually updated. There will be a need for continual updates to reflect plant changes, new failure d.sta, new models suggested by industry events, etc. In the talk, an example was presented of a structure for the PRA that was developed to simplify the task of updating the PRA.
Smaller, simpler event trees were constructed for plant maintenance, configurations during various plant states, support systems, etc. The individual trees could be easily updated in this form and then linked together to give tSe integral result.
6Appendix K contains the view graphs for this presentation.
(
Advanced methodologies such as dynamic interaction models, human cognitive models, and organizational models may be appropriate.
Advanced computer tools need to be implemented for improved reporting speed, query capability, and communication.
Possibilities to explore include artificialintelligence, hypertext, and improved graphics tools.
Living PRA could be used as a tool for performing plant-specific examinations of precursor events and to identify weak spot; in PRAs.
It gives an incentivt for doing cross classification of information coming out of the precursor program to support verification of human error rates, system / subsystem unavailability, etc.
2.7 Trendine Plant Performance: Thouchts on Risk-Based Performance Indicators -
7
~
Joseph R. Fragola Indicators of the safety performance of any plant or system provide information concerning the past, current, and/or future performance of that plant or system. Indicators can be either direct or indirect, leading or lagging. While these appellations are not exact labels, it is true that in the former set direct indicators are those provided by variables themselves or by simple evaluation of performance functions by the direct insertion of variable values.
On the other hand, indirect indicators are those which correlate in a non random fashion with plant or system performance. In this case the analytical relationship responsible for this correlation may be somewhat obscure or even totally unknown. In the latter set of labels, " leading" indicators are those which presage future performance while " lagging" or
" assessment" indicators deal with a determination of the past or current performance. By their nature, lagging indicators tend also to be direct, and correspondingly, leading indicators tend to be indirect, but these general tendencies should not be considered ironclad rules.
Different types of indicators play different roles in providing information on the safety and
" health" of operating commercial nuclear power plants and in detecting trends in that health.
Consider the analogy to human health. In this case, if the body of a particular individual is considered analogous to the plant, and the diagnostician the NRC regulator, what insights L
can be drawn? Firstly, it should be recognized that no diagnostician makes a diagnosis and suggests a course of action on one indicator alone. This is only accomplished via the review of a systematic collection of indicators. The diagnostician must be well av :e of the anatomical systems in the body and their naturally healthful state. When the patient exhibits acute symptoms, they are brought to the attention of the diagnostician. These he must investigate ia light of his knowledge of the normal healthful state of human system and in light of this particular patient's history so that he can determine if the symptoms indicate -
a potentially seriour condition and what actions must be taken to mitigate its development.
Finally, on a regular basis, the diagnostician should review the vital signs of the patient to indicate the presence or absence of potential deviant signals. This should be done in an 7Appendix L contains the view graphs for this presentation.
6 attempt t'o determine which patients might need special attention to investigate their health condition further without'any acute symptoms of which the patient is aware, j
- In the-same sense, these three-tools; knowledge of anatomy and specific health history,
- recognition of and determining the implications of acute symptoms, and recourse to regular,.
3 albeit indirect, Indications of health via vital sign monitoring are - the heart of any
- performance indication system. In the commercial nuclear power plant context, a well structured and implemented PSA or IPE can provide insight into the anatomy of the plant.
- When this is compared with a thorough review of the plant operationa! history, it rovides a sound basis for the first tool in the set. However, the consequence severity and the rare occurrence nature do not allow us to' limit the occurrence of acute symptoms only to an individual plant. To expand the data set, all plants must be considered as sym m in
- generators and insight into the ptential for a serious condition must necessarily be assessed somewhat generically It is in this manner that the Accident Sequence Precursor Program (ASP) provides (to at least some degree) th' second tool.
e As good as these tools might be, they do not provide for the capability of having a reasonably, rapidly reacting. indicator which portends the potential for individual plant w ance degradation in almost real time. Any such indicator must able to act quickly,
- .. a.,e able to be provided in a regular fashion from an objective data set, must be reasonably correlated to_ plant performance, and must produce an acceptable level of false positives and negatives.
One potential-indicator which was considered, after --a
. comprehensive review which investigated over several dozen possibilities, was the average daily power level at a plant. The average daily power level offers advantages in that it is recorded daily and reported monthly. It is also objective in that it could be a metered output of the plant.
The question is whether by using average daily power level and constructing indicators in terms of its variations and fluctuation could a leading indicator of plant performance be provided?-Study seems to indicate, at le'ast after initial review, that combinations of
- measures derived from the average daily power level signal can be of significant value in
~
indicating future individual plant trends. If these ini. 'l results are confirmed, it appears
.that this measure set can be correlated with actual future plant performance in over 75%
of the instances and provide validated predictions within a three to six month time frame.
1The use.of-such an indicator could be of significant value to aid in prioritizing regulatory resources, and when combined with the PSA and ASP tools represents a formidable arsenal to attack the problem of commercial nuclear power plant performance trending.
- 3. DISCUSSION GROUP 1 - RISK SIGNIFICANCE OF OPERATIONAL EVENTS Discussion Group 1 focused on the risk significance of operational events associated with an individual nuclear power plant. The suggested discussion questions are found in Appendix C. Section 3.1 contains a summary of the important information and insights produced by the discussion group, and Section 3.2 contains a more detailed presentation of the information.
3.1 Summary of Discussion Group 1 This group discussed problems in evaluating the risk significance of operational events. The group primarily addressed weaknesses with current approaches, data needs for improved assessments, the role of uncertainty evaluatien in operational event analysis, and general implementation conceras.
The group identified three main areas where improved modeling is needed for the ASP program: (1) developing plant-specific train level models, (2) treating degraded equipment in addition to failed equipment, and (3) improving screening methods to ensure that the important events are efficiently identified. In addition to plant-specific train-level models, it was felt that more detailed models (to the component level) should be developed as information becomes available from the IPEs. These models could be used to supplement routine ASP evaluations with more in-depth evaluations when necessary. Such models would require verification.
In terms of completeness, four potentially important areas, currently not considered in the ASP program, were identified: (1) low probability /high consequence events, (2) external event implications, (3) design and construction errors, and (4) low power / shutdown events.
It was felt that these should not necessarily be analyzed for their risk significance, but flagged as potentially important.
Inadequacies in data reporting were also noted as impacting the quality and usefulness of the ASP results. The validity of the risk estimate is dependent on the quality of the information in the operational events data report. The current data reporting system was felt to be inadequate. Some specific means of upgrading the data were identified. For LERs, a description of the plant configuration at the time of the event should be included.
This should also include the status of all safety or safety-related equipment (e.g., whether
- the equipment is operable, inoperable, or in maintenance). Other data sources besides LERs should be utilized. It was suggested that interactions be initiated with the utilities to bring the n.sdels and failure data they are developing for their IPEs, IPEEEs, etc. into a data base, if feasible.
The role of uncertainties in operational event analysis was felt to be goal dependent. For screening and ranking purposes, point estimates were generally considered adequate.
llowever, for quantitative use in determining risk profiles for decision making, ureertainty/ sensitivity evaluations would be needed to provide a better risk profile.
Some consideration was also given to potential methods for discovering unknown problems.
Such an activity would be complementary to the ASP analyses. An approach, similar to
" black hatting" in sabotage analysis, was suggested in which individuals would look through the events and try to envision t abinations of failures or actions that could lead to problems.
i 3.2.
Details of Discussion Group 1 i
This group discussed problems in evaluating the risk significance of operational events. The group primarily addressed modeling weaknesses with current approaches, data needs for improved assessments, the role of uncertainty evaluation in operational event analysis, and general implementation concerns.
Various uses of ASP were cited throughout the discussions, and in fact, the different possibilities sometimes appeared to add confusion to the discussions when various participants were discussing ASP issues for different uses without recognizing that they were each considering different end uses. The possibilities discussed included screeniag events to identify those that warrant more in-depth evaluation, identifying problem areas at plants, providing estimates of plant risk conditional on the event occurring, providing data for benchmarking plant performance relative to PRA assumptions, and trending plant and/or equipment performance. During the discussion, it was generally agreed that the most appropriate use of ASP is for screening events. Subsequently, it was noted that ASP analyses normally lag the events by a significant time, so isolated events have already been identified for in-depth review before the ASP analyses are performed. ASP has been found to be most useful for identifying groups of events to analyze and as a check that the appropriate events were analyzed. NRR is also using ASP for rapid evaluations of events, with some coordination with the AEOD activities.
The appropriate risk measure to use in precursor studies was discussed. Possibilities ranged from using estimates of health risk to using the probability that the needed systems would be available to respond during the particular event. Concerns were raised regarding data availability for measures that go beyond core damage probability, but it was felt that events with containment bypass potential should be flagged.
The group discussed the need for improved ASP models and determined that the modeling quality needed depended on the particular end use of the ASP results. Three main areas where improved modeling is needed for the ASP program were identified: (1) developing plant-specific train-level models, (2) treating degraded equipment in addition to failed equipment, and (3) improving the screening methods to ensure that the important events are efficiently identified.
The IPEs were viewed as useful for developing train level models, but there was concern over using the specific models and quantification that are submhted. The IPEs were felt to be most useful for identifying system configurations. Significant effort would be required to develop such modeli, for use in the ASP program. Another suggested use was to use the IPEs to' establish plant vulnerabilities that would be textually described (vs. quantitative inclusion) for use by the ASP analyst. The IPEs might also be used to identify plant-specific recovery actions. In addition to plant specific train level models, it was felt that more detailed models '(such as component-level models) should be developed as information becomes available from the IPEs. These models could be used to supplement routine ASP evaluations with more in-depth evaluations when necessary. The IPE models would need verification, and this is not currently being done.
Methods for treating degraded equipment are not currently available for PRAs or ASP.
When degraded conditions are detected without failure, the degraded equipment should probably not be modeled as failed, yet it would have a higher probability of failure than the previously assessed value. New techniques would be needed to adjust the failure rate appropriately.
The selection criteria for ASP events and the labor intensive approach currently needed for performing the reviews were discussed. Generally, multiple failure events are selected, rather than single failure type events. The possibility of further automating the process was discussed but it was felt that it was unlikely that the process could be refined much further without changes to the SCSS program. The problem of estimating generic implications of a specific failure was discussed, that is, a plant other than the plant that experienced the event might have a specific vulnerability which if coupled to the particular event could be a problem for that the plant (but not the plant at which the event actually occurred). It was suggested that p! ant-specific models be implemented to overcome this difficulty, and that attempts be made to involve plant personnel in the evaluation.
It was suggested that the ASP models be sent to the plants for review, and noted that this has been done in the past. The responses from the plants were typically one-sided, pointing out unavailabilities that were too high, but seldom indicating a value was too low.
In terms of completeness, four potentially important areas, currently not considered in the ASP pogrem, were identified: (1) low probability /high consequence events, (2) external event implications, (3) design and construction errors, and (4) low power / shutdown events.
It was felt that these should not necessarily be analyzed for their risk significance, but flagged as potentially important. Conditional core damage probability should be the primary
. ASP measure for evaluation, but other risk measures such as containment failure or health risk should not be ignored. It was felt that ASP is not the appropriate tool for evaluating these other measures, but it could be used to flag events with potentially high consequences such as containment bypass.
NUREG/CP-0124 17 1
-_ ~ - _. _ _ -
Events that are important, when considered with the potential occurrence of an external
_ initiator (e.g., fire, flood, earthquake), should also be flagged. One possible way of doing this would be to identify the types of events that tend to be dominant in external event PRA accident sequences, i.e., failures that are significant in combination with external event initiatcrs.
Design and construction errors and low power / shutdown events should also be flagged for further evaluation. Design and construction errors would be difficult to analyze in ASP because the failure to meet the design basic is identified in the LER, but the margin is not given. The utilities are generally reluctant to provide further details because the problem 4
would normally have been fixed and no longer a concern to them. Guidance from the low power / shutdown PRAs might be needed before these events could be treated in ASP. The number of possible system configurations are significantly greater than at full power.
It -was _ noted that support system failures with interdependencies require significant knowledge of the actual ASP coding to evaluate. The feasibility of using IRRAS/ SARA to overcome these difficulties is being examined by NRR and NRC contractors.
IRRAS/ SARA would also allow uncertainties to be included in the evaluations. Further, importance measures would be available which could be us-ed to identify the important equipment given a particular event has occurred (which could be different from the important equipment identified in an unconditional PRA).
Inadequacies in data reporting were also identified as impacting the quality and usefulness of the ASP results. The current data reporting system was felt to be inadequate. The validity of the risk estimate is dependent on the data quality. Some specific means of upgrading the data were identified. - First, a demiption of the plant configuration at the time of the eve'nt should be included in the Li.is. The utilities should report any other known system unavailabilities at the train level (in addition to the cause of the trip), in addition to inoperable systems. Under the current reporting requirements, the LERs are often so cryptic that they are difficult if not impossible to use. An event should not be split across LERs, as'is currently done in some cases. A flag that would link LERs to the 50.72s previously called into the NRC operations center would be useful. There were felt to be other deficiencies in the LERs, but they were not specifically listed by the group.
It was noted that there might be resistance for improved reporting because it may not acceptable to impose additional reporting requirements. It was noted that the standards for LER reporting are currently being revised, but the revisions will mainly address reportable events, rather than the quality of reporting.
It was suggested that data reporting might be improved by reducing the adversarial relation
- between NRC and the utilities and/or giving more incentive instead of penalty for reporting events.
NUREu/CP-0124 18
-m.
4 1 j
It was felt that other data sources should be tapped for the evaluation. The frequency of
)
entering technical specification action statements was felt to be a good indicator of potential equipmer.t problems; It was suggested that interactions t-initiated with the utilities to bring the data they; are developing for their JPEs, IPEEEs, etc. into a data base.
The role of uncertainties in operational event analysis was felt to be goal dependent. For 1
screening and ranking purposes, point estimates were generally considered adequate.
However, for quantitative use in determining risk profiles for decision making, uncertainty / sensitivity e.aluations would be needed.
It was noted that single train unavailabilities will not be flagged as ASP events since they are not required to be reported by 10 CFR 50.73 (LER rule), and suggested that a parallel
_ program to ASP might be desirable to look at these types of failures as a check for PRAs.
Some consideration was also given to potential methods for discovering unknown problems.
The limited resources allotted to the ASP program was considered to be a problem in this regardi 11 was-felt thato not enough -time was devoted to determining the potential implications of events. An approach, similar to " black hatting" in sabotage analysis, was suggested in which-individuals would look through the events and-try -to envision combinations of failures or actions that could lead to problems. The possibility of essentially sampling the PRAs with varied combinations of failures to identifv important vulnerabilities was discussed. The approach was felt to provide useful information, but at a high cost.
Guided _ searches for postulated types of failures might be more feasible. A related concept would be to use this approach to search for potential errors of commission.
- It was felt that a small cdvisory committee for ASP would be beneficial. The committee should contain about three experts from different areas (e.g., risk analysis, statistics, and nuclear power plant operations). The group would provide review of methods and issues related to ASP.
lit was felt that it-is not appropriate to simply wait for events to occur and then evaluate
~
them, but instead, to augment these evaluations with analyses that search for possible problems at plants. A combined approach was felt to be necessary rather than focusing only on operational events evaluation. It.was noted that NRC performs the event evaluation
- portion of this' combination, but not the analysis portion. It was felt that plant-specific analysis should be performed in addition to' plant-specific event evaluation. It was noted that current NRC priorities in AEOD do not include this activity. The possibility of having the plants themselves perform this analysis was discussed, but the plants suffer from resource limitations.- There was also concern that the plants would not give proper credence to -
events / data from other plants. The need for peer-reviewed PRAs for each plant was aired,
- but the group in general did not believe this would become a reality.
. NUREG/CP-0124 19
m 4.- DISCUSSION GROUP 2 -INDUSTRY RISK PROFILE AND G ENERIC CONCERNS Discussion Group 2 treated the issues ussociated with developing an industry risk profile and also issues with industry trends associated with operational data. The suggested discussion questions are found in? Appendix D. - Section 4.1 contains a summary of the important information and insights produced by the discussion group, and Section 4.2 contains a more
' detailed presentation of the information.
4.1 Summary of Discussion Group 2 Discussion Group 2 focused more on generic concerns of event analysis. The group considered methods for assessing the impact and implications that a single event, which occurred at a particular plant, would have on other plants, as well as the evaluation of a group of similar events (trending). The group also considered the feasibility of generating an industry risk profile. In addition, the group discussed the possibility of using IPE/PRA information for improved ASP modeling and in development of an industry risk profile.
PRA or improved ASP methods can be used to assess the importance of the event from a risk perspective. -For those events important to risk, the qualitative and quantitative implications of the event must be considered in more detail. The cause(s) of failures should be evaluated.
When evaluating a group of events, analyses must be focused in some manner. One way is to use insights from PRAs and IPEs. Thus, risk-h. portant components, identified from PRAs/IPEs, are candidates for trending studies. For components with large numbers of observations, standard statistical techniques are available to assess trends.1lighly reliable components, such as the reactor vessel and batteries, are another group of components which should also be analyzed. Techniques, such as reliability physics methods, can be used to estimate failure rates for these components. The causes of failure should be identified
- in any analysis, and it is probably better to trend causes rather than failures.
Human performance and common cause failures are other quantities which should be analyzed and trended. Models, such as human reliability models and the cause-defense models, can help in such studies. Again it is impartant to identify the causes of failure and trend them if possible.
Event analysis can lead to the identification of new phenomena, such as uew failure mechanisms which were not recognized in the design process. This type of information is important for any component, especially electronic components, even though the component ma not be risk significant. Thus, it is important to monitor the number of failures of components on a regular basis.
The other main topic the group discussed was development of an industry risk profile. The group assumed that severe core damage frequency was the risk measu-1g trended. The NUREG/CP-0124 21
m.
group felt that IPE results and information may be useful inputs te the development of an industry risk profile, liowever, caution mutt be used when combining information from
- different sources. Some things to consider are: (1) scope of the analyses,(2) quality of the various analyses,(3) assumptions used in the analyses, and (4) analyst to analyst variability.
Several concerns were voiced regarding the evaluation of sparse data. When using such data for trending, uncertainty implications must be considered. False trends can appear if uncertainty is not considered in the analysis, it wu noted that there is a tendency to i
consider the occurrence of events as indicating an increased risk, rather than as simply actualizing the predictions. Related to this concern was the tendency to laN1 events as comrnon cause, when again it might simply be realizing the expected number of combined but independent failures. The need for additional guidance and patience when collecting and analyzing operational data was emphasized. It was felt that data are often analyzed in depth before there is sufficient information to produce a meaningful quantitative result.
However, qualitative insights can start to develop with small amounts of data.
4.2 J)hipils of Discussion Grono 2 The major topics discussed were the use of PRA or ASP type approaches for screening events (based on their risk importance) to identify those that warrant in-depth evaluation, considerations when trending operational data, and the potential for generating an industry risk profile, Related toples arose and were considered during the process.
l For evaluating the potential impact of an event at a plant (screening), the importance results from PRAs were felt to be useful, as long as the affected systems were present in the PRA used for the analysis (e.g., not previously truncated out). The group suggested either an upgraded version of the current ASP methodology or train level PRAs for evaluating the i
4 potential impact of an event at one plant on other plants Train level PRAs would be needed rather than the more typical component level PRAs beause operational data is norrnally reported at the train level. It was felt that the ASP metnodoles would need to be upgraded to the level that is demanded of PRAs. Particularly, ASP wald need to be upgraded to include uncertainty and to improve the treatments Aonancn cause failures and human performance.
The possibility of actually using IPE submittals as the basis for improved models in the ASP program and for generating industry risk profiles was discussed. The information required
' for the IPE submittal by the utilities would not be sufficient _to construct models. The utilitbs are required to furnish information such as the event trees, dependency diagrams, and dominant sequence results and descriptions;, they are not required to furnish fault tree drawings, system unavailabilities, cut set listings (dominant and truncated), etc. Additional difficulties would arise when tr ing to use IPEs that were developed using different approaches, particularly, small event tree /large fault tree versus large event tree /small fault
- tree methods. :It was noted that data bases are currently being developed for IPE/PRA results which are intended be used ia explore the similarities and differences among p'cf ats.
NUREG/CP 0124 22 u.
- -.~
1hrough this work, it might be possible to bin plants, rather than having a separate model for each plant.
In discussing the use of ASP for event evaluation, several points were noted. Attempts were made to compare ASP and PRAs. It was generally agreed that ASP actually represents a conditional PRA, and as such, the same rigor should be demanded of it as PRAs. This would include uncertainty analyses for uses that require quantification, but uncertainty analyds would not be needed for screening purposes. Improved treatment of, mmon cause fallutes and human performance was also felt to be necessary for ASP. It was noted that good human performance models are not currently available but that there are better treatments than those currently used in ASP. 1-lowever, no improved treatments were suggested.
Currently, ASP caletdates event importance through a conditional core damage probability.
This was considered appropriate because of the data scarcity; changes in component failure rates vald occur too slowly to be useful. A complementary approach was suggested that
.would pve a different perspective. For a particular event, the impacted equipment would be identified, and then the failure rate would be updated to reflect the occurrence of the failure. Core damage frequencies would then be calculated using the base failure rate and the updated failure rate and compared. This would indicate the risk impact c' the event as a change in core damage frequency.
It was suggested that utilities could determine the impact of a particular event on their plant by using the plant's IPE to evaluate the event, it was also suggested that plants could use their IPE to assess the impact of events which occurred at other plants on their own plant.
This would be a plant specific screening of operational events.
The impliotions of the term " event" were discussed. It was noted that an event can be a complicated set ofinterrelated failures, a single failure of a component, or the identification of the possibility of a failure of a component without it actually occurring, it was i
acknowledged that quantifying this last type of event would be very difficult because the applicability of the observed situation to other situations would require the evaluator to make engineering judgments which require information not readily available, in some cases, it was felt that models could be used to aid in this quantification.
A continualloop between ASP type approaches and snapshot PRAs was suggested in which the PRAs help guide selection and identification of important events. The event analysis leads to updating PRA models and quantification. A difficulty in implementing this process arises because data are reported at the train level, while PRAs are normally modeled at the p
L r
NUREG/C60124 23 u
1
..,...._...~.,,m-,
component level.'llowever, it was felt that approaches could be developed to inch.de train-level data in a component. level PRA."
- When evaluating trends in groups of similar evenu, techniques such as CUSUM and control charts were suggested for components modeled in PilAs that have relatively large numbers of demands and failures. For highl) reliable components, such as the reactor vessel, the scarcity of data would prevent the use of such techniques. For these cases, reliability physics type approaches could be used in which the problem would be decomposed to a level at which a model could be constructed that would be amenable to such methods. Even more difficult would be the evaluation of human performance. The group did not believe current methods adequately model human performance, and that it would probably be quite some time until adequate methods could be developed, llowever, human performance issues should be flagged for now to compile information on the characteristics of these events so they can be used for improved models on human performance.
The treatment of common cause failures in trending analyses and for the identification of potentially important generic issues was discussed.- It was felt parametric models, such as the p factor model, would not be useful for such studies because the) are essentially empirical. A model for the failure would need to be developed using an approach such as the cause defense-framework [2]. - Each observed event would be unique, requiring a separate model to describe it. Once a model has been constructed for a particular event, it could be applied across the industry to test for additional occurrences of the problem.
"ractical considerations limit this approach. That is, it is not feasible to check every plant for each kind of common-cause failure that is identified.
Several concerns were voiced regarding the evaluation of sparse data. First, it was pointed
- out that it is difficult to trend such data on a plant-specific basis, Statistical methods usually require a moderate number of observations to provide useful results with high confidence.
Next, when using data for trending, the uncertainty implications need to be considered.
False trends could appear if uncertainty were not considered. On the other hand, some trends can be masked by data uncertainty. It was also noted that there is a tendency to consider the occurrence of events as indicating an increased risk, rather than as simply actualizing the predictions. Related to this concern was the tendency to label events as common cause failures, when again it might simply be realizing the expected number of combined, but independent, failures. Caution must be used when analyzing and interpreting such data. The trends should be updated as new data occur which will increase the underst= ding and improve the power of the statistical techniques used. It was felt that such data are toe often analyzed and conclusions made before there is sufficient information to produce meaningful and credible results.
i LANL has begun development of approaches that use failure information from 8
components, trains, and systems in the same analysis.
NUREG/CP 0124 24 4
-rr,
.--.m
.--v-,+.--.,-.m..-r-w.--..
u
.r
.m,#+
.--v,r
-*4.r,.,--#,--e.-.wyw--
m.--.. -,. -, -- - -, ~.--,w.-
.--m..,---.-y---,
Determining the time interval to use for trending risk was felt to be difficult. If it is too short, the trends might be overly sensitive to very rare events. Longer intervals would damp out such fluctuations, but would not provide insights as rapidly. It was noted that it is easy statistically to say that a trend is present in a set of data, but much more difficult to determine whether or not a risk-significant trend is present v. hen the uncertainties and model limitations are considered. It was stressed that caution is needed when evaluating rare events because prematurely evaluating isolated events can give a false picture.
The sparsity of data was felt to preclude using the ASP results direct!v to generate an industry risk profile. The previous risk estimates that have been keyed to ASP results were thought to be invalid because of this problem.
It was felt that a reasonable estimate of the industry risk profile might be made by combinig 'he individual plant risks, if the plant submittals for IPE are equivalent to the NURE WIN detail. Ilowever, it was noted that the basis for such combination would depend t,.. enether or not statistical evaluation of the data and modeling assumptions used in the studies indicated commonality. The group of IPEs/PRAs would ideally be explored to attempt to explain trends and differences before attempting to fit models. Several limitations were noted for this approach of using IPEs/PRAs to estimate an industry risk profile. First, the scope of the IPE analyses would need to be considered when performing evaluations related to the risk profile. In addition, the only the IPE analysis process is being reviewed. The rnodels, data, and results are not being reviewed, introducing question as to their accuracy. The level of detail may vary from plant to plant, making integration of results difficult. The influence of different analysts on the results was felt to be an even more important concern. This problem has been demonstrated in standard benchmark exercises (e.g.,1spra reliability benchmark exercises). Risk profiles would also require updating at selected time intervals to reflect plant changes, operations, and practices.
To be most useful, computerized data bases would need to contain much information not required to be reported in the IPEs. For example, a dependency matrix is needed for each plant at the train level, but it need only be reported at the system levelin the IPEs. Among other items, the success criteria used in the IPEs would be needed if it was desired to update results with operational event data. Specific lists of needs were not generated because it was felt that the IPEs will not include the necessary information, and that requests for further information would need to be s:nt to the utilities. It was also felt that it would be usefulif results of importance evaluations were submitted as part of the IPEs.
- 5. DISCUSSION GROUP 3 RISK MONITORING AND RISK.llASED PERFORMANCE INDICATORS The participants in Discussion Group 3 treated the topics of monitoring for trends at an individual nuclear pow r plant and the feasibility of developing risk based indicators of plant performance. The sugh'sted discussion questions for this discussion group are found in Appendix E. Section 5.1 contains a summary of the importan, information and insights produced by the discussion group, and Section 5.2 contains a more detailed presentation t f the information.
5.1 Summary of Discussion Group]
Discussion Group 3 addressed methods for monitoring risk and the development of performance indicators. The strengths and weaknesses of the various characteristics of performance indicators were discussed.
The group felt that risk-based indicators can measure levels of performance vs. goals.
Ilowever, risk based indicators usually need to be accumulated over a long period of time (over a year or more)in order to dlfferentiate trends from random variations. In contrast, information for mdirect indicators (such as daily power loss) can be collected more frequently, and they can indicate short term trends. llowever, the relationship ofindirect indicators to safety is not clear. The group felt that both types of indicators are useful to include in a set of performance indicators.
The group concluded that risk based indicators could be developed for risk important
-components. The information needed for those components 's out of service dates and times and the reasons the components are out of service. Common cause failure (CCF)
- considerations should also be treated in risk based indicators since common cause failures are important contributors to risk and system unavailability.
To aid in the use of risk based indicators, alert levels could be set. The methods to do this included statistics (cumulative, or point-by-point methods), computer simulation (to trade
~
off detection rate versus fals: alarm rate), and experience (trial and error).
Statist; cal issues of operational data were also discussed, it.vas felt that older plant data should always be retained, but it should be treated differently from more recent data.
Several possible statistical methods for doing this exist, and should be examined for the best method for the give t need.
In establishing a vending interval, the following considerations were identified: the time period needed to detect the trend vs. the rate at which the trend develops, the false alarm rate vs. the detection = rate, and the rate of degradation.
' Die need for considering uncertainty properly when determining patterns was noted such that true patterns can be distinguished from those that simply arise from uncertainty.
NUREG/CP 0124 27 m,
x
+,,L.-
CCF data needs for risk based performance indicators were also discussed. LERs would be useful for CCF determination but not necessarily for CCF indication. "Ihey are useful for root cause analysis and provide sequence oriented information.
NPI(DS gives an understanding of proximate cause of failure. Special scarch strategies need to be developed for both SCSS and NPilDS to identify potential common cause failures.
5.2 Details of Discussion Groun 3 Discussion Group 3 addressed methods for monitoring risk and the development of performance indicators. Before discussing the types of performance indicators that might be used, the group first identified the potential uses of the performance indicators. Most likely, they would be used to judgn past performance (assessment indicators) or to project future performance (leading indicators). Assessment indicators could be used both for determining which plants have performed poorly/well and for evaluating whether or not-
- regulations have led to safer plants. It was noted that it is difficult to tell which plants are good / poor performert through assessment indicators, making it even more difficult to validate leading indicators. The group felt that the assessment indicators could be tracked with a higher level of confidence than leading indicators, but acknowledged the need for both.
Both direct and indirect indicators were suggested for these purposes. Direct indicators would have measurable performance that is translated to a risk measure by some risk model.
Indirect indicators would not have this direct connection through an analytical model. It was recognized that there would be a spectrum of possibilities between these extremes. The direct indicators can clearly discriminate good and bad performance because of the direct tie to risk, but require a longer time _ period for data accumulation before trcnds can be established. Indirect indicators can respond over a shorter time period but suffer from lower credibility.
Several levels of indicators were identified, ranging from the business environment down to train and component level indicatoia. It would be more difficult to establish credibility when connecting higher level indicators to risk than if using lower level indicators, llowever, plant specific indicators at the individual component level would not be practical because the current data collection is inconsistent. With higher level indicators, such as average daily power level, trends could be established during a shorter time window than required for a lower levelindicator such as train or component-level indicators. The group felt that the train level would provide the best compromise, and even that would require additional data reporting.
It was noted that the tradeoffs between the noise, variability, and signal strength of the selected indicator need to be recognized. Methods of grouping data that would increase the signal strength would also increase the noise. The uncertainty in the data sets would need to be considered, both in terms of the tolerance perspective and the confidence perspective in the statistics.
28 ww.e--
r
-.wyr aw,new,+..,.---,m-.-ww
,.-e w
,,~.r.
v.
.,-,--e.-
-=.-.rv m------w.,.s,--+--+
am a--m-+-wwww-
%=-ev-,+%-.s.-,.,-ve~w-u-e-
l Approaches for using plant specific PRAs and IPEs in risk monitoring were explored. One possibility would be to use the IPEs and PRAs to determine the importance of safety systems, using the more important systems in the indicator. Then the deviations of the safety system unavailabilities from the values used in the PRAs and IPEs could be tracked.
He need for monitoring CCFs and human performance indicators was recognized, but methods for doing this would need to be developed.
To develop indicators on initiating events, the group felt additional research would be needed. Ilowever, possible avenues to pursue were discussed. For relatively frequent initiators, the data collected over the past 10 years could be examined for all plants, and attempts made to find patterns for categorizing the events. For rare events, such as ATWS, an anproach was suggested that would attempt to identify precursors of the particular event, since data on the event itself would be too searce. Inspection results might provide information on LOCA precursors.
The frequency for updating the risk assessments used in risk monitoring would depend on a combination of factors. Updates would be necessay after any major plant modification (equipment or procedures) or after a deficiency is found through ASP or other methods.
For newer plants, updating would be needed at evey refueling because new plants tend to show changes in performance fo: th: first few cycles. Older plants show less variation so would need less frequent updating, but it was suggested that they be updated at least every 5 years.
In updating risk assessment after plant procedural changes, it was felt that the procedures should be considered similar to procedures at a new plant because they are new to the operators. In addition, the possibility of operators tending to fall back on the "old ways" should be considered.
It was suggested that the current set of performance indicators could be made more risk relevant-by using a risk weighting of the current indicators. A plant specific ASP type review of all events could be used.
The group did not currently know how to handle design and manufacturing errors which are not discovered until a design basis reconstitution or improved surveillance test is conducted by the licensee after years of plant operation, but suggested that future research programs
. examine methods for determining how the discovery of design problems reflects on the number of residual failures in the design. Approaches such as software reliability models that attempt to determine residual defects could be pursued. The importance of keeping reporting non punitive was stressed, such that the utilities would not be reluctant to report information. It was-suggested that the failures be broken into two groups design and
- operational.
LERs and NPRDS were viewed as useful for risk based performance indicators, but not a complete source. LERs would be useful for CCF determination but not necessarily for CCF NUREG/CP 0124 29 f
,~ h v t
. -, - ~
~v' w w -vr em
-+-wvir--r-w m-e= r h +-+
-*e"'
=
"v=-
=r-'-*----m"-"
~ = -"-'e* " - - - * - ' - " " ' " * " * * * - - - - - * " - - " ' ' " " + * - ' - - - - - " " ' ' - ' - - * " - - - - - - - - - - ^ ^
~
t indication. ney were felt to be useful for root cause analysis, especially if a 30 day report has been made. LERs also provide sequence oriented information. NPitDS does not help identify root causes but does give an understanding of proximate cause and whether the failure dates are clustered together. The engineering data in NPitDS can be somewhat useful for common cause failure analysis because it can be used to locate similar i
components after a component problem has been identified by other means. Frequency clustering analysis was suggested using NPitDS data, with analysis to identify which portions are from random and non random phenomena. The non random portions could be examined for common cause failure. %e failure records in NPRDS could be used to l
identify potential common cause component groups, and the work maintenance records
~
could be used to determine the actual cause of the failure.
Statistical issues of operational data were also discussed. The group believed that older f
plant data should always be retained, but treated differently from more recent data. Several possible methods for doing this exist. The key concern was choosing a method that discounts but does not discard old data. It was felt that_ there is no general way for establishing intervals for developing trends, but in developing a specific interval one should consider the time period needed to detect the trend vs. the rate at which the trend develops, the false alarm rate vs. the detection rate, and the rate of degradation.
Several means for establishing alert levels were identified: percentile, CUSUM, computer simulation, or a brute force approach which starts broad and narrows in. The level depends on the false alarm rate, the significance of the false positives and false negatives, and the risk of the item.
The importance of considering uncertainty properly when determining patterns was noted.
Without-it,-true patterns can not be _ distinguished from those that simply arise from uncertainty, it was felt that more rigorous statistical review of current methods was needed for establishing trend intervals.
The group also discussed whether or not suspected outliers should be considered in evaluations, and suggested an approach. First, the group characteristics must be determined using all data for the plants in a group. Then statistical analysis could be used to determine if the suspected outliers are truly outilers o, if they are points on the tails of the distributions. If the suspected outlier is shown not to be an outlier,it should be considered a generic issue, with regulation focused on reducing the variability of the group. On the other hand,if it is shown to be an outlier,it should only be considered for the plant specific evaluation.
L NUREG/CP-0124 30' L
6< OVERAllINSIGilTS The comments expressed in this section were not compiled at the Workshep; they were prepared after careful reading of the notes taken during the discussion groups and listening to the recordings of the actual discussion sessiont (1)
General -
It was recognized that it is not feasible to treat all events, but that certain classes of events should be flagged for further review.
Statistical methods have been demonstrated for trending and as performance indicators for relatively frequent events, but reactor safety concerns often involve sparse data. ; Data scarcity prevents direct use in many cases, but decomposition to a lower level might make analyses possible, it was repeatedly emphasized that care must be exercised when analyzing scarce data or false conclusions may be drawn.
lA common problem identified during the workshop was that NRC needs and uses of analyses were often no! well understood. That is, the roles of ASP, trending analyses, performance indicators, etc., in NRC functions were not generally understood by the workshop participants.
There was a general impression that current evaluations of operational data will not likely identify "what we don't know." increased NRC priority would be needed to focus rescurces on this broader question to identify appropriate methods to perform such analyses in a systematic way.
A universal message frcm all of the discussion groups was to use risk insights from
' PRAs and IPEs to focus trending studies, etc.
L Events which are important from PRA insights should be flagged. Such conditions are events which become important when coupled with an external initiator (e.g.,
fire, flood, earthquake), containment performance considerations, etc.
Common cause failures and human performance concerns were identified as important issues to be further studied in trending analyses and in upgrading of the ASP models.
(2)
Accident SeqqcnctPrecursors There was a general feeling that the ASP program is useful, but that it needs.
improvement in system modeling, treatment _of common cause failures, and NUPEG/CP 0124 -
31
i l
treatment of human performance. Plant specific, train level models would meet this need, r
i In the ASP prograta conditional core damage probabilities are calculated for i
certain events. llowever, the occurrence rate of the prceursor event itself is
?
- generally not calculated. The frequency of precursor events could also be useful as a check on expected occurrence rates based on PRA cstimates.
_ (3) fleneric/ Risk Profile IPE analyses would'be a useful source of information for fulfilling many needs (e.g., developing plant specific, train level models, developing an industry risk profile)c Ilowever, h was realized that much of the information needed for ASP and other uses is not required to be submitted to the NitC in the IPE submittal.
-It was felt that it would be possible for the NitC to develop an industry risk i
profile.
If it is done, it_ will require careful scoping and planning before developing the method., and its implementation.
F (4).-
Performancc_Judka1015 l
Risk based performimce indicators should be developed using risk important
. component and systems. This effort would require new models and data.
.(5) -
.Othn
_ Data reporting was overwhelmingly felt to be a weakness. Information that would-make the failure records more useful include: mode of operation at time of failure,'more complete failure narratives, better root cause informath. time of 3
failure, time equipme'it was restoied to service.
r l
t-I r
i M.~
L NUREG/CP-0124:
32-o l'-
p 5
b ii.
- 7. REFERENCES i
1.
J. W. hiinarick, et al, Precursors to Potential Severe Core Damage Accidents: 1990; A Status Report, NUREG/CR-4674, Volumes 13 and 14 August 1991.
2.
II hi. Paula, et al, A Cause Defense Approach to the Understanding and Analysis of Common Cause Failures, NUREG/CR 5460, hiarch 1990.
3.
S. Kaplan,"On a Two Stage' Bayesian Procedure for determining Failure Rates from Experimental Data,"IEEE Trans.on Power Apparatus and Systems, PAS 102,195 202, 1983.
4.
G. Apostolakls, "The Concept of Probability in Safety Assessments of Technologleal-Systems," Science, 250,1359 1364,1990.
5.
G. Apostolakis, S. Kaplan, B.J. Garrick, and W. Dickter, " Assessment of the Frequency of Failure to Scram in LWRs," Nuclear Safety,20,690-705,1979.
- 6.
N. Siu and G. Apostolakis, "A hiethodology for Analyzing the Detection and Suppression of Fires in Nuclear Power Plants," Nuclear Science and Engineering,94, i
213 226, 1986.
7.
D.L Sanzo and G. Apostolakis, "A Time Dependent hiethodology for Evaluating Component Reliability," Proceedings of the International Nuclear Power Plant Aging Symposium, Bethesda, h!D, August 30 Sept.1,1988, NUREG/CP 0100.
8.
J. Reason, iluman Error, Cambridge University Press, New York,1990.
~ NUREG/CP-0124 33
=-.
APPENDIX A List of Workshop AHendecs NUREG/CP-0124 35 Appendix A l
LIST OF WORKSilOP NITENDEES George Apostolakis, UCIA Bennett Brady, AEOD,
University of California h!NBB-9112 38137 Engineering IV Nuclear Regulatory Comm.
Los Angeles, CA 90024-1597 Washington, DC 20555 (310) 825 1300 (301) 492 4499 Cory Atwood,INEL James Bryce, INEL EG&G Idaho, Inc.
EG&G Idaho, Inc.
hts: 3421 hiS: 2407 PO Box 1625 PO Box 1625 Idaho Falls,ID 83415 Idaho Falls,ID 83415 (208) 526-0431 (208) 526-8231 hichammad Ali Azarm, UNL Robert ">ionitz, FRA Building 130 2000 Center Street Upton, l.ong Isand NY 11973 Suite 418 (516) 282-4992 Oakland, CA 94704 (510) 644 2700 Pat Baranowsky, AEOD MNBB 9112 Allen Camp, SNL Nuclear Regulatory Comm.
Division 6412 Washington, DC 20555 PO Box 5800 (301) 492-4480 Albuquerque, Nhi 87185 (505) 844 5960 Bill Beckner, NRR GWFN-10E4 Al Chaffee, NRR Nuclear Regulatory Comm.
OWFN-11 A1 Washington, DC 20555 Nuclear Regulatory Comm.
(301) 504-1089 Washington, DC 20555 (301) 504-1168 Vic Benaroya, AEOD MNBB-9112 Mike Cullingford, NRR Nuclear Regulatory Comm.
OWFN 12G18 Washington, DC 20555 Nuclear Regulatory Comm.
(301) 492 8318 Washington, DC 20555 (301) 504-1276 Dennis Bley, PLG 4590 MacArthur Blvd.
Suite 401 Newport Beach, CA 92660 (714) 833-2020 NUREG/CP-0124 37 Appendix A 1
l
Mark Cunningham, RES Jack 11eltemes, RES NIE-372 NLS 007 Nuclear Regulatory Comm.
Nuclear Regulatory Comm.
Washington, DC 20555 Washington, DC 20555 (301) 492 3965 (301) 492-3720 John Darby, SEA Don Ilickman, AEOD 6100 Uptown Blvd., NE MNBB-9112 Albuquerque, NM 87110 Nuclear Regulatory Comm.
(505) 884-2300 Washington, DC 20555 (301) 492-4431 Bob Dennig, NRR OWFN 11A1 Tom Ippolito, SEA Nuclear Regulatory Comm.
1700 Rockville Pike
-Washington, DC 20555 Suite 400 (301) 504-1156 Rockville, hiD 20852 (301) 468 7371 Susan Dingman, SNL
- Albuquerque, NM 87185 Nuclear Regulatory Comm.
(505) 844 0099 Washington, DC 20555 (301) 492-3548 Joe Fragola, SAIC 8 West 40th Street -
Bill Jones, AEOD 14th Floor MNBB 9715 New York, NY 10018 Nuclear Regulatory Comm.
(212) 764-2820 Washington, DC 20555 (301) 492-4442 Bill Galyean, INEL EG&G Llaho, Inc.
Ed Jordan, AEOD MS: 2405 -
MNBB-3701 PO Box 1625 Nuclear Regulatory Comm.
Idaho Falls,ID 83415 Washington, DC 20555 (208) 526-0627 (301) 492-4848 Cindy Gentillon, INEL Ernie Lofgren, SAIC EG&G Idaho, Inc.
1710 Goodridge Drive MS: 3421 Tier 2-7-1 PO Box 1625 McLean, VA 22102 Idaho Falls,ID 83415 (703) 821-4492 (208) 526 9891 u
L NUREG/CP-0124 38 Appendix A L
ii e
7,
Erasmia Imis, RES Tom Mitchell, INPO NLS 372 Suite 1500 Nuclear Regulatory Comm.
1100 Circle 75 Parkway Washington, DC 20555 Atlanta, GA 30339 3064 (301) 492-3557 (404) 953 5439
- Steve long, NRR Mohammed Modarres OWFN 10E4 Building 090 Nuclear Regulatory Comm.
Nuclear Engineering Washington, DC 20555 College Park, MD 20742 2115 (301) 504 1077 (301) 405 5226 Fred Manning, AEOD
- Ali Mosleh, U of MD MNBB-9715 Building 090 Nuclear Regulatory Comm.
Nuclear Engineering Washington, DC 20555 College Park, MD 20742 2115 (301) 492-4426_
(301) 405-5215 Ilarry Martz, LANL--
Tom Novak, AEOD Statistics Group.(A 1)'
MNBB-9112 Analysis and Ar sessment Division Nuclear Regulatory Comm.
Ims Alamos, NM 87545 Washington, DC 20555 (505) 667 2687-(301) 492-4484 Gary Mays, ORNL Pat O'Reilly, AEOD Bldg. 9201-3 MNBB 9112 PO Box 2009 Nuclear Regulatory Comm.
Oak Ridge, TN 37831 8065 Washington, DC 20555
.(615) 574-0394-(301) 492-8858 Steve Mays, ACRS Gareth Parry, NUS P-315 -
910 Clopper Road Nuclear Regulatory Comm.
Gaithersburg, MD 20878 Washington, DC 20555 (301) 258-2536 (301) 492 7904 Ilenrique Paula, JBFA
Knoxville, TN 37932 Suite E-103 (615) 966 5232 Oak Ridge, TN. 37830
- (615) 482 6743
, NUREG/CP-0124 39 A pendix A P
Arthur Payne, SNL Pranab Samanta, BNL Division 6412 lluilding 130 PO Box 5800 Upton, Long Isand NY 11973 Albuquerque, NM 87185 (516) 282-4948 (505) 844 7321 lloward Stromberg, INEL Marie Pohida, NRR EG&O Idaho, Inc.
OWFN.10E4 MS: 2407 Nuclear Regulatory Comm.
PO Box 1625 Washington, DC-20555 Idaho Falls, ID 83415 (301) 504 1846 (208) 526 9167 Mike Poore, ORNL Lillian VanSaten, NRC Bldg. 92013 _
W 308
. PO 13cx 2009 Nuclear Regulatory Comm.
Oak Ridge, TN 37831 8065 Washington, DC 20555 (615) 574 0325 (301) 492 8938 Dale Rasmuson, AEOD Gary Wilson, INEL MN13B-9112 EG&O Idaho, Inc.
Nuclear Regulatory Comm. -
MS:
Washington, DC 20555 PO Box 1625
- (301) 492 4490 Idaho Falls,ID 83415 (208) 526 9511 Stacey Rosenberg, NRR OWFN 10E4 Millard Wohl, NRR Noclear Regulatory Comm..
OWFN 11E22 -
Washington, DC ' 20555 Nuclear Regulatory Comm.
C')1) 504-1082
- Washington, DC 20555 (301) 504 1181 Jack Rosenthal, AEOD MNBil 9715
-Jolm Wreathall Nuclear Regulatory Comm._.
. 4157 MacDuff Way Washington, DC - 20555
- Dublin, Oli 43017 (301)'492-4440 (614) 791 9264 Denny Rossi AEOD :
Robert Youngblood, UNL MNB13-3701 -
Building 130
. Nuclear Regulatory Comm.-
Upton, Long Isand NY 11973 LWashington, DC 20555.
(516) 282-2363-
- (301)_492-7361 NUREG/CP-0124 40 Appendix A -
l
\\
,,m
. -. - -,,,,, - - - - ~.,. -.
,r_.----.-
..-,,,#-..--.m,,,--., _.. -, - ~... ~
_.----.-m-.~,...-
E APPENDIX 11 i
NRC Prograrns for Evaluating Operating Data NUREG/CP-0124 41 Appendix 0 l
1
Operationa! Experience and Evaluation Actual operating experience is an essential input to the regulatory process for assuring that licensed activities are conducted safely. Major data sources are reports submitted by licensees to the NRC in compliance with 10 CFR 50.72 ("Immediate Notification l
Requirements for Operating Nuclear Power Reactors"),10 CFR 50.73 (" Licensee Event Report System"), and voluntary reports of component failures submitted to the Nuclear Plant Reliability Data System (NPRDS),which is managed by the Institute of Nuclear Power Operations (INPO). These data are maintained in computerized data bases.
Additional sourus of data include (1) licensees' monthly operating reports, (2) NRC inspection reports (regional reports as well as reports from special evaluations performed by Augmented Inspection, incident investigation, and Diagnostic Evaluation Teams), (3) 10 CFR ' Part 21 reports (" Reporting of Defects and Noncompliance"), (4) preliminary notifications of events issued by the NRC, and (5) foreign reactor events received through international exchange of information. The NRC also obtains operational data from site visits, and from licensee responses to bulletins, generic letters, and 10 CFR 50.54(f) letters.
- Data for NRC sponsored probabilistle risk analyses (PRAs) are usually obtained from site visits, but " generic" sources may also be used.
Specified safety criteria are used to identify events which are Abnormal Occurrences to be reported to Congress (Table 1), significant events for the NRC Performance Indicator Program (Toble 2), important events for engineering analyses and assessments by AEOD's Reactor Operations Analysis Branch (Table 3), and precursors to potential severe core 3
' damage accidents (Table 4) as identified by the Accident Sequence Precursor (ASP) program.
Information on file in the NPRDS is derived from engineering and failure data submitted by nuclear power plant' licensees to INPO. The NPRDS produces failure statistics on components and systems related to nuclear safety. Such statistics are for use in deriving implied "reliabilitt" of components which may be of interest to operators and designers of 4
nuclear power plants, reactor manufacturers, architect engineering and constructor firms and regulatory agencies.Ilowever, the ilata is not sufficient to perform actual reliability and availability _ analyses because of limitations in raw data required to be reported to the system.
TSe NRC considers the NPRDS to be a vital adjunct to the LER system. Its value as an analytle tool is directly dependent upon the accurav and completeness of the data, and the -
degree ofindustry participation.; For 64 plants reviewed by INPO in 1989 and 1990 and one in 1991, the mean completeness of component failure reporting was 70 percent and the median 81 percent.
The primary source of data on operational events used in both routine evaluations and special studies are licensee event reports (LERs), For 1991, about 2000 LERs will be
- submitted covering events with a wide range of significance (e.g., spurious I-IVAC isolations to reactor scrams with complications). About 150 related pieces of data for each LER are NUREG/CP-0124 43 Appendix B L
..a
entered into the Sequenet. Coding and Search Sptem (SCSS) data base. The SCSS facilitates the storage and retrieval of information about each event (e.g., causal and time aspects of occurrences within the event sequence). This system is maintained by Oak Ridge National Laboratory. A separate data base is maintained at the Idaho National Engineering laboratory (INEL); this data base is used to support studies for specific kinds of events and the NRC Performance Indicator Program. The data base is derived from LERs,10 CFR 50.72 reports, and licensees' monthly operating reports and contains operational information such as ESF actuations (including reactor scrams), safety system failures, technical specification violations and shutdowns, and reactor critical hours.
Operational data are reviewed and evaluated to identify (1) significant events and any associated safety concerns and root causes, (2) the trends and patterns' displayed by these events, (3) the adequacy of the corrective action: taken to address these concerns, and (4) l the generic applicability of events and concerns to other plants.
The ASP method models and evaluates plant equipment'and human responses that could affect the progression of an accident, evaluatiag the actual failures that have occurred along with the probabilities for postulated additional failures that could occur. The precursor method uses event tree models to evaluate the likelihood of various possible outcomes (scenarios) for. the events being modeled, resulting in a quantitative estimate of the significance of the event in terms of conditional probability of core damage. The overall 1
ASP analysis process is shown in Figure 1. The precursor event evaluations are presented in ASP NUREG reports which are published annually. The breakdown of precursor events by event type and significance are plotted and provided to the Commission each year to show trends.
Summary information on precursor events are given to NRC senior management to provide another perspective on plant operating experience. NRR has adopted the ASP methodology for evaluation of selected 10 CFD 50.72 reports to assist in the identification of significant events. The ASP models in use by NRR were reviewed and modified to bring them into better quantitative agreement with avcilable PRAs, and ATWS event trees were added.
. Certain shortcomings of the existing ASP models are being addressed by NRR and'their subcontractors SAIC and ORNL For example, when evaluating certain events, mt :eling deficiencies can cause overly conservative estimates of conditional core damage probability.
One deficiency _ concerns not giving proper credit for alternate long term means of core
' decay heat removal and a second deficiency concerns not properly crediting the charging pumps as an alternate to the HPI pumps on certain plants. - Additional event trees are also
- being developed for steam generator tube rupture and ATWS. In addition to correcting known problems, an effort is underway to confirm that ASP modeling of plant system n
- configurations and capabilities are correct and current by verifying them using information from individual plant examination submittals.
Trends and patterns analyses are performed to (1) identify and provide a quantitative context for new safety issuest (2) evaluate the effectiveness of current regulations, regulatory L
44 Appendix B y
I.
1
+..
,#-w,.-<m.* Cme.-~
r m-,m.-,.
,.ww.a,-w..<-.....uw-
.m w.e i-,
.-E,m.-.4w--,em.-
e-.v-.--
.-.r
..-,s-.--
-5
actions and initiatives taken by licensees to resolve safety issue com. erns; and, (3) help guide and focus engineering evahmtions. PRA insights can be helpful to identify components, systems, accident initiators, accident sequences and safety / regulatory issues as candidates for trends and patterns analyses. Also, PRA assessments can be helpful to evaluate the safety significance of the results of trends and patterns analyses.
NRR has begun tiending the results of the ASP evaluations published annually by AEOD.
A summary M. made of tne conditional probabilities for the precursor events for each year.
The total numbers of precursor events per year and the numbers of events exceeding various values have also been considered. The ASP report data was also scrutinized for apparent differences associated with plant age, size of utility company, types of reactors, ett.
The NRC Performance Indicator (PI) Program is another aspec: of efforts to reonitor the
- performance of nuclear power plant licensees. This program currently monitors indMdual plant as well as industry wide _ data on eight Pls and evaluates the data to deterrnine i
performance trends. The eight Pls are (1) the number of unplanned automatic reactor scrams _(trips) while a reactor is critical, (2) the number of safety system actuations, (3) the number of significant events, (4) the number of safety system failures, (5) the forced outage rate, (6) the number of equipment forced outages per 1000 commercial critical hours, (7)
- the collective radiation exposure, and (8) cause codes. Most of these PIs are generated by
- the NRC's computerized data bases. The trends of the Pts are shown on a plant specific
- basis, as well as comparisons to industry-wide averages. These reports are issued quarterly.
In the. fourth quarter report each year, annual industry trends for each PI for the pest several years are presented. Figure 2 shows the trends in the industry averages for the first seven Pls for the years of 1986 through 1990. (Industry wide averages are not calculated for the cause code Pl.):
The Pls are intended to monitor plant operational safety performance. Therefore, they should reflect trends in one of the following three key elements of operational safety: (1) frequency of transients, (2) unavailabuity of safety systems, and (3) potential for common-cause failures.
The development of a risk based indicator of key safety systems unavailability has been studied for some time but has not been implemented because the-needed data is not currently available to the NRC on a routine basis.
The NRC has developed state-of the art software computer systems for use in risk analyses.
The Integrated Reliability and Risk Analysis System (IRRAS)is used to perform a level 1 PRA. Event trees and fault trees are developed and analyzed using IRRAS. IRRAS is being used in the preparation oflow power / shutdown PRAs. The System Analysis and Risk Assessment (SARA) software is designed to perform sensitivity studies on cut sets These programs provide new tools for the NRC to use in ASP studies and event evaluations. The NRC is also loading data from PRAs into the MAR-D data base for use with IRRAS and SARA. : So far 'about 8 PRAs have been loaded into the data base. The key to using -
IRRAS and SARA effectively in NRC appilcations is to develop the event tree and fault tree models to take advantage of the unique features of the codes.
NUREG/CP 0124 45 A pendix B P
~,,un..,,, -. -.,.
-<,~,.,4,ww,
-,...e
. ~., -.. ~
.e..nn,me-.
,. - ~. - -,..
n c,w.
,-~~~w
-w-- - - -
Table 1 Abnormal Occurrence Criteria The following criteria for abnormal occurrence determinations were set forth in an NRC policy statement published in the Federal Ruister on February 24,1977 (Vol. 42, No. 37, pages 10950-10952).
An event will be considered an abnormal occurrence if it involves a major reduction in the degree of protection of the public health or safety. Such an event would involve a moderate or more severe impact on the public health or safety and could include but need not be limited to:
1.
Moderate exposure to, or release of, radioactive material licensed by or otherwise regulated by the Commission; 2.
Major degradation of essential safety related equipment; or 3.
Major deficiencies in design, construction, use of, or management controls for licensed facilities or material.
Examples of the types of events that are evaluated in detail using these criteria are:
For All 1.icensees 1.
Exposure of the whole body of any individual to 25 rem or more of radiation; exposure of the skin of the whole body of any individual to 150 rem or more of radittion; or exposure of the feet, ankles, hands or iorearms of any individual to 375
^
rem or more of radiation [10 CFR 20.403(a)(1)], or equivalent cxposures from internal sources.
2.
An exposure to an individual in an unrestricted area such that the whole body dose received exceeds 0.5 rem in one calendar year [10 CFR 20.105(a)].
3.
The release of radioactive material to an unrestricted area in concentrations which, if averaged over a period of 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, exceed 500 times the regulatcry limit of Appendix B, Table 11,10 CFR Part 20 [CFR 20.403(b)(2)].
4.
Radiation or contamination levels in exces> of design values on packages, or loss of confinement of radioactive material such as (a) a radiation dose rate of 100') mrem per hour three feet from the surface of a package containing the radioactive material, or (b) release of radioactive material from a package in amounts greater than the regulatory limit.
NUREG/CP-0124 46 Appudix B
Table 1 (cont.)
5.
Aa: loss of licensed material in such quantities and under such circumstances that substantial hazard may result to persons in unrestricted areas.
6.
A substantiated case of actual or attempted theft or diversion of licensed material or sabotage of a facility.
7.
Any substantiated loss of special nuclear material or any substantiated inventory discrepancy that is judged to be significant relative to normally expected perforniance and that is judged to be caused by theft or diversion or by substantial breakdown of the accountability system.
8.
Any substantial breakdown of physical security or material control (i.e., access control, containment, or accountability systems) that significantly weakened the protection against theft, diversion, or sabotage.
9.
An accidental criticality [10 CFR 70.52(a)].
10.
A major deficiency in design, construction, or operation having safety implications requiring immediate remedial action.
11.
Serious deficiency in management or procedural controls in major areas.
12.
Series of events (where individual events are not of major importance), recurring incidents, and incidents with implications for similar facilities (generic incidents) at create major safety concern.
For Con 1plercial Nuclear Power Phmts 1.
Exceeding a safety limit of license technical specifications [10 CFR 5036(c)].
2.
Major degradation of fuel integrity, primary coolant pressure boundary, or primary containment boundary.
3.
Loss of plant capability to perform essential safety functions such that a potential release of radioactivity in excess of 10 CFR Part 100 guidelines could result from a alated Wnsient or accident (e.g., loss of emergency core cooling system, loss of
' > a tol rod system).
4.
r;overy of a major condition not specifically considered in the safety analysis e
i sport (SAR) or technical specifications that requires immediate remedial action.
NUREG/CPC 47 Appendix B
_ _ _. ~.
1 4
1 Table 1 (cont.)
5; Personnel error or procedural deficiencies that result in loss of plant capability to cerform essential safety functions such that a potential release of radioactivity in excess of 10 CFil Part 100 guidelines could result from a postulated transient or l
accident (a a, loss of emergency core cooling system, loss of control rod system).
Table 2 Criteria for Significant Events for the Performance Indicator Program Events normally involving one or more of the following:
1.
The degradation of important safety equipment.
2.
An unexpected plant response to a transient or a major transient itself.
3.
A degradati'n of fuel integrity, the primary coolant pressure boundary, or important associated structures.
4.
- A reactor trip with complications.
- 5..
An unplanned release of radioactivity exceeding plant Technical Specifications (TS) or regulations.
6.
Operation outilde the limits of TS.
7.
Other events that are considered significant.
NUREG/CP-0124 48 Appendix B'
=.-,,...
Table 3 AEOD lleactor Operations Analysis liranch Screening Criteria for important Events Events are assigned to one of four categories depending on its safety importance in accordance with *he following criteria:
Category 1 - Those events of such obvious importance that actions should be initiated immediately by AEOD or other office or organization to ensure plant safety.
Category 2 - Those events (or combination of events) which appear to have safety importance but do not require immediate action to ensure plant safety.
Category 3 - Those events (or combinations of events) which require additional consideration by another 1(OAll Section to permit assignmem to Categories 1, 2, or 4.
Category 4 - Those events with little apparent importance to safety.
The criteria used to help identify such events, for operating occurrences and for operating conditions, are listed below. The final determination of significance is based on engineering judgment.
Ontullinto itusner_Cri1cria rc A.
Safety limit violated 11.
Natural phenomenon B.
Alert or higher emergency 1.
Scram / transient /ESF actuation classification with complications g
C.
On-demand failure of safety J.
Scram / transient /ESF actuation system with equipment operable D.
Actual unexpected performance K.
Personnel overexposure or injury E.
Common-mode /cause failure L
Release of radioactivity F.
System interaction M.
An accident G.
Iluman errors N.
Moderate frequency event with the potential for severe core darnage O.
Other NUREG/CP 0124 49 Appendix B u_
I Table 3 (cont.)
_ Operating Condition Criteria A.
Condition which could initiate F.
Procedural or training errors j
an accident or prevent r
. successful mitigation
- D.
.Outside desi,n basis or G.
Potentia 1 f ai1u re or f
requirements degradation of safety equipment C.=
Potential unexpected 11.
Management deficiencies c-
- failure or response l
D.
Potential common mode /
1.
Technical specification cause failure violation E.
Potential system interaction J.
Programmatic defit :encies s
i t
k NUREG/CP-0124 -
50 Appendix B r
k e
e r
--s v,
.,,-.,,e w
m,<sn ee-
~w-m-r-e-w- ' n
,e-r o, - -
,w e-
.s, n
v s
Table 4 Typleal Events Evaluated by ASP Process Unexpected core damage initiators (LOOP and small break LOCA)
All events in which reactor trip was demanded and a safety related component failed All support system failures, including failures in cooling water syste ms, instrument air, instrumentation and control, and electric power systems Any event where two or m e failures occurred Any event or operating conchtlon that was not predicted or that proceeded differently from the plant design basis Any event that, based on the reviewers' experience, could have resulted in or significantly affected a chain of events leading to potential severe core damage The overall precursor selection process is shown in Figure 1.
)
i
- NUREG/CP-0,. 4 51 Appendix B
(
l l
)
i IIIs Requiring Review 1 '
LArs the evad only inv<dve canponent failure (no kas of reduvancy) e kan of redundancy (single systern) seisttk qualificatkuvdesign arw envirotunental quahrptiore - rign ator pee <ritkal evers structaaldegr datkm desigtt error discovered by re analpis o
toumled by trip or LOIW Yes i
- N#M no ap;tecial* safety systesn impact shutdown telated event pastore damage im;* cts only No ir Can event is reasonably analyred by No ideedy as poicidially significant bul PRA leed nusicis?
kn;wacacal to analyze Ikfine im;mct of esent in terms of
- ASP Mo&ls Perkvm detailed rev6ew analysk, and knillakir elserved and trains of systeins quantifg sike unavailabic k
FS
, c4c.
y Modify tuanch prtivabihties to eclist event Calculate cawhtkrsal ptkebildy swtiated with event using modifad y
event trres
! bra (huvatxmal Evers inved,e:
- a c4we damage initiske a total kiss of a systern M
e a kus of redundarwy in Iwe or nuwe systeau 1r He 4
is canistimal pobaNtay 210
-* Rejnt based on k,* peubatality Yes 1
ruumca as. p=mor Figure 1. ASP Analysis Program NUREG/CP 0124 52 Appendix B 5
' '*"c.'6:0 5" "
- w i
$,... -...i..
j 4
p,.
se.
E_EE.
E_EE
,3...
.6,
,sh.
.., e 3 s.n..
c,-i.
. w.1, s,.i.e r -,..
.3.
s..i 2.
I'~ E O O $_0_
I': I m a m.=
e sh7 ph.
oh.
.e e uh? pl oh.
v??C*:,'.,%'L%%'l.
s rac.s u n. not, w Ie.
J u.
e i
10.
"to.
5EEE Gh6 S' 7
,S.B_S Sh9 9'90 086 5'57
.G.S8 8
GS9 9 90 m-i.,.g g c...,.
mi.i,,,,,,,,,,,,,,,,,,,,,,,,,,
ox
<< w i, s,.i.m
.,,., m, -..
.. i mi,..si,,-
w i, s,.i.m
.......i.,,.,..
,y **.
} sm.
{ 400.
v.he,.she s see ne7 n
9 Figure 2. Annual Industry performance averages -- 1986 1990 NUREG/CP.0124 -
53 Appendix B i
l; L
l
l J
l APPENDIX C l-Discussion Group 1 Questions and Pr dicipants JREG/CP-0124 55 Appendix C 1
r WORKSIIOP DISCUSSION GROUP 1 Risk Significance of Events 1
What fundamental screening nr.d modeling aspects of ASP should be reevaluated and improved to increase confidence that important events are not being missed because of biases introduced by the methodology and its implementation, or by limitations in ASP models?
2.
What level of modeling detail should be included in computer codes for ASP analyses?
3,
- What importance measures should be used and how should they be used to identify event significance?
- 4.
110w can ASP be extended to external events: (1) Fire; (2) Internal flood; (3) Seismic; etc?
5.
What kind of human performance evaluation improvements should be made to, or incorporated in ASP models/ procedures?
6.
What can be done to improve ASP analysis efficiency to speed up the process? Can screening criteria and qualitative assessments be developed to reduce the number of detailed ASP /PRA evaluations, especially for those of very low conditional core damage probability.
7.
What are the statistical limitation and problems associated with trending ASP results?
8.
'+ m statistical limitations important when ASP results are used to display an w
rend?
9.
- v the unewainties in ASP? Ilow should uncertainty be factored in to intended v -SP results? How should they be handled? Qualitative treatment versus
. x.e qe n, V.ution.
- 10. What kinds of extrapolations can be done with quantitative and qualitative ASP results?
- 12. Is it practical to use PRAs/IPEs for event (ASP) analysis? How should PRA models and results be structured in order to be most usef al for event analysis? What data is
-needed/
- 13. What is the minimum information needed to perform a credible risk assessment of an event? What kc d of changes should be made to event reporting (50.72/73) to help assure important events will be ider.tified by the ASP screening process?
57 Appendix C
1 DISCUSSION GROUP 1 PARTICIPANTS Moderator:
Allen Camp, SNL
Participants:
Bill Jones, AEOD Fred Manning, AEOD Joe Minarick, SAIC John Darby, SEA
' Bob Budnitz, FRA Steve Long, NRP Marie Pohida, NRR Gary Wilson, INEL Steve Mays, ACRS Cory Atwood,INEL Mike Cullingford, NRR 1
l L
l NUREG/CP-0124 58 Appendix C l
APPENDIX D Discussion Group 2 Questions and Participants I
NUREG/CP-0124-59 Appendix D l
(WORKSHOP DISCUSSION GROUP 2 Industry Risk Profile and Generic Concerns 1.-
Ilow can we use common cause failure, systems interactions, and human performance analysis methodology to analyze industry wide data for trends, and for the identification of potentially important genericissues? What methodologies are most suitable? What data detailis required?
- 2.
What approaches and methodologies can we use to cull the LER and NPRDS data bases to identify risk significant industry trends: at the component level; at the system
' level; at the event level; at the issue level? (other?) How can information from all or a group of plants be used to identify potentially risk significant problems? What other data sources should be routinely screened?
Ilow can we combine or extrapolate from and between LER and NPRDS (or other) data bases to form the most complete picture?
3.
What methodology' and criteria should be used to select a class of events
. (component, system, issue level) for detailed study (e.g. statistical, risk, root cause and engineering evaluations)?
4.
Given the nature of the events reported in the available data bases, what statistical techniques should be used for analyzing trends?
5.
If it were possible to develop a nuclear industry risk profile, what are some ways it could be done and what should it include?
6.
'Given some form of risk estimate is or will be available for most plants, how can they be combined to provide an industry risk profile? What technical issues need to be addressed and what approaches and methods should be used or developed for this
- application?
- 7. -
What approach might be developed using available PRAs and IPE rest.lts to generate a periodic, industry risk profile update (trend). What data would be needed?
8.
What information from PRAs and IPEs should be catalogued in a computerized data base?
9.-
What are the pros and cons of using ASP results ta identify industry historic risk trends. What are the statisticalissues associated with this? What are the statistical implications, confidence level in results?
NUREG/CP@24 61 Appendix D
e
- DISCUSSION GROUP 2 PARTICIPANTS Moderators:
Gareth Parry, NUS Ali Mosleh, Univ, of MD i
Participants:
Pat O'Reilly, AEOD Bennett Brady, AEOD George Apostolakis, UCLA Henrique Paula, JBFA Bob Dennig, NRR -
Iloward Stromberg, INEL All Azarm, DNL Harry Martz, LANL-Dale Rasmuson, AEOD Jack Rosenthal, AEOD Bill Beckner, NRR -
Mark Cunningham, RES Tom Novak, AEOD
- NUREG/CP-0124 62 Appendix D i
i APPENDIX E Discussion Group 3 Questions and Participants l
' NUREG/CP 0124 63 Appendix E e
a e
WORKSHOP DISCUSSION GROUP 3 Risk Monitoring and Risk.Ilased Performance Indicators 1.
If we were to start from scratch with no preconditions, what types of performance indicators should be selected to monitor plant safety / risk? What data would be needed? What practical alternatives are there?
l-2.
What approach should be taken to identifying and developing surrogate or indirect indicators for plant risk monitoring?
3.
What approaches and methods could be developed and used to employ plant specific
- PRAs and IPEs as a risk monitoring tool? Should total risk be monitored or should specific components and systems be monitored? What about monitoring certain component types, or human performance indicators? What types ofindicators should be developed for initiating events?
4.
How often should the risk assessment, or specified elements of it (e.g. system reliability) that are used-for risk monitoring be updated? What data would be required?
5.
What risk methodology might be used to improve the current set of performance indicators? Can they be made more risk relevant?
6.
How should we treat design and manufacturing errors which were not discovered until a design basis reconstitution or improved surveillance test was conducted by a licensee after years of plant operation.
7.
How can we utilize existing LER and NPRDS data in combination to improve on their usefulness in meeting risk-based performance indicator data needs? What other data might be used to fill voids that are inherent th these data sources? Optimally, what data is needed?
8.
What statistical issues should be addressed when developing and implementing risk-based performance indicators? What methods are best suited for routine periodic trending (e.g. rolling average, regression) of risk-based indicators? What approach should be used to select intervals for developing trends? -When is past history to old to be considered indicative of current performance?-
9.
What methods should be used to establish alert levels when monitoring risk? How can we identify significant trends? What method or approach should be used to differentiate actual short term deviations in performance from random variations?
10.
How can we spot patterns of events that point to problems at a particular plant?
NUREG/CP-0124 65 Appendix E
.. \\,
DISCUSSION GROUP 3 PARTICIPANTS Moderator:
Joe Fragola, SAIC
Participants:
Don Ilickman, AEOD Mike Poore, ORNL Dennis Bley, PLO -
Cindy Gentillon, INEL Bill Galyean, INEL Arthur Payne, SNL Carl Johnson, RES Erasmia Lois, RES Pranab Samanta, BNL Ernie Imfgren, SAIC Millard Wohl, NRR John Wreathall, SAIC Stacy Rosenberg, NRR Tom Novak, AEOD NUREG/CP-0124 66 Appendix E
r q
f.;
b 9
APPENDIX F View Graphs for
~
" Accident Sequence Precursor Program' Methods" Joseph W. Minarick l'
H p
67 Appendix F g
' ~ '
f r
ACCIDENT SEQUENCE. PRECURSOR PROGR A M METilODS -
4 Workshop on the Use of PRA Methodology for the r>
Analysis of Reactor Events and Operational Data January 29 30,1992 Joseph W. Minarick Scien:e Applications Intemational Coqoration 1
Definitions
- a Accident sequences of primary Interest in the ASP pmgram are those that, if completed, would have resulted in inadequate core cooling and would have potentially resulted in severe core damage.
_ Accident sequence precursors are events that are importsni elements in such seruences - for g
example, an unusual initiating event or failures of maltiple, components that, when coupled with one or more postulated events, could result in a plant ctooition leanig to severe core damage.
P u
2
- NUREG/CP-0124l 68
-Appendix F
.. ~ -.
. 1.,
w i
Objectives Search operational events for the elements or precursors of severe core damage accident sequences,
. Analyze operational events and rank them na to their likelihood of proceeding to core damage, l,
From operational events identify significant or important sequences that, more likely than others, i 6-could lead to severe core damage.
l-a
- )...
Type of Events Covered
~
.While all off normal plant conditions are associated with some risk, the ASP program concentretes on:
- Unosual initiating events (loss of offsite power, small break loss of coolant accident, cascade
. electrical failures),
' '- Total failures of safety related systems, and '
- Degraded multiple systems NUREG/CP-0124 69-Appendix F p'
w w
q; i
i y-Types of Events Not Cevered i
- Events not addressed include:
C Uncomplicated reactor trips,
- Losses of feedwater without additional failures, Single failures in systems (without an initiating event),
Losses of redundancy In a single system which could be a system failure at another plant (e.g.,
unavailability of a motor-driven and turbine-driven AFW pump at e plant with a three-pump AFW system), and,
Design errors discovered by reanalysis.
i Overall ASP Program Approach
,-p Review IIRs to identify events which satisfy selection criteria as precursors.
'. Determine impact of
- elements" of each event on systems and functions which provide protection
- sgainst core damage. These systems and functions art defined through the use of event sequence nalets (event trees). -
Estimate a conditional probability of subsequent severe core damage for each precursor using event -
tress modified to reflect systems observed to te degraded or failed during the precutscr. Initiating event frequencies and system failure probabilitica developed from the precursors themselves are used when possible.
L Rank pitcursors as to significance and identify attributes of more significant events, a
i NUREG/CP-0124 70 Appendix F 1
I
.~_
3 Review of LERs for Potential Precursors -
~
' All 1984 87 LER: were reviewed by two engineers for potential precursors. Events selected during this review were then subjected to e detailed analysis. Events selected for detailed review included:
core-damage initiators (including LOFWs, LOOPS, and small break LOCAs);
- all events in which reactor trip was demanded; all suppor. system failures, including failures in cooling water systems, instrument air, instrumentation and control, and electric power systems;.
any event wh;te two or more fa!!ures occurred;
~
' any event or operating condit!cn that was not predicted or proceeded differently for the plant design basis; and -
any event that, based on the reviewers' experience, cocid have resulted in or significant y l
affected a chain of events leading to potential severe core damage.
For 1988-89, IERs screened as Category 2 by AEOD and all reactor trips were reviewed for-precursors. - his reduced the number of LER: requiring review by 75% and allowed for additional detailed review and documentation effort. 'llowever, the possibility esists that some potential precursors were not identified to Category 2 events.
Review of LERs for Potential Precursors (cont.)
Use of the SCSS data base to screen LERs for potential prertttsers has been explored in the ASP.
program.: A computer %) screening approach was developed which identified a subset of 25% of LERs which contained almost all precursors which had been identified in 1984-89.- Screening manpower requirements were reduced by 40 50% compared to manual reviews.
Efforts to further confirn the usefulness of SCSS in identifying precuisors are currently underway.
This effo:t involves a manual review of all 1990 LERs.
For 1990,~ the SCSS data base was' screened to identify potential precursors. Rese events were -
reviewed along with AEOD Category 2 events. All events finally selected as precursors were identified using the computerized screening approach..
' NUREG/CP-0124 71 Appendix F vt x
m
--w-y 4
pe<
~
,~
. - -. ~. ~
. ~
+.
Review of LERs for Precuisors: Detailed Review
~ " Die detailed nyiew of selected events considers the immediate impact of an initiating event or the
, potential impact of equipment failures or operator errors on readiness of systems in the plant for.
mitigation of off-normal and accident conditions. 'Dute general scenarios are considered:
- - if the event or failure was immediately detectable and occurnd while the plant was at power, then the event is evaluated according to the likelihood that it and the ensuing plant nispense cculd iend to severe core damage:
If the event or failure had no immediate effect on plact ~jeration (i.e, if no initiating event occurred), then the review considers wichet the plant would require the failed iten.s for mitigation of potential severe core-dx.tage sequences given a postulated initiating event during the failure period; and '
If the event or failure occurs while the plant was not at power, then the event is evaluated according to whether it could have occurred while at power or at hot shutdown immediately
- following power operation or if it could have aly occurred at cold shutdown conditions, if Ni event could have occuned at power it 1: typically evaluated under that condition, Four Sets of Attributes Are Common To ASP Events
- Events are selected and documented as accident sequence precursors if they include one of the following attributes:
a core-damage initiator (such as a LOOP, small steam-like break, or small-break LOCA);
a failure of a system (all trains of a multiple-train system) required to mitigate the
.. consequences of a core-damage initiator.
- degradation in more than one system required to mitigate the consequences of a core-damage initiatort or
- reactor trips and losses of feedwater with a degraded mitigating system (1984 and following),
. and if the estimated conditional probability of subsequent severe core damage t 10-6(1987 and following). Documentation includes 2 3 pages nf descriptive material plus supporting tables, j
1 graphs, diagrams, and computer output sheets. '
Failures in containmentclated systems (total failures and multiple degrades) and other interesting events are also documented.
l:
L i
L l -.
NUREG/CP-0124 72 Appendix F l-i I >
L l
~
. - ~.. - - ~ -
- -~
-. ~. - ~... - -. -...
~...
s-Precursor. Modeling Approach U.S. LWRs have been divided into eight plant classes - five for PWits and three for BWRs 'the classes are defined based on the une of similar systems for accidant sequence n:itigation and simliar response, on a sytym les et, to initiating events.
4
- Trar.sient, loss-of offsite power and small break LOCA event trees were developed for each plant
!g
- c. lass. Each event tire addresses both safety related and non-safety systems v.hich can tx sed to
?; '
tuitigate off normal events. Using audi trees, the impact of system le /el operatico on individual plant classes can be distinguished.
Two undesired end states are included in the event trees:
core damage (inadequate' core coolingh and ATWS (failure to scram).
Example Event ' free Model 0
lPR NO E
CHAL-RESEAT OK OK
-[
il CD 12 CD OK OK r
OK f
13 CO 44 CO OK ok i
1:
CD {1) to CD 17 CD te ATW9
=
I (1)OK fw Class D PWR Classos B and D Nonspecific Reactor Tfip i
NUREG/CP-0124 73 Appendix F
o Estimation of System 1.evel Failures When adequate precursor information exists, system level failure-on-demand probabilities are estimated from the piecursors themselves by at. signing a failure to rtcover (restore) likelihood to each failure, sumining these likelihoods, and thviding by the estimated number of demands:
observed failures on demand
[ p (failum to recover) g i
p(system) =
total demands in observation period Table 1, System Nonrecovery Estimation tlkclihnt af failing to Recovery Descripke ter fun
t"I '
Class R1 Tailut. ad not 81 Tear e be recoverable in required perind, 1,00
- e.ither fmm entrol room or at failed equipment R2 Failure a; teared remverable in requimi perkx1 at failed 0.34 equipment, and equipmem was accessibic; ticovery from catrot runn did not aptwar possible R3 Failure arteered recovmble in required Frkwl from 0.12 control stun, but iecovery was not toutine or involved substantial stren R4 Failure appeared rxovmble in requimi perkul from 0.04 controlroom and was cmedoed toutine or procedurany
- based, a Dese vetoes are used for consistency of analysis. De actuallikelihood of failing to recover from en event at a particular plant is difficult to nuess and may vary substantially fm the values listed above.
NUREG/CP-0124 74 Appendix F
~
M E
is i
' l
'p
-' ]
Conditional Probability of Setere Core Damage L
l
'Indiyldual precursors are ranked as to significance by estimating a conditional probability of subsequent core damage given the failures observed during the event.
Fa!1ures Identifkd during the review of each precursor are mapped onto the plant class event trees
_ hich a.a then used to esthnate a conditional probability of subsequent core damage, given the w
= precursor.-
In this estimation method, the probability of a system failing given that it was observed successful or i
4 not challenged is assumed equal to the failure on demand probability for the system, while the probability of a system falling given that it was observed failed is assumed equal to the likelihood of not successfully restoring the system to operation (non recovery likelihood).
The conditional probability is a measure of the residual protection against core damage which existed during the event, and is a measure of precursor significance.
I a
Precursor Calculational Process
- 1. Event sequences nquiring calculation, if an initiating event occurs as part of a precursor (i.e., the precursor consists of an initiating
=
event plus possible additional fallares), then use the event tree associated with that initiator; otherwlac, use all event trees impacted by the observed unavailability.
L-2. Initiating event probattlity,
- lf an initiating event occurs as part of a precursor, then the initiator probability used it; the
- calculation is the probability of failing to recover from the observed Jnitiating event (i.e., the numeric value of the recovery class for the event).
- If an bitlating event does not occur as part of a precursor, then the probability used for the inhiating event is developed nasuming a constant hazard rate. Event durations (the period of time
- during which the failure existed) are; based on information included in the event report, if
- provided. If the event is dircovered during testing, then one-half of try test period (15 days for a typical 30-day test interval)is assumed, unless a specific failure duration is identified.
NUREG/CP-0124 75' Appendix F v
m
, -. ~ -
... - ~..,.
Precursor Calculational Process (cont.)
- 3. Dranch pabability estimation.
For event tree branches for which no failed or degraded condition is observed, a probabihty equal to the estimated branch failure probability is assigned.
For event : -t r wches associated with a failed system, a probability equal to the numeric value associated t _
+a recovery class is assigned.
For event tree branches that include a degraded system (i.e., a system that still meets minimum operability requirements but with reduced or no redundancy), the estimated failure probability is modified to reflect the loss. of redundancy, but the nominal non recovery probability is not modified. =
7 4._ Conditional probability estimation.
For unavailabilities, a differential measure is cateulated by subtracting the nominal risk over the
. unavailability period from the conditional probability calculated using the modified event trees.
For initiators, the nominal risk over the ml;igation period is not subtracted since it is typically much smaller than the conditional probability calculated with an inillator probability of 1.0.
u Precursor Calculational Process (cont.)
- 5. Support sys_ tem unavailabilities.
stem failures are modeled Systems or trains rendered unavailable as a result of sup ort fed, allimpacted systems (or secognizing that, as long as the affected support system rema ns fai trains) are unavailable; bet if the support system is recovered, all the affected systems are recovered. This can be modeled through multiple calculations which address support system
- failure and success. Calculated core damage probabilities for each case are normalized based on the likelihood of recovering the support system.
NUREG/CP-0124-76 Appendix F m
_._,.~
m l
I i
l Eaample Transient Calculation
~i Postulated Event: Trip a..d Loss of
=
g,,
- ",," {
=
=.
=,
Likelihood of not recovering from trip = 1; likclihood of not recovering
=e a
m -I
'a d'
main feedwater estimsted to be 0.34.
No other failures observed during
=
mitigation.
l i
p(core damage)
+=*
"=
p(seq. ll] +
=
l p[ seq.12] +... +
l i.
- p{ seq.17) i l
= 7.7 x 10-7
- - =
1 p(ABVS)
= p{ seq. I8)
= 3.0 x 10-5 F AILURE 8 OBEERvf a CURING EVIMT; Note: With the exception of relief tarw.a m
,,,,,wnm valve challenge, failure probabilities are mdicated.
Example Unavailability Calculation.
Postulated evens: imavailability of liigh Pressure Injection for 1/2
'O.
. "E E
month Likelihood of non.
"^
' " ~
restoration estimated to be 1.0 m
I 7'
Probability of non recoverable E e,,o Small-Break LOCA in 360 hr period
- mi
= 3.6E-4. Cnnditional probability of
.n co core damage given unavailability of 74 co a
llPI = 1.0.
f 1*
I i
75 im p(core damage) = 3.6E4. nominal
,=
1 on risk for same time period G.0E 7) -
'y"'*
n 3.6E 4 raarneseastaveo cuswea evewr:
Note: A complete analysis of 'his v.,..,
w,,
event would require - postulated
- =**'-'**=*=*a"==8=w**
transients and LOOPS to te addressed._
as well.
NUREG/CP-0124 77 Appendix F
Some Outstanding issues ASP data base is not totally unsistent from year to year.
Event tree models used for core damage calculations have claged from 1984 to 1990.
Screening criteria for selected candidate events has changed froin 1984-1990.
These inconsistencies can te chmina:ed without extensive effort for 1984 and later events.
Improved event tree models are bJr g developed for NRR by SAIC. he modela are based on the ASP models, but renect NUREG il50 insights to a greater degree than the c'irrent ASP models, include additionalinitiating events, and address altemate long. term woling strategies. %ese models will te usable in the ASP program once they are completed.
Inprovements ved to be made to the process of estimating non-recovery likelihoods.
'the potential use of detailed PRAs to analyze operational events needs to be explored.
Potential issues in the use cf precursor conditional probabitales to estimate a retrospectin frequency need to be explored.
=
NUREG/CP-0124 78 Appendix F
.I,g. <~
s 4
-.h I
b
-(
L t
t 1
L
- APPENDIX G l.:
View Graphs
- for -
. Methods for Identifying Risk Significant Trends Gareth W. Parry 1
1 ',;
t
.i
- Final -
79 June 1992
.. z.
I.
F METHbDS FOR IDENTIFYING RISK SIGNIFICANT TRENDS R
o e
presente'd by Gareth W. Party HALLIBURTON NUS Environmenta. Corporation at Workshop on the Use of PRA Methodology for the Analysis of -
Reactor Events and Operational Data -
Annapolis, Maryland DISCUSSION TOPICS-1
~
'. Given an Event Data Base (e.g., LER) and a Component Data Base (e.g.,
- HPRDS), and a PRA Model for each plant, discuss:.
- methods for screening data to identify th'ose data elements that are risk-or safety significant,~-
- methods for analyzing that reduced set to identify trends, data needs to provide meaningful results.
X DL""?,12 NUREG/CP-0124-go Appendix G'
l
~
USE OF PRA MODELS As a general rule, PRAs are an excellent filter for screening risk Important events importance measures Instantanecus risk measuro (failures, initiating events) time averaged risk increase / decrease (unavailabilities)
- However,
- PRAs do not usually address chases of operation other than full power
-do not model all components
-do not model causes of fai'ure O P" " ?.J' 2 L
USE OF PRA MODELS (Continued)
PRA models are structurally static, therefore, trends are identified through parameter value changes (e.g., initiatinD event frequencies, component unavailabilities, human error probabilities)
PRA models often based on specific assumptions. Different assumptions by different analysts can influence the screening for risk imoortance (e.g., assumptions about room cooling).
- PRA models are generally developed down to the level of f ailure mode.
For comparison with the PRA model, the events have to be translated into their impact on the components of the model.
O W.,."
NUREG/CP-0124 81 Appendix G l
I w
. SCREENING CRITERIA
+ Include degraded states as well as (PRA Based) failures
- Degraded states may be indicative of underlying problems or trends
. Screen events against all plant PRAs or just that at which it occurred?
4 1
OI.I.9I" Y J ANALYSIS OFSCREENED DATA
. In its most basic form, data is numbers of events affecting components and/or systems, arid a measure of opportunity. Therefore, analysis primarily focuses on rates.
. Key issues:
-grouping of data
- establishing hypotheses l
esemus NUREG/CP-0124 82 Appendix G -
l
i GROUPING OR POOLING DATA
. Events related to specific components in a specific plant are rare, therefore, stallstical fluctuations ca.i mask trends.
. Increasing sample size increases signal to noise ratio
. Increasing sample size makes sense for:
- generic trends (e.g., aging),
- underlying effects which cut across component types.
9 5 4'8J 5 rp, ju g ESTABLISHING HYPOTHESES To analyze data statistically,it is necessary to have some mental model of effects to be analyzed. The hypotheses give guidance for grouping
' data.'
Examples:
- Exploring aging time origin is taken as start of life, plant data grouped by year since start of life.
Exploring impact of change in regulation - time origin is shifted to date of implementation.
- Exploring impact of maintenance policy cho..., at a plant - all components affected may be regarded as the pool of data.
- Exploring impact of change of a specific piece of equipment - only that component's data is valid.
O WJ55' J*
NUREG/CP-0124 83 Appendix G I
DATA NEEDS
- Emphasis on recent developments in PRA methodology has been explicit consideration of causes as a means of Identifying potential fixes;
- e.g., common cause failure analysis and human reliability
- insights into what data are required to support these analyses highlight the need for a detailed description of the events including all contributing causes and influence factors.
I
@!E537.*J5 THE CAUSE-DEFENSE PICTURE OF CCFs NUREG/CR 5460 stresses the importance of understanding the chain of events that led to failure:
Trigger events Conditioning events
. The role of Defenses, and how they are defeated is crucial Root Cause related to identification of defense against recurrence.
L 915ES$P.LTS l
NUREG/CP-0124 84 Appendix G
3 I
b
^
-INFORMATION REQUIREMENTS Engineering Data
+-
- Description of component, its boundaries, operating and failure modes
. General Requirements for Reliability Parameters
-_ Operational history, exposure, event reports
- For each event an assessment of its impact
. CCF Requirements
- Correlation of event reports for radundant components
- Description of the causal chain leading to failure L
- Method of_ discovery H
- Corrective action -
-Inspection / Testing / Maintenance Practices OO*O 4
m_
.,'?_-
'NUREG/CP-0124 85 Appendix G
a 1
\\.
1 APPENDIX II View Graphs for t
" Approaches for m.alyzing Data to Address Generic Issues Related to Common Cause Failures, Iluman Factors, and Systems Interactions"
-All Mosich l
r 1
1 e-l l
l
- NUREO/CP-0124 87 Appendix H E
..J.
f%
i
?.
APPROACIIES R)R ANALYLING DATA TO ADDRl:SS GENERIC ISSUES RELATED TO COntMON CAUSE FAllAfRES, lit'htAN FACTORS, AND SYSTEMS INTERALTIONS All Mosleh Materlats a44 Nuclear Eegineering Deparunent University of Maryland, College Park Presented at Wrkshop on the Use of PRA Methodology for the Analysis of Reactor Events and Operational Data Annapolls, Jan 28 29,1972 GENERAL OllSERVATIONS O
TO IMPROVE Tl!E QUALITY OF TIIE ACCURACY OF PRA MODELS OPEkATION AL DATA MUST BE USED DOTH QUALITATIVELY AND QUANTITATIVELY O
EQUALLY IMPORTANT DUT MUCil LESS ACKNOWLEDGED IS Tilb NEED FOR AN UNDERLYINO MODEL TO OUlDE DATA C01111710N AND ANALYSIS O
TilESETWO PROCESSES OUOllT TO BE 1RTERACTIVE AND TTERATIVE LEADING TO AN EVOLUTIONARY IMPROVEMENT IN MODELS AND DATA NUREG/CP-0124 88 Appendix H 1
COMMON CAUSE FAILURE ANALYSIS DATA NEEDS O
LITECT!YE USE OF CURRENT MODEl.S REQUIRE
. MORE ACCURATE DESCRiiTION OF Tile EVENTS IN Tr.RMS OF CAUSES AND IMPACT OF Ti!E EVENT
. LEVEL OF REDUNDANCY
. SUCCESS DATA O
IMPROVED MODELS NEED (AS A MINIMUM)1NFORM ATION ON
. COUPLING FACTOR (S)
- BARRIERS AND DEIT.NSES BOTil AOAINST Tile CAUSE AND Tile COUPLING N
. PAILURE TIMES IVTURr MODE 1J WILL NEED, IN ADDITION TO Tile ABOVE, INFORMATION ON O
PilYSICAL NATURE OF Tile ROOT CAUSE AND COUPLING FACTOR OF Tile EVENTS
~
SOME l' ACTS AButTT COMMON CAUSE FAILURE EVFKTS O
CCP EVENTS ARE VERY RARE. A TWO-UNIT PLANT WITil MORE TilAN 22 YEARS OF OPERATIONAL DATA !!AS ONLY EXPERIENCED 6 CCFs WillCII IS ABOLTr $% OF ALL FAILURES EXPERIENCED. OUT OF MORE TilAN 4000 LERs REVIEWED ONLY ABOUT 150 WERE CCFs OF TIIE TYPE MODELED IN PRAs (POWER OPERATION ONLY)
O REYlEW OF Tile CCP DATA INDICATES COMMON CllARACTERISTICS WTTil OENERIC iMPLIC ATIONS PARTICULARLY WITl! RESPECT TO COUPLING FACTORS o
AND DEFENSE STRATEO!ES.
NUREG/CP 0124 89 Append. 11 ix
thiPLICATIONS FOR MODELING AND DATA ANALYSIS O
PLANT SPEC 11iC PRAs MUST CONSIDER INDUSTRY (GENERIC) EXPERIENCE FOR COMPLETENESS OF CCP MODELING AND REALISTIC ASSESSMENT OF PROBABILITIES O
DATA TROM VARIOUS PLANTS NEED TO !!E ANALYZED ACCORDING TO A COMPREHENSIVi! CLASSIFICATION SYSTEM IN ORDER O AIN GENERIC INSIGIITS INTO Tile UNDERLYINO CAUSES OF COM MON CAUSE FAILURES. CURRENT DAT A REPORTING SYSTEMS ONCLUDINO LER AND NPRDS) LACK ADEQUATE RECORDINO AND REPORTINO GUIDELINES FOR CLF EVEPRS.
I PHYSICA'. MODEL OF A COMMON CAUSE EVENT DEGRADED OR FAILED NO 1 COUPLING
[ TRIGOER MECHANISM EVENT
\\
DEGRADED 6
OR FAILED
+
COMPONENT h MODEL PUMP 1 FAILS
- 1. TWO PUMPS FLOOD LOCATED IN X 1
OCCURS
- 2. MOTORS gy SUSCEPTIDLE ROOM X TO MOISTURE PUMP 2 FAILS EXAMPLE t
NUREG/CP-0124 90 Appendix H
_ ~_
l CCF CLASSIFICATION
CCF TYPE II: Components of a system may fall after some time, given a CCF Shock. Components may or may not fall at the same time.
- SPECIFIC CASIZ CASE A: The coup'ing factor couples components in a random fashion so that the components fail conditionally independently.
CASE 11: The coupling factor acts on the components in a dependent fashion.
A TYPICAL TIME. DELAY FAILURE EVENT PILGRIM - MAY 1974 Four salt service water system pumps became or were made inoperable du-ing a -
5 day period. Pump "D" was removed from service because it was making a loud and unusual noise. Upon disassembly, it was observed that the key of the mot _or shaft was sheared at_ the key way. 'Ihe same kind of faults were also obsen>cd
- for other three pumps.
Appendix Ik NUREG/CP 0124 91 a.
~.
- _.... - -.. -. ~. - -
EXAMPLE OF IMPACT VECTOR ASSESSMENT WITH MULTIPLE INTERPRETATION OF EVEttT
,i.. in.iei si.i :
r - ne uri,iio.
c..se.ie,e.t op..
x x
Qt f" J Male.e fantee tower two diesel tenwaters failed to ten due to L.
(Appost191f1 p1vsted resister. fee third wait revistor was also plugged.
'e I??la*? "
(s) (vent (taisteltation Wypotidsis Probattlity f
I I
T
$hath type Feelt Mode 0
I f
3 13 0.9 0
0 1
0 noniethal (s) follyre durtal i
operation i*
3 13 0,1
<f l.
.< > h m
w.,s P V:D. M, '
see,.
ImpacttecIeor til i '. 3 ;
-wx tal Nittple Hypottents loosel tettw assess =*t m
ISSUES RELATED TO HUMAN RELIABILITY MODLIJi AND DAT*.
O
!!UMAN RELIABILITY ESTIMATES AS APPLIED TO NUCLEAR POWER PLANT PRAs ARE ALMOST COMPLETELY BASED ON JUDOEMENT. EVEN IN TilOSE CASES WilERE DATA COLLECTION llAS BEEN ATTEMPTED, MODELS WillCil ARE NOT VALIDATED NOR SUPPORTED BY A THEORETICAL OR EMPIRICAL FOUNDATION DENOMINATE Tile RESULTS.
O Wrfit Tile EXCEI' TION OF A RECENTLY LUNCilED AEOD PROORAM TIIERE HAS BEEN NO SYSTEMATIC EFFORT TO COMPILD AND - ANALYZE ACTUAL OPERATING EXPERIENCE FROM llUMAN PERFORMANCE POINT OF VIEW
'O GENERALLY SPEAKING CURRENT MODELS DO NOT REFLECT ACTUAL OPERATING EXPERIENCE. EVEN QUALITATIVE INS 10 LITS FROM Tile LIMrrED OPERATIONAL DATA
. IIAVE NOT BEEN USED SYST11MATICALLY IN Tile MODELS NUREG/CP-0124 92 Appendix H
f EXAMPLES OF INSIGIITS FROM EVENT REVIEWS PERIVRMED UNDER AEOD PROGRAh!
O EOP INADEQUACY Wfril RESPECT TO HNDLING PARTIAL FAILURES OF SYSTEMS O
DirITRENCE BETWEEN ACTVAL PLANT RESIONSE AND RESIONSE OF SIMULATORS USED TO TRAIN Tile OPERATING CREW O
CREW ERRORIN ASSESSING Tilf NATUREOF PLANT UPSET AND RECOVERY ACTIONS AS A RESULT OF COMMON CAUSE UNAVAILABILTTY OF REDUNDANT INSTRUMENTATION SOME ACit!EVAllLE COALS O
QUANTTTATIVE DATA FOR ERROR PROBADILITY ESTIM ATES IS DIFFICULT ( AT LEAST FOR DIRECT ESTIMATION) SINCE SUCCESS DATA IS VERY DIFFICULT TO 011TAIN, NEVERTHELESS SOME CONSIDERATION $110ULis BE OlVEN TO IDENTIFYING POSSIBLE APPROACi!ES FOR COLLECTING SUCCCSS DATA. Til1$ MIOllT BE EASIER IN Tile CASE OF OPERATOR RESPONSE TO INTR".ATING EVENTS.
O. EFPORTS IN THE AREA OF COLLECTING.- ANALYZING AhD CLASS!FYING HUMAN PERPORMANCE DATA SilOULD BE EXPANDED. Tile DIRECT BENEFir WILL BE IN GAINING INSIGHTS INTO CAUSES AND MODES OP llUMAN ERRORS. SUCil INSIGIITS p.
CAN BE USED TO IMPROVE PLANT SAFETY SOMETIMES WITH MINOR CllANGES IN PLANT OR OPERATING PRACTICES AND PROCEDURES. THEY CAN ALSO PROVIDB MUCil NEEDED 'REAL LIFE' INPUT TO THE IIUMAN RELIABILTTY MODEL BUILDING
- ACTIVmES, NUREG/CP-0124 93 Appendix H 1
l APPENDIX I View Graphs for
" Industry Risk Profiles: Do We Need More Modeling?"
George Apostolakis NUREG/CP-0124 95 Appendix 1 1
ZC M
IU O
m b?
E INDUSTRY RISK PROFILFS: DO WE NEED MORE.
MODELING7 CONCLUSIONS by George Apostolakis Mechanical, Aeros c & Nu ear g eering Department Operational experience is oflimited value University or alifornia unless it is interpreted through validated Los Angeles, CA 90024-1597 modcIs.
Tel:
(310) 325-1300 Fax:
(310) 206-2302 Drawing generic conclusions from operational experience makes the need for models more urgent.
Presented at the Workshop on the Use of PRA Methodology for the Analysis of Reac:or 2
Developing validated modcIs would require Events and Op.Goaal Data significant resources.
Annapolis, Maryland January 29-30,1992 a
c.
Y e
r
- u......
I l
REACTOft SAFI'1T STUDY l
- 11ft RELIAHtLJrY Rt3Ut.T1 %1tlOI WERE (DNIVITD ETRt TO APitY TO A POfVJ110N OF PLACTOR ltANT1 (l'.0) AND i(CNCE IT WAS DF&f1D10 MODEL TilI CVMIONI N1 I A! LURE VARI ABtillY litOM PIN (I TOitANT.*
$0VRCT4; itANDBOOKS RI.FCR11,OPERATINO CXPERIENCE. DI.PARll ttNT OF DETINSE, NA$ A. ET AL OftDER Or MAGNTTUDE ACOURACY A$$(.11ED RANOP. se tDO NORMA 1 Dt3TR!DVrlON
/* *\\
s 4,
,ss,
- 4
's
,d et
\\ /
N
,* 't n
,) ?
' %.L e
<r s
y-te I
jy's 9s e
3710"*
3 xl0~9 per hcarr per pipe section f
0 30
, PD$tt e10 R t...
\\
\\
.-K g
/,j/
~
N P p10 A gs s
s
/
. s to s
/
'N% ~ -.,
,/A
\\
\\.
g 97 1128 il t 10.6 37 4 3t t et t 64 9 Se t 116 4 f Al(Uni AATI Pin MU AND it103 Prior and Posteriet Histograms for Diewt Gene'aws - f ailure 4 Start NUREG/CP-0124 97 Appendix I
e
==
1 l
I E p'#,,-
i cp p/,
h W
- R E
r l
i
[-f-f s ~.. s gl u
_ a%
E a
~
s
\\
l n'
i R*
p*****'
y
_m g
1 i
i f
1 I
_i t
t J
I 2
3
$e U
R 2
9 e
w e
a w
A AI110 v0094 b
O 1
x a
m 4
6 S
D k
4 h
N m
s s9 R
R y
q d
=M o
.ce a
-o s
4 Q
j b
M LIs
- w. '
.c 9
a m
a
+
J E
e E
Y
$ 6
^
s-4 o
e q C
c m
R 4 w x
+
F
-Q 17
- ,w c
- w c
b ti
?
t
.HI g
l ' HI h
y]
o il 2
g g
- ir sIT q
(-
n NUREG/CP-0124 98 Appendix I
--.s i.,.-,,.,
c
,.e-m-.-
...-er
2C 4T 1
a so m
5 E=(t,1-Il g
3 2 n
UNCERTAIN OR DEBATABLE
- n "* *" 'o
- EVIDENCE l
. pit) l
~
fi".w-m,,c1341 O
e FIRE DETECTION 8
{s 2 =
3 j=1 FIRE FREQUENCY RECTIFIABILITY N
t dt v (llt,)
p (T lE )
1 l...
0 0
j=1 i
N e
e t
i c
=.
a I,}
1 s' n pgi I i;
h l
j I
j' !!!!!! ll
- c. ---
1.
[
i'
-3 e
n u
s
~
m E
f
?
- *:/:A
-s 9I
?)
]f i
.e E
li i
N.
f II 3
I j
- ij j
r.t ti 81 i
6 a
i i
f w
u... un,.....
\\
N k
wmo
'k, nwoco k
'a
/.
sivulsens 3
a j
i
\\
onuvoa ev
. o 3
i V
J NUREG/CP-0124 100 Appendix 1
l TAllt!!!I: MODI:1 S I
1.
I.J1EEA1AllillDILE TO CALCULATE T E M Pr.R AT UR E i
Dis 1RIstril0NS TilROUGil A COMPONLt(T DURING NOPM Al.
AND 1RANSIf.NT CONDillONS) 2.
SIRL11:11EALMJ10111L1INCOMPORAllNG 'Itil:kM AL ANil ikRADIAllON CRI'l P. AND $WI,11.INGj 3.
QlEll t
-A t
nc )k
~
. min ( FC(n)/*)( A nc t
g
_A e
c L
,t >0 e
kl R(tl1)=
k=0 1
,t =0
-A t nc )]
C R(tlg) o
[1 - P(1 + min (FC(t));A t
NUREO/CP-0124 101 Appendix I l
l
'~~
a l'
~Z.
Cp f:1 -
O h-i 3
"When both steam generators are dry, the
- d procedure requires the initiation of make-up/high pressure injection (MU/HPI) cooling, or what is The shift supervisor's hesitation to initiate called. the'" feed-and-biced" method for decay heat MU/HPI cooling is essentially treated as a non-event I
removal.
When the hot-leg temperature reached in NUREG/CR-4674.
591* F (normai post-trip temperature is about 550' The emphasts is on component failures and F), the secondary-side operator recommended to the Operator errws (mcorrect actuation of Steam and ekift supervisor that MU/HPI cooling be initiated. At Feedwater Rupture Control System on low steam air.nt the same time, the operations superintendent pressure instead of the desired low steam generator icht th. shift supervisor in a telephone discussion that level) if an auziliary feedwater pump wa.s not providing cooling to on-steam generator within one minute, to "The proper method of manual actuation of the prepare fu MU/HPI cooling. However, the shift SFRCS buttons will be reviewed with all licensed 5
supervisor did not initiate MU/HPI cooling. He operators. The switch layout is being modified to add waited for the equipment operators to recover the additional demonstration of the situation buttons and auxiliary feedwater system-to add actuation guards over the switches.'
The shift sutervisor appreciated the economic
" Operator mterviews md.icated that the shift was consequences of initiating MU/HPI cooling. One fully aware of the core stations and were prepared to t-operator described it as a drastic action Despite his imp ement the bleed-and-feed' core coohng method if l
delay the shift supervisor acknowledged having the auxiliary feedwater was not restored.
confidence in this mode of core cooling based on his l
simulator training; he would have initiated MU/HPI cooling if"it comes to that
LER Text.m NUREGICR 4674,vol
- Iess of Ma.in and Auxih.ary Fentwater Event at ttie Davis.
Besse P! sat on June 9,1985*
NUREG.II54, July 1955.
- C E.
sr e==
.. -. ~...
4 C.
- c MO ORG ANIZATION FACTORS RF1 FVANT TO.iAFETY O7 Strategic A pen
o I.
Goat Pnority 4
2.
Reepossivemess
$l 3.
Safety vs. Beace I.Ane Onentstion 4
Hardware vs_ Haaram Retsvices Emphasis 5.
Regulatory Relationships 6.
Indestry Cearpetition 4
7.
Pubisc Opieses Organizational culture refers to the value s.
usi
.Ma-ge.e.t Rei.ti s I6. ' u"""* *Oe",y "R"U *c*."". "e system of an organization.
11.
Indepeed.- Sefery Engvecerrag Group
- 12. Desise R se Orsamirnessa 2 $- r-- iia 9-Plant policies may set priorities of 14 Coordamers.e/1stestation r
operator actions long before emergencies.
is.. c-ra **
16.
1sterdependence 1
17.
Central ration 1 2*'7,'*d *r*t..s A fundamental management responsibility p;
- 1. i e r. o e,....e... :
is the establislunent of a safety culture w
i.
w.t - coniPoomy 2.
o.orai, es. 82..** **e" covernino the actions and interactions of 3..
1inkages with h
.O vall individuals and organizations engaged 4
s;-d of c cr.c Ree i
U E L" $'.*rio".r.tegr.ti in activities related to nuclear power k M*.*L-e (IAEA INSAG).
d Ergenesmie:
1.'
Esgtmeering Design and 7
- Seppart -
2.
Tolerance for Sub-ScandaA ognpment
]((R 3.
Ineffective Treading 4
Method for Employees se YJest.fy Poemaal PmNews o ee. l...MTg.""" '"**
Practical and validated models for NPP i-Pr=*d-=>
c2 ='i* *'*""
organizations are not available yet.
2.
Procedural Updses c
3.
Fromen,e es Reactrve 4
- '*At Right loc!*
3.
Methods for Setting Work Pnorrt:es 6.
Upda.:seg Documentatice and Drawings 7.
Ae se et Pnenry Sturns
- C 8.
Manegeaient Support for Lower Level Problem Solving g
Pers...ei:
- 3 8.
Accoentability CL.n' l
l.
i1
- li
)j,lj1ll j]
I l
iI III l1 x
w N%
i e
s.
9 r
l 3
CP o
m,.d.m g
sr o
wPI i*g s
E t
e e
=
mt O'
e ar e l(
kk e o ec c e
a i gd C g
rs b
M i
v mTTcUd R
r r ep e
r a
e m = C_
t.
p t
y r
s a
n o
a m_i g
mmSwdd s
C e
m m
A d a ne npn s
g s S'
- e. me i
g e
nr sA g
s s aUa c dd r
e.
n wt.
t e
r i
sl O
e n
y 4;
e a
w ee e
e oalr e g :d g _d s
h L.
_ s v
e s
c t
- s..e r
e a"e es u. A cs =
cgh cd i
r c y
r R
i d M' m mta t e
. e c s e
- .d s
s an e
e c
_ s m
cb tem,W
.e-eK uieR s a m
s
- d. s a
P N t.
y a
so l
l e
a i '" o r
vP e
B o
eg' Fa ML,
m,,U l aC
. eK e
S a
lat h
e T" C t n
s
, e a:
s s
v g
ef < a M
a c
s e3s6E wf ec g
en t oEe s rh
- a. s
. Cec g
e t
e f :
s se 1, teal e lei 4
f m y
e y
v y
e
+
die s, eE
- i yS H e e S
oC,
ae n _1 y e. P a C a A ;; n r
, e e
+
e~
. ar Jcs s
erwh
,y t _
nt 8
c k e y e. 6e 3 E m l.
4 r
n s
s c rAe t
a
.. i u. e n c
n n=gk a
ae e
ru=
e n sI v,
' c t.
- m. e P T e
e
{
a g
g o
e
. ne-t
. F a
n e
m e =
m qn mea
,. 7 r
o T
C.eim*
P c m 5
.lpn-.moe e
a1 rRi s.
v t e
rr a t
ncf s
. o n Oe
. Eg e.t r l:
c ai o
k r
s l
e ml i
i. e eapn.n s r s%
i o
e i u I p r, e
n l
s t mce
,frba s cyc ~ i egt re ade of vf mi:..e a
e mt r
r e
P e'
e oh en oMe e r
r MdicADI T P J T T A,a F R C T e V DEOMTMGSWeREPDP5""e.QTC ;ceRSt u
ce y
r s
. r e n. e o wF e
p m
.e
.. s
.. 01 2 s4 $6
........ 0 1,s 3 4 3 6 D. -..
. 01 l
. 2345'
- S91 1 1 i1 !
- 234567S911
,1 1 1 1 - I I34* 67091 le 1
1 s
c e
s t
i o
(
t 9
2 m.
1 eeo rht c
at g
s n--
i
. % m.
o s 'r r
s
. i t et e
ai-c c e
i n (r m
,,e de l
5m nis
.s 4
ic
.e m.
E eelu e
Mg hdo e
r u
li sc To t
ne.
s e
m r
U' A
. t e gw s
s s
r n1 e,
v icled.
oix t
taaf e
g d (r n )s e
n n
at g
et ir c
d ps t
y3 e) hsC a
a r
r v
t N
nr of m
v3 a et c a r
s t
Es ve Ro s
I 9P yl s m t
rel fad m
k:
nn s oa k pi I
s cut s mg l
a osan y
b ri o d gni
- s t
m.
e ai Fwo0 e
gd e w lI w
r 0
- m
. o l' C
F m
9t al 2
1irni a t
o c
w e
p e
u degd u
m a
a.
e e i
gi=
i r
E t
t u
J ud v
s.
8 Nfw i
d fg v
a e.
e n
i p
m.
Tdai cw
.f {-
u e
- k' 7
.{
yto LY #
2C%mLOQgg.?NM a
a
]
d
- 3
,i
APPENDIX J View Graphs for
'Use of PRAs and IPEs for Event Risk Analysis' Arthur C. Payne, Jr.
NUREG/CP-0124 105 Appendix J
Use of PRAs and IPEs for Event Risk Analysis Arthur C. Payne Jr.
Sandia National Laboratories Presented to:
Workshop on the Use of PRA Methodology for the Analysis of Reactor Events and Operational Data January. 29,1992 What are our goals?
1.-.To be able to assure the public that nuclear power plants are being operated in a safe. manner.
- 2. To develop improved techniques for identifying events that may compromise the predicted level of safety.
-106 Appendix J l'
l-.
5,[
o What Measure Do We Use to Evaluate the Significance of Possible Accidents?
- 1. The bottom line measure should be related to offsite risk to the pubile and environment from possible accidents.
- 2. These measures can be decomposed into several subsidiary measures:
frequency of core damage, conditional probability of vessel breach, e
conditional probability of containment faP~e, e
magnitude of the source term, and -
a consequence to the public.
How Do We Identify Risk Significant Events?
- 1. _ Evaluation of past events, depending on the quality of the data.
- 2. Theoretical analysis techniques such as PRA, systems
. analysis, etc.
s NUREG/CP 0124 107 Appendix J t
l.
What do We E sd in Ocder to Use PRAs/IPEs to Perform Eve;.. Evaluations and Operational Data Trending?
- 1. One needs a set of PRAs on all plants performed to a consistent level of detail and performed with the same goalin mind or
- 2. One noods a system that can account for differtng levels of details and goals.
How Can We Design a System Using a Consistent Level of Detail?
- 1. Up0rado current system -
a.
Select a set of representative plants, incorporating detailed PRA models of those plants into ASP or construct a set of simplified plant models for each plant (ASEP results might be used to generate models or IPEIPRA models might be incorporated directly).
b.
Include models for every plant (Again ASEP or : pes might be used as a bases).
c.
Include all accidents modeled.
- 2. Evaluate all other PRAs/IPEs to see if surrogate models represent each class of plants or if simplified models capture significant characteristics of plants.
NUREG/CP-0124 108 Appendix J i
How Can We Design a System Using a Consistent Level of Detail? (Concluded)
- 3. Upgrade models to include plant to plant variations in design l_
if determined to significant.
l
- 4. Use the current ASP approach to evaluate events. If events are not represented, upgrade models.
How Can We Design a System to Account for Differing Levels of Detalis and Goals?
- 1. Determine the Theoretical Characteristics of Events.
Examples:
a.
Frequency or probability.
b.
Number of components affected, c.
Importance in model.
d.
Detectability of Failure, e.
Diagnosabilty.
f.
Severity of Sequences generated from.
g.
Not thought of, h.
Able to Ana;yze.
Number of plants with precursor.
J.
Complexity may obscure accurate diagnosis.
k.
What if event occurred else where? Different effect in different plants or system.
NUREG/CP 0124 109 Appendix J
How Can We Design a System to Account for Differing Levels of Details and Goals? (Continued)
- 2. Determine characteristics of events that could be detected by data analysis.
- 3. Examine the events that have occurred to see if any characteristics are missing and if you are detecting events with those characteristics.
- 4. List characteristics of event that should be identified by PRAllPEs or assumptions that should limit the identification of events.
How Can We Design a System to Account for Differing Levels of Details and Goals? (Continued)
- 5. Examine PRAs/IPEs to see if events with the appropriate characteristics are being identified, List characteristics of IPEs and PRAs.
Intemal vs. External, Level of detail, Nomencleture, Thermal-Hydraulic code used, support calculations performed, Uncertainty included, Conservative, reellstic, non conservative,
- Current, Scope type of errors not included.
- Method What are assessed in PRAs?, what are missing?'
RVR, multiple tube rupture, reactivity, instrumentation, operator errors of commission, design and construction errors, low power, spent fuel pool.
NUREG/CP-0124 110 Appendix J
l How Can We Design a System to Account for Differing Levels of Details and Goals? (Continued)
- 6. Compare two sets of characteristics to see which each 9
method misses.
- 7. Construct an Al program to classify the characteristics of events determined to be dominant at plants from PRA analysis, other analysis techniques, or data.
Could also enter the assumptions /Ilm!tstions used in the analyses or data gathering,
- 8. Enter the characteristics of data events or theoretical events and see if any characteristics match or assumptions are violated, if does not match, ask for any characteristic changes that would make a match.
The PRA,IPE, data base, and others could be tied to the Al program directly, Aloe piant rianlon information could be included.
How Can We Design a System to Account for Differing Levels of Details and Goals? (Continued)
- 9. Go back and examine tb data to deterrnine if the events could have these characaristics or does the date reporting and analysis need to be changed. Similar consideration for PRA methodology.
10.
- e a master matrix of allidentified events from i nAs,lPEs, and data (etc.). Evaluate these events as to importance or if they have occurred in the data. Do data searches for ones which have not occurred yet but are important.
Use matrix of events thought of or occurred, check off ones occurred, Grade by significance, check trends in data, cct, calculate frequencies and compare to theorotteel frequencies.
NUREG/CP-0124 111 Appendix J
Al ANALYSIS APPROACH
' Determine Characteristics of Events
[
\\
Data Events l PRA Events }*-
k
, Compare to Theory 1
I Modify 4-Method I
Construct Al PRA/IPE Engine and Data
, Data Base,
j g
/_
\\
Compare -
Compare Data to Theory,
. Theory to Data N
I L
Modify Methods 2
t-I Modify Models
/_
Y Reevaluate Reevaluate Data -
Models c
s l
l L
NUREG/CP-0124 112 Appendix J _
=-
+
l i
{
'9 w
+
4m-
.m
,g._
,...,,.,,-.-...,,,,,~..~,.,........,.cy.r-.,,,....v..,.,
,...-,,,.,.,.--,y,
4 l.
APPENDIX K b
View Graphs for "Living PRA Concept" Dennis liley NUREG/CP-0124 113 Appendix K
LIVING PRA kt De, Dennis C. Diey presented at WORKSHOP ON THE USE OF PRA METHODOLOGY FOR THE ANALYSIS OF REACTOR EVENTS AND OPERATIONAL DATA U.S. Nuclear Regulatory Commission Omco for Analyele of Operational Data Annapolls, Maryland January 29 30, 1992
-w.-Est_
~Dh'
=-
g_
Qg,.g nam. u no suwe,s. ~watm wwuns I
"LIVING PRA" OVERVIEW WHAT IS IT7
. ITS USES METHODS DATA REQUIREMENTS POTENTIAL FOR ADOPTION TO NRC USE FOR
' RISK MONITORING AND EVENT ANALYSIS (ASP)
NUREG/CP-0124 114 Appendix K 4
1 l
l l
"LIVING PRA": WHAT IS IT7 l
- CUCHf - ALL THINGS TO ALL PEOPLI
- A SUGGESTED SET OF DESIDERATA
- FULL SCOPE REAUSTIC MODELS - MAINTAIN PERSPECTIVE
- UP TO-DATE DATA AND MODELS
- CONSIDERATION OF UNCERTAINTY l
- ACCES$1DLE AND EASY TO USE l
- EASILY MODIFIED - ALLOW TESTING CHARGES
- A DAY TO-DAY RISK MANAGEMENT TOOL
~.,,
,.....,..q.(
)...,.....,
l
! ' m -,,ar
.~
- "I;"
.;O?
-4 *!, =;.*
l
.'*O *".l.
""l
,, = =
l
. nr l
' EAU l
- '.O. ~ll'.:l.*
2,,.
.wr
- 7
. y
- l% \\=
l tr T
'e
- q=== J
"*"" *.~"./
l m ~
n.
3r
- w~ r~.
1.ac,".=
Paa Rut no..*eemens Appenesten Preeses NUREG/CP.0124 115 Appendix K
- LIVING PRA*--ADDITIONAL INTERPRETATIONS
- ADAPTABLE, CAN ANSWER NE'N ALTERNATIVE QUESTIONS
- E.G., HOW CAN 1 FIND THE RISK FROM CONTAMINATION OF INSTRUMENT AIR?
- UPDATE FREQUENCY
- AFTER MAJOR CHANGES
- ANNUALLY, MONTHLY, DAILY
- AFTER SlONIFICANT INDUSTRY EVENTS
- ABILITY TO COMMUNICATE OUTSIDE THE PRA COMMUNITY m
"Y;.
' ~
+
~ *5
- '. E
- g'-
- p,,.-
a-
..,~~~.~ 71
/,P" 4== s:
able@2:4 40 ME--(vsb.di,-[...
s.1 v
,v.E -:
y ~~
- =L k.
kj$
^~-m W
5
~
'%3?f RLt M W F3 M F.
K'5:
x' d, MtWMalHNMN m
/ mwaceaanz NY"l 1
p-.
y y<~.
- p. y -
v v
s e
a u
J it;'.'?*
y
.wpk These DirneastonalIvent 8equence Olegeam O
NUREG/CP-0124 116 Appendix K
l l
r t=-
.=
~ ~ ~
- =
=
f
=.l*
~-
C.""
O.
.=
.=.=
i._
1 i
i s
_= -- J.-.
3 t - =.
I r
y m.=
/==
1
- -=..====-..=..
x --
e=
3
=.====a=--
I
, r--
'~=~l e
e F
w.- 3~
i c=n ager_.--
i y
.......,...........ma.....,_.....
o 4
EVChT TREE LINKING STRUCTURE FOR SHUTDOWN EVENTS PSA E
-. =
( ~=~=~ \\
l,m G~y
=-
~;.
se-
=-
=c-l1 %4
~
I]
l Q}
"~
( =. ). rve=~
Lt =
l e...-- F H =-- H =.-- H =
H=-=- =.H =. I NUREG/CP-0124 117 Appendix K
'LIVING PRA*--POTENTIAL USES
- LOCATE WEAK POINTS IN PLANT
- SET PRIORITIES AMONG SAFETY PROBLEMS
- EVALUATE CHANGES TO EQUIPMENT AND PROCEDURES
- PLAN MAINTEN A.'
.0 TO OPTIMlZE SAFETY AND PRODUCTION
- OPERATOR AND ENGINEER TRAINING "LIVING PRA"--POTENTIAL USES
- DEVELOPMENT OF SAFETY CULTURE AND RISK AWARENESS
- OPTIMlZE TECHNICAL SPECIFICATIONS
- EVALUATE THE SIGN;FICANCE OF OPERATING EXPERIENCE
- SUPPORT ECONOMIC RISK EVALUATIONS.
SUPPORT EMERGENCY PLANNING AND' RESPONSE
.118 Appendix K
.~.
'LIVDG PRA'--METHODS
- NEW CONCERNS FOR A 'LIVING MODEL'
- CONFIGURATION MANAGEMENT OF MODEL AND DATA
- REVIEW OF DESIGN CHANGE PACKAGES
- BAYESIAN UPDATE OF PREVIOUS DISTRIBUTIONS WITH NEW PLANT-SPECIFIC DATA
- REVIEW OF INDUSTRY EVENTS
'LIVING PRA"--METHODS
- ARE NEW METHODS NEEDED TO SUPPORT SOME USES?
- DISCRETE EVENT SIMULATION
- DYNAMIC INTERACTION MODELS
- NEW HUMAN COC.NiTIVE MODELS
- FASTER ALGORITHMS NUREG/CP-0124 119 Appendix K
- LIVING PRA"--METHODS t
ARE NEW COMPUTER TOOLS NEEDED?
- 'MPROVED PERFORMANCE OR REPORTING e IMPROVED QUERY AND 'WHAT IF?*
- UPDATE _THE USUAL-PRA DATA: FAILURE RATES, INITIATING
. EVENT FREQUENCY, MAINTENANCE FREQUENCY AND DURATION, COMMON CAUSE PARAMETERS
- HOW ABOUT. OPERATING EXPERIENCE AS A CHECK ON HUMAN REllABILITY ASSESSMENT?
NUREG/CP-0124 120 Appendix K l
1 OPERATING EXPERIENCE INSIGHTS FOR PRA i
v,,.......
- s..
w.
4 e... w..-..
Acm.,we op.,
, t i.m e.e.. s... r<
.ncy w
,e Amw...
u.
.,. a = o.
r.e... Tc, e.
co c..
a.e.nn c.=.noa ca r.e..
u.
..w s,m. va. ae w, c.c.,i n
.r.s.
pi.
wei.= v.,i.sa*,
v Am. t.
n.e s.
- s. w t m.
c-r.a n. < u.d.w s.w...
T das
- E..a s.w.=me.
w.u.wo o.mc www
., e.
vo.a.. w s u..
w u.n.. m.u cra.,w n
.em ria A a.co. e. no,
s.v.,s a= wca.g.ee.
.s um.<,..m, o. men soir e
rim.ios a a.e.
o.wc w
Ame ec.
- o.
v.
o p.e. Amu, c=.-
l P. dorm ac.
Twn..i und. Adv.r De.e f.e. u
- 1. S.
c aan a.
a wa EII.cf..f P.c.
ry ACf 4*.
ws. P,.c.e Da/O. 8 eat U d Pfot.du, W.,%
7sshologk.1.ad eliy.m.1 T4.. T. D gno. d r. c.rry si.
o a nu,, m a.
I sp.,' K.6=t.dg.
'LIVING PRA'--POTENTIAL FOR ADAPTATION TO NRC USE FOR RISK MONITORING AND EVENT ANALYSIS (ASP)
- RISK METER *
- PLANT-SPECIFIC EXAMINATION OF PRECURSOR EVENTS
- OFFERS MANY IMPROVEMENTS
. D0 WE NEED NEW CRITERIA FOR PRECURSOR IDENTIFICATION?
- IDENTIFY WEAK SPOTS IN PRA
- INCENTIVI' TO CROSS CATALOG PRECURSORS NUREG/CP-0124 121 Appendix K
SUMMARY
- LIVING PRA MUST BE
- UP-TO-D ATE
- EASILY MODIFIED
- ADAPTABLE
[
- LIVING PRA OFFERS
- PLANT-SPECIFIC RISK-BASED REQULATORY DECISIONS
- A DAY-TO-DAY RISK MANAGEMENT TOOL
- RISK COMMUNICATION AND PERSPECTIVE I
b A pendix K NUREG/CP-0124 122 P
APPENDIX L View Graphs for
" Trending Plant Performance:
Thoughts on Risk-Based Performance Indicators" Joseph R. Fragola NUREG/CP-0124 123-Appendix L
TRENDING PLANT PERFORMANCE TitouGins ON RISK-BASED PERIORhiANCE INDICA'IORS JOSEPH R. FRAGOLA VICE PRESIDENT 3CIEN'E APPUCAn0NS 5N1ERNA110NAt. CORPORA 110N 8 Wrst 4(P Surer 14'*Doon New Yoax, Nr ~oss 10018 i
ALOD WORKS 110P ANNAPOl]S, MARYLAND JANUARY 29 h 30,1992 l-nII zwavz PzarcamAncs InmcAToa uzTzs f. 7c g ;..,. -.'- : d.ago..c I
l
, 4 ",
j-.. _,
_.x (
)
e
.a,.
\\
i.-
k I
m Meter 1
MEE!!!E9"a NUREG/CP-0124 124 Appendix L
PERFORMANCE INDICATOR CONCEPTS U.S. NRC performing R&D on performance indictors since 1980 The term Performance Indicator" reflects a
a set of data that should have correlation with individual plant safety performance (SECY 317)
Performance Indicators are ONE part of a Performance Management System
'l\\vo types of Performance Indicators:
- Direct
- Programmatic l
- .5IUR$YilbE2?'~
l== l l
I i
l ECRAM ~ }
EAFETY$YSTEM i (NCIDENT
[ FRFQUENCT l
, AVAILA311JTY MANAGEMENT 3
I I
1
_ I I
- EYsE (
Ya"N=E 7an'E l h74UrI] f"Eskn$o I
I l *==,8 l l "Jr.#, l l J=.. l co nc,w.
" N A8E8 EQUIPMENT ENOWN Cnnssplual Relationship l
l Al&latsaance to Safety I?ZYfl
_ I a=
l
, M Q f:39213:.>..
NUREG/CP-0124 125 Appendix L
i ItATIONAT P POIl T1IEllMAL EPPICIENCY A9 AN INDICATOft
- Safety and Thermal Efficiency both depend on high quality plant maintenance
- The non-safety related portion of the plant continuously generates Information during operation
- While the safety related portion depends on either passive features or standby systems whose status is only known Intermittently
- Long term neglect of maintenance needed for the plant to fulfill its basic mission - generating electrical energy efficiently-may indicate an even greater neglect of the nuclear safety functions a lleactor scrams and safety system challenges are mainly attributable to BOP system and component failures i
SAllisGE25"~.
Causes of Parameter Fluctuations i
oPERATM)N E.R RoRS
/ MAINTENANCE AND EQUtPMENT ]'
\\
TE5 TERROR $
FAILORES -
M AINTEN A NCE RELATED HEAT SINK LIMITATIONS PERIODIC I
TFJTINo OPERATIONAL EVENTS n
IIEllAVIOR 4 7
/ T EL SC E MANAGEMENT Note anew and line alze denote relative contribution to fluctuations on plant behavior y am::u:. w ::n..
NUREG/CP-0124 12(3 Appendix L
i Average Daily Power level As An Indicator Source The driving factors are hamed on the initial requirements
, wt forth by 'the Commini.non in HECY - M - 317
- Broad Based : Ihe indicator should sense effects from maintenance, operations, engineering, management, etc.
- The indicator must be related to safety
- No new data, or reporting requirements
- The indicator should not be focused on comparing plants
- The indicator should be objective P
.Sg/GJgg@3%.
INITIALANALYSIS
- Fast Fouder Transforms pnd Power Spectrum Densitiep _
+ Standard IIcat Rafe Ahalysis i Otlier Mathematical Tsansformations SlIIQ!?EEG~
NUREG/CP-0124 127 Appendix L
l'I. ANT "X 1985 mourm w rrH n exrs 1100 im.
_._1 s
g I
-_ (. %
T M
q_
N f_
- l-y j,g i
t g
s J~
t l
s5g
_ y 1
-f 400 m.
^
' 1*
l a e e
. ;g_g gy--
1 j
m.
-g._g a_
1
-1 E
a
-g,,-
,a
_. - _] 4.. - >
A..
I
.j
,12
.]
J s
. _.. _._4,_._
e -.
f
! lik-_
2 li i
.ig e
.6
.e m ii, ii. ii. m D**
- Of fh' Y'."' _ ggpg' 'jg.
g Characterizations Of Average Daily Power Level The general plant behavior can be typically characterized by:
O Number and Magnitude of the Power Losises (Fluctuations)
-- SglGff::3.':. 5:y..
3 NUREG/CP-0124 123 Appendix L
' ~ " - ~ - - - - ~,., _. - _
, CILtRACTERIZATIONS OF Tilt OUTPUT 13T.IIAVIOR OF A PLAhT TROkt Tite ANALY$15 OF A\\TRAGE 11\\tLY pot \\TH LE\\TL DATA
- l. II:gh Instability C!narst terved by tnany power lasses
- 2. Iligh Instablitty at low 1%wer levels Characterued by nuny pow er losses at a low puwer leves 3 lhgb instablitty Aner a long Outage Characterved by snany power lossen or seseral strains octwitng rtght After a lengthy outage (.c 4 munttd
- 4. Cantinous Low Power Operuuan ~ Operauon at lower Vatn 60% average power
- 5. Operuuan at Decreasing Power Uvels - Ebwer level decreasing Ior no apparent reason 6 Operunon at a DecreasJng Ivwer level Combused mth a ihgh Instabluty
- 7. Scrains Occurnng at a Low lbwer Level 8, A large Nutuber of Scranas llurge power loss) at Any h>wer Level
- ~ -
--- j M h 5 f h[l.h hUwe
'ur PIANT "II'* 1982 1989 voo T
r1 1
l'f
~
ll j---
suo.
L j
q l
ll l
i i
fi f
~
1 i
l l
j h
l
~
i +
t g
l l
3 1
sm--
l i
i i
h l
l e
l e-F
- i 4
3 i
0*'
l m
- ----t g
t l
m.-_...
_ y l**
- r
.p_
i e
- iaaaeaeaeaett31iiIt33g13 Il l tti Days Of The Year
. ggg,g.,gn,,,,
NUREG/CP-0124 129 Appendix L
~
N
r'~
MF.ASUREMENT OF PAllAMETERS
- Average Power Level Normallred Percent Average Power IA: vel, based on Net Maxir.tum Dependable Capacity
- Time Rate of Change of the Power Level The Dircation of the Slope of the Change in Power Level Per Time (magnitude is ignored)
- Number and Magnitude of the Power Losses (Fluctuntions)
The Number and Magnitude of the Power Losses, for a given time period SQj'.g. gly'. Jo..
PLtNT OUTPUT DEHAVIOR CHARACTEEUZATION " RULES OF THun1D" L power lxxEL 1-uct:(ui, a es
- MODElm. (50, ? 00% and e Bu%
- LOW (1), 2 40% and c 60%
- VERY LOW (V). < 40%
- 2. POWER RATE
- INCIEASING (U)
- 3. POWER 12 VEL INSTALL!LrtT
- (0 e 4:uctuadoni,0% 5% average power loss and a 151osers but < 25 losses
- (F) e Muctuauon 2. 0% 5% average power loss and 2 25 lasses
- (!) e icw instaNutr. 5% - 25% average power loss and a 10 losses j
- #0 s Higfilnstab11 tty, 25% + 50% average power loss and 2 5 losses
- fS) e Signflicant instaNuty, 50% 100% ave. rage power loss and a 3 losses WCTE: !! power le*el = V ($ 40%), a detamtaanon of outage and other saveaugauen la squirsd.
'Diese characterizauona are comNurd to produce an OlffFVT DDIAnTOR ntATRIX, where the Qr54 character identines the Power level the second character idenunes the Power Rate, and the tNrc through atch ercaracters idenury the instabutty.
_ gggt,.g. gig :,,...
NUREQ/CM124 130 Appendix L
a x
A' 4
Combination of Parameters-These individual characterizations are then combined to produce all possible combinations
- of the plant output behavior The com?inations were then placed in a non mathematical matrix to allow for bining of different plant output
. behavior groups.
. The 5 different groups differentiate high quality from poor quality maintenance, as defined in the broad sense here
.1 g Jr g.tga.
L p
131 Appendix L
. -,, ~.. -- -
NRC oosc 315 U 5. NUCLEQQ REGULQTOQV COMMIESION
- 1. REPOR1 NUMBE R Nc$t902 h
mt. m BIBLIOGRAPHIC DATAf SHEET
' (See instntettung on the reverse)-
yGQ[Ql[y,Q}Qf P, flTLE AND SUD14TLE Workshop-on the' Use of PRA Methodology for the Analysis-1 oArtRcPORTeuausHio n
-of-Reactor Events _ and Operational Data j
uo~ m u An June-1992
- 6. T YPE OF REPOftT Final-D.M.: Rasmuson, U.S. Nuclear Regulatory Commission S. Dingman, Sandia National-Laboratories
- i. PchiOo cOvt Reo,=u o,
January 29-30, 1992 8, VE RFORMING ORGANilA1lON - N AML AND ADDRESS In knc.provke O*vmon, Ottu or R*+oot. VS Nwker Rwannwy Comm+ wn. and meamr eahesa. a sonun ner, oravane nesse end memne earnesso -
Division of Safety Programs Office -for Analysis and'Evaluati_on of Operational Data U.S. Nuclear Regulatory Commission Bashington, DC_ 20555 9 $PONSORING ORG ANIZATION - N AME AND ADDH ESS f,' Narc. rype "$eme es eso v'; # reatenfor.provkk N8tC D4vedon, Or/w w Mesma, UK heter Reevantery Comm*esma, i
- omermanneedstressi I
i
- 10. SUPPLCMENTARY NOTES 11, ABSTRACT (w woros pa kast A workshop entitled"The.Use of PRA Methodology for the Analysis of Reactor Events and Operational Data" was held on January 29-30, 1992 in Annapolis, Maryland.
Over~ 50 participants from the.NRC, its contractors, and others participated in the
-;meeti ngs.- During.the first day, presentations were made by invited-speakers to
-discuss issues' in relevant topics.
On the second day, discussion _ groups.were held to focusion three-areas:
(1) risk significance of operational events, (2) industry risk profile and generic concerns,--and (3)- risk-monitoring and risk-based performance _ indicators.
Summaries of the discussion sessions are contained.in the report as= wellias -important -insights gained from;the discussions.
- 12. k,L Y WOHDS/DESCH:P10HS ttest worm ermarsies seer win esern evern ra a warm, te, avoon,i u avActAsisil y si AltetNI Unlimited Probabilistic Risk Analysis n acu,m uAa...cAio~
Performance' Indicators in,r Accident Sequence' Precursors
-Uncl assi f; ed -
- Event: Analysis
,1.
.., n, Unclassi fied Ib. t 'MBEH Of PAGES 16 PHICE
_. N8tc FOHU 335 (?$91 '_
I 1
THIS DOCUMENT Whs PRINTED UgING RECYCLED PAPER u
,gm
.-p.p
- m,,-
a a
w
,A UNITED STATES :-
snctAt rouRTH CLASS RATE 4 ' NUCLEAR REGULATORY COMMISSION
' POSTAGE AND FEES PAC
- WASHINGTON, D.C. 20555-0001 y,,j7f[oja,,
OFFICIAL BUSINESS PENALTY FOR PRIVATE USE, $300 I'1ANIRG" y
7 E!goe:sdaggsucencss sycs WASHINcyCN DC' 205S5 1
a
.,'4^
^
,.,