ML20003E276

PV-2019-10 Post Exam Comments and Analysis

ML20003E276
Person / Time
Site:	Palo Verde
Issue date:	10/11/2019
From:	Kura M Arizona Public Service Co
To:	Kelly Clayton Division of Nuclear Materials Safety IV
References
PV-2019-10
Download: ML20003E276 (32)

Text

Recommended Changes to the 2019 PVNGS NRC Initial Written Exam Question 90 (graded with Q90 removed from exam per discussion with CE)

Given the timeline of events:

At time = 0100: All three units tripped due to a loss of offsite power following an Operating Basis Earthquake:

At time = 0105: Unit 1 had a loss of both EDGs due to Spray Pond piping ruptures in both EDG rooms At time = 0115: The Unit 1 CRS entered 40EP-9EO08, Blackout At time = 0120: Units 2 and 3 each reported that they have lost one EDG and their remaining EDG is supplying their Train A Class 4.16kV Bus At time = 0125: The ECC reported that an offsite line will be available in ~ 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> At time = 0155: The SBOG operator reports that neither SBOG will start Based on the timeline of events, the Unit 1 CRS should ___(1)___, and 40MG-9ZZ07, FLEX Support Guidelines, ___(2)___ required to be performed.

A. (1) remain in 40EP-9EO08, Blackout (2) IS B. (1) remain in 40EP-9EO08, Blackout (2) is NOT C. (1) transition to 40EP-9EO09, Functional Recovery (2) IS D. (1) transition to 40EP-9EO09, Functional Recovery (2) is NOT

Proposed Answer:

A Explanations:

A.

Correct.

B.

First part is correct. Second part is plausible since off-site power will be available well within the PVNGS blackout coping time of 16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br />, however if AC power will not be restored to a unit within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, entry into the ELAP procedure is required.

C.

First part is plausible since Unit 1 is operating on battery power and the Vital Auxiliary safety function will not be satisfied indefinitely, however all safety functions are currently met, therefore transition to the functional recovery procedure would not be warranted at this time. Second part is correct.

D.

First part is plausible since Unit 1 is operating on battery power and the Vital Auxiliary safety function will not be satisfied indefinitely, however all safety functions are currently met, therefore transition to the functional recovery procedure would not be warranted at this time. Second part is plausible since off-site power will be available well within the PVNGS blackout coping time of 16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br />, however if AC power will not be restored to a unit within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, entry into the ELAP procedure is required.

Question Source:

X New Bank Modified Previous NRC Exam Cognitive Level:

Memory or Fundamental Knowledge X

Comprehension or Analysis Level of Difficultly:

4 10CFR55.43:

5 Reference Provided:

N Learning Objective:

10535 - Identify whether or not exit from the Blackout EOP is appropriate

Technical

Reference:

SRO Only Question Guidance from NUREG 1021

Technical

Reference:

40EP-9EO08, Blackout Generally, if a SFSC is not satisfied in an optimal EOP, it is an automatic kickout to the Functional Recovery procedure, however in the case of a Blackout where no class buses will restored within an hour, the correct action is to remain in Blackout and declare an ELAP is in progress.

Technical

Reference:

40EP-9EO08, Blackout Due to the listed conditions, it is clear that power will not be restored within one hour, therefore an ELAP will have to be declared and the CRS will implement 40MG-9ZZ07, FLEX Support Guidelines

Facility Position: In the stem of the question, Unit 1 has been in a blackout for 50 minutes. As such, it is reasonable for the examinee to either determine that performance of 40MG-9ZZ07, FLEX Support Guidelines is NOT required because an hour has not yet elapsed since the start of the blackout condition. It is also reasonable to determine, given the conditions in the stem, that it is likely the blackout condition will exist for greater than one hour, in which case an ELAP would be declared and performance of 40MG-9ZZ07 would be required. The information presented in the stem provides a situation in which an SRO would be within their positional discretion to either wait the full 60 minutes, then implement 40MG-9ZZ07, FLEX Support Guidelines, or, based on their assessment of plant conditions, determine that an ELAP is imminent, which would then require implementation of the FLEX Support Guidelines.

Per Step 13 in 40EP-9EO08, Blackout, IF at least one vital 4.16 kV AC bus is NOT expected to be energized within one hour of the start of the event from either offsite power, an Emergency Diesel Generator, or an SBOG then the contingency actions would be performed. If the applicant expects that power will be available from one of those sources, they would NOT proceed to the contingency actions, which is where 40MG-9ZZ07, FLEX Support Guidelines, is directed.

Step 13.2 (contingency action for step 13) in 40EP-9EO08, Blackout, states, IF AC power will NOT be available from offsite power, an SBOG, or any Units EDG within one hours of the start of the event (ELAP), THEN perform the following: Declare an ELAP is in progress and PERFORM 40MG-9ZZ07, FLEX Support Guidelines. The way this step is worded, the requirement to implement the FLEX Support Guidelines BEFORE one hour has elapsed is based on the assessment of whether or not power may be restored before one hour has elapsed. If the CRS has determined that power may be restored before one hour has elapsed, FLEX Support Guidelines is NOT required. If the CRS has determined that power will NOT be restored before one hour has elapsed, FLEX Support Guidelines IS required.

Exam Author Perspective: When I wrote the question, I intended for the student to assume that power would not be restored within an hour of the event. The reason I wrote the question 50 minutes into the blackout instead of greater than one hour was to enhance the plausibility of distractors B and D. In doing so, I believe that I increased the ambiguity of the question which resulted in the examinees being forced to make an assumption about the restoration of power which would be required to answer the question. In hindsight, I would have placed Unit 1 > one hour into the blackout to conclusively state that an ELAP was in progress to ensure that only one answer could be argued as correct.

Facility Recommendation: Per NUREG 1021, ES-403, Section D.1.b, the question contained an unclear stem that confused the applicants or did not provide all the necessary information to conclusively answer the question.

Although A and B part 2 (IS and IS NOT required) are conflicting, they are each correct based on two different but valid assessments of the given conditions. Based on C and D being clearly incorrect, and either A or B being correct based on CRS judgment, our recommendation is to either accept both A and B as correct or remove the question from the exam since neither A nor B can be conclusively justified as incorrect.

Question 96 (graded question as-is per discussion with CE)

Given the following conditions:

A design change is being proposed for all three units The proposed change is required to be assessed using the 50.59 process (1) Which part of the 50.59 process will indicate if a 50.59 evaluation is required to be performed?

(2) If a 50.59 evaluation is required, what are the MINIMUM qualifications required to perform the evaluation?

A. (1) Screening (2) 50.59 Evaluator qualification ONLY B. (1) Screening (2) 50.59 Evaluator qualification AND an SRO license C. (1) Applicability Determination (2) 50.59 Evaluator qualification ONLY D. (1) Applicability Determination (2) 50.59 Evaluator qualification AND an SRO license

Proposed Answer:

A Explanations:

A.

Correct.

B.

First part is correct. Second part is plausible since one of the minimum education and experience requirements to qualify as a 50.59 evaluator is an SRO license, however having an SRO license is not a requirement in order to perform a 50.59 evaluation, only the 50.59 evaluator qualification is required.

C.

First part is plausible since the applicability determination is used to determine if 50.59 applies or if the change is covered by another regulation, and is one of the two stages in the three step process that proceeds the evaluation, however the applicability determination indicates if a screening is required, not an evaluation. Second part is correct.

D.

First part is plausible since the applicability determination is used to determine if 50.59 applies or if the change is covered by another regulation, and is one of the two stages in the three step process that proceeds the evaluation, however the applicability determination indicates if a screening is required, not an evaluation. Second part is plausible since one of the minimum education and experience requirements to qualify as a 50.59 evaluator is an SRO license, however having an SRO license is not a requirement in order to perform a 50.59 evaluation, only the 50.59 evaluator qualification is required.

Question Source:

X New Bank Modified Previous NRC Exam Cognitive Level:

X Memory or Fundamental Knowledge Comprehension or Analysis Level of Difficultly:

3 10CFR55.43:

3 Reference Provided:

N Learning Objective:

10080 - Describe the purpose of the 50.59 safety screening and evaluation

Technical

Reference:

SRO Only Question Guidance from NUREG 1021 Technical

Reference:

93DP-0LC07, 10CFR50.59 and 72.48 Screenings and Evaluations

Technical

Reference:

93DP-0LC07, 10CFR50.59 and 72.48 Screenings and Evaluations

Technical

Reference:

93DP-0LC07, 10CFR50.59 and 72.48 Screenings and Evaluations Qualification requires a prerequisite, one of which is an SRO license, however there are two other ways to meet the education and experience prerequisite so an SRO license is NOT required to perform a 50.59 evaluation.

Facility Position: The original question submitted for this K/A was graded as LOD 1.

In an attempt to raise the level of difficulty, the second part of the question was modified. In doing so, the exam team lost the operational validity of the question.

The SRO Master Task List includes the 50.59 process, Assess compliance with 10CFR50.59. This is covered in the classroom and on OJT for initial training, and via Computer Based Training (CBT) for continuing training. The objectives for initial 50.59 training are:

Describe operations responsibilities IAW 93DP-0LC07, 10CFR50.59 and 72.48 Screening and Evaluations Describe the purpose of the 50.59 review process Describe when a 50.59 screening is required Describe the NRC definition of change as it applies to 10CFR50.59 Describe the regulatory basis for non-applicability determinations Describe the relationship of 10CFR50.59 to other change regulations

Technical

Reference:

93DP-0LC07, 10CFR50.59 and 72.48 Screenings and Evaluations The operations responsiblities would best correlate, for an SRO, with a PVNGS Section / Team leader. The following indicates what is covered by the first objective listed above:

In the introduction section of the 50.59 lesson plan, it states, It is extremely important that the SM/CRS ensures that the individual(s) performing the Applicability Determination, Screening, or Evaluation are qualified to perform that function, however there is no discussion about what those qualifications are, nor what the prerequisites for those qualifications are. The process for determining if an individual is or is not qualified to perform is to enter the individuals name in to TQUALS (PVNGS qualification verification system) and the system will simply indicate what that person is currently qualified. So while the ability to verify 50.59 qualifications is part of the SRO job function, details of the requirements of said qualifications is not.

The SRO Job Qualification Card contains items related to the 50.59 process as well:

Assess impacts to 10CFR50.59 and 72.48 following a TAPA (Temporary Approved Procedure Action)

Assess impacts to 10CFR50.59 and 72.48 following a change to an Operations Technical Document to align a system or component In both sections, the assess impacts directive determines if the TAPA or change to the TD requires the applicant to determine whether or not use of the 50.59 process is

required. Determining the minimum requirements to perform the applicability determination, screening, or evaluation is not part of the SRO Job Qualification Card.

At PVNGS, 50.59 applicability determinations can be performed by licensed operators as well as other departments (i.e. engineering), however 50.59 screenings and evaluations require a qualification beyond that which is normally expected as an SRO.

Currently, there are 121 people at PVNGS qualified as a 50.59 screener and 26 people qualified as a 50.59 evaluator, none of whom hold an operating license at PVNGS nor do they work for the Operations Department. Being qualified in the 50.59 process is not a requirement to stand watch in an SRO position (currently 26 licensed SROs are qualified applicability determination out of a total 62 of licensed SROs). We believe the second part of the question is beyond the scope of knowledge required for an SRO and is not part of the SRO job function at PVNGS.

Exam Author Perspective: I originally submitted a question based on the 50.59 process (applicability determination, screening, evaluation) that contained the information I felt was appropriate for an SRO applicant to know from memory. The question was graded as LOD-1 and I worked to revise it. I incorporated part of the original submitted question (part 1 of the as-given question) and modified the second part of the question to ask about the required qualifications to perform a 50.59 evaluation. I validated the modified question with six SROs and the feedback was positive. What I did not recognize is that most of the SROs were former engineers and since 50.59 evaluations are an engineering task, they readily knew that an SRO license is not required to perform 50.59 evaluations. By asking the details of what specific qualifications are and are not required in order to perform a 50.59 evaluation, I was outside the scope of what an SRO is required to know to perform their job function.

Facility Recommendation: Per NUREG 1021, ES-403, Section D.1.b, the question is at the wrong license level or not linked to job requirements.

Since performing 50.59 evaluations, and more specifically, knowledge of the minimum requirements to qualify as a 50.59 evaluator, are not a part of the SRO job function, the second part of the question is inappropriate for an SRO licensing exam and should be removed from the exam.

Question 99 (graded with both A and C accepted as correct answers per discussion with CE)

(1) Per 40EP-9EO07, LOOP/LOFC, to meet the Containment Temperature and Pressure Control Safety Function following a loss of offsite power, Containment temperature must be less than a MAXIMUM of (2) Per 40EP-9EO08, Blackout, to meet the Containment Temperature and Pressure Control Safety Function, Containment temperature must be less than a MAXIMUM of A.

(1) 117°F (2) 200°F B.

(1) 117°F (2) 235°F C.

(1) 125°F (2) 200°F D.

(1) 125°F (2) 235°F

Proposed Answer:

C Explanations:

A.

First part is plausible since 117°F is the containment temperature limit per 40EP-9EO07, however only if there is a loss of forced circulation without a loss of offsite power. Second part is correct.

B.

First part is plausible since 117°F is the containment temperature limit per 40EP-9EO07, however only if there is a loss of forced circulation without a loss of offsite power. Second part is plausible since 235°F is the containment temperature limit during a LOCA or if the Functional Recovery procedure is used, however during a blackout, the temperature limit is 200°F.

C.

Correct.

D.

First part is correct. Second part is plausible since 235°F is the containment temperature limit during a LOCA or if the Functional Recovery procedure is used, however during a blackout, the temperature limit is 200°F.

Question Source:

X New Bank Modified Previous NRC Exam Cognitive Level:

X Memory or Fundamental Knowledge Comprehension or Analysis Level of Difficultly:

3 10CFR55.43:

5 Reference Provided:

N Learning Objective:

10319 - Analyze Containment Temperature and Pressure Control to determine if the SFSC acceptance criteria is satisfied

Technical

Reference:

SRO Only Question Guidance from NUREG 1021

Technical

Reference:

40EP-9EO07, LOOP/LOFC Prior to Feb 2018, the containment temp limit was 117°F for a LOOP and a LOFC, however the containment temp limit was raised to 125°F if a loss of offsite power has occurred.

Technical

Reference:

40EP-9EO08, Blackout Technical

Reference:

40EP-9EO03, LOCA

Facility Position: 40EP-9EO07, Loss of Offsite Power / Loss of Forced Circulation, was modified in February of 2018 to reflect the change from 117°F as the maximum containment temperature to satisfy the Containment Temperature and Pressure Control safety function during both a loss of offsite power and a loss of forced circulation. The EOP modification kept maximum containment temperature for a loss of forced circulation at 117°F, but modified the maximum containment temperature for a loss of offsite power to125°F. However, 40DP-9AP12, Loss of Offsite Power / Loss of Forced Circulation Technical Guideline, still indicates that the maximum containment temperature required to satisfy the Containment Temperature and Pressure Control safety function for a Loss of Offsite Power or Loss of Forced Circulation event is 117°F:

Technical

Reference:

40DP-9AP12, Loss of Offsite Power - Loss of Forced Circulation Technical Guideline

Technical

Reference:

PVNGS Technical Specifications EOPs and the associated Technical Guidelines are routinely used for both periodic and NRC written exam. For example, the 2019 NRC Initial Written Exam questions 13 and 36 directly asked about content in the Technical Guidelines, and questions 21, 80, and 85 used the Technical Guidelines in the pedigree to support correct (or plausible incorrect) answers. As a result, the applicants regularly study the Technical Guidelines throughout the program to aid in learning the EOPs.

Exam Author Perspective: When I developed the question, I assumed that the Containment Temperature and Pressure Control safety function limit was the same in the EOP and the EOP Tech Guideline and did not check to ensure they matched. Since the Tech Guideline for each EOP provides amplifying information for the associated EOP, they should match. Although I routinely use the EOP Technical Guidelines for exam questions, I didnt see the need to reference or check it for this question as the information was identified in the EOP. Had I recognized the difference while I was

developing the question, I would have chosen a different safety function to test and generated a Condition Report to address the procedure misalignment.

Facility Recommendation: Per NUREG 1021, ES-403, Section D.1.b, the question contained newly discovered technical information that supports a change in the answer key.

Based on having two approved EOP documents with conflicting information about the maximum allowable temperature following a Loss of Offsite Power event, we recommend accepting both A and C as correct answers due to approved technical documents listing both 117°F and 125°F as the containment temperature safety function limit for a Loss of Offsite Power.

Post Examination Assessment Page 1 of 5 Written Examination Question Assessment Questions 50% of the candidates missed PV-E1298 Ver. 1 15DP-0OT01 Palo Verde Nuclear Generating Station Question #

Success Rate Description 16 33%

Question asked about actions directed from the Alarm Response Procedure following a trip of a Charging Pump.

67% of applicants chose distractor A.

Student feedback indicated that the normal response would be to get a Charging Pump started soon enough to prevent isolation of letdown. Aligning the Charging Pump Selector Switch could be done after the Charging Pump was started so there was no need to start the Charging Pump using the selector switch up front. In other circumstances in which letdown may be lost without prompt action (i.e. a temperature transmitter failure which changes the PLCS setpoint), starting the standby Charging Pump directly is an acceptable action, however per the ARP, the selector switch is used following a trip of a Charging Pump. Most were unaware of the ARP guidance to start the standby Charging Pump using the selector switch. Generated CR to LOIT for analysis of training on immediate actions for various conditions to prevent a loss of letdown. Question was determined to be valid.

Post Examination Assessment (Continued)

Page 2 of 5 PV-E1298 Ver. 1 15DP-0OT01 Palo Verde Nuclear Generating Station Question #

Success Rate Description 23 26%

Question asked about the Reactor trip setpoint for low SG levels and the basis for the setpoint. All applicants knew the setpoint but everyone who missed it did so because they did know the reason for the setpoint.

74% of applicants chose distractor C.

Student feedback revealed that although most, if not all, of the students knew that the top of the SG U-tubes is ~25-45%

NR level, and the reactor trip setpoint for SG level is 44%

WR (which is < 0% NR), they had never heard the information in the second part of the question so they picked the information they had heard before. Basically, the class indicated they would pick an answer they knew was incorrect before they would pick an answer they had never heard before. Question was determined to be valid.

Generated CR to LOIT for analysis of training on the basis for ESF / RPS setpoints.

28 44%

Question asked about the pressure at which the Nitrogen backup valve opens/closes and the impact to letdown on a degradation of Instrument Air pressure.

33% of applicants chose distractor A, 4% chose distractor C and 19% chose distractor D.

Student feedback revealed that some applicants did not know the pressure at which the Nitrogen Backup Valve re-closed and some did not know the IA pressure at which TV-223 fails closed. Both items are well trained, but not routinely reinforced. Question was determined to be valid.

Post Examination Assessment (Continued)

Page 3 of 5 PV-E1298 Ver. 1 15DP-0OT01 Palo Verde Nuclear Generating Station Question #

Success Rate Description 65 30%

Question asked about the method of actuation for STAT-X fire suppression and where a fire in the PCR building will alarm.

7% of applicants chose distractor A and 63% chose distractor D.

Student feedback revealed that they were primarily unaware of how the STAT-X fire protection system is actuated (manually or automatically). Information is clear in the lesson plan and sufficiently trained. Question was determined to be valid.

77 33%

Question asked about whether or not the RCS inventory control and core heat removal safety functions were satisfied during a loss of coolant accident.

33% of applicants chose distractor A and 33% chose distractor B.

Student feedback revealed that because RCS temperatures were lowering and SI flow was adequate, that cooling was in progress and effective, therefore though the safety function(s) was(were) met. The class also asked if the conditions in the stem were plausible. Stem conditions will be evaluated further to ensure the conditions are physically possible. Question was determined to be valid with the possibility for further enhancement.

84 33%

Question asked about which CIVs are and are not covered by LCO 3.6.3, Containment Isolation Valves.

11% of applicants chose distractor B and 56% chose distractor D.

Student feedback revealed a gap in student knowledge regarding what does and does not cause a CIV to be covered by LCO 3.6.3. Generated CR for training to analyze the level of detail in TS training lesson plans for possible enhancement. Question was determined to be valid.

Post Examination Assessment (Continued)

Page 4 of 5 PV-E1298 Ver. 1 15DP-0OT01 Palo Verde Nuclear Generating Station Question #

Success Rate Description 87 22%

Question asked about the parameters required to be bypassed following a failure of a narrow range pressurizer pressure instrument and the accident that parameter is designed to mitigate.

38% of applicants chose distractor B and 13% chose distractor C.

Student feedback revealed that they were unsure about which parameters to bypass because NR pressurizer pressure is an input to DNBR but not to LPD. Those that knew it is not an input to LPD picked the wrong answer for the second part without consideration of the impacts to DNBR. Training on this subject is sufficient, however the style of the question was different than that which the applicants saw during the program. Training will analyze the variety of styles used on programmatic exams as compared to Audit/NRC exams for possible bank question modifications. Question was determined to be valid.

92 44%

Question asked about the minimum number of CETs required to satisfy the PAMI surveillance for each channel of QSPDS.

22% of applicants chose distractor A, 11% chose distractor B, and 44% chose distractor C.

Student feedback revealed that they just didnt know the information. Initial license training will analyze for enhanced focus in the training program. Question was determined to be valid.

Post Examination Assessment (Continued)

Page 5 of 5 PV-E1298 Ver. 1 15DP-0OT01 Palo Verde Nuclear Generating Station Question #

Success Rate Description 97 22%

Question asked about how long a non-standard containment purge is good for and the radiation monitors 33% of applicants chose distractor A, 22% chose distractor C, and 22% chose distractor D.

Student feedback revealed that students did not know the standard duration of a non-standard containment purge release permit is 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. They were also unaware of the radiation monitors required to have a current surveillance prior to venting containment. While performance of a containment vent is on the SRO qualification card, training will analyze whether or not classroom training on this subject requires enhancement. Question was determined to be valid.

All questions missed by any candidate have been reviewed and there are no other issues identified with any other questions.

Administrative Task Assessment No issues identified.

JPM Assessment No issues identified.

Scenario Assessment No generic issues identified. One critical task failed by one crew (hydrogen analyzers failed to be placed in service within 30 minutes from the start of the LOCA)

Written Examination Scores RO Exam Average (for RO applicants): 86.8%

RO Exam Average (for SRO applicants): 90.2%

RO Exam Average (for all applicants): 87.9%

SRO Exam Average: 78.7%

Overall Exam Average: 87.0%