ML19242B821
| ML19242B821 | |
| Person / Time | |
|---|---|
| Site: | Nine Mile Point  |
| Issue date: | 11/30/2018 |
| From: | Peter Presby Operations Branch I |
| To: | Exelon Generation Co |
| Shared Package | |
| ML18338A499 | List: |
| References | |
| CAC00500, EPID L-2018-OLL-0007 | |
| Download: ML19242B821 (23) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 1
H 3
N s
Choice D implausible. TURBINE BUILDING closed loop cooling system is not likely to cool components in the PRIMARY 2
F 3
M E
CONTAINMENT, especially given there is a REACTOR BUILDING closed loop cooling system.
Resolution: Replaced D with Feedwater pump 11 coolers.
3 F
3 B
s Is this Q a KIA match, given that K2.01 tests power supplies to instrument air where this KIA, K2.02, tests power supplies to EMERGENCY air compressors?
This Q seems to help in answering 02 because it states 13 IAC is running when power lost and then asking which SAFETY RELATED compressor(s) are available, with only options being 11 4
F 3
N E
or 12. Does this imply 13 IAC is not safety related and therefore likely cooled by TBCLC vice RBCLC?
Recommend add to justification the impact of PB16A loss (loss of 13 IAC?).
Resolution: Added note to explain KA match. Removed "safety related" from stem. Justification added for PB16A. "If any" deleted from stem.
5 H
3 B
s Is loss of "injection" correct terminology? Resolution: No change.
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOO) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
"Cues": The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
"T/F": The answer choices are a collection of unrelated true/false statements.
"Cred. Dist.": The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
"Partial": One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
"Job Link": The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).
"Minutia": The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
"Backward": The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those KIAs that are designated "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6.
Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" status ratings (e.g., how the Appendix B psychometric attributes are not being met).
1
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Choice B can be argued as the correct answer because an isolation signal is generated to isolate the leak at 53 inches. The fact that it is not successful in isolating the leak because it doesn't 6
F 3
N E
isolate the correct lines does not make this a wrong answer.
Could fix problem by changing stem to "... will first be generated that will isolate the leak."
Resolution: Changed as recommended.
7 H
3 N
s 8
F 3
M s
Q is cueing, refreshing applicant knowledge of circuitry by reminding them of the existence of 1) a low pressure closure and LP closure bypass, 2) MSIV closure scram and bypass.
Recommend change current pressure to 550 psig to better support plausibility of Choice D. Change stem to identify particular IRMs rather than their channel affiliation, thereby letting the applicants determine that one is off Channel 11 and the other is off Channel 12. Change Q and choices to eliminate the cueing as follows:
All MSIVs __ (1) _ and the Reactor _(2)_ scram.
9 H
3 B
E A.
(1) close (2) will B.
(1) close (2) will NOT C.
(1) remain open (2) will D.
(1) remain open (2) will NOT Resolution: Chanced as recommended.
Answer too obvious. If testing the times 3 or square root of 10 knowledge, then lower the value of the indication -
perhaps 42% of scale rather than 75% of scale.
10 H
2.5 B
E Explain how the KA is matched. The Q does not appear to test detector operation.
Resolution: Changed to 50% on current scale and added KA justification 2
ES-401 3
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward 0-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Choice D phrasing "in preparation for" affects choice 11 H
3 B
E plausibility. Recommend change to, "start the Mechanical Vacuum Pump to maintain vacuum in the Main Condenser."
Resolution: Made proposed chanae.
Q not clear as to what is being compared regarding the offsite release. Could interpret as offsite release as compared to before the steam leak. Perhaps change Q to,
"... if any, of the loss of offsite power and EOG response on Reactor Building... "
12 H
3 N
E One could argue pressure will not show positive for a small steam break as leakage and condensation will maintain at about O psid. Also, distractors are weakened with, "by normal RB ventilation." Better to make first column just choice between either "maintained negative" or "NOT maintained negative".
Resolution: Incorporated both recommendations.
13 F
3 B
s 14 H
3 B
s B not plausible, that Reactor would be directly connected to the secondary containment via an ERV tailpipe vacuum 15 F
3 X
M u
breaker. Recommend change to:
If a LOCA occurs, Torus and Drywell pressure will equalize, degrading containment pressure suppression capability.
Resolution: Changed as proposed.
Is this per procedure? Why wouldn't the crew have already 16 H
3 B
E scrammed and closed MSIVs.
Resolution: Reworded 2nd half of C and D to match wordina of B and not specificallv mention MSIVs or TBVs.
17 H
2 B
s 18 F
3 M
s 3
ES-401 4
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Testing GFE knowledge, not site specific, since stem explains system alignment. All that is need is the understanding that lowering the governor (speed) setting will affect frequency when EOG is not connected to infinite bus 19 F
3 M
E and is only generator supplying the loads. Need to make this question test plant specific (NMP) knowledge to match the KA.
Resolution: Changed to surveillance on the EOG, same Q, different answer. Key answer changes from C to 0.
Too low LOO. Distractors A and Bare implausible because it is obvious the associated pumps are not running. And 20 HF 1.5 X
N u
Choice O easily eliminated because controller is in BAL, not MAN.
Resolution: Changed choices to one of four knobs on FWP 13 or master controller.
4
ES-401 5
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Is IAD 12 credible as a distractor? Need to explain why.
Why is it a wrong answer? If dryer available this would be logical. If dryers are available only to associated compressor and Compr 12 is tagged out, then not a credible distractor. Justification states Attachment 4 requires once per day blowdown. However, Attachment 4 just lists valves.
The guidance to blowdown per Attachment 4 comes from Section H, "OFF-NORMAL PROCEDURES", Section 3.0, "To Bypass Instrument Air Dryer 94-168 and 94-169."
Part 2 of Choices A and C have low plausibility because IAD 12 should already be in service so not reasonable to expect applicant to pick the "compensatory" action that describes an already expected alignment of the system. Recommend change initial conditions to have IAD 12 available but not in service. Perhaps state IAD 12 maintenance was completed earlier in the week and that it has been energized for greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> with the power source on and that it is 21 F
3 N
E available for use. Recommend adding procedure to Q stem, as in,"... required compensatory action per N1-0P-20, Service, Instrument and Breathing Air Systems while they are out of service?" to head off any potential argument that placing IAD 12 in service would be an appropriate compensatory action if it were not already in service. The IAC 13 startup section directs placing IAD-12 in service but allows operator to N/A the step. Also, Section 2.3 states, "IAD 12 services IAC 13". Nothing in stem supporting the breathing air distractor. Stem does not indicate breathing air in use. Also, why would there be a breathing air concern if dryers are bypassed? Air doesn't have to be dried to be breathed. Plausibility concerns.
Resolution: 1) Clarified procedure reference in justification statement, 2) Changed compensatory action from "place IAD 12 in service" to "secure use of breathing air header, 3) added procedure reference to stem, 4) Updated justification to better explain the lineup and situation.
Choice C appears to be a subset of Choice A. Recommend 22 H
3 N
E remove the word "inoperable" from Choices A and B. If you bypass 4 detectors you get an auto "inop trip".
Resolution: Reworded choices to address comment.
5
ES-401 6
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
0 LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only 0 and A do not align well. 0 asks "reason" where answer has "height corresponding to". Change Part 2 to the EOP 23 F
3 X
N u
step basis or change the O to ask what the height corresponds to. Check for blowdown setpoint overlap with simulator scenarios.
Resolution: Replaced O with another testinQ same KA.
24 F
3 B
s 0 helps to answer 012 by identifying that RBEVS auto starts, eliminating 012 choices A and B.
What does N1-0P-10 require, Pull-To-Lock or Normal-After-Stop? If not PTL, then conflict in O which states,
"... securing the fan, in accordance with N1-0P-10."
The O no longer gives a reason for the ventilation realignment. Should some general reason be given such as RB Ventilation spuriously isolates and RBEVS automatically starts (assuming this is appropriate automatic response to RB Vent isolation)? Do you need to say anything about RB Ventilation? Perhaps just say RBEVS receives spurious 25 H
3 B
E start signal and do not discuss RB Ventilation system response.
Review of N1-0P-10 shows the only valid RBEVS starts are hi rad on RB Vent Rad or Fuel Pool Hi Range Rad and an auto start of RBEVS automatically stops and isolates RB Ventilation. Therefore, there is no need to say anything about status of RB Ventilation. Recommend 2nd bullet be changed to just say "Reactor Building Emergency Ventilation (RBEVS) automatically starts due to a valid signal."
Resolution: Removed stem bullets 2 and 4. Changed 2nd stem bullet to "RB Vent automatically isolates and RBEVS automatically starts due to a valid siqnal."
26 F
3 B
s 6
ES-401 7
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H}
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N}
(U/E/S}
Focus F
Dist Link KIA Only Typo, inconsistent spacing, in the RRP flows.
Typo. Justification states 6% power, not 66%.
Seems cueing to give indicated flows and then provide only the THREE LOOP P/F map. This leads applicant to realize the other two flows are on non-operating loops, therefore likely reverse flow. Better to hand out P/F maps for multiple recirc pump configurations and let applicant figure it out.
Low LOD as structured. The Q boils down to understanding reverse flow and knowing that ELLA, green region is 27 H
3 M
E allowed, which is somewhat obvious since typical color code applied with green meaning ok, orange/yellow meaning warning, and red meaning do not operate. Recommend providing black and white P/F maps vice color, and redact names of the regions. Applicant should be well enough acquainted with the P/F map to be able to point out the ELLA region as opposed to the RESTRICTED region for example without having the labels.
Please take out the "(3)" in the note. Change note to state, "Power-to-flow maps are provided on the following pages."
Resolution: Provided all three power to flow maps.
28 H
3 M
s 29 H
3 B
s Recommend reducing distractor time from 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> to 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> 30 F
3 M
E because 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> very unlikely procedural requirement.
Battery could be severely depleted in 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> time.
Resolution: ChanQed C and D to 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />.
31 F
3 B
s 32 H
3 B
s 33 F
3 B
s Inconsistent formatting. Q6 uses bold, italics for "first". 034 does not.
34 F
3 B
E Would Choice D be a better distractor at a lower initial power level?
Resolution: Power level makes no different. Leaving Choice Das is. Bolded and italicized "first" in stem.
7
ES-401 8
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Recommend state a leak occurs, rather than a loss occurs since a complete loss did not occur.
What is meant by "(HSA-BV)" in the stem? Is this required?
Seems to cue applicant by stressing this is a Service Air system blocking valve. N1-SOP-20.1 refers to the valve as "BV-94-19, HSA RECEIVER TO IA SEPARATOR, 35 F
3 N
E ABSORBER A.F." Perhaps better to use the name from the SOP.
Remove unnecessary stem valve location information, "(TB 291')".
Consider (your choice) whether "A.F." should be changed "AFTERFIL TER".
Resolution: Changing as recommended.
This Q appears to be testing too simple a level of knowledge, that the SOC System Inlet Isolation Valve inside the OW is normally closed and must be opened to support SOC operation, especially with the added information that the OW is inerted and CANNOT be entered. With only 3 bullets of information provided, the information about not 36 H
3 N
E being able to enter OW seems to telegraph potential need to operate this valve manually. Can setup be changed to a cooldown because of a small line break in the OW? That would convey same information regarding accessibility of the OW wilthout emphasizing a need to enter the OW.
Resolution: Adding to explanation to better justify KA match. Will remove third bullet and add LOCA with OW pressure to stem.
37 H
3 N
E Disconnect between Q and justification (UE vs ALERT)
Resolution: Fixed.
38 H
3 M
s 39 F
3 M
s 40 F
3 N
s 41 H
3 B
s 42 H
3 B
s 43 H
3 B
s 8
ES-401 9
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
0 LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only 44 H
3 N
s 45 H
3 B
s 46 H
3 B
s 47 H
3 N
s 48 H
3 M
s 49 F
3 N
s 50 H
3 B
s 0 wording, "describes the need", implies/shows bias toward bypass. Reword to be more neutral. Perhaps change to, 51 H
3 B
E "whether or not the RWM must be bypassed and why or why not".
Resolution: Implemented recommended chanae.
52 F
3 M
s 53 H
3 N
s Grammar of first bullet? "A control rod movements".
54 H
3 N
E Should this 054 have different power level from 051 to ensure no overlap?
Resolution: Fixed and lowered power to 2% on 054.
9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Choices A and C are implausible. An applicant would not think that the F panel speed controller could control speed if the MG set scoop tube is locked. That knowledge is too basic to be discriminating. Recommend changing Q as shown below (key answer highlighted) to test applicant's understanding of specific causes for scoop tube lockup.
The plant is operating at 85% power, with the following conditions:
Annunciator F2-2-2, REACT RECIRC M-G SET 12, alanms.
The following indication is present for Recirculation Pump 12 on F panel.
[Picture of MG Set 12 Scoop Tube Lockup light and reset pushbutton here]
Which one of the following describes a possible cause of this panel light indication?
55 H
3 X
B u
A.
Instrument air header pressure has lowered to 65 psig.
B.
MG Set 12 Signal Failure Relay RY10 is de-energized.
C.
DC Battery Board 12 Input Breaker has tripped open.
D.
MG Set 12 Air Failure Brake Bypass Valve is closed.
Resolution: Changing to a 2x2 with part 2 as local control available or not available. Changed question to match.
Resolution: 1) changed heading of the Part 2 column and 2) modified the Q statement to ask about ability to take local manual control of the associated Recirc MG set Bailey scoop tube positioner, rather than ability to take local manual control of the associated Recirc MG set.
56 H
3 B
s KA mismatch. Question tests knowledge of instrument locations and does not test ability to identify which are post-57 F
3+
B E
accident instruments.
Resolution: Modified Q to emphasize KA match and explained match in justification.
10
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only 58 F
3 B
s Recommend modify last stem bullet, deleting reference to Unit Supervisor direction, as the wording implies control is outside of control room. Recommend wording as, "Manual control of the Generator H2 Seal Oil pressure regulator bypass is required to raise seal oil pressure."
Question asks for status of pump. Some choices state that 59 H
3 N
E it "automatically starts". Would align better with the question if these choices changed to "has automatically started".
Doesn't the word, "manual" in "manual control of the Generator... " imply outside the control room, especially in the column heading, "Location for Manual Generator...
Regulator Bypass"? Recommend replacing "manual" in three places with "operator.
Resolution: Made recommended changes.
60 H
3 N
s 061 tests understanding of condition where auto closure will have occurred. However, it asks the conditions that will cause closure, setting up situation where applicant could argue no correct answer as none of the choices are at the 61 F
3 M
E closure setpoint threshold value. Recommend minor rewording to address, as "Under which of the following conditions would the Drywell Equipment Drain Isolation Valves have received an automatic closure signal?".
Resolution: Reworded to address comment.
62 F
3 N
s Recommend restating last bullet as follows to get away from implying a control location inside or outside the control 63 H
3 N
E room.
"An operator has taken manual control of... ".
Resolution: ChanQed as recommended.
11
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Is Choice C plausible? Looks like the intent is to distract on whether or not auto isolation occurred. However, the distractor says, "remains unisolated and has lost automatic isolation capability". As written the distractor states system would be left unisolated without isolation capability. Seems like operators would take action upon loss of auto isolation 64 H
3 M
E capability to secure the system. Perhaps better to change C and D to be the opposite of Choices A and B to avoid any inference as to actions taken by the operator. Example:
C.
did NOT automatically isolate and...
D.
did NOT automatically isolate and...
Resolution: Modified as recommended.
KA mismatch. There is no loss or malfunction of reactor 65 F
3 N
s vessel internals.
Resolution: Added to iustification to explain the KA match.
66 F
3 N
s Different format in A/B vs C/D. A/B looks like noun name for an RTIME point where CID look like a paraphrase. Is one point described by computer in all caps where the other is written out in mixed caps?
Recommend us same wording of the allowable range as used in the procedure to preclude post-exam argument of no correct answer per N1-0P-43B:
67 F
3 B
E A.
E635 - CTP-2 HR AVG between 1846 to 1850 MW!.
B.
E635 - CTP-2 HR AVG between 1840 to 1855 MW!.
C.
H305 - 10 Minute average of CTPINST between 1846 to 1850 MW!.
D.
H305 - 10 Minute average of CTPINST between 1840 to 1855 MW!.
Resolution: Capitalized and incorporated 2nd comment.
12
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Recommend modifying choice wording to eliminate the "become acceptable" wording.
Choice D implausible given chart of feedwater temperatures. Recommend:
A.
may be maintained at 85%, at the current feedwater temperature.
68 H
3 N
E B.
must be lowered to 75%, although feedwater temperature is in band.
C.
must be lowered to either 25% or the desired feedwater temperature.
D.
must be lowered to either 75% or the desired feedwater temperature.
Resolution: Replaced "become acceptable" with "return to band" in Choice C and provided entire table.
69 F
3 B
s Given that TS LCO 3.3.2.f provides actions if bulk pool temperature reaches 120°F, recommend raising Distractor 70 F
3 B
E Choice C temperature from 115°F to 150°F, to exceed the expected Heat Capacity Temperature Limit following intentional RPV rapid depressurization.
Resolution: Chanaed Choice C to 125°F.
This Q71 helps to answer Q62 by identifying the power source for the ARMs. Having the specific power supply listed does not seem necessary. Recommend restating stem and choices to eliminate mention of l&C Bus 130.
Instead:
The plant is operating at 100% power when the power source to the Area Radiation Monitors (ARMs) deenergizes 71 F
3 B
E due to a spurious breaker trip.
Which of the following...
A.
go into alarm. When the power source is re-energized, the ARM...
[and so on]
Resolution: Incorporated recommendation.
13
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stern Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Not an appropriate question for a Tier 3 KA. The Q tests specific system knowledge - knowledge of Control Room 72 F
2 B
u Ventilation system procedural requirement to manually initiate CREVS. Needs to test generic, not system specific knowledge.
Resolution: Replaced the Q.
Choice D reenter on Torus temperature is implausible unless Torus temperature is still exceeding EOP entry 73 F
3 B
E condition. Recommend adding stem information about 01 :00 status of Torus temperature, that it is still exceeding its associated EOP-4 entry condition.
Resolution: As recommended.
In the stem, "A plant is operating", should be "The plant is 74 F
3 B
E operating".
Resolution: Fixed.
As written, the Q requires the applicant to read the "higher priority" column in the answer choices in order to determine what is being compared in the stem. Recommend adding the priority comparison items to the stem for clarity and ease of interpretation. Example:
75 H
3 M
s Given that both PCPL and core cooling are currently being challenged, which one of the following identifies the preferred pump... and which issue is higher priority, in accordance with...
Resolution: Left as-is. Recommendation had drawback of adding more words to question.
14
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Q meets KA but it is not discriminating. The applicant need only realize that all APRMs alarming downscale means indicated reactor power is less than 6%. It is like a direct lookup Q.
- Applicant has insufficient information to determine whether Choice C or D could be correct, making these distractors implausible. The applicant is not provided with references to evaluate UE or GE thresholds.
Q76 is coded to 55.43(b)(1) Conditions and limitations in the facility license. However, the Q is related to application of 76 H
3 N
E the emergency plan and should be coded under 55.43(b)(4)
Radiation hazards that may arise during normal and abnormal situations, including maintenance activities and various contamination conditions, as indicated in ES-401.
Resolution: Reworded Q to "... describes the need for EAL" and drop the "if any". Also, in stem "given the following potentially applicable portions of the EAL matrix. Changed C and D to "neither of these EAL declarations are required."
Left 55.43(b)(1) based on prior conversation with BC about EAL tie-in to 55.43.
77 H
3 N
s 78 H
3 B
s Q78 should indicate tie to 55.43(b)(6), vice 55.43(6).
Resolution: Fixed.
79 H
3 B
s 80 H
3 B
s 15
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward 0-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Similar comment to Q77. However, given the KA and the numerous exceeded thresholds, this Q seems ok for SRO written exam.
Recommend change Q from picking the highest to applying SRO E-plan implementation skill in determining the correct call. Change Q statement from "Which of the following identifies the HIGHEST... " to Which of the following identifies the classification that should be communicated to 81 H
3 B
E state and local entities via notification in accordance with EP-CE-111... "
Q81 is coded to 55.43(b)(1) Conditions and limitations in the facility license. However, the Q is related to application of the emergency plan and should be coded under 55.43(b)(4)
Radiation hazards that may arise during normal and abnormal situations, including maintenance activities and various contamination conditions, as indicated in ES-401.
Resolution: Made change for 2nd comment.
The stem information at T=00:20 about 920 psi and stable is leading. The key answer justification states that SOP override actions to maintain stable hot shutdown conditions when fire not under control after 15 minutes. By stating those conditions met at >15 minutes into the fire, the key answer is implied. It doesn't appear this stem information is necessary to answer the Q, therefore recommend deleting the 2nd stem bullet at T=00:20.
Provide explanation in justification section as to how the facility thinks this Q tests SRO-only knowledge, given that a 82 H
1 M
E "confirmed" fire is the "event description" or entry condition for the SOP and that the required reactor control is pretty much the general procedure strategy for major fires, which are those that 1) result in spurious valve operation, 2) loss of equipment control, 3) are not under control in 15 minutes,
- 4) endanger safe shutdown capability, or 5) are so deemed by the Shift Manager.
Q82 should indicate tie to 55.43(b)(5), vice 55.43(5).
Resolution: Added SRO justification for 2nd bullet. Deleting bullet in stem at T=20 about reactor pressure steady and at 920.
Identify the specific page numbers of the TS to be provided.
83 H
3 N
s Q83 should indicate tie to 55.43(b)(2), vice 55.43(2).
Resolution: Fixed.
16
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Inconsistent use of capitalization in choices. Choice B uses 84 H
3 B
E "NOT" where Choice C uses "not".
Q84 should indicate tie to 55.43(b)(5), vice 55.43(5).
Resolution: Fixed. Caoitalized both.
This does not appear to be testing SRO-only knowledge.
SRO is responsible for detailed understanding of EOPs.
However, as described in NUREG-1021, ES-401,, Figure 2-2, if Q can be answered solely by knowing purpose, overall sequence of events or overall mitigation strategy of a procedure, then it is testing RO, not SRO-only knowledge. With water level below top of active fuel and slowly lowering, an RO should know that EOP strategy would not direct the operator to secure the only source of injection. Perhaps can fix by making the answer not so intuitively obvious, by stating in stem that level is +8" and slowly rising. Then within realm of possibility that EOP 85 H
3 M
E might direct isolating the leaking pump.
Is Q wording, "THIS high level", too non-specific? Could it confuse applicant since multiple levels are provided in stem (RPV level, Torus level, Sump level, Corner Room level)?
Recommend change Q wording to, '1he pump that may be the source of leakage and the required control of this pump if...
Q85 should indicate tie to 55.43(b)(5), vice 55.43(5).
Resolution: Changed stem to minus 85 and stable.
Changed room level to 5 feet, which is very close to motor.
Restructured Part 2 to is or is not required to be secured and isolated. Changed Q wording as recommended.
17
ES-401 18 Form ES-401-9 Q does not appear to be SRO-only. It seems answerable with RO knowledge. Coded to 55.43(b)(5) Assessment of facility conditions and selection of appropriate procedures during normal, abnormal, and emergency situations.
However, there is no selection of procedures. Answer can be determined by knowledge of SOC precautions and limitations.
Regarding Choice B, easy for applicant to misunderstand the "CANNOT" emphasis by assuming if other conditions remain as given, which is what the stem states in "acceptability of the conditions". Per P&L 24.0, it appears the crew could cool down below 150°F and then start 3rd pump if major maintenance was to be performed.
Recommend indicate purpose of shutdown in stem, so that answer is wrong for the given conditions.
Choice B seems to stand out from the other choices.
Suggestion for improving balance below.
The choices lead-in ("A 3rd SOC loop... " is leading.
Consider changing Q and adding to justification to address comments as follows.
The plant is in a forced outage to replace an ERV and is shutting down to cold shutdown conditions with the following:
86 H
3 N
E
- Reactor coolant temperature is 338°F and lowering slowly.
Which one of the following describes the acceptability of these conditions for placing SOC loop 13 in service, in accordance with N1-0P-4, Shutdown Cooling?
SOC loop 13 may be placed in service at current...
A.
reactor level when RCS temperature is lowered to less than 212°F.
B.
reactor level when RCS temperature is lowered to less than 150°F.
C.
reactor coolant temperature if reactor water level is raised to a minimum of 10.5 feet on Wide Range level instruments.
- 0.
reactor coolant temperature if reactor water level is raised to a minimum of 0 feet on Flange level instruments.
Add to justification to explain how Choice B is wrong because provision for 3 SOC loop operation is only provided below 150'F if "maior maintenance", a condition specifically 18
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only defined in tech specs, is in progress. Stem conditions indicate fuel will remain in vessel.
Is Q Part 2 technically correct? Is it a choice between all of the suction or all of the discharge valves? Or is it at least either the suction or the discharges on each pump? In other words, would N1-0P-4, D.8.0 be met with, for example, RRP 11 suction closed and RRP 12 thru 15 discharges closed? If so, then Q choices are in conflict with procedure as choice wording states ALL suctions OR ALL discharges must be closed.
Resolution: Changed wording of Q from, "place a 3rd SDC loop in service, which... " to "place SDC loop 13 in service, which... ". This would be consistent with stem bullet above the Q and improves the Q in that the applicant will need to recognize that placing SDC loop 13 in service is placing a 3rd SDC loop in service.
Q87 should indicate tie to 55.43(b)(2), vice 55.43(2).
Specify which TS pages are to be provided to applicants 87 H
3 N
s during examination. They should not have access to TS basis information. Recommend pages 54, 55 and 56.
Resolution: Fixed.
Very similar to Q9. Both Qs test, in part, knowledge of MSIV closure scram setpoint and logic.
Q is leading in that Part 2 of the Q articulates the actual condition of the alarm.
Part 2 is plainly stated in TS LCO 3.6.1.a.(1 ), which is provided to the applicants, that minimum number of trip systems must be operable as given in Tables and for 88 H
3 X
X M
u instrumentation that initiates scram, such as the "MSIV Closure Scram", control rods shall be inserted. With reference provided, identifying whether or not TS requires rod insertion is not discriminating.
Q88 is not SRO-only. This Q tests RO system knowledge (MSIV closure bypass setpoint, impact of failure of that interlock on the reactor trip system) and RO knowledge of a 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or less TS action.
Resolution: Rewrote the Q.
19
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Evaluate and address potential overlap with the operating test. There is a JPM related to a "big 8" valve that fails to auto close during a drywell vent.
Part 2 appears to be a low level lookup with the ODCM provided. And Part 1 is basic system knowledge.
N1-0P-50B, Section H, Loss of Stack Gas Effluent Monitoring states, "IF either OGESMS high range detectors (RN10A or RN10B) are inoperable or out of service, THEN perform the following: 1.1 IF Secondary Containment is required, THEN shut and mark up with a Clearance Caution Tag the following Vent and Purge Valves:[ the 'big 81".
89 H
3 N
E Given this procedural requirement, is it appropriate to test this ODCM note allowance to continue the vent and purge, when the note is in conflict with the procedural requirement?
Q89 does not seem to meet the intent of 55.43(b)(4). See NUREG-1021, Rev 11, ES-401, Attachment 2, pg 22 of 52.
Also, the KA (A2.04) indicates should be testing 55.43(b)(5),
selection of procedures.
Resolution: Adequate difference between Q and aptest. No action needed on 2nd bullet, enough complexity to use of ODCM note. No action on 3rd bullet, Q very focused on what does ODCM require. No action on 4th bullet, fits (b)(4).
Q90, as structured, does not seem to meet intent of 55.43(b )(5) for assessment and selection of procedures, because procedure is provided up to the required step. Q essentially asks what is the next step in the alternate level control leg. Recommend deleting imbedded procedure 90 H
3 N
E section and last stem bullet. Then, can make the argument that SRO must assess conditions and apply his detailed knowledge of EOP-2 to identify the appropriate next course of action, without asking him specifically what is required by next step L-13 of the alternate level leg.
Resolution: Changed last stem bullet to "in alternate level control leg" and deleted imbedded procedure steps.
20
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Evaluate whether this overlaps with an operating test task to respond to a rod double notch.
Is plant is at 90% power to support the rod sequence exchange so that any perturbations can be absorbed without challenging core limits or exceeding rated thermal power limit? Procedures have notes indicating double notching is a possibility. Wouldn't it be within expected RO knowledge to be able to identify that power reduction would not be required and that rod could be placed in desired position without involving reactor engineering? Perhaps could lend plausibility to the power reduction by leaving out the initial power level. The SRO should know the initial power condition required to initiate sequence exchange.
91 F
3 N
E Change stem from "the plant is operating at 90% power" to "The plant has been operating at full power for the past 3 months with the following:... " Change the Band D distractor wording to "Lower Reactor power using Recirculation flow without stating the target power level.
Does the stem need to say the mis-position is due to a timer malfunction? How would the operator know this? By providing this infonnation the Q seems to be cueing the SRO toward his 55.43(b)(5) procedure selection.
Resolution: Changed stem to triple notch and affected Choices, puts them in different response procedure for mispositioned rod, but same actions. For 2nd bullet removed "to approximately 85%" from Band D. For last bullet removed timer malfunction.
92 H
3 B
s 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only Stating that the suction valve breaker is locked out without stating the same for the discharge valve and the discharge bypass valve reduces plausibility of Choice D. Per the explanation provided, expecting the applicant to recognize the valve is only partially closed because of its long stroke time. Should provide support for Choice D plausibility by listing breaker conditions met from TS LCO 3.1. 7.e.1 and 2 in the stem.
Question related to RRP 11 but justification discusses RRP
- 14.
93 H
3 N
E Last sentence of justification for Choice D isn't fully explained.
Recommending improving psychometric balance by changing distractors as follows (proposed changes highlighted):
A.
must remain at or below 85%.
B.
may be raised to a maximum of 90%.
C.
may be raised to a maximum of 90.5%.
D.
may be returned to 100%.
Resolution: No action on first comment. Other recommendations implemented.
Is 2nd stem bullet necessary? Can the plant be operating at 30% power with the Reactor Mode Switch in a position other than RUN?
Do not provide reference to applicants for Q94. Good note 94 H
3 N
E explaining why the Q is not RO level above the line knowledge. However, Q reduces to essentially RO level when TS reference is provided. It becomes very simple reading exercise to identify 3.1.8.c must be applied because 3.1.8.a and 3.1.8.b are not met.
Resolution: Implemented recommendations.
Too low LOO. Applicants are infonmed there is only one correct answer. All applicants will know the purpose of secondary containment is to contain and control radioactive release to the public. All applicants will be able to identify only one of the choices is an activity without risk of causing 95 F
3 B
E a radioactivity release in the secondary containment.
Is this Q topic SRO-only? N1-0P-34, D.33.0 is a general procedure precaution and limitation. Aren't ROs required to know the OP precautions and limitations?
Resolution: Restructured Q to delete obvious distractors B and C.
22
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws 5.0ther
- 6.
- 7.
Q LOK LOO T
Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues I
Cred.
Partial Job-Minutia
- /Units Backward 0-SRO (B/M/N)
(U/E/S)
Focus F
Dist Link KIA Only LOD=1. Not plausible that a license applicant would think 96 H
1 N
u okay to declare an overhauled HPCI pump operable BEFORE performing the PMT "Operability Test".
Resolution: Rewrote timeline to enhance plausibilitv.
97 F
3 N
s This KA related to personnel hazard. Need to ask 98 F
3 B
E something else, such as potential radiation levels if there is an accident.
Resolution: Will add to justification why it is a KA match.
Recommend removing/redacting the curve title from the imbedded curve. Also, change Part 2 of Choices A and B to "Next, an evaluation of pressure suppression capability must be performed." Providing the title to the imbedded curve makes the answer to the 2nd part self-evident. If there is a 99 F
3 B
E limit for pressure suppression pressure, as stated in the curve title, and the plant is at the limit based on given conditions, it is not likely or reasonable to initiate a pressure suppression event by blowing down without evaluating the capability for suppression of blowdown.
Resolution: Implemented Part 2 wording change. Left title on curve.
Change last bullet from "unavailable due to" to "isolated and 100 H
3 M
E tagged for". Applicant should be able to make the availability determination without being told directly.
Resolution: Recommendation implemented.
23