ML20274A092
| ML20274A092 | |
| Person / Time | |
|---|---|
| Site: | Calvert Cliffs  |
| Issue date: | 02/07/2020 |
| From: | Division of Reactor Safety I |
| To: | Exelon Generation Co |
| Shared Package | |
| ML19105A101 | List: |
| References | |
| CAC 000500, ES-401-9 | |
| Download: ML20274A092 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only 1 H 1 N U LOD1, replaced question 2 F 3 B E Removed resulting in a reactor trip from stem 3 H 3 B S 4 H 3 N S Changed stem to have applicant diagnose Loss of Offsite power 5 H 3 N S instead of being told that here is a loss of offsite power. P13001 is lost 6 H 3 N S 7 H 3 N S A2 and C2 implausible and cued answer in stem. .Rewrote stem 8 H 3 X B U to in the CCW system instead of into the CCW system.
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
- Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
- T/F: The answer choices are a collection of unrelated true/false statements.
- Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
- Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4. Check the appropriate box if a job content flaw is identified:
- Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only A2 and C2 are implausible - Heater bank 2 handswitch is in OFF as stem indicated, so they cant auto energize.
9 H 2 X N U Rewrote stem to describe a power supply failure to Prop Heater Bank #2 Removed will mitigate the ATWS from stem 10 F 3 B E Rewrote distractor C to change borate amount to more plausible number (2000 ppm) 11 F 2 B S 12 F 2 B E Changed stem to what sensor is used to differentiate Removed RO is verifying natural circulation 13 F 3 B E Question is Low Cog, not Hi Cog 14 H 3 B S 15 H 3 N S 16 H 2 N S 17 F 3 N S 18 H 3 B E Added to distractors B and D to have a heat removal strategy.
Will change 4th bullet to describe one CEA (CEA #1) 19 F 2 N E continues to withdraw 20 F 2 N S 21 H 3 N S 22 H 3 N S 23 H 2 N S 24 H 3 N S
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only 25 F 3 N S 26 H 3 N S 27 F 2 N S 28 H 3 B E Shortened to 2 cases, and removed Column B 29 H 2 N S 30 F 1 N S 31 H 3 B S 32 F 2 N S 33 F 2 B S 34 H 2 B S 35 F 2 N S 36 H 2 N S 37 F 2 B S Add has/ to have in part 2 of the stem.
38 H 2 N E 39 H 3 B S Removed second to last sentence in stem stating With the given conditions..
40 H 3 N E Separate distractors into two numbered parts.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only 41 H 3 N S 42 F 3 N S 43 H 3 X B E Added to stem per Caution in OI-8E 44 F 3 B S 45 H 3 N S 46 F 2 N S 47 F 3 B S 48 F 2 N S 49 H 3 B S 50 F 3 B S 51 H 2 B S 52 F 2 B S 53 H 3 B S 54 F 2 N S 55 H 3 B S 56 H 3 N E Wrote out PZR 57 H 2 B S
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only 58 H 2 N E Added what the failure was. Edited plausibility statement.
59 H 3 M S 60 H 3 B S 61 H 3 B S 62 H 2 M S 63 F 3 N S 64 H 3 N S 65 F 3 N S 66 F 2 N E Removed first sentence in stem 67 F 3 N S Added details to stem about RO and what his responsibilities 68 F 2 B E are to better match KA 69 F 2 F S 70 F 2 B S 71 H 3 B S 72 F 3 B S 73 F 2 B S 74 F 3 N S
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only 75 F 2 B S 76 H 3 N S The TS is provided as a reference, as well as which TS to use 77 H 1 X N E is listed in the stem. Question is borderline DLU and as a result, LOD1.
78 F 2 B E Change 150% to 125% to make more plausible in B2 and C2 79 H 3 B E A2 and B2, made chargers plural 80 H 3 B S 81 H 3 N S 82 H 3 B S A1 and D1 are implausible since stem says what is the expected effect. A1 and D1 describe no effect.
83 H 3 N E Added if any to effect in stem 84 F 2 N S 85 H 3 B S 86 F 3 N E Added LCO(s) so it doesnt imply plural only.
87 F 3 N S 88 H 3 N S 89 F 3 N S 90 H 2 N S
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q LOK LOD Source Status 8. Explanation (F/H) (1-5) Stem Cred. Job- Q- SRO (B/M/N) (U/E/S)
Cues T/F Partial Minutia #/Units Backward Focus Dist Link K/A Only Part 2 of stem (was) cues that there is only one TS required component inop, therefore C and D are implausible.
91 H 1 N E Added were to part 2 of stem.
92 H 2 B S 93 H 2 B S 94 F 2 N E Changed warehouse workers to equipment operator in stem Removed first sentence in stem 95 F 3 N E Removed capitalized audits in stem 96 H 3 N S 97 F 2 N S 98 F 3 N S 99 F 2 N S 100 F 3 N E Removed first sentence in stem