ML21239A082
| ML21239A082 | |
| Person / Time | |
|---|---|
| Site: | Peach Bottom  |
| Issue date: | 05/18/2021 |
| From: | Todd Fish Exelon Generation Co |
| To: | NRC Region 1 |
| Shared Package | |
| ML20259A337 | List: |
| References | |
| EPID L-2021-OLL-002 | |
| Download: ML21239A082 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
- 1. LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-KIA SRO Only (B/M/N)
(U/E/S)
Focus Dist Link 1
H 2.5 N
s 2
H 3.0 N
s 3
H 3.0 N
s 4
F 2.5 N
s 5
H 2.8 N
s 6
F 3.0 N
s 7
H 3.0 N
s 8
H 2.5 N
s Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty {LOO) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
"Cues": The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
"T/F": The answer choices are a collection of unrelated true/false statements.
"Cred. Dist.": The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
"Partial": One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
"Job Link": The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).
"Minutia": The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
"Backward": The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those KIAs that are designated "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6.
Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status 8, Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Link Minutia
- /Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S) ~
Focus Dist 9
H 3.0 N
s 10 F
2.5 N
s 11 F
2.5 N
s 12 F
3.0 N
s 13 F
2.5 B
s 14 H
3.0 B
s 15 H
3.0 N
s 16 H
3.0 B
s 17 H
2.5 B
s 18 F
3.0 N
s 19 H
2.8 N
E Update picture to clarify provided indications.
20 H
3.0 N
3.0 X
N u
too similar; two possible correct choices. Revised auestion.
22 F
1.0 X
N u
LOD=1. Replaced KIA and Q.
23 F
3.0 N
s 24 H
3.0 B
s 25 H
3.0 N
s
ES-401 3
Form ES-401-9
- 1.
- 2.
- 6. Source
- 7. Status Q
LOK LOO
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other (B/M/N)
(U/E/S)
- 8. Explanation
- -**n
., ~,
26 F
2.5 N
s 27 F
2.5 B
s 28 H
2.5 N
s 29 H
2.5 N
s 30 H
2.5 N
s 31 F
3.0 B
s 32 H
3.0 N
s 33 H
2.5
/
N s
34 F
3.0 N
s 35 H
3.0 N
s 36 F
3.0 N
s 37 F
2.5 N
s 38 H
3.0 N
s 39 H
3.0 N
s 40 F
2.5 N
s 41 F
2.5 N
s 42 F
3.0 N
s 43 F
3.0 N
s
ES-401 4
Form ES-401-9
- 1.
- 2.
- 6. Source
- 7. Status Q
LOK LOD
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other (B/M/N)
(U/E/S)
- 8. Explanation
,r.. "
'A ~,
- 44 H
2.5 N
s 45 H
3.0 B
s 46 H
3.0 B
s 47 H
3.0 N
u Overlap with 071.
Replaced KIA and Q.
48 F
2.5 N
s 49 F
2.5 N
s 50 H
2.5 N
s 51 F
2.5 N
s 52 H
3.0 B
s 53 F
2.5 N
s 54 H
3.0 N
s 55 F
2.5 B
s
- 56 F
3.0 N
s C and D not credible for 57 F
3.0 X
N u
severity of event.
Replaced KIA and Q.
58 H
2.5 N
s 59 F
3.0 N
s 60 H
3.0 B
s LOD=5. Answer requires 61 F
5 X
N u
detailed procedure memorization bevond
ES-401 5
Form ES-401-9
- 1.
- 2.
- 6. Source
- 7. Status Q
LOK LOD
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other (B/M/N)
(U/E/S)
- 8. Explanation
'A r, immediate actions.
Replaced KIA and Q 62 H
3.0 N
s 63 F
2.5 N
s 64 F
2.5 N
s 65 H
2.5 N
s 66 H
3.0 N
s 67 H
3.0 N
s 68 H
3.0 N
s 69 H
2.5 V
N s
70 F
2.5 B
s 71 H
3.0 N
s 72 H
3.0 N
s 73 H
3.0 B
s 74 H
3.0 N
s 75 H
2.5 N
s 76 H
1.0 X
X N
u LOD=1. DLU, given TS.
Replaced KIA and Q 77 H
3.0 N
s 78 F
2.5 N
s
ES-401 6
Form ES-401-9
- 1.
- 2.
- 6. Source
- 7. Status Q
LOK LOD
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other (8/M/N)
(U/E/S)
- 8. Explanation 79 H
3.0 N
s 80 H
3.0 8
s 81 H
3.0 N
s 82 H
3.0 N
s 83 H
2.5 N
s 84 H
3.0 N
s 85 H
3.0 N
s 86 H
3.0 N
s 87 H
1.0 X
8 u
Not SRO-only. Low LOD.
Rewrote Q Question provides info (resin injection or fuel 88 H
3.0 X
X 8
u failure), thus no SRO diagnosis/prediction needed. Replaced KIA and Q 89 H
3.0 8
s 90 H
3.0 N
s 91 H
3.0 N
s 92 H
3.0 N
s 93 H
3.0 N
s 94 H
3.0 8
s 95 H
3.0 8
s
ES-401 7
Form ES-401-9
- 1.
- 2.
- 6. Source
- 7. Status Q
LOK LOD
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other (B/M/N)
(U/E/S)
- 8. Explanation
. ~,
LOD=1. Q requires RO 96 H
1.0 X
N u
knowledge. Replaced KIA and Q.
97 F
3.0 B
s 98 H
2.5 N
s 99 H
3.0 N
s 100 H
3.0 N
s