ML22355A650

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)

ML22355A650
Person / Time
Site:	FitzPatrick
Issue date:	08/30/2021
From:	Brian Fuller Operations Branch I
To:	Kelly D Exelon FitzPatrick
References
EPID: L-2021-OLL-0000
Download: ML22355A650 (1)

Text

ES-401 FitzPatrick 2021 Written Examination Review Worksheet Form ES-401-9 Q

1. LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /Units Backward Q - K/A SRO Only 1

H 2

N S

2 F

3 N

S 3

H 4

B S

4 F

2 N

E Rework the 1st part of options C & D (use info from ST eliminate potential for 2 correct) 5 H

3 X

B E

Remove ref pic (give valve pos instead) 6 H

4 B

S 7

F 2

B E

Change level indication options to NPSH 8

H 4

M S

Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.

3.

Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).

T/F: The answer choices are a collection of unrelated true/false statements.

Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).

4.

Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

Backward: The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)

6.

Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.

7.

Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2

Form ES-401-9 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. / Units Backward Q - K/A SRO Only 9

H 2

N S

10 H

3 N

E Enhance verbiage to improve clarity 11 F

2 B

S 12 F

2 B

S 13 H

3 N

S 14 F

2 N

S 15 H

2 B

S 16 H

2 N

S 17 F

3 N

S 18 H

4 B

S 19 H

3 N

S 20 H

4 B

S 21 H

3 B

S 22 F

2 N

S 23 F

2 B

S 24 F

2 B

S 25 H

3 N

E Revise plausibility statement and change option A verbiage to match B

ES-401 3

Form ES-401-9 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. / Units Backward Q - K/A SRO Only 26 F

2 B

S 27 F

3 N

S 28 H

3 N

S 29 F

2 N

S 30 H

3 B

S 31 H

3 B

S 32 F

2 N

S 33 H

4 B

S 34 F

2 N

S 35 H

2 M

S 36 H

3 B

E Change to 2 double & 2 single options 37 H

2 M

E Make column headers look the same 38 H

3 B

S 39 F

2 N

S 40 F

2 B

S 41 H

4 B

S 42 H

3 B

S

ES-401 4

Form ES-401-9 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. / Units Backward Q - K/A SRO Only 43 H

3 B

E Change to 100% power 44 H

2 N

S 45 F

3 B

E Change option B to cooling for Recirc Pumps 46 H

2 N

S 47 H

3 B

S 48 H

2 N

S 49 F

2 N

S 50 H

3 N

S 51 H

2 N

S 52 H

2 B

S 53 H

3 N

S 54 F

3 M

S 55 F

3 N

E Fix typo 56 F

3 B

S 57 F

2 B

S 58 H

3 B

S 59 H

2 N

S

ES-401 5

Form ES-401-9 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. / Units Backward Q - K/A SRO Only 60 H

2 B

S 61 F

2 B

S 62 H

3 M

S 63 H

3 M

S 64 H

4 M

E Eliminate unnecessary words 65 F

3 B

S 66 F

3 B

S 67 F

2 B

S 68 F

2 N

S 69 H

2 B

S 70 F

2 M

S 71 H

2 B

S 72 F

2 B

S 73 H

2 B

S 74 F

2 B

S 75 F

2 B

S 76 H

3 M

S

ES-401 6

Form ES-401-9 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. / Units Backward Q - K/A SRO Only 77 H

2 B

S 78 H

2 B

E Add 100% power, refueling preparations 79 H

3 B

S 80 H

2 N

S 81 H

2 N

S 82 F

3 N

S 83 H

2 N

E Combine 2nd & 3rd columns into one 84 H

2 N

S 85 H

2 B

S 86 H

2 N

E Add SRO level justification 87 F

2 N

S 88 H

2 M

S 89 H

2 B

S 90 H

2 X

N U

Changed to give voltage values vs. unsat 91 F

2 N

S 92 H

4 M

S 93 H

3 N

S

ES-401 7

Form ES-401-9 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. / Units Backward Q - K/A SRO Only 94 H

2 N

S 95 F

2 N

S 96 H

3 X

X N

U Rewrite to match K/A. Question replaced 97 F

2 N

S 98 H

2 B

S 99 H

3 B

S 100 F

2 N

S