ML20260H150

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)

ML20260H150
Person / Time
Site:	FitzPatrick
Issue date:	10/14/2019
From:	Exelon Generation Co
To:	Todd Fish Operations Branch I
Fish T
Shared Package
ML19105A122	List: ML20043C809 ML20087G852 ML20087H514 ML20260H150 ML20260H151 ML20260H152 ML20260H153 ML20260H154 ML21056A284 ML21056A285 ML21056A306 ML21056A307
References
CAC 000500
Download: ML20260H150 (7)

Text

ES401 JamesA.FitzPatrick Written Examination Review Worksheet FormES4019 2 3.Psychometric Flaws 4.JobContent Flaws 5.Other a 1.LOK 6.Source7.Status LOD 8.Explanation (F/H)(15) (B/M/N) (U/E/S) pStem Cues T/F Cred gi3g Partial Job-Link Minutia #/UnitsBackwardQ K/A -

SROOnly 1 H 3.7 M S 2 F 3.1 B S 3 F 3.2 B S 4 H 3.5 N S 5 H 4.0 N S 6 F 2.2 B S 7 F 2.8 B S 8 H 3.1 B S Refer D ofES-401 toSection andAppendix B foradditionalinformation regarding each ofthe following concepts:

1. Enter thelevelofknowledge(LOK) ofeach questionaseither (F)undamental or(H)igher cognitive fevel.

2. Enter thelevel (LOD) ofdifficulty ofeachquestion a 1 (easy) questions with a difficulty between to5 (difficult); 2 and4 areacceptable.

3. Check the boxif appropriate a psychometricflawisidentified:

"Stem Focus":Thestemlacks sufficient focus toelicit thecorrect answer (e.g.. unclear intent, moreinformation isneeded, ortoomuchneedless information).

"Cues": Thestemordistractorscontaincues (e.g.,

clues,specificdeterminers, phrasing, length).

"T/F":Theanswer choicesare a collection ofunrelated true/falsestatements.

"Cred.Dist.":

Thedistractorsarenotcredible; single implausibledistractors should berepaired, andmorethan oneisunacceptable.

Oneormore "Partial": are distractors partially correct (e.g.,

iftheapplicant canmakeunstated assumptions thatarenotcontradicted bythestem).

4. Check boxif theappropriate ajob contentflawisidentified: '

"JobLink":Thequestionisnotlinkedtothejob requirements the (i.e., question hasa valid K/Abut, as written, isnotoperational incontent).

"Minutia":

Thequestionrequirestherecall ofknowledge that istoospecific for theclosed-reference test mode (i.e.. it isnotrequired tobeknown memory).

from

"#/Units":

Thequestioncontainsdatawithanunrealisticlevel ofaccuracy orinconsistent units (e.g.,

panel meter in percentwith question ingallons).

"Backward":Thequestionrequiresreverselogic orapplicationcompared tothe jobrequirements.

5. Check Iluestions aresampled that forconformance with theapproved K/Aandthose K/As that aredesignated "SRO-only." (K/A andlicenselevel mismatches areunacceptable.)

6. Enter question'ssource:

(B)ank, or(N)ew.

(M)odified, Verify that(M)odified questions meetthecriteria ofFormES-401, Section D.2.f.

7. Based onthereviewer's judgment,isthequestion,aswritten, (U)nsatisfactory(requiring repair orreplacement), inneedof (E)ditorial enhancement, or(S)atisfactory?

8. Ata minimum, any"U"status explain (e.g.,

ratings howthe Appendix B psychometric attributes arenotbeing met).

ES-401 2 FormES4019 1 2 3.Psychometric Flaws 4.JobContent Flaws 5.Other Q LOK 6.Source 7.Status LOD 8.Explanation (F/H)(1-5) em3 Cues T/F @3ed. Job-Link Partial Minutia# /UnitsBackwardQ K/A SROOnly 9 H 3.6 B S 10 H 3.7 B S 11 H 4.1 B S 12 H 3.0 B S 13 F 2.4 B S Q fits Fundamental not 14 H 3.4 M E Reclassifie Order.

Higher as"F".

15 H 3.6 N S 16 H 4.0 B S 17 H 3.7 B S 18 F 2.8 B S Q basically asks 'Where 19 F 1.0 N y CRDpumps?"

arethe newK/Aand Selected newQ.

wrote 20 H 3.5 N S 21 F 2.5 N S 22 H 2.9 B S 23 F 2.2 N S 24 F 3.0 N S

ES401 3 FormES4019 1 2 3.Psychometric Flaws Flaws 4.JobContent 5.Other Q LOK LOD 6.Source 7.Status 8.Explanation (F/H)(1-5) e Cues T/F ed. Partial Job-LinkMinutia#/Units BackwardQ K/A SROOnly 25 H 3.7 M S 26 H 3.4 B S 27 F 2.2 B S 28 H 3.5 B S 29 H 3.3 N S 30 F 2.5 B S 31 H 3.4 B S 32 F 2.2 B S 33 F 2.7 B S 34 F 2.9 M S 35 F 2.8 B S 36 H 3.5 N S 37 H 3.6 B S 38 F 2.6 B S 39 H 3.7 M S 40 H 3.3 B S

ES-401 4 FormES4019 1 2 3.Psyc'hometric Flaws 4.JobContent Flaws 5.Other Q LOK LOD 6.Source 7.Status 8.Explanation (F/H)(1-5) em3 Cues T/F 3ed. Job-Link Partial Minutia# /Units BackwardQ K/A SROOnly 42 F 2.8 B S 43 F 2.8 N S 44 H 37 B S 45 F 2.7 N S 46 F 2.4 N S 47 H 3.4 N S 48 H 3.6 M S 49 H 3.3 B S 50 F 2.7 N S 51 F 2.8 B S 52 H 3.8 N S 53 H 3.3 B S 54 F 2.4 N S 55 H 2.9 B S 56 H 3.3 B S 57 H 3.7 N S

ES401 5 FormES401-9 1 2 3.Psychometric Flaws 4.JobContent Flaws 5.Other Q LOK LOD 6.Source 7.Status 8.Explanation (F/H)(1-5) ed cuesT/F *td.

Partial Job-Link # /Units BackwardQ K/A Minutia -

SROOnly 59 H 3.8 B S 60 H 3.3 B S 61 H 37 M S 62 F 2.7 N S 63 H 3.8 B S 64 F 2.2 N S 65 F 2.5 B S 66 H 3.5 B S 67 H 3.6 B S 68 F 2.5 B S 69 F 2.4 B S 70 F 2.9 B S 71 H 3.8 B S 72 F 2.2 B S 73 F 2.0 B S 74 F 2.0 B S

ES401 6 FormES4019 1 2 3.Psychometric Flaws 4.JobContent Flaws 5.Other Q LOK LOD 6.Source 7.Status 8.Explanation (F/H)(1-5) em, Cues T/F 30td.

Partial Job-LinkMinutia# /Units BackwardQ K/A SROOnly 76 H 3.8 M S Choice implausible

'D' 77 H 3.8 N E (can't anUnusual declare EventandanAlert).

Rewrotedistractor

'D' 78 H 3.4 B S 79 H 3.6 N S 80 H 3.3 M S 81 H 3.5 B S 82 F 2.9 N S 83 H 3.8 M S 84 H 3.4 B S 85 H 3.5 N S 86 F 2.7 N S 87 H 3.8 N S 88 H 3.8 N E Added','ONLY" tochoices A & 'B 89 H 3.9 M S 90 H 3.4 B S 91 F 2.4 X B y Q does "RPV notdiscuss internals."

Wrote newQ.

92 H 3.8 M S

ES-401 7 FormES-4019 1 2 3.Psychometric Flaws 4.JobContent Flaws 5.Other LOK 6.Source 7.Status Q LOD 8.Explanation (F/H)(1-5) em Cues T/F @if38td. Job-LinkMinutia# /Units BackwardQ Partial -

K/A SROOnly 93 H 3.5 N S 94 F 2.5 B S 95 F 24 B S 96 F 2.3 B S 97 F 2.7 B S Notsuitable Minutia.

98 F 2.0 X N U Q.Selected memory-level newK/Aandwrotenew Q.

99 H 3.0 B S 100 F 2.1 B S