Follow
Publications: 52 | Followers: 0

III Choosing the Right Method Chapter 10 Assessing Via ...

Publish on Category: Birds 0

Part IIIChoosing the Right MethodChapter 10 Assessing Via Tests
Traditional Employment TestsWork SamplesSituational Judgment (SJT) TestsTechnologyGlobal Testing
chapter 10 Assessing via Tests
1
Terminology
Test“..objective and standardized procedure for measuring a psychological construct using a sample of behavior”What are some examples?Standardization“..controlling the conditions and procedures of test administration, keeping them constant orunvaryng.”Construct“…fairly well developed idea of a trait.”Most KSAOsKerlinger’sdefinitionsGive an example of a construct and operational definition usingKerlinger’sdefinitions –that’s IO related
chapter 10 Assessing via Tests
2
More terms
Standardization“…controlling conditions and procedures so scores among different people are comparable.”What makes an interview a test rather than just an interview?
chapter 10 Assessing via Tests
3
NORM-REFERENCED AND DOMAIN-REFERENCED TESTING
What’ the difference between the two?(domain-referenced is also call criterion-referenced)What are some examples (or situations) where the purpose would require one or the other? (thought question)Hint: when criticality of performance is important orwhere characteristics of the applicant pool may vary geographically or over time.
chapter 10 Assessing via Tests
4
TRADITIONAL COGNITIVE TESTS
Discrimination and generalizationi.e. recognize or discover relationshipsKnow, perceive, remember, understand, cognitive manipulationProblem solving, evaluation of ideasCompare and contrastIQ, GMA, Cognitive abilityAlso can be considered aptitude and/ or achievemente.g. WPT with NFL players(do you think scores are related to performance?)
chapter 10 Assessing via Tests
5
Traditional Cognitive Tests(Aptitude Tests)
What is the difference between an achievement test and an aptitude test?Should you make one or buy one?GATB(General Aptitude Test Battery)GMAVerbal AptitudeNumericalSpatialForm PerceptionClerical PerceptionMotor/CoordinationFinger DexterityManual Dexterity
chapter 10 Assessing via Tests
6
Cognitive Ability
Off the shelfWPTWatson Glaser Test of Cognitive Ability (WGCTA)DAT (Differential aptitude Test)What is a job you would use each of them for?What are the relative advantages and disadvantages for developing homemade tests vs. commercially available ones?
chapter 10 Assessing via Tests
7
Work Samples and Performance Tests
How can they be both criteria and / or predictors? Give examples.Work Samples and Simulations“standardized abstraction of the work”Give hypothetical examples of a high and low fidelity simulation using an example fromBoeing 737 MAXMCAS)–what level of abstraction Would you use?Developing Work Samples to measure “proficiency”For criterion – which tasks would be measured?For selection –which ones would be omitted?
chapter 10 Assessing via Tests
8
Work Samples and Performance Tests(cont’)
Situational JudgmentsCan be either multiple choice or video (higher fidelity)“should do?”McDaniel & Nguyen, ‘01)or“would do?”Ployhart&Ehrhart, ‘03)With incremental validity over GMA, Personality, job experience(Chan & Schmitt, 02)If situational judgement is used as aconstructfor good judgmentThen it “holds much promise” (Brooks & Highhouse, 2006)
chapter 10 Assessing via Tests
9
Non-Cognitive Performance
Physical Abilities (series ofdistinct physical activities)Sometimes difficult to set valid cut scoresMake sure the cut score does not discriminate unfairly for gender. (can the job be redesigned?)Fitness Testing (measures strength agility, etc.)a two-edged sword for setting standards and not doing so.What is the legal conundrum here?Should fitness testing be required for some jobs on a daily basis?Should drug testing be performed for some jobs and not other?Sensory & Psychomotor Proficiencies (GATB?)
chapter 10 Assessing via Tests
10
Technology and Testing
What is a stenographer? Where did they go?Computerization of testsHi fidelity is more possible with CAT (SJT, etc.)Collection and storing data made easier (lot of data!)Computer Adaptive TestsWhat’s the difference between linear testing and branching algorithms?How can IRT help?
chapter 10 Assessing via Tests
11
Technology and TestingProctored v. Un-proctored
Proctored vs. Un-proctored testing (a range)Proctored1. limits distraction2. Verifies user identity3. Monitors time (also possible with un-proctored)4. prevents unauthorized subs5. prevents cheating
chapter 10 Assessing via Tests
12
Technology and TestingProctored v. Un-proctored
Use of mobile devices for testing online(Arthur, et al.2014)Un-proctored internet-based tests (UIT)Can they be trusted for:for personality?cognitive ability?See two articles on files directory (Arthur et al. 2009 & 2010)AOE Science
chapter 10 Assessing via Tests
13
Technology and TestingSimulations, Games, Gamification
“Gamification”Game elements introduced into traditional testing…as SJT…or cognitive assessments…personality assessed with facial recognition in online interviews, or in basket exercisesVary in levels of fidelityE.g. online interviews with avatars or video interaction with actorsIRT may be implemented here
chapter 10 Assessing via Tests
14
Global TestingStandardization across Countries
Legal Issues…across countries:Definition of protected (or minority groups) varyExamples?Ethical behavior viewed differentlyExamples?“rights of workers” varyExamples?These issues are amplified in a global environment
chapter 10 Assessing via Tests
15
Global Issues
Translations and Equivalence IssuesCross cultural testing a challenge:Different approaches to testingTest administration problemsScore equivalenceConsiderations:Use IRT to establish item equivalenceDevelop “global” measures(Schmitt,Kihm&Robie, 2000)Always use translation to “back translation”
chapter 10 Assessing via Tests
16
Global TestingStandardization across Countries
Translation and EquivalenceScore equivalence is unattainable with literal translationWords don’t mean the same thing in different culturesSome constructs may not exit in some culturesE.g. Island of Bali direction terms are geocentricWhere the speaker is located relative to an object (volcano,e.g)InGuuguYimidhirrlanguage– do not use left-rightAlways know where north is so would say:“a spider is crawling up your southeastern arm”
chapter 10 Assessing via Tests
17
Tests and Controversy
Setting Cut Scores –rarely recommended but often necessary:Civil serviceLicenses & certificationCyclical hiring (needs forecasting estimates, e.g. teachers)Predicted Yield Model (Thorndike, ‘49)Fluctuation of candidate qualifications variesavailability of openings variesDepends upon accurate forecasting (history and research)Regression-Based Methods
chapter 10 Assessing via Tests
18
Tests and Controversy
Do we need licensing tests to establish credentials?Do we need educational proficiency exams?We will always assess - eitherWith tests (more objective)With subjective judgment
chapter 10 Assessing via Tests
19

0

Embed

Share

Upload

Make amazing presentation for free
III Choosing the Right Method Chapter 10 Assessing Via ...