Kim Michaud Rachana Pandey Michael Matthews Robin Jackson Cristina Nazario General Information Authors: Eliot R. Long, Publisher: Victor S. Artese, Wonderlic Inc. Winifred L. Clonts Wonderlic Basic Skills Test Cost: Unavailable Date of Publication: Provided only to administrating organization 1999-2000 2 Description of Purpose and Nature of Test 3 What is its purpose? What does it measure? Purpose: The Wonderlic Basic Skills Test is a brief, multiple-choice test intended to measure the language and math skills of adults, and to be used in assessing the job-readiness of the testtaker WBST Measures: Job Readiness Job Readiness can be defined as: “Having sufficient language and math skills to successfully handle the written and computational requirements of a particular job.” (Donlon & Hanna, 2003) 4 Where might this test be utilized? How would it be used? Often used as part of the hiring process In assessing basic skill levels, one can find a job of “best fit”. In assessing results, potential employers can also find employees that they feel are “best fit” Generally used to assess entry level positions and occupational training programs 5 What type of test is the WBST? Classified as a basic skills achievement test *Handout 6 Who is the WBST Intended For? The WBST is intended for those high school-aged and up Designed for those who wish to assess their basic skills when entering the workforce. Skills assessed can lead to jobs in many vocations 7 What scores does the WBST provide? The Wonderlic Basic Skills Test aims to measure general intelligence or 'g,' which provides insight into how easily individuals can be trained and how well they adjust in work and school settings. Characteristics of Results: Production Machine Tender: Level Two Verbal Level One Quantitative Mathematics Each level indicates one’s attainment ability 8 What do the results tell you about a client? WBST Assesses: Math skills Language skills “coachability” *The Wonderlic measures potential to learn, rather than how well that potential is utilized. 9 About The Test Verbal: 50 Questions – General Educational Development (GED) level – Word Knowledge – Sentence Construction – Information Retrieval Quantitative : 45 Questions – General Educational Development (GED) level - Explicit Quantitative Skills - Applied Quantitative Skills - Interpretive Quantitative Skills 10 About The Test Use one test or both as needed Question Types – Multiple Choice – Increasing order of difficulty Administration time: 20 min. per test No calculators Two equivalent forms: VS1, VS2 and QS-1 and QS-2 11 Critical Evaluation Clean, clear, attractive design and presentation Provides a range of appropriate scores for 900+ entry level jobs Customizable job profiles Appropriate for intended purpose and audience 12 Administration and Scoring Easy administration and scoring Paper and pencil – Individual or group administration – Fax in scoring – Online template scoring Online – Individual or group administration – Online scoring – Results in minutes 13 Critical Evaluation Validity Reliability Content Validity Content should represent what experts say it should for that particular domain WBST based job related language and math skills published by the U.S. Dept. of Labor Reviewers agree that it is sound 15 EXAMPLE excerpted Revised Handbook for Analyzing Jobs (RHAJ) U.S. Dept. of Labor 1991 http://www.skilltran.com/rhaj/chapter7.htm LANGUAGE DEVELOPMENT Level 2 READING: Passive vocabulary of 5,000 - 6,000 words. Read at rate of 190-215 words per minute. Read adventure stories and comic books, looking up unfamiliar words in dictionary for meaning, spelling and pronunciation. Read instructions for assembling model cars and airplanes. MATHEMATICAL DEVELOPMENT Level 1 MATH: Add and subtract two-digit numbers. Multiply and divide 10's and 100's by 2, 3, 4, 5. Perform the four basic arithmetic operations with coins as part of a dollar. Perform operations with units such as cup, pint, and quart; inch, foot and yard; and ounce and pound. 16 17 Normative Validity The test is appropriate for the population it is intended for without gender, cultural or content bias U.S. Dept. of Education reviewed and approved for language/math skills for postsecondary occupational training Expert reviewed for cultural sensitivity Differential item functioning analysis for gender, culture, content bias 1 Reviewer wants more info re: students in school vs. adults at work 18 Construct Validity The scales measure what they are supposed to measure Reviewer’s positives Good grade-progression data Good discussion how item ordering minimized impact of speediness on score Reviewer’s negatives More evidence needed for test wiseness, sensitivity coaching, and context dependence Demonstration that scales accurately represent DOL scales 19 Reliability The measurements reported are accurate Test-Retest & Chronbach alpha 20 Test-Retest - for both same and alternate forms on: (a) same day, (b)30 days, (c) after 30 days - Results Reported: .83 - .93 Reviewer’s positives: Item Response Theory/computer feature which flags significantly different response pattern Item Response Theory enabled standard error of measurements to vary with score level Reviewer’s negatives: Too many small item scales (11 to 12) with insufficient summary information re: how IRT impacted accuracy No standard deviations or means described 21 Chronbach Alpha Internal Consistency Results: .90 - .93 for Verbal /Quantitative, timed/untimed 20 minutes appropriate time & sufficient to measure the verbal and quantitative domains Reviewer’s positives: Chronbach alpha good choice for speed test Reviewer’s negatives: No means and Standard deviations Insufficient summary accuracy information 22 Summary Evaluation Strengths Weaknesses 23 Strengths Gives warning signals of inconsistent patterns. Easy administration, brief test, flexibility in format administration and scoring. Provides accommodations for individuals with disabilities. Evaluates the verbal and quantitative skills often necessary for entry level jobs or programs. Weaknesses The multiplicity of the scales too broad to be specific. Scales do not demonstrate high levels of precision. Not enough detail to determine if the reference group is representative in respects to the national populations of students in school and adult worker populations. More information is needed in the manual to determine the accuracy of the scores. National Football League 26 Questions 27