PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Table of Contents Purpose of This Resource .......................................................................................... 3 Overview of PLATO Test Packs with Prescriptions on PLE ............................. 3 Types of Tests................................................................................................................ 3 State-Specific Fixed Benchmark Tests ....................................................................... 3 National Cumulative Fixed Benchmark Tests........................................................... 3 Criteria for Test Content............................................................................................ 4 Testing Models .............................................................................................................. 4 Test Design ....................................................................................................................4 Test Structure ............................................................................................................... 5 Implementation as Assessment Tool .......................................................................... 5 Implementation as Assessment Tool and Instruction................................................ 6 The Program Structure................................................................................................ 7 Using PLATO Test Packs with Prescriptions in Your Classroom .................... 8 Monitoring Student Progress ...................................................................................... 8 Use Instruction Reports to....................................................................................... 8 Use Assessment Reports to...................................................................................... 8 PLATO Instructional Content..................................................................................... 8 Content descriptors .................................................................................................. 8 PLATO Learning Content Design ............................................................................ 9 PLATO Curriculum Structure ................................................................................10 Understanding the PLATO Learning Curriculum in a Learning Path .................. 10 Sample Structure of a PLATO Learning Title for Language Arts: Reading....... 11 Understanding Curriculum Learning Activities ...................................................... 11 Test-to-Prescription Process...................................................................................... 13 Progressing through Prescribed Instruction ............................................................ 13 Student Flow through an Assessment ...................................................................... 14 Glossary of Terms ......................................................................................................15 Teacher Resource Guide v.1.0 2 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Purpose of This Resource The purpose of this document is to provide teachers with a clear, concise resource to use in conjunction with PLATO® Test Packs with Prescriptions on PLE™. This resource includes general information about Test Packs, including an overview of the product, types of available tests, using the product in your classroom, details around PLATO instructional content, and a glossary of terms. Overview of PLATO Test Packs with Prescriptions on PLE PLATO Test Packs with Prescriptions is an on-grade, subject-based remediation solution that assesses for individual areas of need, targeting instruction to fill each student’s skill gaps. The result is that student assignments specifically focus on their individual instructional needs. PLATO Test Packs provides educators with a tool to measure student achievement based on state or national standards at a given period of time. Teachers may choose to use PLATO Test Packs as an assessment instrument, as instruction, or both. When used as both an assessment tool and instructional tool, students take assessments and are given assignments, or prescriptions, based on their areas of need. Students then complete modules, or assignments, in which instruction is delivered in an online, interactive format. Educators can monitor student progress through instruction and assessment reports. Instruction reports display student information on instruction module use, such as seat time, and completion. Assessment reports display student information on test performance. For more information on reports, see Monitoring Student Progress. Types of Tests PLATO® Test Packs tests are designed to provide information about the strengths and needs of individual students, classrooms, schools, and districts. PLATO Test Packs tests include • state-specific fixed benchmark tests • national cumulative fixed benchmark tests State-Specific Fixed Benchmark Tests State-specific fixed benchmark tests are designed as series of multiple, comparable tests per grade. Fixed benchmark tests are developed using state standards for that state. National Cumulative Fixed Benchmark Tests National cumulative fixed benchmark tests are designed as series of two to three comparable tests per grade in five subjects—reading, writing, math, science, and social studies. These assessments allow for cumulative testing over an entire course. Each national test contains 25–70 test items. The PLATO National Cumulative Teacher Resource Guide v.1.0 3 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Fixed Benchmark tests are available in Writing, English, Math, Science, and Social Studies and span several grades. Criteria for Test Content The criteria used to develop PLATO National Cumulative Fixed Benchmark tests are drawn from national standards and respected professional organizations, such as state departments of education and these national standard-setting sources: • The Thomas Fordham Foundation • National Council of Teachers of Mathematics (NCTM) • National Council of Teachers of English (NCTE) • National Academy of Sciences (NAS) • National Council for the Social Studies (NCSS) • National Center for History in the Schools (NCHS) • The National Assessment Governing Board (NAGB) In addition, PLATO Learning follows a rigorous development procedure based on the Standards for Educational and Psychological Testing, as set forth by the following organizations: • American Educational Research Association (AERA) • American Psychological Association (APA) • National Council on Measurement in Education (NCME) Testing Models • • Cumulative Tests for Annual Progress Measurement Targeted Posttests for Knowledge Acquisition Measurement Test Design Q: How are tests designed? A: Subject matter experts (SMEs) create test structures for each subject area of the tests by synthesizing information from state and national standards and national standard-setting bodies. The standards are analyzed, objectives are determined, and test items are developed using best-practice criteria emphasized by leading assessment experts. Q: What are reporting categories? A: Reporting categories are topics taken from the standards on which students are tested. In both the state-specific fixed benchmark and national cumulative tests, reporting categories represent organized, summative headings of the objectives recorded in detail in the frameworks (see below). Teacher Resource Guide v.1.0 4 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Q: How are frameworks different from reporting categories? A: Frameworks are high-level groupings of related reporting categories. Frameworks cover broad topics. There is a framework summary for each framework that may be used for reporting and other tasks. Q: What does it mean when you say tests are comparable? A: The term comparable refers to each test version measuring the same standards with different test items. Q: What does PLATO Learning do to ensure test items are of high quality? A: PLATO Learning has defined quality by following principles set forth by leaders in the assessment field. Subject matter experts, national standard-setting bodies, Fordham Foundation A-rated state standards, and assessment experts’ best practice guidelines are consulted and synthesized into rigorous guidelines. Q: How do I know that test items are an appropriate readability level? A: Determining readability is not a precise metric or science. PLATO Learning approaches readability in three ways: 1) by consulting several key professional educational resources (e.g., EDL Core Vocabularies, common language found in textbooks and state-released tests) 2) by developing criteria for evaluating readability (e.g., text length, word usage, structure, genre, etc.) 3) by using this criteria coupled with computerized leveling tools (e.g., Lexile, Flesch-Kinkaid, etc.) Test Structure Your school has an implementation plan in which the intended implementation details for Test Packs with Prescriptions on PLE are defined. Teachers may choose to use PLATO Test Packs as an assessment tool, as instruction modules, or both. Implementation as Assessment Tool • • • Students are assigned an assessment instrument (a state-specific fixedbenchmark test or a national cumulative fixed-benchmark test). Developed around the most critical and foundational state or national standards and current Fordham Foundation A-rated state standards, these tests include two to three comparative versions with the same testing categories. Percentage point increases between test versions can provide insight into student strengths, needs, and progress. The percent of correctly answered items can point to areas that require continued remediation and provide guidance to teachers to modify classroom instruction. Teacher Resource Guide v.1.0 5 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Implementation as Assessment Tool and Instruction • • • • • Students take assessments and are given assignments, or prescriptions, based on their areas of need. The assignment process offers a choice to either allow the program to automatically assign the prescription or to do so manually. o Selecting the automatic option means that students will receive their assignments the moment they submit a test for scoring, with the prescription being generated solely based on their test results. o Selecting the manual assignment means that teachers start with the prescription recommendation, consider other student performance, and then manually override the instructional modules when and where appropriate. This approach allows teachers to consider what they might not yet have taught in the classroom. Once the test is submitted for scoring, an assignment for instruction is generated based on individual test item performance. Students then complete modules, or assignments in which instruction is delivered in an online, interactive format. The prescribed instruction, or prescription, is a set of instructional modules generated as a result of those skill gaps identified by the assessment. o The sequence of prescribed modules is fixed, based on PLATO Learning’s organization of how the modules relate to one another in terms of difficulty. o The learning activities included in the prescription are matched to both the appropriate age level and the skill level of the student. o Prescriptions may be sizeable given that they are somewhat cumulative for each specific grade level. Teacher Resource Guide v.1.0 6 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ The Program Structure This diagram represents the path that students follow when Test Packs is used as both an assessment tool and as instructional modules. Teacher Resource Guide v.1.0 7 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Using PLATO Test Packs with Prescriptions in Your Classroom Monitoring Student Progress Ongoing monitoring of student performance and progress is important to ensure student success and to make sound instructional adjustments as needed. Use Instruction Reports to • track student progress against average student progress for a particular assignment; • identify students who are falling behind average progress levels; • identify tests completed by a selected timeframe; • identify students who start assignments close to the due date; • identify students who make numerous attempts to complete certain activities; • track the time it takes for students to complete their activities; • identify students who master activities quickly; • monitor the differences in student activity by single- or dual-demographic categories; and • identify students who spend little time on task or who take too long to complete activities. Use Assessment Reports to • compare results across versions to show trends; • measure student progress from one testing session to another; • identify content areas of greatest need; • target instruction around objectives that have scores from state tests; • identify students with common weaknesses for small-group instruction; • identify demographic-based group gaps; • determine the most common scores, midpoint scores, and average levels of proficiency; • determine range across student scores; • determine if students are on target to complete a test; and • identify problematic concepts across classes. PLATO Instructional Content PLATO online instruction provides dynamic, engaging, digital content which integrates the best that multimedia has to offer with instructional integrity in an engaging environment. Content descriptors Instructional Integrity • age and skill appropriate • best practices implemented • student driven • objectives and tasks are clear Teacher Resource Guide v.1.0 8 PLATO® Professional Services • • • • • • PLATO® Test Packs with Prescriptions on PLE™ varied instructional time self paced instruct, apply, and assess each topic integrates a variety of learning styles promotes individual accountability provides precise measurement of the learner’s progress Engaging Environment • animated modeling • multiple learning modalities • real-life examples • immediate audio feedback • embedded mentors • various tools (manipulatives, calculators, etc.) • step-by-step help formats PLATO Learning Content Design The PLATO Learning curriculum included in a prescription is composed of specific modules across a variety of content titles. A prescription contains the most appropriate modules to teach a particular learning objective. Therefore, the design criteria for the different curricula that make up a prescription vary. These variances reflect the need to accommodate the content students are learning based on the nature of the subject matter and the skills that students need to acquire. Typical Instructional Module Design Tutorial — introduces skills and concepts • Application/Drill— provides practices and reinforces and applies knowledge • Mastery Test— assesses the understanding of the module’s objective • Other Designs Various combinations of the following: Tutorials • Applications/Drills • Practices • Mastery Tests • Offline Activities • Reviews • Skill Builders • Content design does affect how student progress is monitored and reported. • Progress is measured by three criteria: (1) mastery of an end-of-module test (2) completion (3) satisfactorily scored offline activity Teacher Resource Guide v.1.0 9 PLATO® Professional Services • PLATO® Test Packs with Prescriptions on PLE™ Modules do not report a numerical score; only activities report a score. o As a general rule, tutorials will not be scored or mastered; only completion information will be reported. o Applications in some curricula will be scored but not mastered. When the score is available, it will be reported along with completion. o Mastery tests may present a score but will always show mastery and completion. Most modules include a mastery test to check for application of knowledge acquired in the instructional module just completed. This table shows mastery test characteristics: Characteristics of a module mastery test • Mastery level is set at • Immediate response to 80% and fixed within answers most curricula • Correct answers or • Mastery allows the feedback not provided student to proceed to the • Some tests will end early next module if the student cannot • 5–10 randomly selected reach mastery test questions • Some tests give a fixed • One chance to answer number of questions each question regardless of learning progress within the test (pass–fail) PLATO Curriculum Structure Understanding the PLATO Learning Curriculum in a Learning Path PLATO Learning titles are subject-based, sequential learning activities. They are divided into units, each of which consists of several modules. (A module refers to the set of learning activities that addresses a targeted learning objective.) Here is how PLATO Learning titles work as part of a learning path: • A learning path contains the most appropriate instructional modules from a library of PLATO Learning titles to target the learner’s area of need. Learning paths may include one or more instructional modules, or an entire unit, from a given PLATO Learning Title. • With a prescriptive assessment, the learning path assigns particular modules across many titles to target each learner’s area of need. • Without an assessment, teachers would assign the entire title, and learners would work through all the modules in a title. Teacher Resource Guide v.1.0 10 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Sample Structure of a PLATO Learning Title for Language Arts: Reading Understanding Curriculum Learning Activities The curriculum included in the PLATO Learning Test Packs prescriptions, or learning paths, may include different learning activities based on the nature of the subject matter and the skills that learners need to acquire. These variances reflect the need to accommodate the content for specific skills that students are learning. Typical Learning Activity Design Other Designs tutorial application/drill mastery test Various combinations of the following: Tutorials, Applications, Drills, Practices, Mastery Tests, Offline Activities, Reviews, or Skill Builders Teacher Resource Guide v.1.0 11 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Typical Learning Activity Description tutorial The part of a module that contains the instruction. The tutorial always shows completion; it never shows mastery or score. application/drill A practice activity that builds skills but is not presented in a typical module structure. Practice lessons will show completion and score (when available) and occasionally will show mastery. skill activity Single lessons that contain instruction, reinforcement, and assessment all in one lesson rather than having the parts shown in a module. These activities are generally scored. student materials Work sheets that a student needs to print and answer offline to complete certain PLATO Learning activities. mastery test The part of a module that proves mastery of the module objective. It is sometimes scored, although usually only mastery is reported with no score; mastery and completion are always shown. Mastery is normally set at 80%. offline activity The printable part of a module that supports the module objective. Some activities are scored by the teacher using the Answer Keys available online; the teacher marks completion. Others are readings only with no student input required; the student marks these complete within the activity. Teacher Resource Guide v.1.0 12 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Test-to-Prescription Process Here is a graphic representation of the Test-to-Prescription process: Based on key knowledge areas from specific content standards, comparable test versions measure the same standards with different test items. There are two to five questions per standard. Skill gaps in the student’s core knowledge are identified based on whether questions for the various The most appropriate PLATO instructional modules across various content titles are prescribed when the student fails to demonstrate proficiency by incorrectly answering the questions linked to that standard. The student works through the prescription and is tested again using a comparable test with the same test categories. Analysis should show an increase in the number of standards mastered. Prescribed Instruction Standard 1 • Module 1 • Module 2 Standard 2 • Module 1 • Module 2 • Module 3 • Module 4 Standard 4 • Module 1 • Module 2 Progressing through Prescribed Instruction Teacher Resource Guide v.1.0 13 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Student Flow through an Assessment The Learner logs in and selects the Assessment listed in the Assignments Reminder box. ª An Assessment window opens. Many tests have two parts that can be completed together or at different times. Learners progress through a test by clicking question numbers at the bottom. ª The Learner clicks an answer choice and then selects another question. A Test can be stopped and saved at any time. The Grade Test option may also be selected at any time. ª Once a test is graded, the results box appears and the Learner can view the score. Learners receive their prescription assignments by clicking the Back to Assignments button. A personalized prescription is generated based on test scores. ª The prescribed instruction is assigned either automatically or after teacher review. For manual assignments, the Learner receives a message indicating that a test is under review. Upon receipt, the Learner clicks the Open button to view the assignment. ª The Learner accesses each module by clicking the links by each item. Assignment progression is made by selecting the item numbers at the top. Teacher Resource Guide v.1.0 14 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ Glossary of Terms Key Terms Definitions/Descriptions aggregated data These statistics relate to broad classes, groups, or categories, so that it is not possible to distinguish the properties of individuals within those classes, groups, or categories. They are sometimes shown as averages of the group. assessment The comprehensive term for a range of processes to gain information about student learning. assessment reports Assessment reports are the reports generated from test data. They tell the teacher how a student did on a test. Assessment reports • identify student knowledge gaps; • prescribe individual learning; • intervene and remediate gaps within PLATO instructional content; and • provide the ability to evaluate which students need additional instruction and to intervene quickly. There are five types of assessment reports: • Test Progress • Demographic (by assessment) • Score Analysis • Strengths & Needs • Summary Usage average A single value that is used to represent a collection of data, sometimes called measures of central tendency. benchmark A goal statement of reference. Teacher Resource Guide v.1.0 15 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ compacting Students who demonstrate previous mastery spend less time with the regular curriculum and more time with extension and enrichment opportunities. comparable Refers to each test version measuring the same standards with different test items. comparable items Comparable items are written to reflect assessment benchmarks and reporting categories at the objective level. These items consistently represent the same features, parameters, and attributes. Comparable items measure equivalent knowledge and skills at the same cognitive level and with the same degree of difficulty. comparable passages Comparable passages are written according to the same readability and reading ease scales. Comparable passages represent similar genres, length, and textual attributes. Primary genres are expository, narrative, and poetic texts. Determining comparability of passages includes considering appropriate subgenres, topics, stylistics, themes, characterizations, and settings. comparable tests Comparable tests are written as alternate and comparable forms of assessments that align with blueprint specifications. Alternate forms have comparable items with an equivalent format, the same item count, and a distribution that follows designated objectives and reporting categories. When judging test items to make sure the test versions of PLATO Test Packs are comparable, there are four major areas the development team addresses: • Cognitive level • Readability • Plausibility of answer options • Visuals Teacher Resource Guide v.1.0 16 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ contracts Written agreements between teachers and students that outline what students will learn, how they will learn it, in what period of time they will learn it, and how they will be evaluated. Contracts allow students to engage actively in the decision-making process, directing their course of study. criterion-referenced tests Tests that are scored by comparing performance to pre-established criteria, such as objective, benchmarks, or learning goals. The purpose of criterion-referenced tests is to determine how well students have learned specific information. Most states use their state standards tests, which are criterionreferenced tests. cumulative Refers to testing an entire year of standards. formative assessments Assessments that monitor progress during instruction and help teachers make decisions about student learning and provide ongoing feedback to students. framework A structural element within the tests that groups reporting categories into larger groups of like concepts. Often referred to as strands. instruction reports Instruction reports are the reports generated from PLATO content data. They tell the teacher how a student did on PLATO content, such as a prescription assigned after taking a test. Instruction reports • give teachers flexibility and information to guide instruction; • improve communication among teachers, students, parents, and administrators; • provide more accurate data on access and usage of the system; and • track seat time when needed. Teacher Resource Guide v.1.0 17 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ instruction reports (cont.) There are five instruction reports: • Learner Progress • Learner Daily Usage • Class Module Mastery • Class Usage Summary • Demographics Summary (by assignment) learning path A sequence of instructional activities. mean A measure of the general size of the data. Mean = the sum of the values the number of the values median The middle value of a distribution that is arranged in size order. mode The value or values that occur most often in a distribution. national cumulative fixed benchmark tests National cumulative fixed benchmark tests are designed as series of two to three comparable tests per grade in five subjects— reading, writing, math, science, and social studies. The cumulative tests were developed against a consolidation of information from these sources: • National standards • Current Fordham Foundation A-rated state standards • National standard-setting bodies, including the National Council of Teachers of Mathematics (NCTM), the National Council of Teachers of English (NCTE), the National Academy of Sciences (NAS), the National Council for the Social Studies (NCSS), and the National Center for History in the Schools (NCHS) Teacher Resource Guide v.1.0 18 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ norm-referenced tests Scores are ranked and compared to each other. The results of these tests compare a student’s score to scores from a sample of representative students. They are designed to classify students according to what they know compared to other students. The SAT test and Stanford Achievement Tests are examples of norm-referenced tests. objective An observable educational outcome. prescription options Prescriptions may be automatically assigned by the system or made available for the teacher to edit prior to manual assignment. prescription summary Identifies specific instructional modules with corresponding test items and the number of items that must be correctly answered not to receive the module as part of a prescribed assignment. printing options The entire test may be printed, or just the reading passages (if applicable). This produces browser-based printing (e.g., not MS Word, PDF). quick reference cards (QRCs) Brief references that provide information on reports and other topics. They are found at: http://support.plato.com/PLE range The difference between the highest and lowest values in a distribution. reporting category A structural element within the tests which groups the concepts tested into like categories for reporting purposes. scheduling options Tests may be scheduled to be taken within a particular time window for district- or schoolwide calendar purposes. Teacher Resource Guide v.1.0 19 PLATO® Professional Services PLATO® Test Packs with Prescriptions on PLE™ standard A measure against which other items can be compared for the purpose of determining accuracy or quality; a measure indicating that a student has minimum competency in understanding or mastering a benchmark or objective. state-specific fixed benchmark tests State-specific fixed benchmark tests are designed as series of multiple, comparable tests per grade. Fixed benchmark tests are developed using state standards for that state. summative assessments Assessments that occur at the end of instruction. They measure performance against the intended learning goals after they are taught. test design Provides information about standards addressed, number of questions for each standard, and answer distribution; shows the entire test with questions organized by standard. test A particular type of assessment given at a fixed point in time. test information Provides summary information about the test, including date of publication, subject, grade, total number of items, and estimated duration. tutorials Tutorials for user support are longer presentations of key information, such as how to create a user. They are found at: http://support.plato.com/PLE Teacher Resource Guide v.1.0 20