Assessment & Accountability Briefing Book FCAT School Accountability Teacher Certification Tests Florida Department of Education www.fldoe.org Florida Department of Education FLORIDA DEPARTMENT OF EDUCATION STATE BOARD OF EDUCATION JIM HORNE Commissioner of Education F. PHILIP HANDY , Chairman T. WILLARD FAIR, Vice Chairman Members LINDA J. EADS, ED.D. CHARLES PATRICK GARCÍA JULIA L. JOHNSON April 23, 2004 WILLIAM L. PROCTOR , PH.D. LINDA K. TAYLOR Dear Friends: I am pleased to present this Assessment and Accountability Briefing Book for your information. There is no greater challenge facing Florida’s citizens and their elected representatives than public education reform. As Education Commissioner, I take seriously my responsibility to make sure that no child is left behind. I also want to foster cooperation and goodwill among our education community. This means parents and educators will all have a say in the education of our children. The FCAT has been part of the landscape of education reform in Florida for several years. It plays a key role in the ongoing effort to raise standards. However, it is important to remember that the FCAT is not an end in itself, but a means to an end. The result is an improved education for Florida’s children and increased accountability for its schools. I hope this Briefing Book proves useful to you. The Department staff and I stand ready to assist you in any way possible. Working together, we can make sure that all Florida’s children receive a world-class education. Sincerely, Jim Horne 325 W. GAINES STREET SUITE 1514 TALLAHASSEE, FL 32399-0400 (850) 245-0505 www.fldoe.org Table of Contents I — Introduction and Chronology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 II — The Florida Comprehensive Assessment Test (FCAT) Frequently Asked Questions About the FCAT. . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 The Financial Cost of Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Educator Involvement in FCAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 The Question of “Teaching to the Test”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Reporting Student Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Reliability and Validity of FCAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 III — School Accountability in Florida Critically Low Schools – 1995 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A+ School Grades – 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Raising the Bar for A+ – 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . No Child Left Behind & Adequate Yearly Progress . . . . . . . . . . . . . . . . . . . . . . 29 29 30 31 IV — Other K-12 Assessments The Reading Diagnostic Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 School Readiness Uniform Screening System (SRUSS). . . . . . . . . . . . . . . . . . . 34 College Board’s Preliminary Scholastic Aptitude Test (PSAT) and ACT’s Educational Planning and Assessment System (PLAN) . . . . . . . . . . 35 V — Certification Examinations for Florida Educators Florida Teacher Certification Examinations (FTCE) . . . . . . . . . . . . . . . . . . . . . . Florida Educational Leadership Examination (FELE) . . . . . . . . . . . . . . . . . . . . FTCE/FELE Test Administrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . FTCE/FELE Registration and Fees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 41 43 45 VI — Other Postsecondary Assessments Florida College Entry-Level Placement Test (CPT) . . . . . . . . . . . . . . . . . . . . . . 47 Florida College Basic Skills Exit Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 The College-Level Academic Skills Test (CLAST) . . . . . . . . . . . . . . . . . . . . . . 47 VII — Appendices A—FCAT Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 B—Scoring of FCAT Performance Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 C—Terms and Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Glossary of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Content Assessed by the FCAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Assessment and Accountability Briefing Book iii Sunshine State Standards (SSS) Tested on the FCAT for Grades 3-5. . . . . C-2 FCAT Norm-Referenced Test (NRT) Content for Grades 3-5 . . . . . . . . . . C-3 Sunshine State Standards (SSS) Tested on the FCAT for Grades 6-8. . . . . C-4 FCAT Norm-Referenced Test (NRT) Content for Grades 6-8 . . . . . . . . . . C-5 Sunshine State Standards (SSS) Tested on the FCAT for Grades 9-10. . . . C-6 FCAT Norm-Referenced Test (NRT) Content for Grades 9-10 . . . . . . . . . C-7 D—Statutes and Rules, Assessment and Accountability . . . . . . . . . . . . . . . . . . D-1 iv Assessment and Accountability Briefing Book Introduction and Chronology I - Introduction and Chronology Introduction The purpose of this Briefing Book is to provide readers with a guide for understanding the state’s educational assessment and accountability programs. This publication is intended to serve as a resource for detailed information about the Florida Comprehensive Assessment Test (FCAT), the School Grading system, and the Certification Examinations for Florida Educators, as well as to give an overview of how the different areas of student and educator assessments and school accountability results are integrally related. Better decisions are being made about education because FCAT data provides information to help educators understand how well students are learning and schools are improving over previous years. Because Florida has educational assessments, citizens have standard measures to use in determining student progress, evaluating schools, and ensuring that qualified educators are teaching in and leading our schools. Accountability for Schools and Students – A Chronology Florida’s focus on educational accountability began well before 1998 and the first administration of the FCAT. The key events of the development of the state’s focus on improving student achievement are described in the following chronology. This summary briefly describes the origin of the student assessment and school accountability systems, including the origin of the Sunshine State Standards and the development, administration, scoring, and reporting of the FCAT.1 1968 State legislation, Section 229.551, F.S., instructed the Department of Education (DOE) to improve educational effectiveness. 1969 The Legislature appropriated an annual sum for sponsoring an educational research and development program in the DOE. 1970 The Legislature enacted Chapters 70-339, Laws of Florida, which authorized the Commissioner of Education to develop a plan for evaluating the effectiveness of educational programs. 1971 The Educational Accountability Act was enacted (Section 229.57, F.S.) to implement the Commissioner’s plan for educational assessment in Florida, called the Statewide Assessment Program. Local educators reviewed and evaluated the objectives to be assessed, and in September 1971, the State Board of Education adopted the objectives. The first statewide assessment in Florida collected educational data on students in Grades 2 and 4. 1974 The Educational Accountability Act of 1971 was amended to require Reading, Writing, and Mathematics assessments by 1976. 1976 The Educational Accountability Act of 1976 expanded the provisions of the 1971 and 1974 state legislation. Changes included developing assessments for students in Grades 3, 5, 8, and 11. In addition, this legislation authorized the nation’s first high school graduation test by requiring students in the graduating class of 1978-79 to pass a state-administered functional literacy test before receiving a high school diploma. _____________________________________ 1 A detailed history of assessment and accountability in Florida is available on the Internet at http://www.firn.edu/sas/hsaphome.htm. Assessment and Accountability Briefing Book 1 Introduction and Chronology 1978 In October, approximately 435,000 students in Grades 3, 5, 8, and 11 were tested. Since the graduating class of 1979 was the first to be required to pass the test, twelfth graders and adults who had failed one or both sections of the 1977 Functional Literacy Test were retested. Approximately 35,000 twelfth graders and 4,400 adults were retested. The first legal challenge to the statewide assessment program was filed by the National Association for the Advancement of Colored People in Dade County contesting the Department’s right to limit public access to the Functional Literacy Test. The case was dropped in June 1979. The second legal challenge to the statewide assessment program was brought by Florida citizen John Brady, who disputed the legality of the scoring system used for the test. Both the hearing officer and the District Court of Appeals ruled in favor of the State Board of Education and upheld the decision to implement the scoring procedures adopted by the Board. In the third legal challenge to the statewide assessment program, Mr. Brady, joined by plaintiff Blount et al., argued that the testing criteria were arbitrary and unfair. The hearing officer refused to address the issues surrounding the test and ruled in favor of the State Board of Education. The decision was appealed to the District Court of Appeals, where it was upheld. 2 1979 The fourth legal challenge was filed in federal court in Tampa by the Bay Area Legal Services and the Center for Law and Education at Harvard against the SSAT-II as a requirement for high school graduation in May 1979. The Appeals Court upheld the test but delayed the implementation of the SSAT-II as a graduation requirement until the 1982-83 school year. In the meantime, the SSAT-II was used by the DOE to identify students needing remedial assistance. 1980 In the fifth and sixth legal challenges, two groups, representing plaintiffs in Love v. Turlington and Debra P. v. Turlington, filed cases against the state basic skills testing program, alleging that its tests had not been properly validated and that its implementation schedule was unfair. These cases were resolved in 1982 and 1983 respectively. 1981 In April, the SSAT-II was first administered to tenth graders. Moving the test from eleventh to tenth grade provided additional testing opportunities for students. 1982 In October, the Love v. Turlington case was settled out of court. As part of the settlement, the plaintiff was awarded a high school diploma. 1983 In the Debra P. v. Turlington case, the U.S. District Court ruled that the State could deny diplomas to members of the graduating class of 1983 and beyond who did not demonstrate the minimum competencies assessed by the SSAT-II. 1984 Beginning with a March assessment, tenth graders took a new version of the SSAT-II that raised standards to encourage students and teachers to reach higher achievement. The test was based on revised Minimum Student Performance Standards adopted by the State Board of Education. The name of the test was changed to the High School Competency Test (HSCT). Assessment and Accountability Briefing Book Introduction and Chronology 1991 The 1991 School Improvement and Accountability legislation, commonly referred to as Blueprint 2000, established the Florida Commission of Education Reform and Accountability and called for sweeping changes in schools. The intent of the legislation was to ensure higher levels of achievement for all students and more accountability for schools. The legislation also committed the state to rewarding higher performing schools and providing assistance to unsuccessful schools. School boards were required to identify and report on the status of schools not making adequate progress. At the end of the 1990-91 school year, school boards reported 72 schools in 65 districts were not making adequate progress. 1992 In October, the HSCT administration for students was changed from tenth to eleventh grade. Students could continue to retake the HSCT through a thirteenth year as high school students or as adults, as long as they were continuously enrolled. The Florida Writing Assessment Program (FWAP) was administered for the first time to fourth graders in 1992. Students wrote an essay in response to a designated writing prompt. Students were assigned one of two types of prompts, narrative or expository. In 1993, the program was expanded to include eighth-grade students, and in 1994, tenth grade was added. 1995 The Florida Commission on Education Reform and Accountability recommended procedures for assessing student learning in Florida that would raise expectations for students and help them compete for jobs in the global marketplace. The State Board of Education adopted the recommendations, called the Comprehensive Assessment Design (CAD), in June 1995. Also in 1995, the State Board of Education established student achievement criteria and identified critically low schools based on these criteria. The criteria included norm-referenced test (NRT) scores at Grades 4 and 8, writing scores at Grades 4, 8, and 10, and HSCT scores at Grade 11. Schools were identified as being critically low when average scores in all three subject areas were below the minimum criteria for two consecutive years. In November, there were 158 critically low performing schools or 7% of the total number of schools reported. Identifying these schools resulted in a commitment to improving student achievement for all schools and all students. 1996 Florida’s curriculum framework (content standards) were adopted by the State Board of Education for seven subject areas. New legislation (Section 229.565, F.S.) recognized the Performance Standards as the academic standards for Florida students and authorized the Florida Comprehensive Assessment Test (FCAT). In the second year of the accountability program, 71 critically low schools were identified as not meeting the student achievement criteria. The 1995-1996 Performance Status of Schools Report provided results for all schools on these criteria. 1997 Schools not meeting the accountability criteria were identified and reported for the third year. The number of critically low schools decreased to 30, indicating the positive impact school accountability was having on student achievement. The Florida Legislature created, but was unable to fund, a statewide program to recognize schools with exemplary or improved performance. In February, the FCAT was field tested in Grades 4, 5, 8, and 10. Assessment and Accountability Briefing Book 3 Introduction and Chronology 1998 The FCAT was administered for the first time in January to students in Grades 4 (Reading), Grade 5 (Mathematics), and Grades 8 and 10 (Reading & Mathematics). Tests at these grade levels established baseline data and included performance tasks. Achievement levels 1 through 5 were set for FCAT scores. Although the FCAT results were not used for accountability purposes in 1998, school results were reported. Based on the accountability criteria, only four schools were identified as not making adequate progress in 1998, compared to 158 three years earlier. The Florida School Recognition program was funded for the first time at $5.4 million and 140 schools received recognition and monetary rewards. 1999 The Florida Legislature revised several of the statutory requirements for the state assessment program and enacted bold new accountability legislation known as the A+ Plan for Education (CS/HB 751). This legislation increased standards and accountability for students, schools, and educators. The concept of annual learning gains was added to the accountability system with the addition of tests at grades 3 through 10. The revisions also included the addition of a science assessment for students in Grades 5, 8, and 10; a norm-referenced test at grades 3-10: the use of the FCAT for graduation; and the development of a system for calculating the academic growth of each student over a year’s time. Consistent with the new legislation, the State Board of Education identified five school performance levels as letter grades, and the 1999 FCAT results were used to assign school grades. In the first year that school performance grades were issued, 78 schools were designated as “F” schools. Students in two schools that were designated critically low performing in 1998 and received “F” performance grades in 1999 were eligible for and some received opportunity scholarships. Approximately $30 million was disbursed to 319 schools for meeting the school recognition criteria. 2000 The FCAT results for grades 4, 5, 8, and 10 were again used for assigning school grades. All 78 “F” schools from the previous year moved up at least one grade level. Four schools were issued “F” performance grades for the first time in 2000. In the third year of school recognition awards, 1,015 schools received financial incentives for earning an “A” or improving at least one letter grade. 2001 In the third year of issuing school grades, the number of “A” and “B” schools increased from 21% in 1999 to 41% in 2001. In the same time period, “D” and “F” schools decreased from 28% to 12%. There were no “repeat” or new “F” schools in 2001. In addition, 842 schools received school recognition awards for “A” grades or improving by at least one letter grade. In August, the State Board of Education established the FCAT passing scores that students had to earn as one of their requirements for receiving a regular high school diploma. In December, achievement level standards for reading and mathematics were also established by the State Board of Education. Five levels of achievement were identified for FCAT scores in grades 3, 4, 5, 6, 7, and 9 to complement the achievement levels established in 1998 for grades 4, 5, 8, and 10. 4 Assessment and Accountability Briefing Book Introduction and Chronology 2002 During the 2002 special legislative session, Section 1008.25 (5) (b), F.S. was amended to require the retention of third-grade students who failed to achieve above a level one in Reading. For the first time, it was possible to report annual growth scores for FCAT Reading and Mathematics by using a developmental scale. These scores represented achievement across Grades 3-10, with scores ranging from approximately 0 to 3000. The developmental scale score (DSS) and change in this score provide parents and educators with a measure of student learning gains over a year’s time. The use of the DSS growth score was included as part of the A+ Plan school performance grading system for the first time in 2002. Using this new system, a record 1,311 schools earned school recognition awards for earning an “A” or improving a letter grade. However, 64 schools received “F” grades. FCAT Science was added to the battery of tests. In a legal challenge to the law that protected the security of the state mandated assessments, a Pinellas County student who failed the FCAT and his family sued to see the graded booklets. The Leon County Circuit Judge ruled in favor of granting access to the tests; however, the District Court of Appeals overturned the lower court decision. 2003 The sixth administration of the FCAT tests of the SSS in Reading and Mathematics for tenth graders occurred in 2003. The graduating class of 2003 (ninth graders in 1999-2000) were required to attain passing scores on FCAT as one of the requirements for high school graduation. During the 2003 legislative session, Florida Statute 1008.22 (9)(9) permitted some senior high school students to graduate with a regular diploma by substituting designated SAT or ACT scores for the required FCAT scores. Accountability for Educators – A Chronology Florida also has a long history of ensuring that high-quality teachers and administrators are in every classroom across the state because teachers are the key to providing a high quality education for Florida’s students. The state has implemented measures of teacher quality that ensure teacher competency in the basic skills of reading, writing, and mathematics, professional pedagogy, and subject-matter expertise. In addition, assessments for undergraduate students have been implemented to ensure basic skills acquisition of all Florida students seeking professional credentialing. Assessments in all of these areas have been modified over time to ensure that increased expectations for professionals are consistent with increased expectations for public school students. 1980 State legislation, Section 231.087, F.S., established the Florida Council on Educational Management (FCEM). The duties of the FCEM included identifying relevant management competencies of school managers, developing standards and procedures for evaluating manager performance, and specifying procedures for school manager certification, which became the basis for creating the Florida Educational Leadership Examination. All teacher candidates seeking certification in the state of Florida were required to take the Professional Education Examination and the original Florida Teacher Certification Examination (FTCE-O), consisting of Reading, Writing (essay), and Mathematics subtests. Assessment and Accountability Briefing Book 5 Introduction and Chronology 6 1984 For the first time, colleges and universities were required to give a test of basic college-level skills to all students prior to the award of an associate of arts degree and for admission to upper-division status in a state university in Florida. The minimum competencies expected of college students in public institutions in Florida were set forth in Rule 6A-10.0316, FAC. 1986 The Florida Educational Leadership Examination (FELE) was implemented in accordance with Rule 6A-4.00821, FAC. This test was developed for the certification of educational administrators. Also in 1986, legislation (Section 1012.56, F.S.) changed the FTCE basic skills requirement to the Florida College Entry-Level Placement Test (CPT). Beginning July 1, 1988, the CLAST was to be used for testing the basic reading, writing, and mathematics competencies for teacher certification as described in Rule 6A-4.0021 (2)(d), FAC. Teacher candidates applying for initial certification were also required to pass both a Professional Education Examination and a subject area examination for each area of certification sought. 1991 For the first time in Florida, computer-adaptive testing was available to teacher candidates needing to demonstrate the basic skill competencies assessed by the CLAST. Any student or teacher candidate needing to retake subtests in English Language Skills, Reading and/or Mathematics could take advantage of this method. The Essay portion of the CLAST was not available via computer. 1996 Effective in January, Section 240.107 (9) (c), F.S. provided students an exemption for all or parts of the CLAST if they earned a cumulative grade point average of 2.5 on a 4.0 scale in college courses identified by the Postsecondary Planning Commission, and achieved a passing score on the CLAST. The CLAST, however, continued to be required for all teacher certification candidates. 1997 Section 240.107 (9) (c), F.S. was revised to eliminate the CPT as a requirement for exempting the CLAST, and additional mechanisms were instituted for demonstrating mastery of college-level academic skills: minimum scores on the Scholastic Aptitude Test (SAT I) or the American College Test (Enhanced ACT), in lieu of CLAST scores. 2000 Effective in July, Section 1012.56, F.S. was revised to replace the CLAST with the General Knowledge Test to meet basic skills requirements for teacher certification. 2002 The revisions of Section 1012.56, F.S. were implemented. Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) II - The Florida Comprehensive Assessment Test (FCAT) Frequently Asked Questions About the FCAT 1. What is the FCAT? The Florida Comprehensive Assessment Test (FCAT) is part of Florida’s overall plan to increase student achievement by implementing higher standards. The FCAT, administered to students in Grades 3-10, contains two basic components: criterion-referenced tests (CRT) measuring selected benchmarks in Mathematics, Reading, Science, and Writing from the Sunshine State Standards (SSS) and norm-referenced tests (NRT) in Reading and Mathematics measuring individual student performance against national norms. 2. Why do students take the FCAT? The FCAT is given to measure achievement of the Sunshine State Standards. The skills and competencies outlined in the Standards are also embedded in the material of the student’s core classes. The best understanding of a student’s academic achievement comes from looking at multiple pieces of evidence (including FCAT scores) collected over time. 3. What is the legislative authority for the FCAT? In the early 1970’s, the statewide assessment of students in selected grades was authorized. In 1976, the Florida Legislature approved assessments in Grades 3, 5, 8, and 11, including the nation’s first high school graduation test. Since then, the Legislature has continuously supported assessment and evaluation activities in the state’s public school system. The purpose and design of the statewide assessment program is articulated in Section 1008.22, F.S., and the pupil progression plan is in Section 1008.25, F.S. 4. How does the FCAT fit into the A+ Plan for education in Florida? Accountability for student learning is the key focus of Florida’s system of school improvement. Results from the statewide assessment program are the basis of Florida’s system of school improvement and accountability. Student achievement data from the FCAT are used to report educational status and annual progress for individual students, schools, districts, and the State. The A+ school grades are based on the percent of students meeting high standards and the percent of students who make learning gains. 5. Who takes the FCAT? Approximately 1.6 million public school students in Grades 3-10 participated in the 2003 administration of the FCAT. These students, including limited English proficiency students (LEP) and exceptional education students (ESE) with disabilities, are all working toward a regular high school diploma. Opportunity and McKay Scholarship students also take the FCAT and, at the request of their parents, home-schooled students may take the FCAT. Beginning in 2004, Florida Virtual School students also take the FCAT. 6. What does the FCAT cost to administer, score, and report results? The current cost of FCAT is $16.57 per student. This includes the costs for developing test questions, holding review meetings with Florida educators, field testing, production and printing of tests, shipping and return of test materials, scoring, and reporting scores to parents, schools, districts, and the State. The cost of testing at all grades is less than one-third of one percent of the state’s K-12 educational budget. Assessment and Accountability Briefing Book 7 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) 7. How mny times has the FCAT been administered? The FCAT has been administered annually since 1998. FCAT Writing has been administered twelve times to Grade 4 students. The 2004 administration was the seventh test administration at some grade levels, but only the third at others (Grades 3, 6, 7, and 9). The tenth-grade FCAT has been administered three times per year since the 2001-2002 school year. 8. When is the FCAT administered? The tests are administered during two separate time periods. Students take the FCAT Writing assessment in February and take the Reading, Mathematics, and Science FCAT in March. Students needing to retake the Grade 10 FCAT have that opportunity in October, March, and June. 9. How much time does it take to administer the FCAT to a student? The total amount of time required to administer the FCAT depends on the grade level. In general, students spend approximately ten hours over a two-week period taking different parts of the FCAT. 10. What subject areas are measured by the FCAT? The FCAT presently includes Reading, Writing, Mathematics, and Science. In 2003, Science was added for students in Grades 5, 8, and 10. 11. What is the passing score for the FCAT? FCAT scores affect students in Grade 3 Reading (for promotion and retention purposes) and Grade 10 Reading and Mathematics (for high school graduation). Some districts also have required achievement on FCAT as part of their local pupil progression plan adopted by their local School Boards. Students in Grade 10 must earn a scale score of 300 to pass the FCAT Reading and Mathematics. The equivalent developmental scale scores are 1926 on FCAT Reading and 1889 on FCAT Mathematics. Students in Grade 3 must score in Level 2 or higher on FCAT Reading to be promoted. There are also alternate assessments and good cause exemptions that apply to Grade 3 students. 12. How are the FCAT results reported? The FCAT results provide data for each student, school, and district, and for the State. At the school, district, and state levels, the program produces average scores and reports showing the percentage of students performing at the five achievement levels. For each student, achievement levels, scale scores, and developmental scale scores are reported, as well as performance on specific content strands; each student’s norm-referenced scores indicate the student’s ranking against national norms. 13. How are achievement levels and scale scores different from developmental scale scores (DSS)? Reading and Mathematics FCAT student scores are reported in three ways, that is, by achievement level, by scale score, and by developmental scale score. The scale scores, which range from 100 to 500 for each subject area and grade level, are divided into five categories, from 1 (lowest) to 5 (highest), called achievement levels. The developmental scale score was introduced to track student progress over time and across grade levels. New student “growth” reports (started in 2002) include student scores reported as an achievement level, a scale score of 100 to 500, and as developmental scale scores. The FCAT developmental scores range from 0 to 3000 across Grades 3-10. By using FCAT developmental scores, parents can monitor their student’s academic progress from one grade to the next. 8 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) 14. What are some considerations when using developmental scale scores? When using developmental scale scores to determine and interpret student “learning gains” across grade levels, subject areas, and school years, it is important to be aware of the following: • Developmental scores are available only for FCAT Reading and Mathematics. • Learning gains can be determined only for students in Grades 4-10 who have two years of FCAT data. • Developmental score scales typically show larger increases (more student growth) at the lower grade levels and less (student growth) at the higher levels. • Annual “growth” information should be considered within the total context of the student’s annual academic record of achievement. • Some students may show no “learning gains” based on two years of FCAT scores. • Learning gains and achievement level scores across two years are not always concordant because of the different scaling methodologies from which they are derived. 15. Is a passing score on the FCAT Writing assessment a graduation requirement? At the present time, FCAT Writing is not being used to determine eligibility for a high school diploma. 16. Do students have to pass both the Reading and Math sections of the Grade 10 FCAT before they graduate? Yes, but students need to retake only the section they did not pass. Students will have multiple opportunities to retake the FCAT until they earn passing scores. Students may retake the FCAT in June, October, and March during their eleventh- and twelfth-grade years. 17. Is performance on the FCAT required for grade-to-grade promotion? Local school boards establish their own rules for promotion or retention of public school students. Florida statutes specifically mention retention for students who do not read well at the end of Grade 3. Each school board has certain latitude in how it implements this requirement; however, school personnel must change the educational plans of all students who score in level 1. The district Pupil Progression Plan (available at local schools and school board offices) specifically outlines the grade-level requirements. 18. What can you tell me about the new science assessment? In 2003, FCAT Science was given for the first time to all students in Grades 5, 8, and 10. The Science assessment includes life science, physical and chemical science, earth and space science, and scientific critical thinking questions. It is similar in format to the Reading and Mathematics FCAT. 19. Who writes the questions for the FCAT? Professional item writers employed by the DOE’s test-development contractors prepare the first draft of all test questions. Committees of Florida classroom teachers and curriculum supervisors, working with DOE staff, review and revise each test item. Before a test question appears on the FCAT, it is reviewed for community sensitivity and possible bias. From 300 to 400 Florida teachers, administrators, and citizens participate in the FCAT development process annually. Assessment and Accountability Briefing Book 9 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) 20. What are the FCAT questions like? Most of the test questions are multiple choice, but some of the mathematics questions require students to “bubble in” their answers on a grid. Other reading, mathematics and science questions ask for a short, written answer, and a few ask for a more detailed or extended answer. In some parts of the test, students write answers in response to an article or story, to assess how well they understand what they have read. The table below shows the various item formats on FCAT by the subject and grade level assessed. T Y P E S Item Format O F B Y F C A T S U N S H I N E S TAT E S TA N D A R D S Reading Essay MultipleChoice Q U E S T I O N S O N G R A D E L E V E L Writing Mathematics Science 3-10 5, 8, 10 5-10 8, 10 NRT 4, 8, 10 3-10 GriddedResponse ShortResponse 4, 8, 10 5, 8, 10 5, 8, 10 ExtendedResponse 4, 8, 10 5, 8, 10 5, 8, 10 3-10 The reading and mathematics questions are drawn from different subject areas such as Social Studies, Science, Mathematics, Reading, Health/Physical Education, the arts, and the workplace. Many questions include graphics and illustrations to help students decide on the correct answer. The FCAT questions and performance tasks incorporate thinking and problem-solving skills that match the complexity of the Standard being assessed. 21. When are FCAT results returned to districts? The results of Writing, administered in February, and Reading, Mathematics, and Science, administered in March, are usually sent to the districts by early May. Results for students in Grades 3 and 12 are prioritized and returned first. 22. Are the FCAT scores for schools adjusted for the socioeconomic status of students? Schools are responsible for teaching all students, regardless of their socioeconomic status. All students are capable of making adequate learning progress, and all schools are held to equally challenging performance standards. 23. How are the FCAT scores used for school accountability? The A+ school accountability program is designed to offer financial incentives to all Florida schools and to encourage students and teachers to attain higher standards. Providing financial resources is part of the task, of course, but sometimes student learning does not improve even 10 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) though more money has been spent on education. This is why Florida has the A+ Plan and why the federal government has enacted the No Child Left Behind Act of 2001 (NCLB). Students, teachers, and school administrators can improve their performance if they have a clear understanding that their first obligation is academic achievement. 24. What is the Adequate Yearly Progress (AYP) Report? The No Child Left Behind legislation requires all states to report student achievement based on results of reading, mathematics, and writing statewide assessments and, also, high school graduation rates for all schools, districts, and the State. The AYP Report provides a breakdown of achievement test results for major racial groups, economically disadvantaged students, students with disabilities, and students with limited English proficiency. All groups must reach the annual proficiency target for their schools to make Adequate Yearly Progress. 25. Do students with disabilities receive accommodations on the FCAT? Yes, every effort is made to provide a level playing field for students with disabilities taking the FCAT and seeking a standard high school diploma. Section 1007.02 (2), F.S., permits testing accommodations for a student who: • has been assigned to a special program, according to State Board Rule 6A-6.0331, FAC, and • has a current Individual Educational Plan (IEP). Federal law (the Individuals with Disabilities Act of 1997) requires the inclusion of ESE students in regular assessment programs. The school, district, and state FCAT score averages represent all students taking the test, including students with disabilities. Exemption from the graduation test requirement for students with disabilities seeking a high school diploma is described in Section 1003.43 (11) (b), F.S. 26. Do students with limited English proficiency (LEP) take the FCAT? LEP students are expected to take the FCAT. State Board Rule 6A-6.09091, FAC, exempts some LEP students from participating in the statewide assessment program: • if the student has been receiving services in an approved district LEP plan for one year or less, and • if the student’s LEP committee determines that the FCAT is not appropriate. LEP students may take the FCAT using accommodations appropriate for the particular need of the student. It is the responsibility of local school educators to work with students and parents to identify the allowable testing accommodations. 27. What are the legal issues surrounding the FCAT? There are at least three clear legal issues regarding high-stakes tests like the FCAT: (1) Students must have advance notification of the testing requirement. (2) Students must have opportunities to retake the test if they initially earn a failing score. (3) Students must be provided opportunities to learn the skills being tested; therefore, schools must be able to demonstrate that the skills being tested are being taught, using evidence such as lesson plans, texts, and curricular offerings. 28. Are the FCAT scores available on the Internet? Yes. State, district, and school reports are available on the Internet. The website address is: http://www.firn.edu/doe/sas/sasshome.htm. Assessment and Accountability Briefing Book 11 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) The Financial Cost of Assessment1 The statewide assessment program is funded from a combination of general revenue and federal sources. DOE employees direct and monitor the various activities required to implement a largescale assessment program; however, contracts are necessary to implement the program. As the FCAT program has changed and expanded, the cost to provide the associated services has changed as well. The current testing program includes Reading and Mathematics (both CRT and NRT components2), Writing (essay), and Science. Additional details about these components can be found in other sections of this briefing book. Note that when making a transition from one contractor to another, there is overlap in the contracted dates to allow for start-up and close-out activities. Two tables are provided in this section. The first table shows the separate contracting costs for development, administration, scoring, and reporting, as well as per student costs that have impacted the cost of the program since 1995. The second table indicates the program and contract changes during the same period. K - 1 2 T E S T I N G P R O G R A M C O S T S Development and NRT Administration, Scoring, Reporting Total Fiscal Year Cost Cost per K-12 Student 1996* $4,805,687 $2,434,906 $9,675,499 $4.44 1997* $6,900,000 $2,561,594 $12,023,188 $5.37 1998* $5,974,768 $3,210,615 $12,395,998 $5.40 1999* $8,770,666 $2,977,593 $14,725,852 $6.30 2000 $2,779,500 $24,188,657 $28,016,442 $11.79 2001 $5,625,942 $21,749,558 $28,159,278 $11.59 2002 $5,792,768 $31,501,746 $38,076,440 $15.26 2003 $7,115,060 $33,663,006 $41,700,354 $16.45 2004 $10,449,697 $31,824,095 $42,945,838 $16.57 2005 $13,008,010 $30,518,314 $44,061,305 $16.67 (Est.) * Development also included some administration, scoring, & reporting costs. ________________________________ These figures include the cost of the HSCT and the cost of the Florida Writing Assessment before it was included as part of the FCAT. CRT stands for criterion-referenced test and NRT stands for norm-referenced test. The Florida CRT, or the standards-based test, assesses student learning of the Sunshine State Standards. The Florida NRT is the Stanford Achievement Test series published by Harcourt Educational Measurement. 1 2 12 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) P R O G R A M Year 1996 1997 1998 A N D C O N T R A C T C H A N G E S Tested Subjects & Grade Levels Contracts HSCT – Contract with DRC2 ending HSCT – New contract with NCS2 Writing – Contract with NCS2 Read/Math – New contract with CTB2 HSCT – 11 Writing – 4, 8, 10 HSCT – 11 Writing – 4, 8, 10 Reading – 4, 8, 10 (field test) Mathematics – 5, 8, 10 (field test) HSCT – 11 Writing – 4, 8, 10 Reading – 4, 8, 10 Mathematics – 5, 8, 10 HSCT – Contract with NCS Writing – Contract with NCS Read/Math – Contract with CTB HSCT – Contract with NCS Writing – Contract with NCS Read/Math – Contract with CTB 1999 HSCT – 11 Writing – 4, 8, 10 Reading – 4, 8, 10 Mathematics – 5, 8, 10 HSCT – Contract with NCS Writing – Contract with NCS Read/Math – Contract with CTB Read/Math – New contract with HEM4 for program expanded by A+ legislation 2000 HSCT – 11 Writing – 4, 8, 10 Reading – 4, 8, 10 & field test new grades Mathematics – 5, 8, 10 & field test new grades NRT – 3 -10 HSCT – Contract with NCS Read/Math – Contract with HEM Read/Math – New contract with NCS5 2001 HSCT – 11 Writing – 4, 8, 10 Read/Math 3 -10 (SSS & NRT) HSCT – Contract with NCS Read/Math – Contract with HEM Read /Writing/Math – Contract with NCS Science – New contract with NCS6 2002 HSCT – 11 Writing – 4, 8, 10 Read/Math – 3 -10 (SSS & NRT) Science – 5, 8, 10 (field test) FCAT Re-takes begin HSCT – Contract with NCS Read/Math – Contract with HEM Read /Writing/Math – New contract with NCS7 Science – Contract with NCS 2003 HSCT – 11 Writing – 4, 8, 10 Read/Math – 3 -10 (SSS & NRT) & Re-takes Science – 5, 8, 10 2004 HSCT – 11 Writing – 4, 8, 10 Read/Math 3 -10 (SSS & NRT) & Re-takes Science – 5, 8, 10 HSCT – New contract with FSU8 Read/Writing/Math/Science – New contract with HEM9 Read/Writing/Math – Contract with NCS Science – Contract with NCS HSCT – Contract with FSU Read/Writing/Math/Science – Contract with HEM Read/Writing/Math – Contract with NCS Science – Contract with NCS Assessment and Accountability Briefing Book 13 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) 2005 HSCT – 11 Writing – 4, 8, 10 Read/Math 3 -10 (SSS & NRT) & Re-takes Science – 5, 8, 10 HSCT – Contract with FSU Read/Writing/Math/Science – Contract with HEM Read/Writing/Math – Contract with NCS Science – Contract with NCS DRC – Data Recognition Corporation; CTB – CTB McGraw-Hill; HEM – Harcourt Educational Measurement; NCS – NCS Pearson, Inc. The competitive bidding process used by the DOE has helped to keep FCAT as cost effective as possible, considering the scope of the program (SSS and NRT tests in reading and mathematics at eight grade levels and tests in writing and science at three grade levels). If the services provided for the FCAT NRT were purchased using current catalog prices, the cost of that program alone would be approximately $15 per student. Overall, the NRT component accounts for only one-third of the cost of the FCAT program; therefore, the full catalog price for the FCAT would be approximately $30-45 per student. The cost of FCAT at all testing grades is less than one-third of one percent of the state’s K-12 educational budget. ________________________________ This contract included administration, scoring, and reporting of the test. This contract included development administration, scoring, and reporting for three grade levels. 4 This contract included development for eight grade levels and the NRT. 5 This contract included administration and scoring of the reading and mathematics tests for eight grades for writing. 6 This contract included the development, administration, and scoring for three grades. 7 This contract included the administration, scoring, and reporting for all tests. 8 This contract included the administration, scoring, and reporting for as long as students still require the test. 9 This contract included the development of reading, mathematics, science, and writing+ tests for all grades. 2 3 14 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) Educator Involvement in FCAT The FCAT subject area content is defined by the practicing classroom teachers and curriculum specialists who advise the DOE personnel as test items are written, reviewed, and validated. The FCAT includes reading, mathematics, science, and writing assessments, but the test questions are placed in the context of social studies, science, mathematics, reading, the arts, health/physical education, and the workplace, and employ real-life situations to check student skills in the various subject areas. Committees of practicing classroom teachers and curriculum staff review all items, and committees of educational leaders and citizens make recommendations. People meet in structured conferences, inspect test questions, consider the performance of students on the test, and make recommendations as to what they believe should be reasonable standards for students. It takes careful effort and time to develop and implement an assessment program with the complexity and accuracy of FCAT. S T A N D I N G C O M M I T T E E S Rotating membership 1. Reading Content Advisory Committee – This committee is composed of 15–20 Reading and/or Language Arts professionals from schools, school districts, and universities. They advise the DOE about the scope of the Reading assessment. Their recommendations may include which benchmarks should be assessed on FCAT, the item types recommended for each benchmark, the types of reading materials to be used, the range of difficulty for passages to be used on FCAT, and the number of benchmarks, passages, and items to be assessed per grade level. This committee meets once or twice a year. 2. Writing Content Advisory Committee – This committee is composed of 15–20 Language Arts professionals from schools, school districts, and universities. They advise the DOE about the scope of the Writing assessment, including the benchmarks that should be assessed and the item types recommended for each benchmark. In years prior to 2000, this committee was constituted as separate grade-level committees and was used to advise the DOE about the implementation of the Florida Writing Assessment Program. In 2000-2001, the title FCAT Writing was used, and their discussions were broadened to include comprehensive writing assessment topics. This committee meets once or twice a year. 3. Mathematics Content Advisory Committee – This committee is composed of 15–20 Mathematics professionals from schools, school districts, and universities. They advise the Department about the scope of the Mathematics assessment, including the benchmarks that should be assessed and the item types recommended for each benchmark. This committee meets once or twice a year. 4. Science Content Advisory Committee – This committee is composed of 15–20 Science professionals from schools, school districts, and universities. They advise the DOE about the scope of the Science assessment, including the benchmarks that should be assessed and the item types recommended for each benchmark. This committee meets once or twice a year. Assessment and Accountability Briefing Book 15 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) 5. Technical Advisory Committee – This committee is composed of 10–15 professionals with expertise in psychometrics. The members include Florida district test directors, representatives from the FCAT Reading and Mathematics Content Advisory Committees, Florida university faculty members, and representatives of universities and state agencies outside Florida. In addition, the psychometric advisors of the DOE’s contractors participate in the meetings of this committee. Committee members assist the DOE by reviewing technical decisions and documents and by providing advice regarding the approaches the DOE should use to analyze and report FCAT data. This committee meets once or twice a year. A N N U A L A N D A D H O C C O M M I T T E E S Convened periodically; includes previous and new participants 16 1. Prompt Writing Committee – The purpose of this committee is to develop new writing prompts for the fall administration of the FCAT Writing Pilot Test. Participants include language arts teachers from the targeted grade level, school and district curriculum specialists, and university faculty from the discipline area. This committee usually meets during late spring. 2. Prompt Review Committee – This committee reviews the prompts and student responses from the FCAT Writing Pilot Test. The review ensures that prompts selected for FCAT employ clear wording, are of appropriate difficulty and interest level, and are unbiased. The purpose of the committee is to select prompts for the FCAT Writing Field Test. Participants include language arts teachers from the targeted grade level, school and district curriculum specialists, and university faculty from the discipline area. This committee meets in the fall after the Pilot Test. 3. Community Sensitivity Committee – Florida citizens associated with a variety of organizations and institutions review all passages, prompts, and items for issues of potential concern to members of the community at large. This review ensures that the primary purpose of assessing achievement is not undermined by inadvertently including in the test any material that may be deemed inappropriate by parents and other citizens. Reviewers are asked to consider whether the subject matter and language of each reading passage, writing prompt, or test item will be acceptable to Florida students, their parents, and other members of Florida communities. The question posed to each participant is: “Considering the variety of cultural, regional, philosophical, political, and religious backgrounds throughout Florida, will the subject matter and language of this reading passage, writing prompt, or test question be acceptable to Florida students, their parents, and other members of Florida communities?” Participants in these committees include representatives of statewide religious organizations, parent organizations, community-based organizations, and cultural groups (e.g., Hispanic or American Indian), school boards, school district advisory council members, and leaders in business and industry from across the state. Each Community Sensitivity Committee meets once a year. 4. Bias Review Committee – Groups of Florida educators representative of Florida’s regional, racial/ethnic, and cultural diversity review passages, prompts, and items for potential bias. Reviewers look for the following types of bias: gender, racial/ethnic, linguistic, religious, geographic, and socioeconomic. The question posed to each participant about each passage, prompt, or item is: “Might this passage/prompt/item offend or unfairly penalize examinees on the basis of personal characteristics such as gender, race, ethnicity, religion, socioeconomic Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) status, disability, or geographic region?” A test item, prompt, or passage is considered biased if characteristics of the item, unrelated to the skill being measured, result in an unfair advantage or disadvantage for a particular group of students. Participants in these committees include representatives of Florida school districts, universities, and statewide organizations that serve the various groups that are potentially affected by the types of bias described, for example, Title I, English for Speakers of Other Languages (ESOL), and Equal Education Opportunity (EEO). Every attempt is made by the Department to represent the various groups potentially affected by bias at a level well above their representation in the general population. In addition to this professional judgment model, differential item functioning (DIF statistic) is examined for all FCAT items. Each Bias Review Committee meets once a year. 5. Item Content Review Committee – Content reviews are conducted for reading passages and reading, mathematics, and science items to determine whether the passages/items are appropriate for the grade level for which each is proposed. In addition, participants are asked to evaluate whether the items measure the benchmark, are clearly worded, have one and only one correct answer, or are of appropriate difficulty. Participants include teachers from the targeted grade level and subject area, school and district curriculum specialists, and university faculty from the discipline area. The Item Content Review Committees usually meet once a year. 6. Rangefinder Committee – After experimental performance items (short- and extendedresponse) and writing prompts are field tested on FCAT, scoring of a representative set of student responses for each item/prompt is conducted to establish guidelines for the handscoring of all responses. Participants establish the range of student responses that represent each score point of the rubric for each item or prompt. As a result of these meetings, training materials for handscorers are assembled. Participants include teachers from the targeted grade level and subject area, school and district curriculum specialists, and university faculty from the discipline area. The Rangefinder Committees meet after spring testing and prior to handscoring of field test performance items. 7. Rangefinder Review Committee – After performance items and writing prompts are selected for use on FCAT, a scoring and review of a representative set of student responses is conducted to establish guidelines for the handscoring of all responses. Participants discuss and verify the range of student responses that represent each score point of the rubric for each item or prompt. As a result of these meetings, training materials for handscorers are reviewed and revised. Participants include teachers from the targeted grade level and subject area, school and district curriculum specialists, and university faculty from the discipline area. The Rangefinder Review Committees meet in the fall. 8. Gridded-Response Adjudication Committee – A review of all field-test responses to griddedresponse questions is conducted to determine whether all possible correct answers have been included in the scoring key. The various responses are examined and judged as either incorrect or correct. Committee members are asked to evaluate the possibility of finding the answer through an alternate process and determine if resulting answers are acceptable. Based on their advice, the DOE establishes rules for how each gridded-response item will be scored. Participants include teachers from the targeted grade level and subject area and school and district curriculum specialists. Assessment and Accountability Briefing Book 17 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) 9. Standards Setting Committees – From time to time, DOE staff seek the advice of district educators and business/community representatives to recommend achievement standards for FCAT. For example, committees were used to recommend the FCAT Achievement Levels currently in place. For these committees, selection is made from persons familiar with FCAT from prior committee participation and persons who may be unfamiliar with FCAT but have an interest in the standards being established. Participants include teachers from the targeted grade level and subject area, school and district curriculum specialists, school and district administrators, university faculty from the discipline area, and business and community leaders. 10. Special Ad Hoc Committees – On occasion, groups of parents, teachers, school/district administrators, and others are convened to review various aspects of the testing program and to advise the DOE on appropriate courses of action. These committees provide advice on topics such as score reporting, norm-reference tests, and interpretive products. 18 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) FCAT 2002-2003 Educator Involvement 577 participants 64 school districts 38 weeks of meetings 82 meetings WHAT? Bias Committee – 86 Members • Reading & Math • Science • Writing Sensitivity Committee – 51 Members • Reading & Math • Science Reading Committee – 130 Members • Item Review • Passage Review • Item Scoring • Scoring Review Math Committee – 106 Members • Content Advisory • GR Review • Item Content • Item Scoring • Scoring Review Science Committee – 79 Members • Content Advisory • Expert Review • GR Review • Item Review • Item Scoring • Scoring Review Writing Committee – 125 Members • Content Advisory • Prompt Review • Prompt Writing • Item Scoring • Scoring Review FCAT D E V E L O P M E N T Ethnicity • Asian 3% • African Am. 23% • Caucasian 67% • Hispanic 7% • Other 1% Gender • Female • Male WHO? 80% 20% Region • Crown • East Central • Panhandle • South • West Central 14% 20% 23% 24% 19% County Size • Small • Medium • Large 23% 35% 42% County Type • Rural • Urban • Suburban 21% 42% 37% WHERE? Assessment and Accountability Briefing Book 19 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) The Question of “Teaching to the Test” Teaching to the test is not and has never been the intent of the educational accountability program. Teachers should be focusing on incorporating all of the Sunshine State Standards (SSS) in their classroom instruction, not just those benchmarks on the test. The State Board of Education adopted the Standards as policy and, according to Florida law, the Standards must be part of each school’s curriculum. Teachers should never cease instruction in their ordinary lessons to spend time teaching the content required by the FCAT. The skills and competencies outlined in the Standards should be taught throughout the day-to-day instruction of students. For a standards-based test like the FCAT, teaching the Standards assessed is an important aspect of educational accountability. The negative connotation of “teaching to the test” should not be associated with teaching the Standards that are being tested. It is instruction that focuses only on teaching the items on the test that is to be avoided. This is one reason why the Department has not released the specific questions on the FCAT and has only released sample questions. The Standards can and should be taught across the entire curriculum. For example, students taking a history class could be learning and reinforcing reading skills, analyzing the opinions of the author, writing answers to questions posed by the daily lessons, and using mathematics skills to draw graphs of population trends. A well-planned field trip in health education or sociology could involve reading as preparation, discussion and analysis, writing summary reports, and preparing data analyses of observed phenomenon. When teachers incorporate the Standards successfully, student learning and success on the FCAT should follow. Teaching the subject matter required by the Sunshine State Standards and tested by the FCAT is entirely appropriate and desirable. No school should ignore its responsibilities or spend long hours in activities called FCAT preparation. All-around good instruction will provide students the knowledge and skills needed to be successful on the FCAT. Districts are charged with instructional validity responsibilities to ensure that the Standards are being taught in every classroom; however, each teacher has a high degree of autonomy in organizing class content. The state cannot monitor each student’s daily classroom instruction, although every effort is made to ensure that teachers introduce academic information at the appropriate time during the school year. If parents have concerns about the curriculum taught at their child’s school, they should offer their opinions to the educators and school board members in their district. All districts have had the opportunity to review and influence the Standards content. 20 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) Reporting Student Test Results FCAT results are reported at the individual student, school, district, and state levels. Various combinations of summary reports are provided at the school and district levels to assist local educators in determining the performance of their students. Reading and Mathematics performance is reported in three ways, that is, by achievement levels, by scale scores, and by developmental scale scores. (See Appendix A for 2003 FCAT results.) A C H I E V E M E N T L E V E L S C O R E S Achievement levels for each subject area and at each grade level are reported on a scale of 1 (lowest) to 5 (highest) and describe the success a student has attained on the Florida Sunshine State Standards tested on the FCAT. The five levels (defined by cut-scores on a 100-500 scale) were adopted by the State Board of Education. There are no scale scores lower than 100. Level I performance was designated by the Commissioner of Education to be low enough to question the student’s academic progress. The state average score was about 300 when the test was first administered in 1998. Since then, the average score has fluctuated as students have been tested. The levels do not indicate passing scores. The table below lists achievement levels for FCAT SSS Reading and Mathematics, along with the scale score ranges associated with each achievement level and grade. The achievement levels for FCAT SSS Science will be determined in the future. FCAT ACHIEVEMENT LEVELS Reading Mathematics Level 1 Level 2 Level 3 Level 4 Level 5 Grade Level 1 Level 2 Level 3 Level 4 Level 5 100-258 259-283 284-331 332-393 394-500 3 100-252 253-293 294-345 346-397 398-500 100-274 275-298 299-338 339-385 386-500 4 100-259 260-297 298-346 347-393 394-500 100-255 256-285 286-330 331-383 384-500 5 100-287 288-325 326-354 355-394 395-500 100-264 265-295 296-338 339-386 387-500 6 100-282 283-314 315-353 354-390 391-500 100-266 267-299 300-343 344-388 389-500 7 100-274 275-305 306-343 344-378 379-500 100-270 271-309 310-349 350-393 394-500 8 100-279 280-309 310-346 347-370 371-500 100-284 285-321 322-353 354-381 382-500 9 100-260 261-295 296-331 332-366 367-500 100-286 287-326 327-354 355-371 372-500 10 100-286 287-314 315-339 340-374 375-500 Assessment and Accountability Briefing Book 21 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) D E V E L O P M E N T A L S C A L E S C O R E S A different measurement, called a developmental scale score (DSS), is used to understand whether a student has “gained” in achievement. For the individual student reports (started in 2002), student scores are linked to the score scale of 100 to 500 and converted to scores on the developmental scales. The FCAT developmental scores range from 0 to about 3000 across Grades 3-10 and link two years of student FCAT data that track student progress over time. By using FCAT developmental scores, parents and educators can assess changes in scores across years and monitor the student’s academic progress from one grade to the next. Each year, student scores should increase according to the student’s increased achievement. In comparing student “gains” across grade levels, subject areas, and school years, it is important to consider the following limitations of the developmental scale: • Developmental scores are available to students in Grades 3-10, who have two years of FCAT data; Grade 3 students tested in 2003 will not have annual learning gains because there are not two years of FCAT data for those students. • Developmental score scales typically show larger increases (more student growth) at the lower grade levels and less student growth at the higher levels. • Student FCAT data reflect only one year of FCAT “growth” information, which should be considered within the total context of the student’s annual academic record of achievement. • Some students may show no “growth” based on their two years of FCAT scores. The table below lists the FCAT developmental scale scores for each achievement level. FCAT ACHIEVEMENT LEVELS FOR THE DEVELOPMENTAL SCALE Reading Mathematics Level 1 Level 2 Level 3 Level 4 Level 5 Grade Level 1 Level 2 Level 3 Level 4 Level 5 86-1045 1046-1197 1198-1488 1489-1865 1866-2514 3 375-1078 1079-1268 1269-1508 1509-1749 1750-2225 295-1314 1315-1455 1456-1689 1690-1964 1965-2638 4 581-1276 1277-1443 1444-1657 1658-1862 1863-2330 474-1341 1342-1509 1510-1761 1762-2058 2059-2713 5 569-1451 1452-1631 1632-1768 1769-1956 1957-2456 539-1449 1450-1621 1622-1859 1860-2125 2126-2758 6 770-1553 1554-1691 1692-1859 1860-2018 2019-2492 671-1541 1542-1714 1715-1944 1945-2180 2181-2767 7 958-1660 1661-1785 1786-1938 1939-2079 2080-2572 886-1695 1696-1881 1882-2072 2073-2281 2282-2790 8 1025-1732 1733-1850 1851-1997 1998-2091 2092-2605 772-1771 1772-1971 1972-2145 2146-2297 2298-2943 9 1238-1781 1782-1900 1901-2022 2023-2141 2142-2596 844-1851 1852-2067 2068-2218 2219-2310 2311-3008 10 1068-1831 1832-1946 1947-2049 2050-2192 2193-2709 Additional information concerning student academic “growth” at each grade level in Mathematics and Reading based on FCAT scores is available on the Florida Department of Education website at http://www.firn.edu/doe/sas/fcat/fcat_score/index.htm. 22 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) G R A D U A T I O N R E Q U I R E M E N T Passing scores on the Grade 10 FCAT are required for high school graduation. In 2001, the State Board of Education adopted administrative rule 6A-1.09422 that specified passing scores on the Reading and Mathematics FCAT. The Board acted on recommendations from the Commissioner of Education that provided for input from the education community as well as from citizens throughout the state. As a result, students who expect to graduate from high school must earn passing scores on the Reading and Mathematics Sunshine State Standards portion of the Grade 10 FCAT. The current Grade 10 passing scores, as determined by the State Board of Education, are as follows: FCAT SSS Reading Test FCAT SSS Mathematics Test 300 (developmental scale score of 1926) or above 300 (developmental scale score of 1889) or above Passing scores of 287 (developmental scale score of 1856) or higher on the FCAT SSS Reading test and 295 (developmental scale score of 1868) or higher on the FCAT SSS Mathematics test were required of seniors in the graduating class of 2002-2003 who were sophomores in 2001. Performance on FCAT is not the sole criteria in determining eligibility for graduation. The law is very specific in that no student can receive a standard high school diploma from a Florida public school unless that student has met all academic requirements. Students must take required courses, earn the requisite number of credits, maintain a grade point average of 2.0, and pass Grade 10 Reading and Mathematics FCAT before graduating. If students do not earn passing scores on the FCAT, they have additional opportunities to take the test. The Grade 10 FCAT is administered in October, March, and June to 11th and 12th graders who have not passed. Most students in Grades 10 through 12 have six opportunities to pass the FCAT. Assessment and Accountability Briefing Book 23 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) Reliability and Validity of FCAT The Florida Comprehensive Assessment Test (FCAT) was designed to assess student achievement of the Sunshine State Standards (SSS). The test meets all professional standards of psychometric quality traditionally associated with standardized achievement tests. Two constructs that are generally used to indicate the quality of a standardized test are reliability and validity. In this section several measures of the technical quality of FCAT show that scores from FCAT are both reliable and valid. More detailed technical information than presented here is available from the Florida Department of Education upon request. Test Description The FCAT is used to assess the levels of students’ knowledge and skill in Reading and Mathematics at Grades 3-10. Two types of scale scores are reported on FCAT: (1) scale scores for each grade level (100-500 points), and (2) developmental scale scores (DSS) that span all grade levels (0-3000 points). In addition, five levels of proficiency are reported. Level 1 is low and level 5 is high. High school students must attain a scale score of 300 on the Grade 10 FCAT in order to satisfy the testing component of the graduation requirements. At Grade 3, students must attain a satisfactory reading score before being promoted to Grade 4 without good cause. Initially students must attain a score above level 1; however, alternate methods of demonstrating reading skill are also provided. R E L I A B I L I T Y Reliability can be represented in several ways, but the concept essentially means that the test provides consistent measurement of an examinee’s knowledge. Reliability measures help users generalize student performances from one time to another. Four kinds of reliability coefficients can be used in relation to FCAT: (a) internal consistency (b)test-retest reliability (c) inter-rater reliability (d)reliability of classifications For any of the four types of reliability, the coefficient is expressed as a number from zero to one (0.0-1.00). A value of zero indicates a lack of reliability that results in inconsistent scores from one test administration to the next. On the other hand, a value of one indicates perfect consistency. The most common measure of reliability is the internal consistency reliability coefficient. Test-retest reliability requires a special study where students take FCAT twice in a very short period of time. Because the internal consistency measures of reliability have been shown to produce similar results using only one administration per student, this method is used for ongoing testing programs. Summaries of inter-rater reliability and the reliability of classifications are provided in the FCAT technical reports. Internal consistency reliabilities for FCAT are reported using two methods: Cronbach’s Alpha and Item Response Theory (IRT) marginal reliabilities. Cronbach’s Alpha coefficients are reported for the FCAT-SSS tests and for the FCAT-NRT (KR-20 is used) found in Table 1. The IRT marginal reli24 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) abilities are available only for the FCAT-SSS and are found in Table 2. Both of these methods are used to estimate the reliability of test scores from a single test. Cronbach’s Alpha (and the KR-20) coefficients are based on classical test theory. The KR-20 formula is used with tests that contain only dichotomously scored items (right or wrong). Some performance items on the FCAT are scored on a scale from 0-4; therefore, Cronbach’s Alpha is the more appropriate statistic for the FCAT-SSS test. Table 1 shows FCAT reliability coefficients using Cronbach’s Alpha for the FCAT-SSS component and KR-20 for the NRT component. The FCAT-NRT reliability coefficients are those reported by the test publisher. The data in Table 1 confirms that FCAT is a highly reliable test for assessing the educational achievement of Florida students. C L A S S I C A L T A B L E 1 R E L I A B I L I T Y READING O F F C A T MATHEMATICS Cronbach’s Alpha – SSS KR-20 Cronbach’s Alpha - SSS KR-20 2001 2002 2003 NRT1 2001 2002 2003 NRT1 3 .91 .91 .91 .94 .89 .89 .88 .90 4 .90 .90 .90 .93 .89 .89 .88 .90 5 .88 .87 .90 .93 .92 .92 .92 .90 6 .91 .89 .89 .92 .87 .88 .87 .90 7 .92 .91 .91 .93 .90 .88 .89 .90 8 .90 .89 .89 .94 .92 .93 .93 .91 9 .91 .87 .89 .94 .92 .91 .89 .87 10 .89 .88 .88 .93 .93 .92 .92 .88 I R T M A R G I N A L ( R T A B L E 2 ) R E L I A B I L I T Y I J READING-SSS O F F C A T MATHEMATICS-SSS 2001 2002 2003 2001 2002 2003 3 .88 .90 .91 .88 .89 .88 4 .91 .89 .91 .88 .89 .88 5 .89 .87 .90 .94 .93 .93 6 .90 .89 .90 .88 .89 .87 7 .90 .90 .91 .90 .88 .89 8 .91 .87 .90 .94 .93 .93 9 .90 .88 .89 .91 .91 .90 10 .90 .89 .88 .94 .93 .92 _____________________________________________________ 1 KR-20 data are found in the technical materials for the Stanford 9, published by Harcourt Educational Measurement. Assessment and Accountability Briefing Book 25 T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) The data in Table 2 provide additional confirmation that the FCAT is a highly reliable test. In IRT, marginal reliabilities are used to represent the variability of test scores for a specific group of examinees. These marginal reliabilities estimate the standard error of measurement (SEM) for the test and can be interpreted in the same way as Cronbach’s Alpha. Table 2 shows the reliabilities using the average SEM for all students. V A L I D I T Y We usually do not talk about a test having validity. Instead, we say that interpretations of test scores are valid. The test score tells us something about the student, and we want those interpretations to be valid. In general, validity refers to the extent to which the test measures the characteristic it is supposed to measure. FCAT is intended to measure a student’s achievement of the skills and content described in the Sunshine State Standards. Validity cannot be directly observed; therefore, we depend on various pieces of evidence that indicate the presence or absence of validity. The types of validity evidence are often grouped into these three interrelated categories: (a) content-related evidence (b) criterion-related evidence (c) construct-related evidence Content validity evidence refers to the degree to which an assessment reflects the content it was designed to assess. The American Psychological Association (APA) standards for Educational and Psychological Testing (1985) stated that: Content-related evidence of validity is a central concern during test development…. Expert professional judgment should play an integral part in developing the definition of what is to be measured, such as describing the universe of content, generating and selecting the content sample. And specifying the item format and scoring system. (p. 11) The FCAT is designed to assess Sunshine State Standards (SSS) that were developed with involvement of instructional specialists. Annual reports of participation of Florida educators in this process are available upon request. To ensure high content validity of FCAT, the Department of Education has implemented the following steps for all of the items included on FCAT: • Educators and citizens judged the standards and skills acceptable. • Item specifications were written. • Test items were written according to the guidelines provided by the item specifications. • The items were pilot tested using randomly selected groups of students at appropriate grade levels. • All items were reviewed for cultural, ethnic, language, and gender bias and for issues of general concern to Florida citizens. • Instructional specialists and practicing teachers reviewed the items. • The items were field tested to determine their psychometric properties. • The tests were carefully constructed with items that met specific psychometric standards. • The constructed tests were equated to the base test to match both content coverage and test statistics. 26 Assessment and Accountability Briefing Book T h e F l o r i d a C o m p r e h e n s i v e A s s e s s m e n t Te s t ( F C AT ) Because FCAT assesses the content of the SSS and is developed using credible and trustworthy methods, the content validity of the test is substantiated. Evidence of criterion-related validity is presented as the correlation of one test with a criterion. Criterion validity usually is presented as either concurrent evidence or predictive evidence. Concurrent validity refers to the comparisons of test performance with an external criterion that is obtained at relatively the same time as the administration of the test. Predictive validity compares test performances with an external criterion that is obtained at some point in the future. Concurrent validity is more relevant for FCAT than predictive validity and can be examined by the correlation of scores on the criterion-referenced portion (SSS) with scores on the norm-referenced portion (Stanford 9). Both components of the FCAT are administered at approximately the same time. The data presented in Table 3 confirm that the FCAT demonstrates concurrent validity with the Stanford 9 test; however, the validity coefficients do not indicate that the tests provide exactly the same information. T A B L E 3 C O R R E L A T I O N S B E T W E E N T H E A N D N R T T E S T S READING F C A T S S S MATHEMATICS 2001 2002 2003 2001 2002 2003 3 0.84 0.84 0.85 0.85 0.84 0.84 4 0.80 0.83 0.82 0.83 0.82 0.81 5 0.84 0.84 0.84 0.82 0.84 0.83 6 0.83 0.84 0.83 0.84 0.84 0.82 7 0.83 0.82 0.82 0.84 0.83 0.83 8 0.82 0.83 0.82 0.81 0.82 0.83 9 0.82 0.81 0.82 0.82 0.80 0.81 10 0.80 0.80 0.78 0.79 0.77 0.76 Construct-related evidence of validity is the degree to which the test measures the skills intended to be measured. Confirmatory and explanatory factor analysis, and correlational methods are often used to evaluate construct validity. Another approach to establishing construct validity is to conduct convergent and discriminant analyses. FCAT technical reports present detailed information regarding these types of validity and provide evidence that both FCAT-SSS reading and mathematics tests have substantial convergent validity. Summary The evidence of reliability and validity supports the claim that FCAT is technically sound and meets or exceeds the professional standards for standardized achievement tests. Assessment and Accountability Briefing Book 27 28 Assessment and Accountability Briefing Book School Accountability III - School Accountability in Florida Florida’s current school accountability system is being implemented according to the guidelines provided in the 1999 A+ legislation and the federal No Child Left Behind Act of 2001. Prior to 1999, the Florida school accountability system was called “Critically Low Schools.” Each of these systems is described briefly in this section. Critically Low Schools – 1995 In November 1995, Florida identified 158 schools as having critically low student performance. These schools had low student performance for two years in a row in all three areas tested, Reading, Writing, and Mathematics. After schools were identified as low performing, they received focused technical assistance and additional resources from the district and State. Student performance at these schools improved because of these additional efforts; however, most schools did not remain on the list because of their improvement in Writing. In the first year school grades were assigned, only four of the original 158 critically low-performing schools received a grade of “C” or higher, but by 2003, the number receiving a “C” or higher had increased to 88. A+ School Grades – 1999 In 1999, the Florida Legislature passed the Bush/Brogan A+ plan (Section 1008.34 F.S.). The legislation required increased public accountability for schools in the form of school ratings from “A” to ”F” and included several other important features. • Students achievement—Schools in Florida must be child centered, not school or district centered. Educational programs should revolve around the individual needs of each child. An academic improvement plan was required for all students with low scores on the Florida Comprehensive Assessment Test (FCAT). • Student gains—The FCAT was expanded to include standards-based and norm-referenced assessments of Reading and Mathematics all Grades 3-10. Subsequently, the standardsbased tests were designed to accurately assess annual student learning gains based on the benchmarks in the Sunshine State Standards. • Choices for parents—School choice and opportunity scholarships were required for students in schools receiving a grade of “F” in two of four consecutive years. • Resources for low performing schools—Schools were required to implement comprehensive school improvement plans to help students reach the goals set forth in the A+ Plan. The school district and the Department provided additional funds and assistance to “D” and “F” schools. • Rewards for improvement and success—Funds were made available for successful schools earning a grade of “A” or improving a letter grade from one year to the next. • Program changes for students—When students do not make progress during the year, schools are required to offer an educational program that is different from the educational program students received in the current year. These changes are documented in an academic improvement plan. Assessment and Accountability Briefing Book 29 School Accountability The A+ Plan called for a transition to student learning gains by the 2001-2002 school year. At that time, Florida implemented a “value-added” system that tracks individual student progress across grades 3-10. After meetings across the state and input from teachers, principals, and education groups such as the Florida PTA, the Florida School Board Association, the Florida Superintendents Association, and community leaders, the State Board of Education unanimously approved the components of the school grading system in December 2001. For the first time, in 2002, Florida’s school grading system contained the use of annual learning gains. This was an original and integral component of the Bush/Brogan A+ Plan and was enthusiastically embraced by educators and parents alike. Three years of research and development contributed to reaching this goal. With new methods available for determining student learning gains, school grades could be based on both the progress of students from one year to the next and their achievement of high academic standards. The school grading system begun in 2002 also emphasized Reading in half of its six components. Schools earn points for each of the six components according to the percent of students who attain each of the criteria: 1. Reading – Meet challenging content standards on FCAT (> Level 3) 2. Mathematics – Meet challenging content standards on FCAT (> Level 3) 3. Writing – Meet challenging content standards on FCAT (> Level 3/3.5) 4. Reading – Show annual learning gains 5. Mathematics – Show annual learning gains 6. Reading of the lowest 25% of students – Show annual learning gains Raising the Bar for A+ – 2003 On November 18, 2003, the State Board of Education amended Rule 6A-1.09981, “Implementation of Florida’s System of School Improvement and Accountability,” and Rule 6A-1.09422, “Florida Comprehensive Assessment Test Requirements.” The changes to these rules raised the bar for student and school performance in requirements of the school grading system and for maintaining high achievement level standards on FCAT. Changes effective for the 2003-04 school year • All new schools will be graded in their first year as long as the school has at least 30 eligible students with valid FCAT assessment scores in Reading and at least 30 eligible students with valid FCAT assessment scores in Mathematics for both the current and previous years. • Schools will receive a school letter grade based on guidelines issued by the Commissioner. The district accountability coordinator is responsible for verifying that each school is appropriately classified before school grades are issued. • District average writing scores for all students will be used as the school writing score when a school does not contain the grade level assessed by writing (Grades 4, 8, or 10) or when fewer than 30 students are tested. • Scores of the lowest performing students will be used when fewer than 30 students are among the lowest scoring 25%. Scores for all students at or below the identified cut point will be included. To be included in the lowest performing group, students must have scored at or below FCAT level 3 in the previous year. • The Commissioner can assign a lower letter grade than represented in the points earned when the percent of students tested is less than 90%. 30 Assessment and Accountability Briefing Book School Accountability Changes effective for 2004-05 and 2005-06 • The criterion score for FCAT Writing will increase to 3.5. • All students will be included in determining learning gains. Students with disabilities, limited English proficient (LEP) students, and standard curriculum students will all be included in the components addressing annual learning gains. The three components addressing challenging content standards will continue to include only standard curriculum students. Changes effective for 2006-07 and beyond • The criterion score for FCAT Writing will increase to 4.0. • Science will be added to the school grading calculation. Science will be the seventh category for calculating school grades, and the grading scale will be adjusted. The category will be achieving challenging content standards in Science for grades 5, 8, and 10. Science will not be used to assess annual learning gains. No Child Left Behind & Adequate Yearly Progress The Federal No Child Left Behind Act of 2001 requires states to evaluate the performance of all students in all public schools in order to determine whether schools, school districts, and the State have made adequate yearly progress (AYP). Florida’s approved accountability plan uses the same FCAT scores and definitions of “grade level” used in the A+ Plan. In addition, the plan identifies the specific criteria for determining and reporting AYP for all schools. Not making AYP means that a school has not met the proficiency criterion or the percent passing standard for at least one group of students. The measures included in determining AYP are reading and mathematics proficiency, writing score and graduation rate improvement, and whether or not schools tested enough students in each group (see also the section that follows called AYP Criteria). Although all schools in the state are identified as making or not making AYP, only Title I schools not making AYP in two consecutive years will be identified as “schools in need of improvement.” For the 2003-04 school year, the “schools in need of improvement” were the Title I schools that received a performance grade of “F” for the 2001-02 school year and that did not make AYP for the 2002-03 school year. Students attending these schools were eligible for public school choice options for the 2003-04 school year. Title I schools that do not make AYP for more than two consecutive years are required to provide supplemental educational services to students and to implement strategies for improving school performance. All of the criteria for AYP must be evaluated for nine separate groups of students: all students in the school, White, Black, Hispanic, Asian, and American Indian students, economically disadvantaged students, students with limited English proficiency, and students with disabilities. To make AYP schools must: • Test 95% of the students on the statewide assessment (FCAT) or via an alternate assessment method. • Meet the reading proficiency target2 (31% of students scoring at Level 3 or above on FCAT or the alternate assessment). Assessment and Accountability Briefing Book 31 School Accountability • Meet the mathematics proficiency target2 (38% of students scoring at Level 3 or above on FCAT or the alternate assessment). • Improve at least 1% in the percentage of students scoring 3 and above on FCAT Writing • Improve at least 1% in the graduation rate for high schools. Safe Harbor Provisions: Schools or districts with subgroups that do not meet the annual objectives for Reading or Mathematics can meet AYP by reducing the proportion of non-proficient students in that subgroup by at least 10 percent from the prior year. Schools rated as a “D” or “F” under the A+ school grading system do not make AYP. ______________________________________ 1 Schools must have more than 10 students in the “all students” group for inclusion in the AYP calculation. Subgroups must have 30 or more students for inclusion in the AYP calculation. 2 Annual proficiency targets are described in Florida’s No Child Left Behind accountability plan. The annual targets are two- and three-year incremental steps toward the goal of having 100% of all students proficient by 2013-14. These annual targets identify the percentage of students who must be proficient in reading and mathematics. 32 Assessment and Accountability Briefing Book Other K-12 Assessments IV - Other K-12 Assessments The Reading Diagnostic Program The Florida Legislature has identified Reading as a subject of great importance for elementary children. Section 1008.22 (formerly 229.57), F.S., specifically indicates the need for students to develop proficiency in Reading before entering the fourth grade. In an effort to help school districts increase student achievement in Reading, the Florida Department of Education has implemented the Reading Diagnostic Program. The Reading Diagnostic Program provides reading assessment materials to all Florida public school districts for students in Grades K-3. Districts may currently choose from three diagnostic instruments for each grade level. These materials are intended to be used by teachers to assess student pre-reading and early reading skills and to plan appropriate instruction that addresses deficiencies for the lowest twenty-fifth percentile students. These assessments measure student skills in the areas of (1) phonological awareness, (2) word recognition, (3) oral reading accuracy, and (4) comprehension of text. Early Reading Diagnostic Assessment (ERDA) – The ERDA is a battery of tests that evaluates the reading skills of students in Grades K-3. The tests measure a range of readiness and reading skills (phonological awareness, alphabetic principles, word recognition, oral reading accuracy, and comprehension of text) and provide classroom teachers with an assessment tool to determine student reading strengths and weaknesses. This assessment is offered for use in the public school system by the DOE and The Psychological Corporation. Diagnostic Assessment of Reading (DAR) – The DAR is an assessment tool that can be used to evaluate student pre-reading skills through Grade 12; however, the DOE provides this instrument to districts for students in Grades 2-3. Most students will be assessed on word recognition, word analysis, oral reading, spelling, and word meaning. This assessment is offered for use in the public school system by the DOE and Riverside Publishing. Fox in a Box – The Fox in a Box is an assessment approved as a reading diagnostic tool in Florida schools for students in Grades K-3. The learning strands in this assessment include the Florida requirements of phonological awareness, word recognition, oral reading accuracy, and comprehension of text. CTB McGraw-Hill is the publisher for this product. Assessment and Accountability Briefing Book 33 Other K-12 Assessments The School Readiness Uniform Screening System (SRUSS) The School Readiness Uniform Screening System (SRUSS) is intended to gather information about the readiness of all public school children as they enter kindergarten. Section 411.01(10), Florida Statutes (F.S.), entitled “School Readiness Uniform Screening,” required that the Department of Education implement the SRUSS beginning with the 2002-03 school year. The corresponding requirement in school law may be found in Section 1008.21, F.S., (formerly Section 229.567, F.S.), entitled, “School readiness uniform screening (kindergarten).” Public school system educators are required to administer the SRUSS, Early Screening Inventory – Kindergarten (ESI-K), to students entering kindergarten in the fall 2002-03 school year. Additionally, the law requires the Department to submit school readiness results to the K-20 data warehouse for the purpose of longitudinal tracking. The ESI-K is a developmental screening instrument that takes approximately 20 minutes to administer and gives a quick overview of a student’s development in three areas: visual motor/adaptive, language and cognition, and gross motor skills. The ESI-K uses three categories to describe screening results: “Ok,” “Rescreen,” and “Refer.” The information derived from the screening system serves two important purposes. It will assist teachers in understanding the abilities of children entering school and in planning classroom instruction to help children become successful learners. Screening information may also be helpful in improving school readiness programs and services for children prior to school-entry age; however, the results of the screening cannot be used to serve as a barrier to students entering school, nor should results be used for the purpose of “ability grouping” of children. Data from screening instruments provide information about a child that can be shared with parents. Significant decisions that influence educational services for any child should be based on multiple sources of information obtained over time. Before 2002-2003, school districts completed a “School Readiness Checklist” for students entering kindergarten. Based on the results of the checklist, a determination was made regarding whether the child was “ready” for kindergarten. Since the SRUSS replaces the previous “School Readiness Checklist,” school districts are no longer required to complete a data element for the student database that indicates whether the child is “ready” for school. 34 Assessment and Accountability Briefing Book Other K-12 Assessments College Board’s Preliminary Scholastic Aptitude Test (PSAT) and ACT’s Educational Planning and Assessment System (PLAN) Schools are being strongly encouraged to offer and promote rigorous course work to help increase students’ skills necessary for success in postsecondary education and the workforce. As a result, there has been more interest in identifying students who are likely to be successful in rigorous college preparatory courses. The State of Florida entered into a partnership with the College Board and ACT, Inc. to provide free Preliminary Scholastic Aptitude Test (PSAT) or PLAN testing for interested tenth-grade students. Florida Statute 1008.22 and a directive of the 2003 Florida Legislature in Specific Appropriation 109 provided $1,600,000 to the statewide administration of the PSAT and PLAN. Funding is allocated each year to the Florida Department of Education’s Office of Assessment and School Performance via the Governor’s One Florida Initiative that pays for the PSAT and PLAN testing for all 72 public school districts. The Office of Equity and Access personnel at Florida Department of Education work with school district personnel to encourage 100% participation of Grade 10 students in this program. High school sophomores are at a particularly important transitional stage in their high school experience. The College Board’s PSAT and ACT’s PLAN assessment are excellent practice opportunities for students wishing to take the SAT and ACT. The PSAT and PLAN have been found to be good predictors of student achievement on the SAT and ACT assessments. The diagnostic reports schools and parents receive from the PSAT/PLAN tests provide useful information to assist in determining student readiness for Advanced Placement and/or Level 3 courses. Level 3 courses include honors, international baccalaureate, advanced placement, and other college-preparatory classes. The PSAT and PLAN, typically given in Grade 10, serve as a midpoint review of a student’s academic progress. The PLAN assessment covers four skill areas: English, mathematics, reading, and science. The PSAT measures critical reading, math problem-solving, and writing skills that students have developed throughout their primary and secondary educational careers. PSAT and PLAN results help tenth graders build a solid foundation for future academic and career success and provide information needed to address school districts’ high-priority issues. Most importantly, scores on the PLAN and PSAT help students make the most of their remaining years in high school and guide them in planning for their post-graduation transition to further education or the workplace. These tests give students first-hand practice for the ACT and SAT and the chance to qualify for scholarship and recognition programs. Students taking the PSAT and PLAN come to the attention of colleges and universities, if the students so chooses. PSAT and PLAN results can help all students—those who are college-bound as well as those who are likely to enter the workforce directly after high school. Assessment and Accountability Briefing Book 35 36 Assessment and Accountability Briefing Book F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s V - Certification Examinations for Florida Educators Florida Teacher Certification Examinations (FTCE) Florida law requires that teachers demonstrate mastery of basic skills, professional knowledge, and content-area specialization. Testing requirements for teacher candidates seeking certifications are described in Section 1012.56, Florida Statutes (F.S.) and in Rule 6A-4.0021, Florida Administrative Code (FAC). History of the FTCE Early Teacher Certification Exam Program Beginning in 1980, all teacher candidates seeking certification in the state of Florida were required to take the original Florida Teacher Certification Examination, which was known as the FTCE-O, and the Professional Education Examination. The FTCE-O included a basic skills component that consisted of writing an essay and multiple-choice tests in Reading and Mathematics. The professional educator component was a multiple-choice test assessing general knowledge in five areas: human development, appropriate student behavior, planning instruction, implementing instruction, and evaluating instruction. All candidates for new certificates (academic, degreed vocational, and Occupational Specialist) were required to pass both tests. The original basic skills test has been phased out and is no longer an accepted measure of basic skills for teacher candidates. Adding CLAST as the Basic Skills Test In 1986, legislation was enacted to amend the FTCE basic skills requirement (Section 1012.56, F.S.). Under this legislation and beginning July 1, 1988, the College-Level Academic Skills Test (CLAST) was used for testing the basic Reading, Writing, and Mathematics competencies of teacher candidates. See Rule 6A-4.0021 (2)(d), FAC. The CLAST is a basic skills test administered to students in public community colleges and universities. (Many private institutions also require their students to take and pass the CLAST.) To become certified on or after July 1, 1988, teacher candidates had to receive passing scores on all four subtests of the CLAST. Because the CLAST is currently required for education majors in state universities, teacher candidates who need to take the test are mostly from out of state or graduates from Florida’s private schools. Adding Subject Area Examinations Also included in the 1986 legislation (Section 1012.56, F.S.) was a requirement for subject area examinations. These tests assess knowledge of specific content areas and have been required for teacher certification since 1988. During the early stages of development, committees of educators from throughout the state reviewed existing commercial tests and determined that the tests available were not sufficient to meet the Florida legislative mandate. At this point the Florida Department of Education established contracts with Florida universities for the development of the exams. The universities included Florida State University, the University of Central Florida, the University of Florida, the University of South Florida, and the University of West Florida. Assessment and Accountability Briefing Book 37 F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s Current Testing Requirements for Teacher Certification In 2000, the Florida Legislature mandated the creation of a new basic skill test (Section 1012.56, F.S.). Effective July 1, 2002, teacher candidates applying for new certificates in Florida are required to pass the General Knowledge Test, instead of the CLAST. Candidates must also pass the Professional Education Test and a subject area test in each content area for which certification is sought. General Knowledge Test The General Knowledge Test is the required basic skills test teacher candidates must pass in order to become certified. The General Knowledge Test is aligned to the content standards Florida teachers must teach and for which schools are accountable, the Sunshine State Standards. Like the CLAST, the General Knowledge Test contains four subtests: multiple-choice tests of English Language Skills, Reading, and Mathematics, and a written Essay. The Essay subtest measures general writing skills, such as formulating a thesis statement, providing supporting details, using a logical organizational pattern, and following conventions in word choice, sentence structure, grammar, spelling, capitalization, and punctuation. The English Language Skills subtest measures knowledge of conceptual and organizational skills, word choice, sentence structure, grammar, spelling, capitalization, and punctuation. The Reading subtest includes passages with questions measuring literal and inferential comprehension. The Mathematics subtest includes number sense, concepts, and organization; measurement; geometry; algebra; and data analysis and probability. G E N E R A L K N O W L E D G E T E S T Subtest Item Format Items Passing Score≤ Mathematics Multiple-choice 45 200 (Scale Score) Reading Multiple-choice 40 200 (Scale Score) English/Lang. Skills Multiple-choice 40 200 (Scale Score) Writing Essay Prompt 1 of 2 prompts 6È ≤ Passing scores were established for the first time during 2002-2003. Scale scores are derived for each administration to ensure comparable difficulty of subsequent test forms. È Two readers rate each essay on a rating scale from 1 to 6 points. The readers’ scores are then combined to achieve a total score, which ranges from 2 to 12 points. A candidate must receive passing scores on all four subtests of the General Knowledge Test; however, examinees need only to retake the subtest that was not passed. There is no limit to the number of times an examinee may retake a subtest, but State Board of Education Rule 6A-4.0021 (3)(d) does prohibit examinees from retaking a test within 31 days. 38 Assessment and Accountability Briefing Book F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s The General Knowledge Test is offered throughout the state during regular administrations in January, April, July, and October. In addition to the regular administrations, teacher candidates also have the opportunity to take the test during supplemental administrations that are given in four cities— Jacksonville, Miami, Orlando, and Pensacola—during February, May, September, and December. Examinees also have the opportunity to take the Reading, Language, and Mathematics exams on computer at designated locations. The Essay portion, however, cannot be taken on computer. Professional Education Test The Professional Education Test assesses general knowledge of pedagogy and professional practices. On February 18, 2003, the Florida Board of Education amended the Rule 6A-4.0021, FAC to include revisions to the competencies and skills in Professional Education. A revised, updated version of the test was administered for the first time in July 2003. The Professional Education Examination is a multiple-choice test containing 120 items. It is offered throughout the state during regular administrations in January, April, July, and October. In addition to the regular administrations, teacher candidates also have the opportunity to take the test during supplemental administrations that are given in four cities—Jacksonville, Miami, Orlando, and Pensacola—during February, May, September, and December. Examinees also have the opportunity to take the Professional Education Test on computer at designated sites. Subject Area Examinations In 1998, there were 16 subject area tests, but the number has grown steadily since then. By the 1996 testing year, 54 subject area tests had been developed. There are currently 57 subject area tests. These are listed in the following table. F T C E S U B J E C T Agriculture 6-12 Art K-12 Biology 6-12 Business Education 6-12 Chemistry 6-12 Computer Science K-12 Drama 6-12 Earth/Space Science 6-12 Economics 6-12 Educational Media Specialist PK-12 Elementary Education 1-6** Emotionally Handicapped** English 6-12 ESOL K-12 Exceptional Student Education K-12 Family and Consumer Science 6-12 French K-12 Geography 6-12 German K-12 Guidance and Counseling PK-12 A R E A E X A M I N A T I O N S Marketing 6-12 Mathematics 6-12 Mentally Handicapped K-12** Middle Grades English 5-9 Middle Grades General Science 5-9 Middle Grades Integrated Curriculum 5-9 Middle Grades Mathematics 5-9 Middle Grades social Science 5-9 Music K-12 Physical Education K-8 Physical Education K-12 Physical Education 6-12 Physically Impaired K-12** Physics 6-12 Political Science 6-12 Prekindergarten/Primary PK-3 Preschool Education (Birth-Age 4) Primary Education K-3** Reading K-12 School Psychologist PK-12 Assessment and Accountability Briefing Book 39 F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s Health K-12 Hearing Impaired K-12 History 6-12 Humanities K-12 Indust. Arts/Tech. Education 6-12 Journalism 6-12 Kindergarten-Grade 6 Latin K-12 Social Science 6-12 Sociology 6-12 Spanish K-12 Specific Learning Disabilities K-12** Speech 6-12 Speech-Language Impaired K-12 Varying Exceptionalities K-12** Visually Impaired K-12 ** = Subject areas will no longer be tested after June 30, 2004. Most of the subject area examinations are composed solely of multiple-choice items that are scored right or wrong, with the total “raw” score being equal to the total number of correct items. Some subject area tests include performance components in addition to multiple-choice items: Middle Grades English 5-9, English 6-12, French, German, Spanish, and Speech. For these examinations, a composite score is determined by combining the raw multiple-choice score and the performance score for all tests except German and Speech, for which examinees are given separate pass/fail scores for the performance sections. All test results that are reported as scale scores are mathematically derived from the number correct or raw score. The minimum passing scale score is set at 200. The subject area examinations are offered throughout the state during regular administrations in January, April, July, and October. In addition to the regular administrations, teacher candidates also have the opportunity to take the test during supplemental administrations that are given in four cities—Jacksonville, Miami, Orlando, and Pensacola—during February, May, September, and December. Examinees also have the opportunity to take some of the subject areas on computer at designated locations inside and outside the state of Florida. The FTCE Registration Bulletin contains details about the availability of these tests. Test Development All of the tests required for teacher certification are developed according to the same procedures. Statewide committees for the various subject areas determine the body of knowledge to be measured through extensive literature reviews, surveys of practicing educators across the state, and professional discussion among committee members. The committees then develop the competencies (broad categories of knowledge) and skills (more specific abilities) to be tested, as well as the test blueprints (the percentage of the test that comes from each competency). At each stage of the test development process, separate teams of content area experts develop the competencies, skills, blueprints, and items; then other teams evaluate, revise, and validate the first teams’ products. Teams include practicing classroom teachers, district supervisors, principals, and university professors. 40 Assessment and Accountability Briefing Book F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s Florida Educational Leadership Examination (FELE) Candidates seeking certification in Educational Leadership in the state of Florida are required to take and pass the Florida Educational Leadership Examination (FELE) as specified in Section 1012.56, F.S., and in Rule 6A-4.00821, FAC. History of the FELE In 1983-84, panels of experts, including university professors and practicing school leaders, organized the 19 principal competencies identified in studies commissioned by the Florida Council on Educational Management into the eight domains tested on the FELE. The teams then developed the competencies (broad categories of knowledge) and skills (more specific abilities) to be tested, as well as the test blueprints (the percentage of the subject test that comes from each competency). At each stage of the process, separate teams developed the competencies, skills, blueprints, and items; then other teams evaluated, revised, and validated the first teams’ products. Each team included education administrators, district-level personnel, and university professors. The FELE was pilot tested by the University of West Florida in 1986. Then, in 1987, it was given to candidates under “norming” administrations. Under these administrations, all examinees taking the test were considered “passing.” The results from these special administrations were valid for certification purposes only for a period of two years from the administration date. The first administration for which passing scores were required occurred in November 1988. The University of West Florida administered forms of the FELE twice a year, in May and November, from November 1988 through May 1991. Other universities have assisted the Department of Education with the administration of the FELE since 1991. The Institute of Instructional Research and Practice (IIRP) at the University of South Florida administered the FELE twice a year from November 1991 through November 1993. The University of Florida administered the test from April 1994 through September 2001 and administered forms of the test four times a year, along with the regular FTCE administrations. Starting in June 1996, it also became possible to take the FELE during the four supplemental testing dates. Beginning with the October 2001 administration, the IIRP at the University of South Florida has administered the FELE. Assessment and Accountability Briefing Book 41 F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s Current Requirements for FELE The FELE examination covers the eight domains of the Florida Educational Leadership core curriculum specified in Rule 6A-4.00821, FAC. The eight areas are divided among three subtests and include the number of items indicated in the following table. F L O R I D A E D U C A T I O N A L L E A D E R S H I P E X A M I N A T I O N Subtest 1. 2. 3. Domain School Management School Communications School Operations Items Testing Time Leadership 40 2 hours Management 40 Personnel 40 Multiple Choice 27 Essay 1 Curriculum 40 Finance 40 Law 40 Technology 40 2 hours 2 1⁄2 hours An examinee must receive passing scores on all three subtests of the FELE; examinees can retake a failed subtest on another administration date. Each subtest, including School Communications (subtest 2), must be retaken in its entirety. For the essay part of the School Communications subtest, two qualified readers rate each examinee’s essay on a rating scale from 1-4 points. The readers’ scores are then combined to achieve a total essay score, which ranges from 2-8 points. Any rater discrepancies that may occur are refereed by a third reader. The FELE is administered four times a year, along with the regular FTCE administrations in January, April, July, and October, and is also administered four times a year, along with the FTCE supplemental administrations in February, May, September, and December. 42 Assessment and Accountability Briefing Book F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s FTCE/FELE Test Administrations The FTCE and the FELE are administered four times a year at multiple sites across the state. In addition, four supplemental administrations are available, but only at four test sites. The FTCE regular examinations are given at about 50 centers located in 24 cities throughout Florida. The FELE is given at 11 of the centers. The FTCE Professional Education Examination and Subject Area Examinations each take two hours and 30 minutes, with the exception of the K-6 examination, which takes 4 hours and 20 minutes. The General Knowledge part of FTCE takes 3 hours and 50 minutes. The FELE examination takes 6 hours and 30 minutes. The number of examinees taking certification tests has increased significantly over the past two test-administration cycles (October to September). For example, the number of Subject Area Examinations given in 2001-2002 was 20,355, and this amount doubled to 42,472 in 2002-03 . These increases are due to many factors including revised certification requirements, the highly qualified teacher requirements of the No Child Left Behind Act, and class size reduction efforts. Data on the number of tests given during the past two administration cycles are presented in the following two tables: October 2001 to September 2002 and October 2002 to September 2003. Number of Tests or Subtests Administered From October 2001 through September 2002 Regular Administrations Supplemental Administrations Computer-based Test Total Subject Area Exam 19,461 888 6* 20,355 Professional Education Exam 11,162 492 N/A 11,654 5,750 5,440 5,433 7,058 393 386 261 286 N/A 6,143 5.440 5,433 7,058 1,239 1,308 1,228 1,868 114 132 97 241 N/A 1,353 1,440 1,325 2,109 1,446 1,463 1,604 43 45 45 N/A 1,489 1,508 1,649 Test CLAST Essay English Language Skills Reading Math General Knowledge Essay English Language Skills Reading Math FELE Subtest 1 Subtest 2 Subtest 3 ≤ CLAST was administered through June 30, 2002. N/A = Not applicable * Computer-based testing began in August 2002. Assessment and Accountability Briefing Book 43 F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s Number of Tests or Subtests Administered From October 2002 to September 2003 Regular Administrations Supplemental Administrations Computer-Based Test Total Subject Area Exam 36,031 4,368 2,073 42,472 Professional Education Exam 13,803 1,263 700 15,766 8,165 8,370 7,898 11,598 1,125 1,006 857 1,241 1,361 1,645 1,354 413 137 408 Test General Knowledge Essay English Language Skills Reading Math N/A 1,553 1,411 4,096 FELE Subtest 1 Subtest 2 Subtest 3 N/A N/A = Not applicable. 44 Assessment and Accountability Briefing Book 9,290 10,929 10,166 16,935 1,774 1,782 1,762 F l o r i d a Te a c h e r C e r t i f i c a t i o n E x a m i n a t i o n s FTCE/FELE Registration and Fees In 2001, the Department of Education contracted with the Institute for Instructional Research (IIRP) at the University of South Florida for the development and administration of the FTCE. All examinees’ registration forms and fees are processed by IIRP. Examinees can obtain a registration bulletin and application for all certification exams from any of the following: • • • • • A university college of education, A local school district office, The Bureau of Educator Certification office at the Department of Education, The FTCE office at the Department of Education (850/245-0513), or Online at www.cefe.usf.edu. Applicants must correctly complete the form and sign the application included in the bulletin. The application, along with appropriate payment, must be submitted to the certification examination office (IIRP) at the University of South Florida via a postal delivery service as described in the bulletin. Upon receipt of the correctly completed application form and associated fees, each candidate is sent an acknowledgment postcard after his/her registration has been processed. The fee for FTCE applications received by the regular administration deadline is $25.00 for each test (General Knowledge, Professional Education, or subject area). A late registration charge of $15.00 is assessed for each test for the regular administrations received after the deadline. Supplementary examinations require an additional charge of $100.00 for each test to cover the additional expense of offering these expedited registrations and administrations. The fee for the FELE is $50.00 if an application for the regular administration is received by the registration deadline. The test fee and charge for late registration is $80.00. Supplementary examinations require an additional charge of $100.00 to cover the additional expense of offering these expedited registrations and administrations. Registration and payments for computerized tests must be done online at the certification examinations Website, www.cefe.usf.edu. Examinees usually receive their results within one month. Examinees who fail an examination may retake the test at a subsequent administration date as long as it is 31 days after the first examination (Rule 6A-4.0021, FAC). Examinees must pass all appropriate tests and subtests of the FTCE or FELE for certification purposes as required by the Official Statement of Status of Eligibility issued by the Florida Bureau of Educator Certification. Assessment and Accountability Briefing Book 45 46 Assessment and Accountability Briefing Book Other Postsecondary Assessments IV - Other Postsecondary Assessments Florida College Entry-Level Placement Test (CPT) Section 1008.30, F.S., requires the State Board of Education to develop and implement a common placement testing program to assess the basic computation and communication skills of students who intend to enter a degree program at any public community college or state university. This policy has been implemented by the Board through Rule 6A-10.0315, FAC. Beginning August 1, 1995, postsecondary institutions began implementing the common placement testing program using one common test, the Florida College Entry-Level Placement Test (CPT). Areas tested on the CPT are reading comprehension, sentence skills, and elementary algebra. The CPT test books and related materials are provided to institutions by the College Board through a contract awarded by the Florida Department of Education. Using these materials, institutions administer and score the tests. Students who do not achieve cut scores on the CPT, as specified in Rule 6A-10.0315, FAC, are required to successfully complete the appropriate college preparatory coursework and pass the Florida College Basic Skills Exit Test. Florida College Basic Skills Exit Test Through Section 1008.30, F.S., the 1997 Legislature made passing an exit test a condition for meeting basic college computation and communication skills requirements. As a service to the institutions offering college preparatory programs, the Florida Department of Education developed test forms and related materials to meet the requirements of the legislation. The Florida College Basic Skills Exit Test comprises subtests in writing, reading, and mathematics. Students who do not achieve passing scores on the Florida College Entry-Level Placement Test (CPT) must pass the appropriate remedial course(s), as well as the Florida College Basic Skills Exit Test. Institutions are responsible for the administration of the exit test. This includes maintaining test security and setting test dates, length of administration time, and passing requirements. The College-Level Academic Skills Test (CLAST) The College Level Academic Skills Test (CLAST) is part of Florida’s system of educational accountability that satisfies the mandates of Section 1008.29, F.S. The CLAST measures students’ attainment of the college-level communication and mathematics skills that were identified by the faculties of community colleges and state universities through the College-Level Academic Skills Project (CLASP). The skills have been adopted by the State Board of Education and are listed in Rule 6A10.0316, FAC. Since August 1, 1984, students in public institutions in Florida have been required to demonstrate achievement of these skills for the award of an associate in arts degree and for admission to upper-division status in a state university in Florida. Rule 6A-10.0311, FAC, describes the requirements for alternatives to CLAST. Since 1996, students have been able to exempt one or more of the CLAST subtests by achieving certain scores on the Scholastic Achievement Test (SAT) or the American College Testing Program (ACT), or by earning a 2.5 grade point average in qualifying college courses. The CLAST consists of four subtests: Essay, English Language Skills (ELS), reading, and mathematics. Each subtest yields a single score that is reported to the student and to the institution needing the scores. Assessment and Accountability Briefing Book 47 48 Assessment and Accountability Briefing Book Appendix VII - Appendices A—FCAT Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 B—Scoring of FCAT Performance Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 C—Terms and Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Glossary of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Content Assessed by the FCAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Sunshine State Standards (SSS) Tested on the FCAT for Grades 3-5. . . . . C-2 FCAT Norm-Referenced Test (NRT) Content for Grades 3-5 . . . . . . . . . . C-3 Sunshine State Standards (SSS) Tested on the FCAT for Grades 6-8. . . . . C-4 FCAT Norm-Referenced Test (NRT) Content for Grades 6-8 . . . . . . . . . . C-5 Sunshine State Standards (SSS) Tested on the FCAT for Grades 9-10. . . . C-6 FCAT Norm-Referenced Test (NRT) Content for Grades 9-10 . . . . . . . . . C-7 D—Statutes and Rules, Assessment and Accountability . . . . . . . . . . . . . . . . . . D-1 49 Appendix A Appendix A FCAT Results The FCAT has been administered to selected grades from 1998 to 2004. While there are several ways to describe student performance on the Sunshine State Standards testing component, the following two tables are particularly useful to track changes in the state average scores and changes in the percent of students scoring in each of the five FCAT Achievement Levels for Reading and Mathematics. S U N S H I N E F C A T R E A D I N G S T A T E S T A N D A R D S T E S T Grade Year Number of Students Mean Scale Score 3 2001 2002 2003 186,139 188,387 188,107 289 293 298 29 27 23 14 14 15 32 32 33 21 23 25 4 5 5 57 60 63 4 1998a 1999 2000 2001 2002 2003 150,246 174,923 183,733 188,696 191,866 193,391 294 288 293 298 299 305 32 36 33 31 30 25 18 17 16 16 15 15 32 29 29 28 28 31 17 17 19 18 21 23 2 2 4 7 6 6 53 55 60 5 2001 2002 2003 187,570 192,604 192,881 282 285 290 31 28 25 17 18 18 29 30 33 18 19 21 5 4 4 52 53 58 6 2001 2002 2003 187,234 194,125 196,333 292 291 295 30 30 28 18 18 18 29 28 30 18 18 18 5 5 5 52 51 53 7 2001 2002 2003 183,272 191,991 197,417 292 294 297 32 29 28 21 21 21 28 29 29 14 16 17 5 5 6 47 50 52 8 1998a 1999 2000 2001 2002 2003 136,011 161,752 170,139 174,016 184,483 192,116 298 295 290 295 295 301 26 28 32 30 29 26 30 28 29 27 26 26 30 31 27 26 28 30 12 12 11 13 14 16 1 1 1 4 3 3 43 45 49 9 2001 2002 2003 191,518 204,728 205,965 286 287 291 46 44 43 26 27 27 16 17 18 7 8 8 5 4 5 28 29 31 10 1998a 1999 2000 2001 2002 2003 117,023 131,070 144,789 144,471 150,135 167,396 299 302 298 307 303 302 36 33 35 31 32 33 35 37 36 31 33 32 19 19 19 20 21 20 6 6 6 8 8 8 4 5 4 9 7 8 37 36 36 NOTE: Percent of Students by Achievement Level 1 2 3 4 5 Achievement Level Three & Above a The 1998 data include only standard curriculum students. NOTE: The 1999 - 2003 data include students from all curriculum groups. This tends to lower the average score. NOTE: Achievement Level information was not reported in May 2001 for grades 3, 5, 6, 7, and 9. The data shown here reflect retroactive application of the Achievement Level criteria. A-1 Appendix A F C A T M A T H E M A T I C S S U N S H I N E S T A T E S T A N D A R D S a T E S T Grade Year Number of Students Mean Scale Score 3 2001 2002 2003 186,336 188,606 188,487 291 302 308 24 21 19 24 20 19 33 34 34 16 20 22 3 5 7 52 59 63 4 2001 2002 2003 188,633 192,366 193,503 286 294 298 31 26 22 24 24 23 29 32 34 13 15 16 3 4 4 45 51 54 5 1998a 1999 2000 2001 2002 2003 145,734 173,105 182,300 187,623 192,472 192,692 300 303 314 314 318 320 36 33 26 27 25 23 32 32 29 25 27 26 21 21 24 22 23 24 10 12 17 20 19 21 1 2 5 6 6 7 48 48 52 6 2001 2002 2003 187,054 193,948 196,134 291 298 302 39 35 31 21 22 22 24 25 27 12 13 14 4 5 6 40 43 47 7 2001 2002 2003 183,131 191,786 197,161 290 292 296 35 33 31 20 21 21 24 26 26 15 14 15 6 7 6 45 47 47 8 1998a 1999 2000 2001 2002 2003 136,637 161,073 170,287 174,067 184,379 191,656 299 296 303 308 305 310 30 33 29 24 25 22 24 23 20 21 22 22 30 27 28 31 31 32 11 11 13 14 14 14 5 6 10 10 8 10 55 53 56 9 2001 2002 2003 191,094 203,911 205,079 284 286 293 30 28 23 24 24 25 24 26 28 15 15 17 7 6 6 46 47 51 10 1998a 1999 2000 2001 2002 117,693 131,493 144,830 144,236 149,784 302 308 311 321 319 33 27 26 20 19 25 26 23 21 21 20 24 23 24 25 17 19 22 25 27 4 4 6 10 8 59 60 2003 165,624 320 19 20 24 27 9 60 NOTE: Percent of Students by Achievement Level 1 2 3 4 5 Achievement Level Three & Above The 1998 data include only standard curriculum students. NOTE: The 1999 - 2003 data include students from all curriculum groups. This tends to lower the average score. NOTE: Achievement Level information was not reported in May 2001 for grades 3, 4, 6, 7, and 9. The data shown here reflect retroactive application of the Achievement Level criteria. The FCAT Reading and Mathematics results are reported as scale scores ranging from 100 to 500 at each grade. These scores are divided into achievement levels. At level 5, students have success with the most challenging content of the Sunshine State Standards and correctly answer most of the test questions. The percentage of students scoring in each achievement level, along with state and district mean scores, are reported to districts and schools. A-2 Appendix A F C A T R E A D I N G A N D M A T H E M A T I C S N O R M - R E F E R E N C E D T E S T S C O R E Sa S T A T E W I D E C O M P A R I S O N F O R 2 0 0 0 - 2 0 0 3 READINGa Grade MATHEMATICSa Year Scale Score Medianb NPRc Scale Score Median NPR 3 2000 2001 2002 2003 616 622 624 629 49 56 57 61 612 615 618 623 56 59 62 66 4 2000 2001 2002 2003 640 643 644 645 56 56 57 58 626 632 634 637 57 59 62 64 5 2000 2001 2002 2003 647 652 654 656 45 51 52 55 654 651 653 654 63 59 61 63 6 2000 2001 2002 2003 656 659 662 663 43 49 52 53 655 662 664 666 55 61 63 64 7 2000 2001 2002 2003 671 678 680 681 48 54 56 57 668 677 680 681 48 61 65 66 8 2000 2001 2002 2003 691 696 697 695 54 59 60 58 680 684 687 688 56 62 64 65 9 2000 2001 2002 2003 683 688 688 689 38 44 44 44 693 702 704 705 52 63 65 66 10 2000 2001 2002 2003 683 700 701 696 33 49 50 46 701 711 714 713 54 64 67 66 a NOTE: These scores are from the Norm-Referenced Test (NRT) portion of the FCAT. b NOTE: Median is the score that identifies the middle point. c NOTE: NPR is the National Percentile Rank and indicates the percent of students who earned the same score or lower. Students who score at the national average earn a NPR of 50. The achievement of Florida students can be compared to that of a national sample of students in grades 3-10, because the FCAT administrations include nationally norm-referenced tests (the SAT9). The median national percentile rank is shown for each grade level and subject area in the shaded columns. A-3 Appendix B Appendix B Scoring of FCAT Performance Tasks Student responses to the FCAT writing essays and the reading, mathematics, and science performance tasks are scored using a process called “handscoring.” As this name implies, trained scorers read and evaluate the student responses using a “hands-on” process. Scoring involves comparing student responses to a scoring rubric and assigning each paper a single, holistic score. The scoring rubric describes the characteristics of the answer required for each possible score. What is holistic scoring? The term “holistic” is used to emphasize the importance of the whole work, including the interdependence of its parts. A rubric is the guideline for evaluating student responses to each task. Different rubrics are used for the different subjects and the different types of tasks (short- and extendedresponse). The Writing rubric considers four elements of writing (focus, organization, support, and conventions) as they work together to support the whole work. The Reading rubric considers the student’s use of text-based information and details from the passage. For Mathematics and Science, the rubrics address the level of understanding related to the task that is demonstrated by the student’s work. Rubrics for each subject area are published in the document, Understanding FCAT Reports, found on the Department of Education website at www.firn.edu/doe/sas/fcat/fcpress.htm. What is the process for scoring? Florida educators score tasks. Teams of Florida educators (classroom teachers and content specialists) meet to review each task on the FCAT. These teams apply the rubric to the specific tasks to determine the scores for student papers. This process is referred to as “rangefinding,” because educators identify the range of responses that are acceptable for each score point in the rubric. After the educators have scored a sufficient number of papers, they select the rangefinder papers (also called anchor papers) to be used to train the professional readers who will score each student’s paper. Professional “scorers” are hired. The FCAT scoring contractor is responsible for hiring professional staff to read and score student papers. The Florida DOE has established minimum qualifications for each scorer. The DOE’s requirements include that scorers have a bachelor’s degree in a field related to the subject area being scored (reading, writing, mathematics, or science). For example, persons scoring reading or writing papers could have degrees in English, communications, journalism, or literature, and readers of mathematics items might have degrees in accounting, statistics, mathematics, or another quantitative field. Scorers must take a qualifying exam. All “scorers” must participate in rigorous training designed by Department staff and delivered by contractor staff. The training involves developing an understanding of how to use the rubric, how to score holistically, and how to apply the scoring standards established by Florida educators. Scorers are trained during a thorough, multi-day training program. During this time, they read and score many papers under the supervision and instruction of an experienced scoring director and a DOE staff member. In addition, no scorer is permitted to score student papers until they have passed a qualifying examination. The qualifying examination is a previously scored set of papers and the prospective scorer has to answer 80-90% correct. This process is called qualifying. Florida educators have previously scored the papers used for qualifying as well as those used in the training materials. The qualification process occurs at the conclusion of the multi-day training program. B-1 Appendix B All papers are read twice. Each student’s paper is read by at least two qualified scorers. For writing and for the reading, mathematics, and science extended-response tasks, the student’s score is the average of two exact or adjacent scores. If the scores of the first two scorers are not exact or adjacent, a third scorer assigns a score without knowing the two previous scores. The score of the third scorer that matches either of the previous two scores becomes the final score. Scores on the reading, mathematics, and science short-response tasks must always be identical. If exact scores are not obtained, a third scorer is required. The scorers never know the names of the students, the schools they represent, or the scores assigned by other scorers. Scorers are monitored every day. Although intensive training is required at the beginning of the scoring process, training does not end there. Training concepts are reviewed throughout the scoring session. Periodically, papers previously scored by Florida educators are presented to all scorers to check on their scoring accuracy. These papers are called validity papers because they confirm whether scorers are assigning valid scores. The reliability of ratings is also monitored daily to determine the amount of agreement between scorers. The resulting validity statistics and the inter-rater agreement (reliability) statistics are key elements in the quality-control measures utilized during scoring. Supervisors also read behind their team of scorers to check on the accuracy of the scores given to each paper. When the quality control procedures in place indicate areas of concern, these are addressed in daily training sessions (called calibration sessions) conducted for the large group or for individuals. In addition, specific papers and scorer questions are discussed daily throughout the scoring process. If scorers cannot maintain scoring accuracy, they are dismissed. Do machines grade student papers? Professional scorers who are trained and qualified according to the process described above do the scoring of FCAT essays and performance tasks. The scorers view actual student responses using an electronic image-based scoring system. Answer documents are scanned using imaging technology (book pages must be scanned), and images are routed to computer workstations, where they are scored by scorers. The electronic system for distributing papers to scorers permits student papers from one school to be spread across an entire room instead of giving them to a single table of scorers. Responses also can be randomly redistributed for the second reading. Scorers score a set of two to three different reading, mathematics, and science tasks during each day of scoring, rather than just one item over and over. How long does it take to score FCAT performance tasks? The answer to this question depends on how many scorers are hired to do the scoring and how successful and skilled they are at scoring. If the targeted number of scorers is successfully qualified, it takes four to five weeks to complete the training and score all the tasks. To score the writing essays, it takes 800 scorers. To score the reading, mathematics, and science tasks it takes 1,500 to 2,000 scorers working in double shifts to complete the scoring. Because the scoring of FCAT performance tasks does not require a year-round commitment, the FCAT contractor relies on temporary professional employees. Scoring must be conducted in numerous locations around the country, since a sufficiently large pool of professionals seeking temporary employment does not exist in a single large city. The scoring of FCAT performance tasks in 2003-04 was conducted in eight cities. B-2 Appendix B F C A T W R I T I N G S T A T E W I D E G R A D E S 4 , 8 , A N R E S U L T S D 1 0 The FCAT writing component, formally known as Florida Writes!, has been administered in grades 4, 8, and 10 since 1993. The statewide results for the subsequent years are shown below. Grade 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 4 8 10 2.0 3.0 NA 2.2 2.7 2.9 2.4 3.1 3.3 2.5 3.5 3.3 2.6 3.4 3.6 3.0 3.3 3.6 3.0a 3.3a 3.5a 3.2a 3.7a 3.9a 3.4a 3.7a 3.8a 3.4a 3.8a 3.8a 3.6 3.9 3.8 a NOTE: The data for 1999 - 2003 are for all curriculum students. Prior years are for standard curriculum students. F C A T W R I T I N G Description of Grade 4 Writing Scores H O L I S T I C Description of Grade 8 Writing Scores R U B R I C S Description of Grade 10 Writing Scores 6.0: The writing focuses on the topic, is logically 6.0: The writing focuses on the topic, is logically organized, and includes ample development of organized, and includes substantial development supporting ideas or examples. It demonstrates a of supporting ideas or examples. It demonstrates mature command of language, including precision a mature command of language, including preciin word choice. Sentences vary in structure. sion in word choice. Sentences vary in structure. Punctuation, capitalization, and spelling are generPunctuation, capitalization, and spelling are ally correct. generally correct. 6.0: The writing focuses on the topic, is logically organized, and includes substantial development of supporting ideas or examples. It demonstrates a mature command of language with freshness of expression. Sentences vary in structure. There are few, if any, usage, punctuation, or spelling errors. 5.0: The writing focuses on the topic with adequate 5.0: The writing focuses on the topic with ample development of supporting ideas or examples. It development of supporting ideas or examples. has an organizational pattern, though lapses may It has an organizational pattern, though lapses occur. Word choice is adequate. Sentences vary in may occur. Word choice is adequate. Sentences structure. Punctuation, capitalization, and spelling vary in structure. Punctuation, capitalization, and are generally correct. spelling are generally correct. 5.0: The writing focuses on the topic, is logically organized, and includes ample development of supporting ideas or examples. It demonstrates a mature command of language. Sentences vary in structure. Usage, punctuation, and spelling are generally correct. 4.0: The writing focuses on the topic, though it may contain extraneous information. An organizational pattern is evident, but lapses may occur. Some supporting ideas contain specifics and details, but others are not developed. Word choice is adequate. Sentences vary somewhat in structure, though many are simple. Punctuation, capitaliza- tion, and spelling are usually correct. 4.0: The writing focuses on the topic, though it may contain extraneous information. An organizational pattern is evident, but lapses may occur. Some supporting ideas contain specifics and details, but others are not developed. Word choice is adequate. Sentences vary somewhat in structure, though many are simple. Punctuation, capitaliza- tion, and spelling are usually correct. U: The writing is unrelated to the assigned topic or cannot be read. U: The writing is unrelated to the assigned topic or cannot be read. 4.0: The writing focuses on the topic, though it may contain loosely related information. An organizational pattern is apparent. Some of the supporting ideas contain specifics and details, but others do not. Word choice is adequate. Sentences vary in structure. Usage, punctuation, and spelling are generally correct. 3.0: The writing generally focuses on the topic, though it may contain loosely related information. An 3.0: The writing generally focuses on the topic, organizational pattern is demonstrated. Develop3.0: The writing generally focuses on the topic, though it may contain extraneous information. An ment of the supporting ideas maybe uneven. though it may contain extraneous information. An organizational pattern has been attempted, but Word choice is adequate. There is some variation organizational pattern has been attempted, but in sentence structure. Usage, punctuation, and lapses may occur. Some of the supporting ideas lapses may occur. Some of the supporting ideas spelling are generally correct. or examples may not be developed. Word choice or examples may not be developed. Word choice is adequate. Sentences vary somewhat in struc- is adequate. Sentences vary somewhat in struc2.0: The writing addresses the topic, though it may ture, though many are simple. Punctuation and ture, though many are simple. Punctuation and lose focus by including extraneous information. capitalization are sometimes incorrect, but most capitalization are sometimes incorrect, but most An organizational pattern is demonstrated. commonly used words are spelled correctly. commonly used words are spelled correctly. Development of the supporting ideas may be nonspecific. Word choice may be limited. Frequent 2.0: The writing may be slightly related to the topic or 2.0: The writing may be slightly related to the topic or errors may occur in sentence construction, offer little relevant information and few supporting offer little relevant information and few supporting usage, punctuation, and spelling. ideas or examples. There is little evidence of ideas or examples. There is little evidence of an organizational pattern. Word choice may be an organizational pattern. Word choice may be 1.0: The writing addresses the topic, though it may limited or immature. Sentences may be limited to limited or immature. Sentences may be limited to lose focus by including extraneous and loosely simple constructions. Frequent errors may occur in related ideas. The organizational pattern is weak. simple constructions. Frequent errors may occur punctuation, capitalization, and spelling. Ideas are presented through lists and limited or in punctuation, capitalization, and spelling. inappropriate word choice. Frequent errors may 1.0: The writing may only minimally address the topic 1.0: The writing may only minimally address the occur in sentence construction, usage, punctuabecause there is little or no development of topic because therein little or no development of tion, and spelling. supporting ideas or examples. No organizational supporting ideas or examples. No organizational U: The writing is unrelated to the assigned topic or pattern is evident. Ideas are provided through lists, pattern is evident. Ideas are provided through cannot be read. and word choice is limited or immature. Unrelated lists, and word choice is limited or immature. information may be included. Frequent errors Unrelated information may be included. Frequent in punctuation, capitalization, and spelling may errors in punctuation, capitalization, and spelling impede communication. may impede communication. B-3 Appendix C Appendix C Terms and Content Assessed on The FCAT G L O S S A R Y O F T E R M S Achievement Levels—Five categories of achievement that represent the success students demonstrate with the content assessed on the FCAT Sunshine State Standards (SSS). Benchmark—A specific statement that describes what students should know and be able to do. The benchmarks are part of the SSS. Cluster—A grouping of related benchmarks from the SSS. Clusters are used to summarize and report achievement for FCAT SSS Reading and FCAT Science. Content Subscores—The number of raw score points earned by a student in each sub-area of FCAT SSS Reading, Mathematics, and Science. Content subscores are reported for clusters or strands within each content area. For example, in Mathematics, subscores are reported for number sense, measurement, geometry, algebra, and data analysis and probability. Developmental Scale Score (DSS)—A type of scale score used to determine a student’s annual progress from grade to grade. The FCAT Developmental Scale for Reading and Mathematics ranges from 86 to 3008 across Grades 3–10. On the Student Report, the DSS is called the “FCAT Score.” National Percentile Rank (NPR)—A score that shows the percent of students who earned the same or a lower score. NPRs show the rank of an individual compared to the national sample of students or norm group, although NPRs do not compare an individual to Florida students who took the test. Norm-Referenced Test (NRT)—A test designed to compare the performance of one group of students to the national sample of students, called the norm group. Raw Score—A score that reports the number of points a student earned on each test question. Students earn one raw score point for each correctly answered multiple-choice item and gridded-response item and up to four raw score points on performance tasks. Raw scores are reported by content subscores. Rubric—The scoring guidelines or criteria used to evaluate all FCAT performance tasks and essays. The rubric describes what is required for each possible score point. Scale Score—A score used to report test results on the entire test. FCAT SSS scale scores range from 100 to 500 and are determined by which test questions the student responded to correctly. FCAT NRT scale scores are solely determined by raw score point totals. Stanine—Standard scores that divide a distribution into nine parts. The word “stanine” comes from the fact that it is a Standard score on a scale of nine units. Strands—The broad divisions of content in the SSS. For example, in the Language Arts SSS, there are seven strands (reading, writing, listening, viewing, speaking, language, and literature). Sunshine State Standards (SSS)—Florida’s curriculum framework that includes curriculum content areas, strands, standards, and benchmarks. The SSS provide guidelines for the educational curriculum in Florida. C O N T E N T A S S E S S E D B Y T H E F C A T The FCAT content is derived from the Sunshine State Standards developed by committees of practicing classroom teachers and curriculum specialists and adopted by the State Board of Education. The Standards are broad statements of what students should know and be able to do. The Standards are subdivided into smaller units called benchmarks. The FCAT measures some of these benchmarks in reading, writing, mathematics, and science, although not all benchmarks can be measured on each test annually. While students are expected to know how to conduct library research and write a research paper, the FCAT could never assess such a learning outcome. The benchmarks measured by the FCAT are indicated on the nine following pages. C-1 Appendix C Sunshine State Standards (SSS) Tested on the FCAT for Grades 3-5 W R I T I N G C O N T E N T T E S T E D For FCAT Writing, students demonstrate their writing skills by producing, within 45 minutes, a draft response to an assigned topic (prompt). This type of writing is called demand writing. Fourth grade students are randomly assigned one of two prompts that ask students to tell a story (narrative writing) or to explain (expository writing). Prompts are carefully selected to ensure that the subject matter is interesting and appropriate for fourth grade students, and all prompts are reviewed by a Bias and Sensitivity Committee. The Sunshine State Standards benchmarks assessed by FCAT Writing for Grade 4 emphasize the areas of focus, organization, support, and conventions. Writing Process R E A D I N G C O N T E N T T E S T E D The FCAT Reading tests employ a wide variety of written material to assess students’ reading comprehension as defined in the Sunshine State Standards. These tests include informational and literary reading passages. Informational passages are written to provide readers with facts about a particular subject and may include magazine and newspaper articles, editorials, and biographies. Literary passages are written primarily for readers’ enjoyment and may include short stories, poems, folk tales, and selections from novels. Tests for Grades 3, 4, and 5 assess the following SSS reading comprehension skills and processes: Words and Phrases in Context • uses strategies to increase vocabulary through word structure clues (prefixes, suffixes, roots), word relationships (antonyms, synonyms), and words with multiple meanings • uses context clues to determine word meanings The student drafts and revises writing in cursive1 that • focuses on the topic • provides a logical organizational pattern, including a beginning, middle, conclusion, and transitional devices Main Idea, Plot, and Purpose • determines the stated or implied main idea or essential message in a text • includes ample development of supporting ideas • demonstrates a sense of completeness or wholeness • identifies relevant details and facts • demonstrates a command of language including precision in word choice • recognizes and arranges events in chronological order • identifies author’s purpose in a text • indicates a general knowledge of the correct use of subject/verb agreement and verb and noun forms • recognizes when a text is intended to persuade • includes, with few exceptions, sentences that are complete except when fragments are used purposefully • understands plot development and conflict resolution in a story • uses a variety of sentence structures and • demonstrates a knowledge of the basic conventions of punctuation, capitalization, and spelling. _______________________________________________ 1 Language Arts Writing Benchmark 1.2.2 for Grade 4 states that students should write in cursive. For FCAT Writing, students may print or write in cursive. Comparisons and Cause/Effect • recognizes the use of comparison and contrast • recognizes cause-and-effect relationships • identifies similarities and differences among characters, settings, and events in various texts Reference and Research • reads, organizes, and interprets written information for various purposes, such as making a report, conducting an interview, taking a test, or performing a task • uses maps, charts, photos, or other multiple representations of information for research projects C-2 Appendix C M A T H E M A T I C S C O N T E N T T E S T E D The FCAT Mathematics tests assess the achievement of the Sunshine State Standards in mathematics. FCAT Mathematics tests for Grades 3 and 4 include only multiplechoice questions. The FCAT Mathematics test for Grade 5 combines gridded-response questions with multiple-choice questions and also includes several performance tasks that are scored on 2-point and 4-point rubrics. Approximately the same number of questions is used for each of the five strands in Grades 3, 4, and 5. FCAT Mathematics assesses what students in Grades 3–10 know and are able to demonstrate in the following content strands: Number Sense, Concepts, and Operations • identifies operations (+, –, x, ÷) and the effects of operations • determines estimates • knows how numbers are represented and used S C I E N C E C O N T E N T T E S T E D The FCAT Science tests assess the achievement of the Sunshine State Standards in science. Approximately the same number of questions is used for each of the four clusters: Physical and Chemical Sciences, Earth and Space Sciences, Life and Environmental Sciences, and Scientific Thinking. The FCAT Science test for Grade 5 consists of mostly multiple-choice questions and also includes short-response tasks and extended-response tasks that are scored on 2-point and 4-point rubrics. Physical and Chemical Sciences • understands that matter can be described, classified, and compared • traces the flow of energy in a system • identifies the differences between renewable and nonrenewable energy sources • describes, predicts, and measures the types of motion and effects of forces • identifies the types of force that act upon an object Measurement Earth and Space Sciences • recognizes measurements and units of measurement • • compares, contrasts, and converts measurements understands that changes in climate, geological activity, and life forms can be traced and compared Geometry and Spatial Sense • recognizes that Earth’s systems change over time • describes, draws, identifies, and analyzes two and three-dimensional shapes • identifies the cause of the phases of the moon and seasons • visualizes and illustrates changes in shapes • recognizes the role of Earth in the vast universe • uses coordinate geometry Life and Environmental Sciences Algebraic Thinking • understands that living things are different but share similar structures • describes, analyzes, and generalizes patterns, relations, and functions • • writes and uses expressions, equations, inequalities, graphs, and formulas recognizes that many characteristics of an organism are inherited • explains the relationship and interconnectedness of all living things to their environment • understands that plants use carbon dioxide, minerals, and sunlight to produce food (photosynthesis) Data Analysis and Probability • analyzes, organizes, and interprets data • identifies patterns and makes predictions, inferences, and valid conclusions • uses probability and statistics C-3 Scientific Thinking • uses scientific method and processes to solve problems • recognizes that most natural events occur in consistent patterns • understands the interdependence of science, technology, and society Appendix C F C A T N o r m - R e f e r e n c e d T e s t ( N R T ) C o n t e n t f o r G r a d e s 3 - 5 R E A D I N G C O N T E N T T E S T E D M A T H E M A T I C S C O N T E N T T E S T E D * * Initial Understanding—Demonstrate the ability to comprehend explicitly stated relationships in a variety of reading selections. Interpretation—Demonstrate the ability to form an interpretation of a variety of reading selections based on explicit and implicit information in the selections. Critical Analysis—Demonstrate the ability to synthesize and evaluate explicit and implicit information in a variety of reading selections. Strategies—Demonstrate the ability to recognize and apply text factors and reading strategies in a variety of reading selections. The above objectives are measured within the following contexts: Recreational—material typically read for enjoyment. Textual—material typically found in grade-appropriate textbooks and other sources of information. Functional—material typically encountered in everyday life situations. Concepts of Whole Number Computation—Demonstrate an understanding of the fundamental operations of arithmetic and their properties. Number Sense and Numeration—Demonstrate an understanding of the system of whole numbers and the basic principles of arithmetic. Geometry and Spatial Sense—Demonstrate an understanding of geometric principles. Measurement—Demonstrate an understanding of the principles of measurement. Statistics and Probability—Demonstrate an understanding of the relationships in data sets and the laws governing chance. Fraction and Decimal Concepts—Demonstrate an understanding of representations of rational numbers. Patterns and Relationships—Identify missing elements in numeric and geometric patterns. Estimation—Determine the reasonableness of results and apply estimation in problem solving. Problem Solving Strategies—Demonstrate an understanding of the process of solving conventional and non-routine problems. _______________________________________________ * Currently, the FCAT NRT is the SAT-9, published by Harcourt Educational Measurement. Source: Stanford Achievement Test Series, Ninth Edition, Compendium of Instructional Objectives Note: Printed with permission by Harcourt Educational Measurement, August 1999. C-4 Appendix C Sunshine State Standards (SSS) Tested on the FCAT for Grades 6-8 W R I T I N G C O N T E N T T E S T E D For FCAT Writing, students demonstrate their writing skills by producing, within 45 minutes, a draft response to an assigned topic (prompt). This type of writing is called demand writing. Eighth grade students are randomly assigned one of two prompts that ask students to persuade (persuasive writing) or to explain (expository writing). Prompts are carefully selected to ensure that the subject matter is interesting and appropriate for eighth grade students, and all prompts are reviewed by a Bias and Sensitivity Committee. The Sunshine State Standards benchmarks assessed by FCAT Writing for Grade 8 emphasize the areas of focus, organization, support, and conventions. Writing Process The student drafts and revises writing that R E A D I N G C O N T E N T T E S T E D The FCAT Reading tests employ a wide variety of written material to assess students’ reading comprehension as defined in the Sunshine State Standards. These tests include informational and literary reading passages. Informational passages are written to provide readers with facts about a particular subject and may include magazine and newspaper articles, editorials, and biographies. Literary passages are written primarily for readers’ enjoyment and may include short stories, poems, folk tales, and selections from novels. Tests for Grades 6, 7, and 8 assess the following SSS reading comprehension skills and processes: Words and Phrases in Context • uses various strategies, including contextual and word structure clues, to analyze words and text • draws conclusions from a reading text • recognizes organizational patterns Main Idea, Plot, and Purpose • determines the stated or implied main idea or essential message in a text • identifies relevant details and facts • recognizes how an organizational pattern supports the main idea • focuses on the topic, is purposeful, and reflects insight into the writing situation • conveys a sense of completeness and wholeness and adherence to the main idea • provides an organizational pattern with a logical progression of ideas • includes support that is substantial, specific, relevant, concrete, and/or illustrative • identifies and uses the author’s purpose and point of view to construct meaning from text • demonstrates a commitment to and an involvement with the subject • recognizes persuasive text • recognizes and understands how literary elements support text (e.g., character and plot development, point of view, tone, setting, and conflicts and resolutions) • presents ideas with clarity • employs creative writing strategies appropriate to the purpose of the paper Comparisons and Cause/Effect • demonstrates a command of language (word choice) with freshness of expression • recognizes the use of comparison and contrast • recognizes cause-and-effect relationships • includes sentences that are complete except when fragments are used purposefully • • uses a variety of sentence structures and locates, organizes, and interprets written information for a variety of purposes • • contains few, if any, convention errors in mechanics, usage, and punctuation. uses a variety of reference materials to gather information for research projects (e.g., indexes, magazines, newspapers, journals, and card and computer catalogs) • checks validity and accuracy of research information (e.g., strong versus weak arguments, fact versus opinion, and how authors’ personal values influence conclusions) • synthesizes and separates collected information into useful components C-5 Reference and Research Appendix C M A T H E M A T I C S C O N T E N T T E S T E D S C I E N C E C O N T E N T T E S T E D The FCAT Mathematics tests assess the achievement of the Sunshine State Standards in mathematics. FCAT Mathematics tests for Grades 6 and 7 include multiple-choice questions and gridded-response questions. The FCAT Mathematics test for Grade 8 includes multiple-choice and gridded-response questions, as well as several performance tasks that are scored on 2-point and 4-point rubrics. Approximately the same number of questions is used for each of the five strands in Grades 6, 7, and 8. The FCAT Science tests assess the achievement of the Sunshine State Standards in science. Approximately the same number of questions is used for each of the four clusters: Physical and Chemical Sciences, Earth and Space Sciences, Life and Environmental Sciences, and Scientific Thinking. The FCAT Science test for Grade 8 consists of mostly multiple-choice questions and gridded-response questions, and also includes short-response tasks and extended-response tasks that are scored on 2-point and 4-point rubrics. FCAT Mathematics assesses what students in Grades 3–10 know and are able to demonstrate in the following content strands: Physical and Chemical Sciences • recognizes the differences between solids, liquids, and gases identifies operations (+, –, x, ÷) and the effects of operations • contrasts physical and chemical changes • identifies atomic structures • determines estimates • recognizes properties of waves • knows how numbers are represented and used • describes how energy flows through a system Measurement • • recognizes measurements and units of measurement describes, measures, and predicts the types of motion and effects of force Earth and Space Sciences • compares, contrasts, and converts measurements • recognizes that forces within and on Earth result in geologic structures, weather, erosion, and ocean currents Number Sense, Concepts, and Operations • Geometry and Spatial Sense • describes, draws, identifies, and analyzes two and three-dimensional shapes • explains the relationship between the Sun, Moon, and Earth • visualizes and illustrates changes in shapes • • uses coordinate geometry understands that activities of humans affect ecosystems • compares and contrasts characteristics of planets, stars, and satellites Algebraic Thinking • describes, analyzes, and generalizes patterns, relations, and functions Life and Environmental Sciences • writes and uses expressions, equations, inequalities, graphs, and formulas • identifies the structure and function of cells • compares and contrasts structures and functions of living things • understands the importance of genetic diversity • recognizes how living things interact with their environment Data Analysis and Probability • analyzes, organizes, and interprets data • identifies patterns and makes predictions, inferences, and valid conclusions Scientific Thinking • uses probability and statistics • uses scientific method and processes to solve problems • recognizes that most natural events occur in consistent patterns • understands the interdependence of science, technology, and society C-6 Appendix C F C A T N o r m - R e f e r e n c e d T e s t ( N R T ) C o n t e n t f o r G r a d e s 6 - 8 R E A D I N G C O N T E N T T E S T E D * M A T H E M A T I C S C O N T E N T T E S T E D * Initial Understanding—Demonstrate the ability to comprehend explicitly stated relationships in a variety of reading selections. Concepts of Whole Number Computation—Demonstrate an understanding of the fundamental operations of arithmetic and their properties. Interpretation—Demonstrate the ability to form an interpretation of a variety of reading selections based on explicit and implicit information in the selections. Number Sense and Numeration—Demonstrate an understanding of the system of whole numbers and the basic principles of arithmetic. Critical Analysis—Demonstrate the ability to synthesize and evaluate explicit and implicit information in a variety of reading selections. Geometry and Spatial Sense—Demonstrate an understanding of geometric principles. Strategies—Demonstrate the ability to recognize and apply text factors and reading strategies in a variety of reading selections. The above objectives are measured within the following contexts: Measurement—Demonstrate an understanding of the principles of measurement. Statistics and Probability—Demonstrate an understanding of the relationships in data sets and the laws governing chance. Recreational—material typically read for enjoyment. Fraction and Decimal Concepts—Demonstrate an understanding of representations of rational numbers. Textual—material typically found in gradeappropriate textbooks and other sources of information. Patterns and Relationships—Identify missing elements in numeric and geometric patterns. Functional—material typically encountered in everyday life situations. Estimation—Determine the reasonableness of results and apply estimation in problem solving. Problem Solving Strategies—Demonstrate an understanding of the process of solving conventional and non-routine problems. _______________________________________________ * Currently the FCAT NRT is the SAT9 published by Harcourt Educational Measurement. Source: Stanford Achievement Test Series, Ninth Edition, Compendium of Instructional Objectives Note: Printed with permission by Harcourt Educational Measurement, August 1999. C-7 Appendix C Sunshine State Standards (SSS) Tested on the FCAT for Grades 9 and 10 W R I T I N G C O N T E N T T E S T E D For FCAT Writing, students demonstrate their writing skills by producing, within 45 minutes, a draft response to an assigned topic (prompt). This type of writing is called demand writing. Tenth grade students are randomly assigned one of two prompts that ask students to persuade (persuasive writing) or to explain (expository writing). Prompts are carefully selected to ensure that the subject matter is interesting and appropriate for tenth grade students, and all prompts are reviewed by a Bias and Sensitivity Committee. The Sunshine State Standards benchmarks assessed by FCAT Writing for Grade 10 emphasize the areas of focus, organization, support, and conventions. R E A D I N G C O N T E N T T E S T E D The FCAT Reading tests employ a wide variety of written material to assess students’ reading comprehension as defined in the Sunshine State Standards. These tests include informational and literary reading passages. Informational passages are written to provide readers with facts about a particular subject and may include magazine and newspaper articles, editorials, and biographies. Literary passages are written primarily for readers’ enjoyment and may include short stories, poems, folk tales, and selections from novels. Tests for Grades 9 and 10 assess the following SSS reading comprehension skills and processes: Words and Phrases in Context • selects and uses strategies to understand words and text The student drafts and revises writing that focuses on the topic, is purposeful, and reflects insight into the writing situation • makes and confirms inferences from a reading text • interprets data presentations (e.g., maps, diagrams, graphs, and statistical illustrations) • provides an organizational pattern with a logical progression of ideas Main Idea, Plot, and Purpose • includes effective use of transitional devices that contribute to a sense of completeness • determines stated or implied main idea • identifies relevant details • includes support that is substantial, specific, relevant, and concrete • identifies methods of development • determines author’s purpose and point of view • demonstrates a commitment to and and involvement with the subject • identifies devices of persuasion and methods of appeal • • employs creative writing strategies appropriate to the purpose of the paper identifies and analyzes complex elements of plot (e.g., setting, tone, major events, and conflicts and resolutions) • demonstrates a mature command of language with freshness of expression • recognizes the use of comparison and contrast • uses a variety of sentence structures and • recognizes cause-and-effect relationships • contains few, if any, convention errors in mechanics, usage, punctuation, and spelling. Reference and Research Writing Process Comparisons and Cause/Effect • locates, gathers, analyzes, and evaluates information for a variety of purposes • selects and uses appropriate study and research skills and tools according to the type of information being gathered or organized • analyzes the validity and reliability of primary source information and uses the information appropriately • synthesizes information from multiple sources to draw conclusions C-8 Appendix C M A T H E M A T I C S C O N T E N T T E S T E D S C I E N C E C O N T E N T T E S T E D The FCAT Mathematics tests assess the achievement of the Sunshine State Standards in mathematics. The FCAT Mathematics tests for Grades 9 and 10 include multiplechoice questions and gridded-response questions. The Grade 10 test also includes several performance tasks, scored on 2-point and 4-point rubrics. At Grades 9 and 10, the Geometry and Spatial Sense strand and the Algebraic Thinking strand have slightly more questions than the other three strands. The FCAT Science tests assess the achievement of the Sunshine State Standards in science. Approximately the same number of questions is used for each of the four clusters: Physical and Chemical Sciences, Earth and Space Sciences, Life and Environmental Sciences, and Scientific Thinking. The FCAT Science test for Grade 10 consists of mostly multiple-choice questions and gridded-response questions, and also includes short-response tasks and extended-response tasks, scored on 2-point and 4-point rubrics. FCAT Mathematics assesses what students in Grades 3–10 know and are able to demonstrate in the following content strands: Number Sense, Concepts, and Operations • identifies operations (+, –, °—, ÷) and the effects of operations • determines estimates • knows how numbers are represented and used Measurement • recognizes measurements and units of measurement • compares, contrasts, and converts measurements Geometry and Spatial Sense Physical and Chemical Sciences • describes and explains the structure atom and its interactions with other atoms • recognizes and explains chemical reactions • describes how energy flows through a system • describes, measures, and predicts the types of motion and effects of force Earth and Space Sciences • recognizes that forces within and on Earth result in geologic structures, weather, erosion, and ocean currents • identifies and explains the interconnectedness of Earth’s systems • describes, draws, identifies, and analyzes two- and three-dimensional shapes • understands that activities of humans affect ecosystems • visualizes and illustrates changes in shapes • • uses coordinate geometry compares and contrasts characteristics of planets, stars, and satellites Algebraic Thinking • describes, analyzes, and generalizes patterns, relations, and functions • writes and uses expressions, equations, inequalities, graphs, and formulas Data Analysis and Probability • analyzes, organizes, and interprets data • identifies patterns and makes predictions, inferences, and valid conclusions • uses probability and statistics C-9 Life and Environmental Sciences • contrasts and compares the structure and function of major body systems • recognizes that structures, physiology, and behaviors of living things are adapted to their environment • identifies and explains the role of DNA • explains the relationship and interdependence of all living things and their environment Scientific Thinking • uses scientific method and processes to solve problems • recognizes that most natural events occur in consistent patterns • understands the interdependence of science, technology, and society Appendix C FCAT Norm-Referenced Test (NRT) Content for Grades 9-10 R E A D I N G C O N T E N T T E S T E D * M A T H E M A T I C S C O N T E N T T E S T E D * Initial Understanding—Demonstrate the ability to comprehend explicitly stated relationships in a variety of reading selections. Concepts of Whole Number Computation—Demonstrate an understanding of the fundamental operations of arithmetic and their properties. Interpretation—Demonstrate the ability to form an interpretation of a variety of reading selections based on explicit and implicit information in the selections. Number Sense and Numeration—Demonstrate an understanding of the system of whole numbers and the basic principles of arithmetic. Critical Analysis—Demonstrate the ability to synthesize and evaluate explicit and implicit information in a variety of reading selections. Geometry and Spatial Sense—Demonstrate an understanding of geometric principles. Strategies—Demonstrate the ability to recognize and apply text factors and reading strategies in a variety of reading selections. The above objectives are measured within the following contexts: Recreational—material typically read for enjoyment. Textual—material typically found in gradeappropriate textbooks and other sources of information. Functional—material typically encountered in everyday life situations. Measurement—Demonstrate an understanding of the principles of measurement. Statistics and Probability—Demonstrate an understanding of the relationships in data sets and the laws governing chance. Fraction and Decimal Concepts—Demonstrate an understanding of representations of rational numbers. Patterns and Relationships—Identify missing elements in numeric and geometric patterns. Estimation—Determine the reasonableness of results and apply estimation in problem solving. Problem Solving Strategies—Demonstrate an understanding of the process of solving conventional and non-routine problems. _______________________________________________ * Currently the FCAT NRT is the SAT9 published by Harcourt Educational Measurement. Source: Stanford Achievement Test Series, Ninth Edition, Compendium of Instructional Objectives Note: Printed with permission by Harcourt Educational Measurement, August 1999. C-10 Appendix D Appendix D Statutes and Rules, Assessment and Accountability R E L E V A N T S T A T U T O R Y A U T H O R I T Y The following references in the Florida Statutes (F.S.) are directly related to the statewide assessment Program. • Section 1008.21, F.S., requires the Department of Education to implement the School Readiness Uniform Screening System (SRUSS) for students entering kindergarten and to report the results of the screening. • Section 1008.22, F.S., defines the statewide K-12 assessment program, its purposes, and its components. • Section 1008.22 (3) (a), F.S., requires the State Board of Education to approve student performance standards in various subject areas and grade levels. These standards form the basis for the statewide assessment tests. • Section 1008.22 (3) (c) (5), F.S., requires public school students to earn passing scores on the Grade 10 statewide assessment test before receiving a high school diploma. • Section 1008.23, F.S., provides for the confidentiality of the assessment instruments, including developmental materials and work papers; it also removes the tests from the provisions of s.119.07(1) and s.1001.52. • Section 1008.24, F.S., specifies that the various tests administered in accordance with Section 1008.22, F.S., shall be maintained in a secure manner. • Section 1008.25, F.S., requires districts to have pupil progression plans that incorporate statewide assessment results; participation in the statewide assessment tests is required for all students. • Section 1008.29, F.S., requires the Department of Education to implement the College-level communication and mathematics skills test (CLAST), which serves as a mechanism for students to demonstrate mastery of their academic competencies prior to upper-division undergraduate instruction. • Section 1008.30, F.S., authorizes the Board of Education to develop and implement a common placement test for the purpose of assessing basic computation and communication skills of students who intend to enter a degree program at any public postsecondary educational institution. • Section 1008.31, F.S., authorizes a performance accountability system to assess the effectiveness of Florida’s seamless K-20 education delivery system. • Section 1008.34, F.S., requires the Commissioner of Education to prepare annual state, district and school reports of results of the statewide assessment program. • Section 1012.56, F.S., specifies the requirements for educator certifications. D-1 Appendix D The following laws and regulations relate to the various assessment and accountability programs. Assessment Programs and Information Florida Statutes State Board of Education Rules Graduation Requirement for High School Graduation Requirement for Certain Students with Disabilities Requirements Regarding Sunshine State Standards Associate of Arts Degrees, Issuance Universities, Admissions of Students F.S.1003.43 F.S.1003.438 F.S.1003.41 F.S.1007.25 F.S.1007.261 6A-1.0995 6A-1.0996 6A-1.09401 None None School Readiness Uniform Screening System (SRUSS) F.S.1008.21 None Student Assessment Programs High School Competency Test (HSCT) Requirements Florida Comprehensive Assessment Test (FCAT) Requirements Statewide Assessment for Students with Disabilities Procedures for Special Exemption from Graduation Test PSAT/PLAN F.S.1008.22 F.S.1008.22 F.S.1008.22 F.S.1008.22 F.S.1008.22 F.S.1008.22 6A-1.0942 6A-1.09421 6A-1.09422 6A-1.0943 6A-1.09431 None Access, Maintenance, and Destruction of Assessment Materials Maintenance of Test Security F.S.1008.23 F.S.1008.24 6A-1.0944 6A-10.042 Assessment of Limited English Proficient Students Pupil Progression Plan F.S.1008.25 F.S.1008.25 6A-1.09432 None University, Community College, and School District Articulation Procedures for College-Level Academic Skills Test (CLAST) Attainment of CLAST Communication and Computation Skills Minimum Standards of CLAST Skills Application of CLAST Skills in State Postsecondary Systems College Preparatory Testing, Placement, and Instruction CLAST Computation Skills Participation in the CLAST by Non-public Institutions F.S.1008.29 F.S.1008.29 F.S.1008.29 F.S.1008.29 F.S.1008.29 F.S.1008.29 F.S.1008.29 F.S.1008.29 6A-10.024 6A-10.030 6A-10.0311 6A-10.0312 6A-10.0314 6A-10.0315 6A-10.0316 6A-10.0317 Florida College Entrance-Level Placement (FCEP) F.S.1008.30 None School Improvement and Accountability School Grades F.S.1008.31 F.S.1008.34 None 6A-1.09981 Florida Teacher Certification Examination (FTCE) Florida Educational Leadership Examination (FELE) Confidentiality of Assessment Instruments F.S.1012.56 F.S.1012.56 F.S.1012.56 6A-4.0021 6A-4.00821 None D-1 Notes Assessment & Accountability Briefing Book FCAT School Accountability Teacher Certification Tests Florida Department of Education www.fldoe.org Florida Department of Education