Diverse Models and Challenges: Assessment for Learning in Online Settings SAS Center for Educational and Instructional Technology Research (CEITR) Presented By: Mansureh Kebritchi, Ph.D. & Kate Andrews, Ph.D. Context Online education changes all components of teaching and learning including assessment. Many empirical studies were conducted in examining effective assessment techniques. Problem Lack of literature reviews to provide an integrative report on effective assessment techniques and existing challenges in assessment of learners Problem Often online educators have to go through daunting task of sifting the increasingly expanding literature to identify these strategies and issues. Purpose Presents the results of a critical literature review on models, best practices, and challenges of assessment of learners in higher education online courses. Method Cooper’s (1988) procedure was used: Formulate the problem Collect data Evaluate the appropriateness of the data Analyze and interpret relevant data and Organize and present the results Method Empirical studies and literature reviews conducted for online courses between 2004 and 2014 The keywords: “Online assessment”, “Assessment models”, “Assessment Techniques” Searching sources: oOnline journals oAcademic search engines and databases (EBSCO HOST, ERIC, Google Scholar) oRelated conference websites Method: Data Analysis and Evaluation 26 articles were found: 12 focused on issues related to assessment models 8 discussed assessment techniques 4 focused on assessment challenges The remaining articles that did not discuss specific issues in the above three aspects were excluded from this review Findings: Models Using mix of various assessment models: Dynamic assessment Formative assessment Summative assessment Diagnostic assessment Findings: Dynamic Assessment Recommended Model Has four components: Individual differences Assessing the very processes of systematic thinking, learning, and problem solving not the product Assessing learning potential, educability, and the zone of proximal development Mediation (Haywood, 2012) Findings: Dynamic Assessment Individual Differences Identify each individual’s unique qualities, strengths, weaknesses Discover individually useful methods for maximizing individuals’ learning effectiveness. Findings: Dynamic Assessment Assessing the Process “an important way to learn about the characteristics of events that are not directly observable is to change those events and then to observe the effects of the changes”. (Haywood, 2012, p.220) Direct observation of learning Comparison of effectiveness of help or instruction Use of transfer paradigm Findings: Dynamic Assessment Assessing Educability What is versus what could be Learners with larger zone of proximal development (ZPD) will do much better in school Cognitive development as a dynamic phenomenon (Vygotsky, 1934,1986) Need for assistance to be provided for learners in the assessment of their ZPD Findings: Dynamic Assessment Mediation A very special kind of interaction between a learner/performer and a more cognitively competent person (e.g., mentee and mentor) Goals: To discover obstacles to effective performance, To neutralize those obstacles whenever possible, To assess the effects of “obstacle removal” on subsequent performance on the same or similar tasks. Findings: Dynamic Assessment Practical Framework: Assess Feedback Intervention Review (Lauchlan, 2013) Findings: Techniques Techniques including: Use of proctored tests Maintain continuous communication with learners Maintain dynamic interaction Modify traditional tools for online assessment Use of alternative assessments such as: authentic, performance, portfolio assessment (Gaytan, 2005) Findings: Challenges Cultural diversity o attitudes toward and approaches to learning o failure to have developed a strong cultural identity; often associated with transculturality Knowledge gap o Inadequate store of information Ineffective motivation components o Emphasis on task-extrinsic sources of satisfaction and consequent failure to have developed a task-intrinsic motivational orientation (Haywood,2012) Conclusion This literature review identified assessment models, techniques, and challenges in online settings Online educators and instructional designs, may consider the identified issues to improve assessment of online courses References - Black, P., & Wiliam, D. (2009). Developing the Theory of Formative Assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ829749&site=ehost-live - Clark, I. (2012) Formative assessment: Assessment is for self-regulated learning. Educational Psychology Review, 24(2), 205-249. - Crossouard, B., & Pryor, J. (2012). How theory matters: Formative assessment theory and practices and their different relations to education. Studies in Philosophy and Education, 31(3), 251-263. - Dodge, J. (2013). What are formative assessments and why should we use them? Tips for using formative assessments to help you differentiate instruction and improve student achievement. Scholastic Inc. Retrieved from http://www.scholastic.com/teachers/article/whatare-formative-assessments-and-why-should-we-use-them References - Feuerstein, R., Feuerstein, R. S., Falik, L. H., & Rand, Y. (2004).The dynamic assessment of cognitive modifiability. Jerusalem: International Center for the Enhancement of Learning Potential. DOI: 10.1177/0734282906290365 - Fullan, M. & Donnelly, K. (2013). Alive in the swamp, assessing digital innovations in education. Retrieved from http://www.michaelfullan.ca/wp-content/uploads/2013/06/13_Alive_in_the_Swamp.pdf - Gaytan, J., & McEwen, B.C. (2007). Effective online instructional and assessment strategies. American Journal of Distance Education, 21 (3), doi: 10.1080/08923640701341653 - Griffith, D. & Mellor, M. (2014). Making accountability meaningful. Retrieved from http://www.ascd.org/publications/newsletters/policy-priorities/vol20/num02/Making-AccountabilityMeaningful.aspx - Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. - Haywood, H. C. (2012). Dynamic assessment: A history of fundamental ideas. Journal of Cognitive Education and Psychology,11(3), 217-229. References - Herold, B. (2014). Testing digital tools to improve formative assessments. Retrieved from http://www.edweek.org/ew/articles/2014/03/13/25personalized.h33.html - Lai., E.R. & Viering, M. (2012). Assessing 21st century skills: integrating research findings . Retrieved from http://researchnetwork.pearson.com/wp-content/uploads/Assessing_21st_Century_Skills_NCME.pdf - Morrissette, J. (2011). Formative assessment: Revisiting the territory from the point of view of teachers. McGill Journal of Education, 46(2), 247-265. - Niemivirta, M. (2006). Assessing Motivation and Self-Regulation in Learning within a Predictive Design: Incorporating Systematic Elements of Change. Educational Psychology Review, 18(3), 255–259. Retrieved from 10.1007/s10648006-9020-5 - Offerdahl, E. G., & Tomanek, D. (2011). Changes in instructors’ assessment thinking related to experimentation with new strategies. Assessment & Evaluation in Higher Education, 36(7), 781-795. - Price, M., Carroll, J., & O'Donovan, B. (2011). If I was going there I wouldn’t start from here: A critical commentary on current assessment practice. Assessment & Evaluation in Higher Education, 36(4), 479-492. References - Randall, L., & Zundel, P. (2012). Students’ perceptions of the effectiveness of assessment feedback as a learning tool in an introductory problem-solving course. Canadian Journal for the Scholarship of Teaching and Learning, 3(1), 1-17. - Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189. - Stecher, B. & Hamilton, L.(2014). How to assess 21st century competencies 12 key lessons. Retrieved fromhttp://blogs.edweek.org/edweek/global_learning/2014/02/how_to_assess_21st_century_compe tencies_skills_12_key_lessons.html - Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3-14. - Wolsey, T. (2008). Efficacy of instructor feedback on written work in an online program. International Journal on ELearning, 7(2), 311–329. Thank you! Any Questions? Contact Information: Mansureh Kebritchi: Mansureh.kebritchi@phoenix.edu Kate Andrews: Kateandrewss@email.phoenix.edu SAS Center for Educational and Instructional Technology Research (CEITR) EducationalTechnology@Phoenix.edu