The Competitive Edge: Web-based
Assessment of Learning
Joyce A. Scott, Trustee
Richard C. Thompson, Dean
Jones International University
© Jones International University®, Ltd. 2012
Copyright Jones International University, 2012.
This work is the intellectual property of the
author. Permission is granted for this material to
be shared for non-commercial, educational
purposes, provided that this copyright statement
appears on the reproduced materials and notice
is given that the copying is by permission of the
author. To disseminate otherwise or to republish
requires written permission from the author.
© Jones International University®, Ltd. 2012
Jones International University
• Founded in 1993
• First totally online university
• Accredited in 1999 by the
North Central Association’s HLC
• “…the success of our students and the viable,
global democratization of education are our
urgent concerns.” (excerpted from JIU Vision)
© Jones International University®, Ltd. 2012
Student Demographics1
•
•
•
•
74% work full time and 11% are not employed 2
67% are age 30 or older
57% are women
Diversity
– 36% African American
– 3.7 % Hispanic
– 32% White
• 49% earned $40,000+ annually prior to enrollment
1
Data as of January 1, 2012
2 85% of respondents provided employment data
© Jones International University®, Ltd. 2012
Adult Learning
• “…is subsidiary and supplemental to a primary
productive role in society”—Verner (1965)
• Students need to know relevance, utility, application
• Adult students need
– To be self-directing
– Task-centered learning
– Time and conditions for reflection
– Feedback to guide learning, correct mistakes,
improve learning, model professional practice
© Jones International University®, Ltd. 2012
Assessment
• Involvement in Learning (1984)
– Set standards of learning
– Publish standards so students can internalize
– Give feedback to guide/redirect effort
• Integrity in the College Curriculum (1985)
– Faculty should examine learning outcomes
– Evaluate efficacy of students, faculty, programs
• Levels of assessment: From the student to the
institution (2005)
© Jones International University®, Ltd. 2012
JIU Course Design Model
• Philosophical outlook
– Adult learner focus
– Consistent student experience
– Centralized curriculum development
• Key Elements
– Assigned responsibilities
– Specific format (look/feel, pedagogical construct, etc.)
– Assessment built into courses
© Jones International University®, Ltd. 2012
JIU Assessment Infrastructure
Institutional Learning Outcomes→Workplace Competencies
– Communicate effectively using a variety of methods and disciplines
↓
Program Learning Outcomes (MBA)
– Apply selected principles of business communication to a variety of
management challenges
↓
Course Learning Outcomes
– Communicate effectively/persuasively both orally and in written form, in
a variety of multi-cultural contexts.
[BC607 Leading from a Global Perspective]
Institutional Learning Outcomes (ILO)
Each graduate will have demonstrated the ability to:
– think and act like a global citizen with personal integrity, respect
for diversity, and ethical behavior
– think and act like leaders and supportive teammates
– adapt and apply theories to solve “real world” problems using
innovative and creative solutions
– communicate effectively using a variety of methods and
disciplines
Institutional Learning Outcomes-2
– communicate effectively using a variety of methods and
disciplines
– think analytically, critically, and systemically to prepare for a life
of learning and accomplishment
– leverage information and technology to amplify personal and
organizational achievement, and promote information literacy
– design a professional development strategy to formulate a
career pathway leading to future success
Workplace Competencies (WC)
• Convert Institutional Outcomes to measurable
indicators.
• Workplace Competencies use concepts from
specific elements of the ILOs and translate into
practical indicators:
- Completeness
- Critical Thinking
- Timeliness
- Synthesis
- Originality
- Clarity
- Feedback Incorporated
© Jones International University®, Ltd. 2012
Program Learning Outcomes
(PLO)
• At JIU, PLOs are anchored whenever possible in:
– Standards from regulatory bodies, or
– Standards of relevant professional associations, or
– When all else fails, standards are developed by a Program
Advisory Committee convened by the university
© Jones International University®, Ltd. 2012
PLOs
• Example--outcomes for MBA-Project Management :
– Knowledge
• Examine different ethical reasoning concepts, theories, and issues
as they apply to the conduct of business in global and ecommerce environments.
– Skills
• Assess budgeting, cash management, credit administration,
investment analysis, borrowing funds and financial forecasting
tools for effective business decision making.
– Project Management Proficiencies
• Develop timelines, conduct evaluations, and measure overall
strategic impacts for projects.
© Jones International University®, Ltd. 2012
Course Learning Objectives (CLO)
• CLOs:
– developed by faculty, reviewed by curriculum committees
– built for each course within the context of a curriculum map.
– connect each course to the Program Learning Objectives.
• CLOs naturally delve into details of learning outcomes
within the course
© Jones International University®, Ltd. 2012
Competitive Edge or Advantage
• Resources and Capabilities create Distinct
Competencies
• Lead to unique value creation
• Assessment system creates value for both the
students and the university
–
For value added competitive advantage models see Porter’s collective work on Competitive
Advantage or Wernerfelt’s Resource-Based View (RBV).
Assessment & The Competitive
Edge – Student View
• Up-front knowledge of learning expectations
• Directly measured against those expectations
• Student Success Data show progress, current
and progressive
• Individual
• Compared with peers
Individual Feedback
Student view
of performance
in a specific
course
© Jones International University®, Ltd. 2012
Individual Student
Comparative Growth
© Jones International University®, Ltd. 2012
Student Summary Feedback
Student view of performance across all course
outcomes and Workplace Competencies
© Jones International University®, Ltd. 2012
The Learner’s Real-World Portfolio
–
–
–
–
–
–
–
Final course assessments
Real-world experience in the profession*
Professors’ summative evaluations
PSP report or document*
External sponsor’s evaluation*
Projects*
Resume and other job-related documents
* For students engaged in project-based courses or final
professional synthesizing projects
© Jones International University®, Ltd. 2012
JIU’s Competitive Edge
• Student Success Data and Student End of Course Surveys are
aggregated at:
– Course Level
– Specialization and Program Level
• Combination of two assessments provides a rich view of the
performance of courses and programs and is used by:
– Faculty during course revision
– Curriculum Committees
– Program Review Committees
Bi-Annual Assessment Report
School of Business: Student Achievement of Course Learning
Objectives and Workplace Competencies by Course
Legend:
Exemplary: 91%-100% of Course Learning Objectives are assessed as Proficient or Advanced
Average: 75%-90% of Course Learning Objectives are assessed as Proficient or Advanced
Needs Review: 0% - 74% of Course Learning Objectives are assessed as Proficient or Advanced
© Jones International University®, Ltd. 2012
Student Success Data:
Aggregate Achievement of Workplace Competencies and Course
Learning Objectives at the Program Level
© Jones International University®, Ltd. 2012
Student End of Course Survey
Results
Student End of Course Survey 2011
MBA Program and Project Management Specialization
Scale is 1 to 5
MBA
Dec'11
Course Quality Questions
This course was well designed.
4.10
This course was appropriate for my profession.
4.29
Learning objectives were clearly stated at the beginning of
this course.
4.45
My work in this course was high quality.
4.28
I learned what I was expected to learn in this course.
4.24
This course allowed me to be an active learner.
4.26
This course allowed enough time for me to be successful.
4.17
This course's materials and readings aided my learning.
4.23
MBA571 The Project Management Framework
MBA572 Planning Projects to Manage Outcomes
MBA573 Project Controls and Tracking Project Progress
MBA574 Strategic Thinking and Project Management
MBA
2011
Ave.
Project
Management
Specializatio
MBA571 MBA572 MBA573 MBA574 n
4.26
4.40
4.53
4.38
4.55
4.47
4.42
4.68
4.77
4.53
4.63
4.65
4.49
4.34
4.34
4.39
4.17
4.30
4.57
4.51
4.46
4.57
4.09
4.47
4.63
4.57
4.59
4.63
4.48
4.43
4.34
4.47
4.44
4.34
4.29
4.47
4.70
4.63
4.63
4.52
4.45
4.63
4.56
4.55
4.53
4.52
4.33
4.50
Next Phase
• Implement a packaged assessment system (such as
TaskStream or LiveText) to
– Integrate/organize assessment data from current systems
– Provide structured portfolios for students
– Organize/aggregate student work for accreditation reporting; e.g.
HLC, NCATE
• Develop a Business Intelligence system to
– Incorporate packets of data from disconnected systems
– Allow for rich university evaluation of assessment data by
student demographics, financial performance data, and even
national databases
References
•
•
•
•
•
•
•
Association of American Colleges (AAC). (1985). Integrity in the college
curriculum: A report to the academic community. Washington, D C.
Brookfield, S. D. (2006). The skillful teacher: On technique, trust and
responsiveness in the classroom, (2nd ed.) San Francisco: Jossey-Bass.
Knowles, M., Holton, E., & Swanson, R. (1998). The adult learner: The
definitive classic in adult education and human resource development (5th ed.).
Houston: Gulf Publishing Co.
Miller, R. & Leskes, A. (2005). Levels of assessment: From the student to the
institution. Washington, DC: AACU.
U.S. Department of Education. (1984). Involvement in learning: Realizing the
potential of American higher education. Washington, D.C.: Author
Verner, C. (1965). Programming. Review of educational research, 35(3), 177-184.
Wernerfelt, B. (1984). A Resource-based View of the Firm. Strategic Management
Journal, 5(2), 171–180.
For further information about this presentation:
www.cyberschools.com
For further information about Jones
International University (JIU):
www.jiu.edu
© Jones International University®, Ltd. 2012