Assessing reflection
R.Hallett
Assessing reflection: Classroom based research and professional
development processes and practices
Abstract:
Victoria University Melbourne offers a Graduate Certificate in Tertiary Teaching as a professional
development activity for staff. The primary focus of the graduate program is to improve teaching
practice by developing understandings of learning practice through reflection for and on teaching.
Mechanisms for understanding the learning of students in the classes taught by students taking this
course are integral to the course structure. Classroom based research as defined by Angelo and Cross
(1993) is a key mechanism used for this purpose. One subject – one third of the program – requires the
conduct of classroom based research over a semester. Participants are introduced to and required to
use, a variety of techniques that elicit feedback about the quality of the student learning experience. In
addition to integrating these techniques with teaching practices, participants are also guided through a
series of face-to-face and online activities designed to stimulate content, process and premise reflection
on their work as tertiary teaching professionals. Assessing reflection is thus a key task for professional
developers teaching this program. The focus in both a written report and an oral presentation is the
demonstration of reflection on and in teaching rather than the ‘mechanics’ of the research process.
This paper describes the structure of the Graduate Certificate program, how the classroom based
research component is organised within that structure, and how the classroom based research activity
is managed through face- to- face and online approaches. It discusses the linkage between the
assessment of reflection and professional development processes and practices in tertiary settings.
Background
Victoria University is a dual sector institution comprising TAFE and Higher
Education divisions. The Graduate Certificate in Tertiary Teaching is offered HECSfree, as a voluntary professional development activity for teachers (.5 and above) in
both divisions. It is therefore tied to institutional goals that include the development of
a teaching culture that is inclusive of both divisions. As a result of this institutional
orientation, the course is referenced to a definition of learning and teaching that is
inclusive of TAFE and Higher education policies, processes and practices as they
apply to in Higher Education and TAFE disciplines and fields of practice. In the 5
years since its inception, over 100 VU VET teachers and Higher Education academics
have graduated from the program. The majority of graduates are TAFE teaching
professionals. Contract and sessional teaching staff do not gain HECS-free access,
although each year, one or two participants in this category enrol. It is currently
offered as three subjects over a one-year timeframe, although completion in one year
is not a requirement. The Certificate is part of the School of Education postgraduate
program, and is taught by staff from the University’s Centre for Educational
Development and Support (CEDS).
The program draws on the ‘educator as reflective practitioner’ model of professional
development derived from the work of Schon (1987), and later modified by others
(Boud & Walker 1998, for example). In recognition that the concept of the ‘reflective
practitioner’ is loosely defined and is open to a range of interpretations, progressive
program iterations have attempted to frame stated learning outcomes in terms of what
it means to be reflective, and to align these with assessment processes and practices.
These efforts have been informed by the conceptual basis of reflection offered by
Eraut (1995), as these ideas apply in the VU context. Assessment practices include the
use of Structured Observations of Learning Outcomes (SOLOs), developed by Biggs
(1999). In this program, SOLOs are used to determine levels of achievement as well
as provide feedback to participants about their development as tertiary teaching
professionals.
1
Assessing reflection
R.Hallett
In the context of a professional development program such as the Graduate Certificate
offered to academics and TAFE teaching professionals at VU, assessment has two
purposes. The first is focused on the individual and is aimed at facilitating the
learning of academics through reflection so that professional development is ongoing.
It is developmental and thus formative. This purpose lends itself to process-orientated
professional development approaches that engender qualitative assessment
methodologies aimed at uncovering how well students understand learning and
teaching in their context. The second purpose is summative. It aims to assess skills
and knowledge of learners so that accreditation can be granted. Summative
assessment lends itself to quantitative approaches that aim to determine how much
students know about learning. The focus of this assessment is on institutional goals
rather than the needs of individuals. Assessment of learning in programs such as the
Graduate Certificate thus needs to meet these two goals simultaneously. There is a
danger that, because the process of reflection is difficult to capture, assessment
processes and practices could focus on quantitative measures (‘how much’). This is
the basis of current critique of programs that seek to accredit teaching performance in
Higher Education (for Nicholls 2001 in the UK). The challenge is to ensure that
quantitative assessment measures are used to capture the intent of process-orientated
professional development programs (‘how well’), while also reporting performance to
‘standards’. There is thus a tension between individual and institutional demands that
is evident in assessment processes and practices.
Conceptual framework for Graduate Certificate
The approach adopted in the Graduate Certificate offered at VU to resolve the tension
between the personal and institutional perspectives is to locate reflection in terms of
the personal and professional growth of academics. Critical reflection on practice is
located as central to the learning process for academics (Boud , Keogh and Walker,
1985). Critical reflection is about questioning current beliefs, and identifying new
ways of acting, and is transformative (Mezirow, 1992: 5). It has potential to stimulate
changed practices both at the individual and institutional levels. It has greater potency
for academics already engaged in practice, as there is an established practice to reflect
on. For those new to teaching, the focus of professional development programs is on
induction at the institutional level so that critical reflection can arise at a later date
(add reference). Professional development programs involving staff already teaching
such as the Graduate Certificate, therefore have capacity to bring about change both
for individuals, but also on institutional processes and practices, if individuals have
capacity to effect change. In this program, critical reflection is located as a thematic
link between the three subjects comprising the Graduate Certificate.
Critical reflection is achieved by reflecting on practice. According to Mezirow’s
transformation theory, critical reflection is achieved through engaging in content,
process and premise reflection. Content reflection focuses individuals on the content
or description of a problem and prompts the question ‘what should I do, know or find
out?’. Process reflection focuses on strategies to solve identified problems and
involves asking ‘how do I know if it works/if I am effective?’. Premise reflection asks
‘what is the relevance of the problem itself ?’. The Graduate Certificate is organised
so that participants engage in each type of reflection in different ways and in different
combinations through out the course. In each subject, assessment tasks require
2
Assessing reflection
R.Hallett
content, process and premise reflection. However, the focus of assessment in each
subject is on one of the types identified by Mezirow.
The focus on each type of reflection in each set of assessment tasks are aligned to the
teaching and learning activities devised for each subject comprising the Graduate
Certificate. The teaching and learning approaches adopted, and the assessment tasks
in each subject are:
 classroom-based activities such as lectures and small group work delivered in
an intensive program (An Introduction to Tertiary Education). The assessment
task required for the classroom – based-activities is a Teaching Portfolio,
which is conceptualised primarily as a content reflection. Participants are
asked to think about ‘what should I do, know or find out about how to
develop myself as a tertiary teaching professional?
 Problem Based Learning (PBL) activities delivered over one semester (Issues
in Tertiary Teaching). The assessment task for the (PBL) activities is a
proposal for change within the participant’s immediate teaching environment.
It is conceptualised primarily as a process reflection. Participants are
encouraged to answer ‘how do I know if systems work and teaching can be
effective?’.
 and work-based classroom research delivered over one semester (Improving
Tertiary Teaching). The assessment tasks for the work-based research activity
are conceptualised as a premise reflection. Participants answer ‘what is the
relevance of the problem I have identified in my classroom? Have I asked the
right question?’. These questions are answered in both assessment tasks – a
report on the classroom based research and a presentation at a conference.
Thus, although premise, process and content reflection are required in each subject,
there is an attempt to deconstruct the notion of ‘reflection’ through conceptualising
each assessment task against one of these types.
Mezirow’s types of reflection are also linked conceptually in design of assessment
tasks, to the purposes of reflection identified by Schon (1983) and Cowan (1998). In
order to achieve these, each subject addresses the development of the hierarchy of
knowledge forms identified by Biggs (1996). According to Biggs, functioning
knowledge (knowledge used in practice) needs to have a foundation of procedural
(knowing how) and declarative (knowing what) knowledge, that is then used to form
conditional knowledge (knowing when and why). In each subject, the relationship
between the purpose of assessment and knowledge forms is outlined below:



The purpose of the teaching portfolio, as a content reflection, is to encourage
reflection for action (Cowan). Classroom based activities in that subject are
focused on filling the gaps in procedural (knowing how to engage in
classroom management and evaluation) and declarative knowledge (knowing
what current learning and teaching theories are).
The purpose of the policy proposal, as a process reflection, is to encourage
reflection on action (Schon 1983). The focus of these activities is on building
conditional knowledge (knowing when and why). The PBL process provides
opportunities for participants to experiment with possible solutions to work
based ‘problems’ in a low risk environment.
The purpose of classroom based research assessment, as a premise reflection,
is to encourage reflection in action (Schon 1983). Work based learning
3
Assessing reflection
R.Hallett
processes aim to develop awareness of practice, and as such, focus on
functioning knowledge. 1
Across the program, assessment tasks are designed to develop capacities to articulate
assumptions underpinning practice critically question those assumptions and then
make changes on the basis of that critique.
Engagement in reflection is finally linked in this program to ideas about the
scholarship of teaching originally described by Boyer (1990). Although meanings
remain open to debate (Kreber 2002), critical reflection has emerged as pivotal to
emerging definitions. There is broad agreement that critical reflection along with
scrutiny by peers and inquiry into one’s own practice, are elements of the emerging
definitions (Martin & Ramsden 2000, Trigwell et al 2000, Kreber & Cranton 2000,
Andresen 2000). Critical reflection is achieved through engaging in content, process
and premise reflection (Mezirow 1992). In the Graduate Certificate, assessment is
structured so that participants are introduced to, and guided through a process of
critical reflection on their practice. This process theoretically, provides opportunities
for the demonstration of engagement in the scholarship of teaching. However, not all
assessment tasks require demonstration of all elements identified as essential to
definitions of the scholarship of teaching, most notably, communicating the outcomes
of inquiry into practice through peer review (Trigwell et al 1999). The scholarship of
teaching in this course privileges reflection into one’s own teaching practice and the
learning of students within the context of a particular discipline, and the scholarly
contributions of others on teaching and learning (the other two characteristics
identified by Trigwell et al). This suggests that the scholarship of teaching in this
course is one of many forms of scholarship that can exist (Cross & Steadman 1996).
The VU approach draws on a model developed by Kreber & Cranton (2000). This
model utilises Mezirow’s reflection typology to suggest that there are nine possible
ways in which teachers can reflect on their practice by focusing reflection on
research-based and or experience-based knowledge. In this model, teachers engage in
content, process and premise reflection in the areas of instructional design,
pedagogocial knowledge and curriculum knowledge. The approach adopted at VU is
similar. However, the knowledge domains identified at VU are in terms of
declarative, procedural, conditional and functioning knowledge. Whereas the Kreber
model distinguishes the knowledge forms into instructional, pedagogocial and
curriculum types2, the approach adopted at VU is integrative, with the emphasis on
functioning knowledge being the result of integration of each of these knowledge
forms. Thus for example, Schulman’s pedagogical content knowledge, separated in
the Kreber model, is conceptualised in the VU approach as integrated with all three
knowledge forms that comprise functioning knowledge. The conceptual framework
for the Graduate Certificate is functioning knowledge – expressed as the functioning
knowledge that teachers in the program come into it with. The focus is on building
awareness in teachers about the scope and nature of their functioning knowledge.
Each subject thus explicitly focuses on building aspects of functioning knowledge
(declarative, procedural and conditional) in particular ways. The focus on reflection in
2
Instructional knowledge is identified in terms of understandings of all aspects of instructional design,
pedagogical knowledge is knowing how students learn and how to facilitate learning and curriculum
knowledge is knowledge of goals purposes and rational for classes and courses by Kreber.
4
Assessing reflection
R.Hallett
this framework is thus not either research-based or experience-based knowledge, but
on their integration as functioning knowledge. In this approach, the scholarship of
teaching is referenced to functioning knowledge, and as such, recognizes the multiple
forms of scholarship of teaching (Cross and Steadman, 1996) in terms of how teachers
integrate espoused theories and theories in use as defined by Schon (1983).
Classroom based research
The Graduate Certificate subject, Improving Tertiary Learning and Teaching,
explicitly assesses reflection on functioning knowledge. The focus is on research into
learning as it arises in participant’s own classrooms, and as such, is a discipline-based
approach to the scholarship of teaching. Classroom based Research as defined by
Angelo & Cross (1993), Cross & Steadman (1996) and McKernan (1996) is primarily
concerned with improving student learning through the investigation of practical
issues that arise in the classroom rather than with generating research publications. It
sits conceptually, on a continuum between informal evaluation and major educational
research (Healy 2000). Participants in this subject investigate their own practice by
identifying a practical issue in a particular class they teach. They are led through the
process of classroom-based research outlined by Angelo and Cross (1993). They are
therefore required to use the Teaching Goals Inventory (TGI) to clarify their
instructional goals, and Classroom Assessment Techniques (CATs) as a means of
eliciting student feedback about what and how their students are learning. According
to Angelo (1995), improved critical thinking can arise for teachers as well as students
when teachers engage in collecting and acting on feedback from students when they
undertake CATs.
In addition to using the tools developed for the conduct of Classroom based
Research, participants are required to undertake other tasks that aim to stimulate
reflection. These include a series of self-reflections (Cranton 2002), interviews with
their peers about how students learn in their discipline, database searching for
discipline-specific information about teaching, and exercises to determine teaching
styles and approaches (Bennett 1999). Face-to-face activities (once a month) are
discussion sessions, which are continued online. The tasks are undertaken out of class
time, and are reported online to the whole group and are periodically peer reviewed.
The SOLO statements provide clear descriptions about what is required to achieve
levels of attainment from an H1 to a pass (Biggs 1996). .3 These SOLOs, like others
used in the Certificate program, provide feedback to participants about the type of
reflection engaged in that particular element of the course. That is;
3
A written report – worth 90% of the assessment -outlines the process undertaken to investigate
practice, and is assessed against the following criteria: planning classroom research, planning and
designing learning activities, using feedback for continuous improvement, and articulating what was
learned both about themselves as teachers and how their students learn. The remaining 10% of marks
are for a 10-minute talk at an internal teaching symposium. Assessment criteria for this task are: clarity
of explanation of the research process, and articulation of the impact of the research on them as tertiary
teaching professionals in terms of what they have learned about how students learn in their
discipline/field of practice.
5
Assessing reflection
R.Hallett
Professional development processes and practices
The reflective model for professional development, though pervasive, still remains
vague. The program described in this paper attempts to clarify what reflection is and
how it can be assessed in ways that are meaningful for individuals engaged in it as a
process-orientated professional development activity, as well as for accrediting
bodies. The conceptual framework for the program rests on a model for reflection that
utilises Mezirow’s (1992) reflection typology as well as knowledge forms identified
by Biggs (1999). This approach differs from a similar one developed by Kreber &
Cranton (1997) in that the functioning (or working) knowledge that teachers bring to
the program is the starting point for conceptualising their knowledge needs. The
SOLOs that guide assessment in each subject describe different ways that reflection
can be achieved when participants engage in reflection for action, on action and in
action. These provide a set of descriptions of ways to reflect in particular contexts that
can be used in conceptualising the reflective model as a framework for professional
development support in tertiary education. In addition to pointing to ways of
conceptualising professional development support for those new to teaching as well as
those already in it, these descriptions also raise issues about the scholarship of
teaching in these contexts.
Programs for teaching in new contexts.
The approach adopted in the Graduate Certificate program provides a conceptual
framework for professional development programs for staff new to teaching – or those
new to teaching in a particular context. Assessment of a content reflection aims to
assist teachers to review their prior experiences and anticipate what they may do to
extend their procedural and declarative knowledge domains in light of information
they have received through instruction in the program (including institutional policies
and processes). Assessment tasks (in the case of the Graduate Certificate, a Teaching
Portfolio) need to guide new teachers through an analysis of what they decide to do to
develop their skills. In order to avoid the danger that assessment could amount to an
exercise in counting how many skills that teachers need to acquire (or how many they
already have), the assessment tasks should also introduce the ideas of process and
premise reflection and include consideration of these in the final assessment. SOLOs
that provide qualitative descriptions of different levels of understanding of how the
content reflection manifests in the execution of the task avoids these dangers. At the
higher levels of understanding, qualitative descriptions also include elements of
premise and process reflection (on one’s career to date in the case of a Teaching
Portfolio), for example. These activities are conceptualised as an introduction to the
scholarship of teaching rather than an engagement with it.
Programs for ongoing professional development
The assessment of content, process and premise reflection also provides a framework
for ongoing professional development processes and practices. For those already
operating in a particular context, reflection in terms of each type brings aspects of
functioning knowledge as it relates to teaching into awareness. Functioning
knowledge is primarily defined in terms of disciplinary knowledge for academics in
particular, and because of this, professional development processes and practices that
develop pedagogical content knowledge (Shulman, 1987) – knowledge of how
students learn and how to facilitate learning- in terms of discipline knowledge are
6
Assessing reflection
R.Hallett
recognised as the most desirable (Beaty 2001, Trowler & Cooper 2002, Neuman
2001, Nicholls, 2001).
In the Graduate Program described here, content, process and remise reflection for
those already teaching involves exploring their discipline knowledge, and possible
new professional identities. For those involved in content reflection and the
development of a Teaching Portfolio, it involves preparing for teaching in similar
ways to how those new to teaching or the institutional context undertake the process.
Content reflection in this framework assumes an orientation to aspects of work that
they have previously not considered. For those involved in subjects where process and
premise reflection are the focus, reflection involves engagement with discipline-based
information about learning and teaching. Again, SOLOs provide teachers with
descriptions of their performance of in terms of capacities to reflect. These tasks are
conceptualised as scholarship of teaching at the higher levels of qualitative
descriptions of the SOLOs in the subjects where process and premise reflection are
the focus of assessment. However, it is recognised that the scholarship engaged by
those in the program does not include strong elements of peer review and
dissemination, and therefore does not assess all of the characteristics identified by
Trigwell et al as high-level scholarship.
Professional development in a dual sector institution
The approach described here has been developed in a dual sector setting, and has thus
involved conceptualising functioning knowledge in terms of the work of TAFE
teachers as well as Higher Education academics. The approach described above has
included both TAFE teachers and Higher Education academics. One of the major
impediments arising for participants in the program is access to discipline-specific
literature about how students learn. This is particularly the case for TAFE
participants, as there is a limited literature dealing with learning and teaching in the
TAFE classroom. This currently represents a barrier to encouraging the engagement
of the scholarship of teaching through reflecting on one’s own practice by comparing
it with what is written in the literature. As a result of this, and because of the focus on
functioning knowledge in this approach, engagement with the literature is being redefined in terms of both the educational discourse (general issues) and the discourse
of practice (issues specific to the discipline )– which participants from both divisions
are required to explore. The inclusion of the discourse of practice as a source of
information about how students learn – and thus an activity that engages the
scholarship of teaching, is as aspect that has arisen in this program.
References
Angelo, T (1995), Beginning the dialogue: thoughts on promoting critical thinking:
Classroom assessment for critical thinking, Teaching of Psychology, Vol 22, No 1,
Feb.
Angelo, T & Cross, K (1993) Classroom Assessment Techniques: a Handbook for
College Teachers (2nd ed.) San Francisco: Jossey-Bass.
Beatty, L (2001), Teaching Evaluation and Accreditation, New directions for
Teaching and Learning, No. 8, Winter.
7
Assessing reflection
R.Hallett
Biggs, J. 1999. Teaching for quality learning at university: what the student does.
Buckingham : Society for Research into Higher Education & Open University Press.
Boud, D & Walker D, (1991), Experience and learning : reflection at work,
Geelong, Vic : Deakin University : distributed by Deakin University Press.
Boud, D, Keogh R, Walker, D (19985), Reflection, turning experience into learning
London : Kogan Page .
Boyer, E (1990), Scholarship Revisited. Princeton, NJ: Carnegie Foundation for the
Advancement of Teaching.
Cowan, J (1998), On becoming an innovative university teacher : reflection in action.,
Buckingham ; Philadelphia, PA : Society for Research into Higher education & Open
University Press.
Cranton , P (2001), Becoming an authentic teacher in higher education
Malabar, Fla. : Krieger Pub. Co.
Cross, K & Steadman, M (2000), Classroom research: Implementing the scholarship
of teaching. Jossey-Bass, San Francisco.
Eraut, M (1994), Developing Professional Knowledge and Competence, Falmer,
London.
Healy, M (2000), Developing the Scholarship of Teaching in Higher Education: a
discipline-based approach, Higher Education Research & Development, Vol 19, No.
2.
Kreber, C and Cranton, P (1997), Teaching as Scholarship: a model for instructional
development, Issues and Inquiry in College Learning and Teaching, 19 (2), pp. 4-13.
Kreber, C and Cranton, P (2000), Exploring scholarship of teaching, Journal of
Higher Education, 71, pp. 476-496.
Mezirow, J (1992) Transformative Dimensions of Adult Learning, Jossey-Bass, San
Francisco, CA.
Nicholls, G (2001), Professional Development in Higher Education, Kogan Page Ltd,
London.
Neumann, R (2001), Disciplinary differences and university teaching, Studies in
Higher Education, Vol. 26, Issue 2.
Schon, D (1983), The Reflective Practitioner: How Professionals Think in Action.
New York: Basic Books
Shulman , l (1987) knowledge and teaching, Harvard Educational Review, Vol 57,
pp. 1-22.
8
Assessing reflection
R.Hallett
Trigwell, K., Martin, E., Benjamin, J., Prosser, M. (2000) scholarship of teaching: a
model, Higher Education Research and Development, 19, pp. 155-168.
Trowler, P & Cooper, A (2002), Teaching and Learning Regimes: Implicit theories
and recurrent practices in the enhancement of teaching and learning through
educational development programs, Higher Education Research and Development,
Vol 21, No. 3.
9