Process for mapping and analysing a major

advertisement
Process for Mapping and Analysing a Major
Introduction
This stage of the mapping process is focused on the characteristics of the current major and how
individual courses contribute to this. There are two tasks in the process. The first requires the
individual course reviews to be collated in an Excel spread sheet. Initially, this was undertaken under
the Curriculum Renewal Project, but subsequent mapping or revisions to existing mapping will need
to be organised through the school or faculty. The second task involves discussion of the major
among discipline staff using, among other sources of information, the collated documentation.
Because the configuration of majors varies from highly structured (eg languages) to very open across
the Faculty, in the first task two different sets of proformas have been developed to facilitate the
collation processes and these should be adapted as required for specific majors.
Mapping the major is the second step in reviewing the curriculum in a broad process of curriculum
renewal that involves the following stages:




Reviewing the current curriculum
 course coordinators map and analyse each constituent course in the major
 discipline academics map the major by collating the individual course reviews and
collectively analysing the results
Agreeing on the preferred curriculum
 discipline academics collectively determine the preferred characteristics of the major
(content, weightings of the DSLOs, progression of skills, etc). It may be useful to
consider how other universities or international higher education systems construct
similar majors.
Renewing the curriculum
 course coordinators reshape the curriculum of their courses (learning outcomes,
assessment, and teaching and learning arrangements) to fit the expectations of the
major
Documenting the changes
 course coordinators complete necessary documentation and submit through the
appropriate channels.
The next stage of the process Agreeing on the preferred curriculum, is closely related to this stage,
and it may be helpful to undertake them together, in quick succession, or to capture ideas as they
emerge from the discussion of the review of the major.
Documents
Task 1 is the collation of all course reviews in the major using the relevant Excel proforma (for either
high or low levels of structure or multi-stranded awards). Completed course review documents need
to be provided to the person undertaking the collation process. Basic knowledge of Excel would be
an advantage.
Once the course reviews have been collated these documents need be provided to the Heads of
Discipline and made available to other Discipline staff of the major for Task 2.
Task 2 uses the completed proformas, among other resources, as the basis of detailed discussion
and analysis by Discipline staff of the major. To support the group discussion you may find the
following documents useful:
1
Process for Reviewing Majors






your Discipline-Specific Learning Outcomes (access restricted to HUMSS staff)
review documentation for all courses in the major prepared through the previous process,
and in particular the tables and analysis
Research Skills Development Framework
Assessing graduate attributes
Teaching and learning activities and graduate attributes
Academic literacies and e-learning skills
Purpose of Mapping a Major
Mapping the major has two purposes.
First, it provides an opportunity to see the likely learning outcomes of studying the current major.
Although majors are taught by staff as individual courses, students experience them within the
context of a major and, more broadly, an award. To collectively comprise a ‘coherent body of study’
as stipulated by the University, a major needs to be more than a collection of individual courses.
Where majors are very open structurally this can be challenging.
Second, mapping provides the opportunity to identify how individual courses contribute to the
overall outcomes of the major. That is, it displays how students experience increasing academic
demands throughout their studies: greater complexity in the discipline specific discourse,
introduction to new academic skills and literacies, varieties of assessment, and exposure to contextspecific applications and practices.
Background
The course mapping process is intended to get some sense of the emphases and intentions of the
courses and how this impacts on the major. This is of course a highly subjective and individual
activity. The course reviews do have value at the course level but the main focus of this stage of the
process is the major. In that context the collated course reviews enable a perspective across the
constituent courses and inform decisions in relation to the major. The quantification is simply a way
of aggregating the course data to get this higher level perspective.
The review process produces very rough data. The data varies a lot across courses and this depends
to a significant extent on how the learning outcomes are written and the nature of the assessment.
If the learning outcomes are focused statements and there are just a few points of assessment, the
review process is less complicated.
The key point here is that the course review data in both its individual and aggregated forms is an
estimate of the emphases of the major. The approach is to use the numerical values in a broad way
to identify major gaps or over-emphasis. Its value is in facilitating a dialogue around shared insights
and interests rather than as a process driven by the data.
In addition to mapping the emphases of the major, information is also gathered on the development
of research skills, academic literacies and e-learning to be used in the documentation in program
reviews. The summary of issues table on the final page of this document makes provision for
comment for this purpose. The original process for mapping research skills was problematic for a
number of reasons and so the consideration of research is best undertaken as part of general
discussion. The Research Skills Development Framework may be a useful tool in this respect. Some
disciplines may find it useful to use other frameworks.
2
Process for Reviewing Majors
The review of the major focuses on the following overlapping themes:







current emphases of the curriculum
alignment between the learning outcomes and the assessment
nature of the assessment
structure and pre-requisites
development of academic literacies and expectations
development of research skills
support for assessment through teaching and learning arrangements
The current emphases of the major are the result of choices that have been made about the
inclusion (and also exclusion) of particular knowledge, skills and application. Because not everything
in the field of study can be taught in the major, it raises questions about what is chosen to be taught,
how those choices are made and what justification there is for that choice. A key question is
whether the current approach is an appropriate expression of the field or feasible in the current
circumstances.
In developing the Discipline-Specific Learning Outcomes, your discipline has identified in broad terms
the characteristics of the major. Reviewing the current major against the DSLOs will display the
extent to which the current emphases are a manifestation of the stated characteristics. It may be
that some of the DSLOs are not present at all in the course learning outcomes or assessment, or
some aspects might dominate over others.
Furthermore, by comparing the aggregated scores for the major’s course learning outcomes with the
aggregated scores for assessment, a rough estimate of the extent of alignment across the major can
be seen. That is, it demonstrates in general terms whether the intentions of the major (as expressed
in the course learning outcomes) are consistent with the assessment; having similar scores means
good alignment. The proformas also aggregate the data at the level of the Graduate Attributes,
providing a higher-level perspective of the data. In the design of a curriculum, alignment is an
important issue because it is concerned with the validity of the assessment.
The nature of the assessment across the major is a central issue. The types of assessment tasks (eg
essay, presentation, report, poster) need to be appropriate in making judgments about the
achievement of certain kinds of learning outcomes. For example, it is not appropriate to assess
information literacy through an exam, or oral communication skills only through the text of a
speech. In addition, the variation in the types of assessment provides students with the opportunity
to develop a range of communication and other skills that enhance their learning experiences and
often their career prospects as well.
A large part of how students experience individual courses is related to the structure of the major
and the way this supports the developmental aspects of learning within the discipline. If the major is
highly structured the progression of skills and understandings will be apparent through the prerequisites and co-requisites and other formal mechanisms.
In majors where there is minimal structure, the benefits of student choice and curriculum flexibility
need to be balanced with ensuring students have the necessary knowledge and skills to successfully
undertake advanced level courses without completing relevant courses at lower levels. Where there
is minimal formal structure and few pre-requisites or co-requisites, students may gain informal
advice through the local student networks or from advice by staff. In majors where this is the case, it
might be useful to make these implied or preferred (but not required) pathways more accessible by
openly advocating sequences, recommending complementary courses, or identifying patterns that
3
Process for Reviewing Majors
might lead to particular employment outcomes. To some extent, courses with large enrolments can
be seen as a proxy structure and so it may be useful to load these courses with particular outcomes
of the major in order to maximise the number of students achieving these outcomes.
Compulsory courses at Level 1 such as the BA core course or discipline-based foundation courses
provide the opportunity for all students to gain some entry level knowledge and skills. Where this
kind of structure does not occur, it is important that provision is made for the systematic
introduction of knowledge and skills to prepare students for the greater demands of higher level
courses. Although it reduces flexibility, there may be some advantage in introducing some structure
elements, particularly in preparation for capstones.
Related to structure is the development of academic literacies and expectations. This includes
progressively increasing word counts in essays, requiring higher order thinking skills, introducing a
variety of assessment types so that students gain a range of communication and other skills,
ensuring there is consistency in expectations, and providing adequate levels of support for particular
activities at varies stages of the student life cycle.
Where majors are highly structured the transitions between courses or year levels are predictable
and can be managed in direct ways. Where majors have minimal structure, the approach needs to
identify the learning outcomes and then to create various pathways that will achieve those
outcomes. This will involve providing multiple opportunities to learn particular skills in order to
maximise students’ exposure. It also involves ensuring easy access to support often through an
agreed set of resources. For example, a number of courses may include the assessment requirement
to use a particular database, or to engage students in various kinds of group work (projects, studentmanaged groups, presentations), or to use the online assessment facilities on MyUni. Such a
systematic and intentional approach encourages the development of skills over time and in different
contexts, and is conducive to learning for students with a range of learning styles and life
experiences.
One of the defining characteristics of the University of Adelaide is its focus on research for all
students and so the systematic development of research skills is a central feature of all
undergraduate degrees. As part of their induction into a discipline, it is expected that students will
experience the discipline-specific approach to research, and engage in research-based processes and
activities in an intentional and measured way appropriate to their level of study. Because the
approach to research differs across disciplines, the various majors will undertake this developmental
process in different ways. The Research Skills Development Framework (Level 3 and above) may be a
useful tool in undertaking this and a spreadsheet using this structure is provided in the templates for
this purpose.
Finally, the ways in which assessment is supported through teaching and learning arrangements,
including access to resources, is central to the design of a curriculum. This is predicated on the idea
that students need to be explicitly supported in achieving the assessment; and this includes more
than just the mastery of content. Academic literacies including writing, thinking, accessing resources
and evaluating their relevance, and working in groups are developmental learning processes that
need to be intentionally, systematically and actively taught in the context of courses and in
association with assessment.
4
Process for Reviewing Majors
TASK 1 Collation of the Course Mapping
Three templates have been developed for the collation of the courses mapping:



Template 1 - Minimal Structure Major/Program
Template 2 - High Structure Major/Program
Template 3 - Multi-Stranded Major/Program (including foreign languages)
Using the appropriate Excel template, arrange for the data from all courses in the major to be collated.
This includes all Level 1 and Advanced Level Courses, but not Cross Listed Courses.
In all templates Tables 1, 3 and 7 are locked to protect the formulae that sit behind the cells. For this
reason they cannot be altered (including adding and deleting rows or columns) without using the
password HUMSS. The other templates can be amended to facilitate organisation of the data.
The templates are dynamic; they will automatically total scores in Tables 1 and 3, and carry across these
totals into Table 7. Although the templates have been made as simple as possible, some basic knowledge
of Excel would an advantage, particularly if you want to delete surplus rows in protected sheets.
Excel Spreadsheet
Comment
Table 1 Learning Outcomes for Courses in Major using the DSLOs
Enter learning outcome scores from course maps. The total
scores will automatically appear in the coloured line at the
bottom of the table and in the yellow column on the right. The
numbers in the yellow column on the left should total 10 for a 3
unit course and 20 for a 6 unit course. If this is not the case
there is an error in the courses data.
This sheet is locked to protect the formula
that sits behind the cells. To make changes to
the spreadsheet such as adding or deleting
rows first ‘unprotect’ the sheet using the
password HUMSS.
Table 2 Learning Outcomes for Courses in Major using the RSDF
In the discipline meeting (Task 2) map the learning outcomes
statements that make reference to research skills. Use the
Research Skills Development Framework (proforma provided) or
an alternative approach appropriate to your discipline.
Table 3 Assessment for Courses in Major using the DSLOs
Enter assessment scores from the course maps. The total scores
will automatically appear in the coloured cells at the bottom of
the table and in the yellow column on the right. The numbers in
the yellow column on the left should total 10 for a 3 unit course
and 20 for a 6 unit course. If this is not the case there is an error
in the courses data.
This sheet is locked to protect the formula
that sits behind the cells. To make changes to
the spreadsheet such as adding or deleting
rows first ‘unprotect’ the sheet using the
password HUMSS.
Table 4 Assessment for Courses in Major using the RSDF
In the discipline meeting (Task 2) map the assessment that
requires students to demonstrate research skills. Use the
Research Skills Development Framework (proforma provided) or
an alternative approach appropriate to your discipline.
Table 5 Types of Assessment in Courses in the Major
Collate the types of assessment identified in the course reviews.
Add and delete rows and columns as required.
Typically, different terms are used to
describe similar kinds of assessment. The list
may need to be modified to take account of
these variations.
Table 6 Learning and Teaching Arrangements
Collate the teaching and learning arrangements identified in the
course reviews. Add and delete rows and columns as required.
5
Process for Reviewing Majors
Typically, different terms are used to
describe similar kinds of arrangements. The
list may need to be modified to take account
of these variations.
Table 7 Alignment of CLOs and Assessment
The totals from Table 1 (Learning Outcomes) and Table 3
(Assessment) will automatically update into Table 7.1 and 7.3
respectively. The scores for the Graduate Attributes will
automatically appear in the purple section for each.
This sheet is locked to protect the formula
that sits behind the cells. All cells are
populated automatically and so this sheet
should not need to be unprotected.
Table 8 Summary of Analysis
As a record of the curriculum renewal exercise complete the left
hand side of the table. Identify the changes that have been
agreed in the right hand side.
A word version of this table is also included at
the end of this document.
TASK 2 Analyse and Discuss the Major within the Discipline
The purpose of Task 2 is to facilitate the analysis of the major by academic staff through scrutiny of
the collated course reviews. In undertaking their course reviews, Course Coordinators will have
some insights and ideas for change in their own courses that should be part of the discussion.
In undertaking the analysis, it is useful to consider how the major has developed over time and why
it has particular characteristics and constituent courses.
The discipline-specific learning outcomes statement developed by the staff of the major/program
encapsulates the learning outcomes framed within the Graduate Attributes. The mapping process
identifies the extent to which the DSLO statement is fulfilled through the constituent courses.
Analysis of the data of majors/programs where the structure is very open, will need to take account
of local knowledge about the enrolment patterns of students, courses with large classes and courses
offered to limited groups of students.
The analysis covers the following themes:







current emphases of the curriculum
alignment between the learning outcomes and the assessment
nature of the assessment
structure of the major
progression of academic literacies
development of research skills
support for assessment through teaching and learning arrangements
Process
The central part of the curriculum renewal process is the analysis and discussion of the mapping by
discipline staff which is best undertaken in a meeting called for the purpose. The meeting needs to
be managed by a staff member who has undertaken some initial analysis of the aggregated data and
determined an appropriate process. The reflective questions below provide some suggestions for
focusing discussion. It is accepted that the process is iterative and that curriculum renewal may
occur progressively over time with some issues given priority over others.
In addition to the discussion of the aggregated data, the development of research skills needs to be
discussed and mapped.
Details of the discussion should be recorded using the table at the end of this document.
6
Process for Reviewing Majors
Reflective Questions
The following questions may assist you in analysing the aggregated course review data. The tables
referred to in the reflective questions are the collated course reviews from the mapping templates.
Current emphases of major






Has there been a decision at some point to frame the major in a particular way? If this is the
case, what were the factors that shaped it?
Do the overall emphases of the major as reflected in the collated course learning outcomes
and assessment represent what you think happens in the major? (Table 7 and DSLO
statement)
Does the major have enough emphasis on preparing students for the kinds of professional
practice undertaken by your graduates? (Graduate Attributes 7 and 8 in Table 7 and DSLO
statement)
Do you consider the overall emphases of the major as reflected in the collated course
learning outcomes and assessment to be appropriate? How does it compare with similar
majors in other universities? (Table 7 and DSLO statement)
Are there current issues or developments in your discipline which might require you to rethink the current emphases?
Do the overall emphases of the major as reflected in the collated course learning outcomes
and assessment accurately represent what the promotional and marketing information
publishes? (Table 7 and promotional material)
Alignment







Do the course learning outcomes cover all the identified DSLOs for the major? (Table 7)
Does the assessment cover all the identified DSLOs for the major? (Table 7)
Is there alignment between the course learning outcomes and the assessment? That is, do
the course learning outcomes and assessment have similar scores? (Table 7)
Are some DSLOs assessed more than is necessary? Are some DSLOs assessed less than is
necessary? (Table 7)
If there is an issue with alignment what needs to change? Do the learning outcomes or the
assessment, or both, need to be reconsidered.
Which courses could carry any re-alignment that is necessary?
Which courses have large enrolments and could, therefore, be effective in carrying specific
outcomes for the major? What would those outcomes be?
Nature of Assessment







Is there a comprehensive range of assessment types? Would it be useful to introduce other
forms of assessment? (Table 5 and Assessing graduate attributes)
Does the range of spoken and written genres occurring in the courses reflect the stated DSLOs?
(Table 5)
Is group work assessed? (Table 3, column 4c, and Table 5).
To what extent does the major contribute to ‘professional’ outcomes as outlined in Graduate
Attributes 7 and 8? (Table 7 columns 7a, 7b, 8a, 8b)
Are the assessment types appropriate in judging the achievement of the course outcomes?
See the document (Tables 3 and 5 and Assessing graduate attributes)
What kinds of essays (descriptive, comparative, analytical, critical, etc.) are assessed?
Is the number of points of assessment consistent across courses? (Table 5, line 5.4)
7
Process for Reviewing Majors
Structure and pre-requisites

















How have the current courses come to constitute the major?
Is there an overarching framework or rationale for the courses taught?
Which courses have a formal place in the structure and are central to the delivery of the
outcomes for the major? Eg Compulsory core, Small Group Discovery Experience, Capstones,
or Level 1. (Table 1)
Is the current structure appropriate?
In your experience, do some courses make a significant contribution to particular DSLOs? Is
this reflected in the current learning outcomes and assessment? (Tables 1 and 3)
Do the pre-requisite arrangements ensure students enrol in appropriate units?
If the structure is very loose how do students make appropriate enrolment decisions?
Would the learning experience of students be better if there were more/less formal
structure in the major? What would you do to put this in place?
Which courses do not have a formal place in the structure but are considered central to the
delivery of the outcomes of the major?
Would a compulsory foundational Level 1 course be a useful component of the major?
Are there some courses which make a marginal contribution to the overall outcomes of the
major because they are seldom offered or there are very small enrolments?
Are there courses that are currently at Level 1 that should be at a higher level?
Are there courses currently at Advanced Level that should be at Level 1?
Are there any courses that could be re-badged at Level 3 because of their complexity and
the demands of their current assessment?
Are there ways that you might provide more open forms of advice such as suggested
sequences or complementary courses, or patterns that might lead to particular employment
outcomes?
How will the proposed capstones fit into the structure?
Are there other actions you could take that might increase the coherence of the major?
Development of Academic Literacies and e-Learning

Is there appropriate sequencing of academic literacies so that students build their expertise
in logical and orderly ways?
 Do Level 1 courses introduce students to the basic academic literacies and skills (see
Academic literacies and e-learning skills), appropriately sequenced and supported, including:
 e-Experiences (Tables 5 and 6)
 online resources in Level 1 courses? (Table 5)
 spoken and written communication and skills (Tables 1 and 3, DSLOs 4b)
 problem solving (Tables 1 and 3, DSLOs 3a, 3b, 3c, 3d)
 teamwork (Tables 1 and 3, DSLO 4c)
 ethical, social and cultural competence (Tables 1 and 3,DSLO 7a, 7b, 8a, 8b)
 Do Advanced Level courses continue to develop relevant academic literacies and skills (see
Academic literacies and e-learning skills) appropriately sequenced and supported, including:
 e-Experiences (Tables 5 and 6)
 online resources in Level 1 courses? (Table 5)
 spoken and written communication and skills (Tables 1 and 3, DSLOs 4b
 problem solving (Tables 1 and 3, DSLOs 3a, 3b, 3c, 3d)
 teamwork (Tables 1 and 3, DSLO 4c)
8
Process for Reviewing Majors
 ethical, social and cultural competence (Tables 1 and 3, DSLO 7a, 7b, 8a, 8b)
 How are students prepared throughout the major for capstones and honours?
Development of Research Skills







To what extent are research skills articulated in the course learning outcomes? (Table 2)
To what extent are research skills present in the assessment? (Table 4)
To what extent are research skills actively taught?
Is there alignment between the course learning outcomes and the assessment in relation to
the Research Skills Development Framework, Level 3 or above?
Is there a clear and intentional developmental process around research skills?(Tables 2 and 4)
What is the longest piece of writing students are required to write and at what point does that
occur? (Table 5, line 5.2)
How many pieces of writing of 3 000 words or more does a student write? (Table 5, line 5.3)
Teaching and Learning





Is there appropriate teaching and learning activity to support the understanding of content?
To what extent is this support available in flexible forms of delivery? (Table 6)
How are the various forms of teaching and learning arrangements distributed across the
courses? Are there patterns in particular clusters of courses? Do some forms of teaching
dominate? (Table 6)
Do the kinds of teaching and learning arrangements reflect the types of assessment? For
example, is there explicit support for group work, specific written genres, and social and
cultural competence aspects of assessment? (Table 6)
Are there resources that are or could be shared? (Table 6)
To what extent are students required to engage with online resources and processes such as
online submission and Turnitin? (Table 6)
Summary of Issues
Characteristics
Issues needing to be addressed:
General comments
Current emphases of the curriculum
Alignment between the learning outcomes and the assessment
Nature of the assessment
Structure of the major and pre-requisites
Development of academic literacies across the curriculum
Development of e-learning across the major
9
Process for Reviewing Majors
Development of research skills
Support for assessment through teaching and learning arrangements
10
Process for Reviewing Majors
Download