PROGRAM REVIEW SUMMARY ASSESSMENT REPORT
University of Saskatchewan
Interdisciplinary Graduate Program in Toxicology, Toxicology Centre
EXECUTIVE SUMMARY
On 22 and 23 March, 2011, a review of the Interdisciplinary Graduate Program in Toxicology
was undertaken by a Program Review Panel that consisted of Dr. Chris Kennedy (Professor,
Department of Biological Sciences, Simon Fraser University, external U of S member), Dr. Joe
Bidwell (Professor, Discipline of Environmental Science and Management, University of
Newcastle, Australia, external U of S member), and Dr. A.J. (Fred) Remillard (Associate Dean
Research & Graduate Affairs and Professor of Pharmacy, College of Pharmacy and Nutrition,
internal U of S member). In addition to reviewing the self-study report prepared by the
Toxicology Centre, the Program Review Panel toured the Centre’s facilities and met with faculty
and graduate students associated with the Program. The general assessment of the Program
Review Panel is that the Graduate Program in Toxicology is meeting the expectations for a
quality graduate program. It is the Panel’s unanimous opinion that the Program and the
Toxicology Centre easily rank within the top 5 toxicology programs at the international level. The
academic staff associated with the Program exhibit an outstanding commitment to research and
graduate education and the Centre facilities are world class. All members of the panel would
recommend that students apply to this program and that the program produces graduates who are
highly employable and sought after by potential employers.
The six quality assurance categories the program review panel considered in the review are
specifically discussed below, with strengths and key recommendations outlined for each. The
panel did identify a number of key areas that could be focused upon to further improve the quality
of the program including the appointment of a permanent director, development of formal
policies for a number of key areas in the program (e.g. development of student learning outcomes
and key graduate policies), instituting more consistent scheduling for graduate courses, and
transitioning to a committee to administer the graduate programs.
OVERVIEW OF PROGRAM QUALITY CATEGORIES
1.0 CURRICULUM
Overall Assessment: Meets Basic Quality Standards but Improvements Needed
1.1 Overview
The range of graduate courses listed by the Toxicology Graduate Program provides sufficient
breadth within the general field of toxicology. If all courses were available on a consistent basis,
most students associated with the program would have a good range of options to choose for their
programs of study and could select courses that would directly augment the research they are
undertaking. A current bias toward ecological toxicology versus human/biomedical toxicology
currently exists in the program and this issue was identified in the self-assessment document that
was provided for this program review. Focus on human/biomedical toxicology is expected to be
enhanced with a new faculty position that is expected to be filled in 2011.
A review of syllabi from selected graduate courses indicates students are expected to demonstrate
critical thinking skills and have opportunities to gain experience in giving presentations and
conducting reviews of the primary literature. These courses also appear to be providing students
1
with current content and theory as it relates to toxicology. At least one course listed under the
program focuses specifically on techniques in the field. While it is probable that basic training in
study design is provided by the interactions students have with their supervisor and advisory
committee, graduate students in the program did express concern over a lack of relevant
coursework in statistics. Based on the type of research the students are doing and the self-study
document provided to the program review panel, students are also gaining skills through the
research they are undertaking including both field and laboratory-focused projects. One of the
most impressive features related to program curriculum was the willingness of the highly
research-active core faculty to teach and the clear enthusiasm they exhibited for graduate
education.
It is the opinion of the program review panel that the program curriculum suffers from three key
deficiencies. First is the lack of consistency of course offerings that currently appear to be
haphazard with regard to scheduling and availability. A number of the graduate students indicated
that the current situation makes it difficult to develop a true focus with course work and leads to
their taking what classes are available simply to meet credit requirements. This has direct
implications for developing and timely submission of student programs of study. It also has
implications for initiatives such as the non-thesis Masters program. Part of this issue is an
inherent problem associated with a multidisciplinary program that relies on faculty from other
departments to teach courses. However, the issue is also due to the loss of faculty who were
teaching into the program. This not only includes individuals who have physically left the
program due to retirement, taking positions at other universities, or changing affiliations within
the University of Saskatchewan, but also those who have taken on significant administrative
responsibilities in the Toxicology Centre or elsewhere in the University (further detailed under
Quality Category 4.0). There was also indication that some faculty from other departments who
teach into the toxicology program are not fully supported by their department heads and are left to
organize their own teaching schedules to facilitate participation in the program.
Second, is an apparent lack of clearly articulated student learning objectives (SLOs). While
faculty who met with the program review panel stated they basically all “knew” what skills
students coming out of the program should have, this does not take the place of well-defined
SLOs which can be used to develop courses, insure linkages in content between courses, and
facilitate course reviews.
Finally is the perception by the graduate students who enter the program without an
undergraduate degree in toxicology that courses to augment their background are arbitrarily
assigned at the discretion of the Graduate Program Chair. The lack of a general introductory
course in toxicology that all incoming graduate students are required to take also reduces the
potential for students to develop a cohort that they can identify with.
1.2 Strengths
1.2.a The range of courses listed under the graduate program, while focused on ecological
toxicology, provide sufficient breadth and facilitate key activities such as data
synthesis and interpretation, and presentation skills.
1.2.b
A new faculty hire with expertise in biomedical/human toxicology is anticipated to
enhance research and teaching activities in this area.
1.2.c
The theory component of the courses reviewed was current and relevant.
2
1.2.d
The core faculty affiliated with the Graduate Program in Toxicology has a clear
commitment to quality teaching. This is a particularly commendable attribute of the
program.
1.3 Recommendations
1.3.a A schedule of course availability over a two-year time frame should be developed
and maintained. This will facilitate the development of student Programs of Study
and eliminate the need to revise these programs due to changing course availability.
1.3.b
Procedures for determining teaching schedules for faculty from other departments
who teach into the toxicology program should be established between the Director of
the Toxicology Centre and the head of the academic department of the participating
faculty. The faculty members themselves should not have to conduct these
negotiations.
1.3.c
A series of student learning outcomes (SLOs) should be developed for the program as
a whole and for each course offered. These SLOs should be included as part of the
general curriculum policy for the program.
1.3.d
A policy regarding the specific background students entering the program are
expected to have should be established included the types of courses students would
be expected to take to address any deficiencies in previous training.
1.3.e
The Toxicology Group Executive should explore the possibility of offering a
common introductory course that all graduate students entering the program would be
expected to take. However, the Program Review Team recognizes that developing the
content of such a course may prove challenging since students entering the program
may have significant differences in their background in toxicology.
2.0 LEARNING ENVIRONMENT
Overall Assessment: Meets Basic Quality Standards
2.1 Overview
From the discussion the program review panel had with the graduate students, it appears that the
quality of supervision (mentorship, teaching, research skills) from the supervisors is good
although there will always be some personality conflicts. In general the supervision by the core
toxicology faculty appears to be more positive while there is a trend for the quality of supervision
to diminish the farther faculty are away from the toxicology centre (i.e. adjuncts outside the
university). The closer students are located to their supervisors, even if it is a different college,
the better the supervision. The quality of the advisory committee is more variable. Some students
have little contact with members of their committee beyond the meetings themselves while others
have had members who are very supportive. It was generally agreed that it is a graduate student’s
responsibility to actively seek advice from their committee members. Another concern with the
advisory committee is the fact it is tightly controlled by the Graduate Chair which may or not be
of benefit to the student. Regarding course instructions, there is a goal that course work should
promote and evaluate critical thinking and several class syllabi reflect that. However this outcome
is not formally assessed/ evaluated and therefore may not apply to all students. There is a process
to evaluate the quality of all graduate courses (students are administered an evaluation form and
results submitted to the Director and instructor), however it is not clear how effective this process
is.
3
2.2 Strengths
2.2.a Students felt there were adequate opportunities for knowledge transfer (seminars,
attending and presenting at conferences etc.) Several travel funds, including
individual faculty support, are available for students to attend national and
international conferences.
2.2.b
Students were very pleased with the resources available to them especially the stateof-the-art common laboratory facilities and the aquatic toxicology laboratory as well
as library resources, computer access/availability and classroom equipment.
2.3 Recommendations
2.3.a It is not unusual for some graduate students to experience difficulty or have
significant issues with their supervisors. Such examples were presented to the review
team and the resolutions were unacceptable. The team was informed that the Chair of
the Graduate Program provides verbal direction on dealing with such matters when
they enter the program. Our recommendation is to have a written policy and
procedure manual that clearly addresses sensitive and important issues that will lead
to a satisfactory resolution for all parties involved.
2.3.b
The Advisory committees are chaired by a strong chair with a long history of the
graduate program and possessing extensive knowledge on policy and procedure,
which is an asset for the program and the students. At the same time there is a
perception that only one individual is responsible for everyone’s graduate program.
See section 4.3.b for recommendation of the formation of a Graduate Studies
Committee that would be responsible for making important graduate student
decisions beyond that of the advisory committee chair.
3.0 FACULTY PROFILE AND MENTORSHIP
Overall Assessment: Meets Basic Quality Standards
3.1 Overview
The Toxicology core faculty appear to have strong research backgrounds, are successful in
obtaining prestigious Tri-Council funding as well as from other funding sources. They regularly
attend and present at major toxicology conferences and are invited to speak at other university/
institutions. Most faculty supervise an appropriate number of graduate students as well as
participate in numerous advisory committees. The low ratio of graduate students to faculty does
not reflect a true picture as only aggregate data are presented. However even with this
summarized data, the average number of peer-reviewed journal publications per faculty per year
(4) seems reasonable.
3.2 Strengths
3.2.a The Toxicology Centre has attracted CRC chairs (and continue to prepare
submissions for future chairs) with strong research backgrounds with proven ability
to obtain grants. This allows for excellent opportunities to share those skills with
graduate students.
3.2.b
Resulting from the above, the Toxicology Centre was able to obtain the necessary
funding which lead to a major expansion and provided state of the-art-research
facilities and equipment, which can be used by all faculty and graduate students
4
3.2.c
Despite their extensive commitment to research, the core faculty all have a passion
especially for teaching in order to offer their graduate students the best opportunity
possible as well as mentoring their students.
3.3 Recommendations
3.3.a The Toxicology Centre needs to develop clear polices and expectations (teaching,
grants, supervision of graduate students) with the home Colleges so as not to
discourage new faculty and research chairs whose dissatisfaction may lead to
unexpected departures from the centre
4.0 ADMINISTRATION
Overall Assessment: Meets Basic Quality Standards but Improvements Needed
4.1 Overview
The Toxicology Centre is at present, headed by the Acting Director, Dr. David Janz, following
several acting Directors who report to the Vice-President, Research. At present, the Graduate
Chair (B. Blakely) oversees the administrative and academic aspects of the Toxicology Graduate
Program and reports to the CGSR. The duties of the Graduate Chair include acting as the main
point of contact for all admission inquires, is responsible for receiving and assessing applications
to the program, the selection of potential supervisors for new students, the submission of PhD
student programs of study and progress reports to the College of Graduate Studies and Research
(CGSR), organizing committee structure, organizing meetings, recording of student funding,
facilitating the Ph.D. qualifying and comprehensive examinations and collating grades for these
exams, and serving as chair on all Ph.D. Advisory Committees. Since 2007, the Assistant
Director of the Toxicology Centre (M. Pietrock) serves in a similar capacity for all M.Sc. students
with the exception of admissions. The Graduate Secretary (S. Rolleston) assists both the Graduate
Chair and Assistant Director by managing the GSIS, updating student files, grade entry,
corresponding with committee members and students, scheduling committee meetings, and
setting examination dates.
4.2 Strengths
4.2.a Although early in his position, Dr. Janz shows extreme enthusiasm for maintaining
and improving the functioning and maintaining the excellent reputation of the
program in light of the losses of the contributions of several faculty members (loss of
position [e.g. Dr. Sits, Dr. Dube], or secondment to administrative duties [e.g. Dr.
Liber, Dr. Janz]).
4.2.b
The Administrative staff includes Ms. Adriana Brown, Manager of Finance and
Operations, Shanda Rolleston, Lab/Office Manager, and Fiona Price, Clerical
Assistant Chief Financial Officer. Ms. Brown plays an integral role in almost all
aspects of the program; her extensive knowledge and expertise in the functioning of
this unit are absolutely critical. Ms. Rolleston, in addition to above mentioned role,
acts as the Graduate Secretary and works closely with the Graduate Program Chairs
in all aspects of the M.Sc. and Ph.D. graduate student programs. Ms. Price as well,
acts as the Undergraduate Secretary & CREATE Program Coordinator.
4.2.c
In light of local living costs and tuition, students within the toxicology program
receive stipend that meet or exceed those in other units. The level of funding afforded
students is in accordance with the profile of the program, and above that of other
similar programs elsewhere.
5
4.3 Recommendations
4.3.a It is understood that the position of Dr. Janz as Acting Director is limited. It is
recommended that this position be returned to Director status as soon as possible.
The unit has undergone several faculty losses and needs strong leadership to
continue the units success, which is best served by a full time permanent Director.
4.3.b
The Graduate Chairs are responsible for the majority of all organization,
communication and processing of students through their programs of study. This is
an enormous amount of work and the Chairs should be recognized for their
contributions. There are perceived inconsistencies in how students are dealt with
during their programs of study, and the review panel recommends the formation of a
Graduate Studies Committee (GSC) who would be responsible for those duties now
performed by the Chairs. We recommend that the GSC consist of regular faculty
members, as well as a Graduate Chair, and a student representative elected from a
graduate caucus. Faculty should be much more involved in all aspects of their own
graduate student’s programs including the development of individual programs of
study, scheduling of exams, and other aspects of graduate student not directly related
to research projects.
4.3.c
Many of the clerical and organizational responsibilities currently handled by the
Graduate Chairs, and in the future by a GSC, should be mainly facilitated by the
graduate studies secretary.
4.3.d
Graduate student perception, opinions and points of view are paramount to proper
functioning of any graduate program. It is recommended that a graduate student
caucus be formed with student positions on major committees such as the GSC.
This is common practice in other Universities, and perceived issues of
‘confidentiality’ should not be a barrier to such inclusion. Students may be asked to
withdraw from meetings during sensitive discussions that warrant such action.
4.3.e
Even though students in the Toxicology Centre are funded well, in light of increases
in the cost of living over the years, there could be a revisiting of award amounts to
meet rising costs of living in Saskatoon.
4.3.f
Most operating procedures and structures of the Toxicology Centre are consistent
with University standards, although there have been a number of situations in which
the Toxicology Centre policies have not been in accordance with standard practice
and disruptions of student progress have occurred. The first is confusion between
the policies of the Toxicology Centre and the College with respect to M.Sc. to Ph.D.
transfer times. As well, the review panel heard several student concerns regarding
reporting procedures for student-supervisor complaints and issues. Currently the
Graduate Chair hears such complaints. It is recommended that all policies of the
Toxicology Centre be in line with the College in regards to the first example. With
respect to the second, students need to be aware of complete reporting lines for all
complaints, disputes or other issues which should be heard first at the level of the
GSC, with clear and transparent lines to higher levels including Human Resources, if
necessary.
4.3.g
Information tools such as the website and graduate student handbook are available to
students, and students are advised by the Graduate Chairs at the initiation of their
programs of these tools. It is recommended that these tools be regularly updated.
6
4.3.h
In the past, recruitment efforts of the program were larger and succeeded in
establishing the recruitment of excellent students to the toxicology program. With
the continued success and growth of the Toxicology Centre, efforts in this respect
were reduced. It is recommended that efforts in this regard be renewed.
5.0 STUDENT VIABILITY AND PROGRAM PERFORMANCE:
Overall Assessment: Meets Basic Quality Standards
5.1 Overview
The program has a good mix of graduate students from within the province of Saskatchewan,
other Provinces in Canada, and other countries. The self-assessment document the Program
Review Team was provided with did not indicate the grade point averages or other credentials of
the students entering the MSc or Ph.D. programs and it appeared that evaluation of applicants and
admission to the programs was largely at the discretion of the two individual Graduate Chairs.
The undergraduate program at the University provides a good source of students to the
Toxicology Graduate Program.
The program review panel felt the reputation of the program is high and easily ranks within the
top 5 toxicology programs internationally. Based on the Team’s discussion with the graduate
students, the majority were attracted to the program because of its outstanding reputation at the
national and international level.
The time to completion for the MSc and Ph.D. degrees indicated in the program’s self-assessment
document seems long at first glance, particularly for the MSc (3-4 years for MSc and 4-6 years
for Ph.D.). However, these data must be interpreted in light of the fact that many students
undertake field-based research programs that often take longer to complete than laboratory-based
projects. In addition, employer demand for students from the program is such that some students
are taking positions prior to completing their thesis that has extended the reported completion
times for the MSc program in particular. This has been a particular issue in some cases that
involved adjunct faculty who were acting as the primary supervisor for a graduate student.
Due to the interdisciplinary nature of the program, teaching fellowships for students enrolled in
the program are limited. All student stipends are guaranteed from research grants or scholarships
when entering the Program. In the past 5 years, 3 students have received National scholarships.
Among the Tri-Agency scholarships, 4 students have been awarded prestigious NSERC Canada
Graduate Scholarships (CGS) in the past 5 years. Students have also received several
International awards. Travel funds and awards for all graduate students are provided through the
Toxicology Centre, the Toxicology Graduate Student Association, and various other University
of Saskatchewan colleges and departments. The Toxicology Centre has two endowed awards
each year, the Sisodia Graduate Scholarship and the Schiefer Travel Award. In addition, the
AREVA travel award, administered through the Toxicology Centre, supports graduate student
travel associated with visiting foreign research institutions.
The employment prospects for students associated with this program are outstanding. As
mentioned above, this has on occasion led to situations in which students leave the program early.
5.2 Strengths
7
5.2.a
The Toxicology Graduate Program has an outstanding international reputation and
attracts students from both Canada and other countries.
5.2.b
Potential supervisors are expected to have sufficient funding to support students they
accept into their programs. The core faculty associated with the program is very
successful in attracting nationally competitive grants and industry-based funding
which translates into support for students.
5.2.c
Graduates from the program are highly sought after by potential employers.
5.3 Recommendations
5.3.a Students are not admitted into the program unless a full financial commitment is
made by supervisors, however, several examples of inadequate funding or loss of
funding by Adjunct Professors have occurred which leave students in difficult
situations. It is recommended that all Adjunct Professors have a Co-supervisor from
the Toxicology Centre on committees who can ensure funding is covered in all cases.
Funding limitations should be discussed up front with all students at the start of their
program. In addition to this, recording and monitoring of all funding in yearly
committee meeting reports, signed by all committee members and reviewed by GSC,
must occur to ensure all students are receiving consistent and adequate funding.
5.3.b The majority of M.Sc. students complete their program in 3-4 years and Ph.D.
students complete their program in 4-6 years. There have been circumstances where
Toxicology Centre graduate students have taken longer to complete their program
(both student and supervisor-related); however, the members have recognized this
problem and are considering means to rectify this issue and reduce the overall time to
completion. Emphasis is made on the importance of completing programs in a timely
manner by the student’s supervisor, the Graduate Chair, and the Advisory
Committee, and all parties are responsible for ensuring students complete on time. It
is recommended that students and supervisors (through the GSC) sign a ‘contract of
funding and expectations’ at the start of programs so each is aware of the limitations
of funding and lengths of times expected for completion.
5.3.c
The graduate program needs to ensure that Ph.D. students have an opportunity to
obtain real teaching experience, not just lab instruction as many are interested in
pursuing academic careers.
6.0
STUDENT OUTCOMES:
Overall Assessment: Meets Basic Quality Standards but Improvements Needed
6.1 Overview
The graduate program in Toxicology collected 6 replies over a total of 4 years: 2 survey replies in
2005/06 for the Program Comments on Student Satisfaction index; 1 reply in 2006/07; 2 replies
in 2007/08 and 1 reply in 2008/09. The responses for subsequent years under review have not yet
tabulated by the College of Graduate Studies and Research. Internal review documents on the
quality of the Program and overall satisfaction of students in their Exit Surveys indicated that out
of 15 students (2006-2009), 9 were very satisfied, 5 were satisfied, and 1 was dissatisfied with the
Program. There were several mixed levels of satisfaction for survey questions however, the
number of replies to this survey were low and an accurately assessment cannot be made on
whether this reflects overall student satisfaction with the Program over the past 5 years or not.
The meeting with graduate students indicated a similar trend in responses to similar questions (as
8
in the survey) from the review panels, that being very satisfied with the program.
6.2 Strengths
6.2.a The high level of specific toxicological training and education graduate students
receive in the Toxicology Program continue to make students highly sought after by
government, industry, and the private sector. Many students are hired before
completion of their project (which contributes to longer completion times). M.Sc.
students enrolled in the Toxicology Graduate Program can specialize in the areas of
environmental toxicology or biomedical toxicology. Due to the interdisciplinary
nature of the Program, students can receive training and education in a wide range of
areas associated with toxicology, including analytical toxicology, aquatic/terrestrial
toxicology,
ecotoxicology,
forensic
toxicology,
immunotoxicology,
molecular/biochemical toxicology, nutritional toxicology, radiation/radionuclide
toxicology, and veterinary toxicology. Students enrolled in the interdisciplinary
M.Sc. Program receive research experience and training that will aid them in
addressing applied toxicological issues. Based on regular internal program reviews,
novel academic and research initiatives are discussed and recommended, and several
new courses have recently been added to the curriculum. A large number of M.Sc.
students begin their career after graduation in environmental consulting, while others
remain in academia in pursuit of Ph.D. degrees. Many Ph.D. graduates in the
Program continue on as postdoctoral researchers. The primary objectives of the
Ph.D. Program are to deliver excellence in interdisciplinary training, to encourage
independent research, to develop academic and leadership qualities, and to integrate
students into the broader toxicology scientific community. To aid in this objective,
Ph.D. students in the Toxicology Graduate Program have access to world-class
research facilities at the Toxicology Centre, Western College of Veterinary Medicine,
College of Medicine, College of Pharmacy and Nutrition, College of Agriculture and
Bioresources, and the Canadian Light Source Synchrotron, which include extensive
animal care facilities, analytical chemistry instrumentation, and molecular biology
laboratories. Due to the larger research component and lengthier program, Ph.D.
students have a great opportunity to take advantage of foreign exchange programs
with partnering universities. Also, as part of the CREATE HERA Program, Ph.D.
students have the opportunity for advanced course work in risk assessment, the
opportunity for cooperative work placement, and attendance in the Summer Institute
of Risk. As a result, the Program has grown into one of the most advanced
interdisciplinary toxicology programs in North America.
6.2.b
The majority of conference presentations are made by graduate students. As a metric
of graduate student success in this regard (although many different conferences and
workshops are attended each year), presentations made by members of the
Toxicology Graduate Program at the Society of Environmental Toxicology and
Chemistry compared with all other institutions and agencies (2008/09) were behind
only the United States Environmental Protection Agency and Environment Canada.
Thus, the Program gave the most presentations at SETAC meeting of any university
in the world. Publications arising from student projects are numerous and seem to be
the normal practice in the Program. Equally impressive are the number of graduate
student scholarships and awards given to students in the Program. The quality of the
theses defenses is acceptable, and the process of external examiner reports and
bringing in experts for defenses ensures high standards.
9
6.2.c
The level of student satisfaction with their graduate experience and outcomes are
reflective of a quality program and valued educational experience from what was
expressed at the graduate student meeting. Most students had a very positive attitude
towards their experience and training in the program and the opportunities provided
for their future careers in this area. There were limited exit surveys to draw
conclusions from, but the majority of these expressed similar views. It is clear from
the interviews that the outcomes listed (systematic understanding of knowledge and
an awareness of current challenges and new insights, critical thinking skills etc.) are
being transferred appropriately to students through their educational experience.
6.3 Recommendations
6.3.a There is limited information to accurately reflect if a correlation between the
employment obtained by program graduates and the career paths that the program is
designed to support, however, anecdotal evidence suggests that graduates are using
their training towards careers in this area. The program should attempt to retain more
information from graduates on exit surveys to adequately address the subject of
student outcomes.
6.3.b
One particular area that seemed to be similar to the surveys was the inconsistency of
supervisor availability. This is certainly not unique to this program and internal
documents state that Faculty in the Program realize that there are certain instances
where supervisors may not be readily available. This should be part of the annual
committee meeting for students, recorded and assessed by the GSC for potential
remedial action.
10