evaluation framework template - Washington & Jefferson College

advertisement
NAME: Contreras, Biology Dept
INSTITUTION: Washington & Jefferson College
HHMI Long-term Ecological Monitoring (LEM) Equipment
Inputs
Strategies
Outputs
Use of equipment and current
technology employed by professionals
and researchers in Ecology,
Environmental Science, and
Information Technology
Training of students by peer, postdoc,
and faculty mentors in the use of
equipment and technology
Number of students, faculty and
projects utilizing equipment and
technologies
Summer on-campus research by
undergraduate interns will reach more
undergraduates through the
contribution of information (and the
development of research
protocols/experiences) to existing
Biology courses, EVS courses, and
other courses in the W&J curriculum
(through the use of a dedicated server
and website for data access—Webbased database for LEM project
(WebLEM)
Postdoc mentor and student mentees
will set up and maintain WebLEM for
field collected spatial and biological
data
Postdoctoral researcher in Information
Technology
Peer and faculty mentors
Equipment:
1) GPS (Global Positioning
System; Trimble GeoXT and
accessories)—spatial data
collection
2) Ruggedized handheld
computers w/ GPS cards (Trimble
Recons)—biological and spatial
data collection
3) Server and related software-hosting web-based database for
Long-term Ecological Monitoring
(LEM) project data and information
4) Fondreist water quality
monitoring station—remote data
collection for stream monitoring
5) Digital Nikon camera—data
collection
6) 15” Mac Book Pro—laptop—for
downloading and processing data
Training of students by peers,
postdoc, and faculty mentors in the
organization, construction,
maintenance and querying of a large
and complex database (spatial and
biological data--WebLEM)
Training of students by peers,
postdoc, and faculty mentors in the
use of GPS and GIS (Geographical
Information System) for relating spatial
data to biological data—necessary for
examining ecological processes
Mentoring experience for Postdoc
through work with undergraduate
researchers on WebLEM
WebLEM serves as a searchable
database for faculty and students who
will use the data for other research
projects and in courses currently
offered at W&J
WebLEM serves as a searchable
database for faculty and students who
will use the data for other research
projects at other institutions doing
similar research
(Short Term-Learning)
Outcomes
(Medium Term-Action)
Students will become competent in the
use of the equipment for collecting and
processing data in the field and then
uploading the data to the WebLEM on
a daily basis
Postdoc’s experience results in
tenure-track teaching position after
completion of postdoctoral experience
Students will be able to train peers
and faculty in the use of current
technology used in ecological studies
Knowledge of the appropriate use of
technology will increase student
acceptance rates into graduate
programs and professional positions
following graduation
Use and knowledge of advanced
technology used in ecology will
increase the probability of successful
completion of graduate and
professional programs/positions in
Biology and other disciplines related to
the LEM project
Impacts
(Long Term-Conditions)
Through the use of technology all
participants in the LEM project will
have a greater understanding of
science and the nature of biological
and ecological research
NAME: Contreras, Biology Dept
INSTITUTION: Washington & Jefferson College
Washington &Jefferson College On-campus Summer Student Research
Evaluation Questions for OUTCOMES
Possible Indicators/Measures
Possible Data Collection Methods and Information Sources
1. Was the training on the use of the technology
appropriate for student researchers, interns, and
faculty?
1. Students and faculty are able to successfully incorporate
the use of technology into ecological (and related)
research and coursework.
1. Successful mentoring and training by students/faculty of
other students/faculty in the use the technology—peer
mentoring and training
2. Does the training and use of technology contribute
to our student’s understanding of ecological
research and science.
2. Students and faculty can successfully train others in the
use of the technology.
2. Written evaluation of student research project which
includes a metric for measuring student competency in
using technology (GPS, GIS, and/or database
management) through formal testing or presentation of
the project results as a poster or oral presentation
3. Will the use of this technology strengthen existing
programs (and new) programs in the sciences and
other disciplines at W&J?
4. Will the training and use of technology make
students more attractive to potential graduate and
professional programs/employers?
3. Use of technology expands beyond use in the LEM
project into other research projects and courses that
address questions in ecology and environmental science
4. Students using the technology do better in ecology and
environmental science courses (and related courses)
5. a) Continue to major in science
b) Graduate with science degree
c) Pursue additional research experiences before
graduation
d) Expect to continue in research post-research
experience
e) Value summer research experience that includes
technology training—satisfaction surveys (SURE)
6. a) Pursue science and research-related activities after
graduation, particularly through their job/education (postgraduation surveys—1, 5, and 10 years)
b) Pursue science and research-related activities five
years after graduation, particularly through their
job/education (post-graduation surveys—5 years)
c) Pursue science and research-related activities ten
years after graduation, particularly through their
job/education (post-graduation survey—10 years)
7. a) Applied, received, and accepted academic faculty
positions, including type of faculty positions (e.g. tenuretrack)
b) Employed in research or science jobs (e.g. academic
research, industry, government, NGO’s, associations,
and societies)
d) Mentoring undergraduate students
e) Earned honors and awards in research and
education
8. Success in desired career track
3. Using surveys to track the number of students and
faculty using the technology and the WebLEM in their
research and courses
4. Tracking over time the individual and average grades for
courses utilizing the technology and tracking Major Field
Test in Biology scores in the ecology and evolution
sections of the test
5. a) Student Exit Questionnaire—for research and courses
b) Interview with student about the value of the
experience and the use of technology and it importance
in their research
c) Supervisor, Mentor, Summer Program Director, and
Selector Feedback
d) Focus Group
e) SURE
6. a) Tracking Questionnaire
b) Contacting Undergraduate Institution and Selector
c) Alumni search
d) National Surveys (NSF, AAMC, etc.)
e) Grants, honors, and awards
e) Publication searches
f) Citation Impact
7. a) Tracking Questionnaire
b) Alumni search
c) Interview
d) Focus Group
e) Peer review through panels, awards, etc. gauging
research and mentoring contributions
8. a) Student, faculty, and administration surveys and
interviews
b) Increase in support for undergraduate student
research activities
9. a) Tracking of number and quality of undergraduate
student research experiences at a given institution
b) External awards/funding for undergraduate student
c) Institutional funding for undergraduate research
d) Faculty research grants
Rank/Priority (include brief rationale)
Information to answer questions 1, 2, and 3 should be relatively
easy to answer since in-house and external surveys (SURE) can
be administered to students/faculty immediately after, or during,
their research/course experiences. Question 4 will take time to
answer and will require more long-term assessment through
surveys of alumni. These kinds of surveys often allow only
limited analysis since the percentages of responses are often
very low.
Download