Running Head: Untangling Learning Environment and Outcomes Complex Relationships
Untangling the Complex Relationship among Variables Affecting both Perceived and Actual
Canadian Medical Student Learning Outcomes: A Mixed Methods Approach
Executive Summary
Joan Forder
1
CJSTL Analysis
2
Background
What are the components of an effective learning environment that will lead to positive
undergraduate medical student behaviours that will then lead to successful completion of
learning outcomes? A literature search did reveal a number of learning environment variables
including (both perceived and actual) workload, course content load, amount of class time,
degree of stress, degree of resilience/coping, readiness for change, freedom of learning, and
instructional styles/types. A number of student behaviors were also reported to include depth of
learning, degree of self-directed learning, approach to study, and the number of classes
attended/absent. However, very few studies focused on the relationship among all three aspects
(learning environment, student behavior, and learning outcomes) and none explored all variables
and outcomes.
The literature highlights a large number of possible variables that together create the
learning environment for undergraduate students. One variable widely studied is the degree of
stress experienced by the student. Many mitigating factors have been attributed to the increase in
this one variable alone. These include: the socioeconomic-health gradient (Adler et al., 1994);
the quality and continuity of stress symptoms over a six-year medical students programme
(Niemi & Vainiomaki, 2006); systematic review of articles reporting on depression, anxiety, and
burnout among U.S. and Canadian medical students (Dyrbye, Thomas, & Shanafelt, 2006);
student burnout as a function of personality, social support, and workload (Jacobs & Dodd,
2003); causes, consequences, and proposed solutions to medical student distress (Dyrbye,
Thomas, & Shanafelt, 2005); the quality of education as it relates to student stress (Pena & Reis,
1997); an analysis of stress and academic performance in medical school (Stewart, Lam, Betson,
Wong, & Wong, 1999); and specific analysis of stress in various academic levels of medical
students within four Canadian Schools of Medicine (Toews et al., 1997). From the literature
review, the following learning environment variables were chosen for the proposed study:
workload, course content load, amount of class time, degree of stress, degree of
resilience/coping, readiness for change, freedom of learning, and instructional styles/types. The
following student behaviors were also chosen: depth of learning, degree of self-directed learning,
approach to study, and the number of classes attended/absent. Both perceived (qualitative) and
actual (quantitative) student outcomes will be measured and used in the analysis of data.
The interrelationship among all variables and the factors that influence the learning
environment and outcomes has not been undertaken to date. This complex web of learning
environment relationships influences another web of student behaviors. The degree to which
these variables together influence learning outcomes and the weighting of each variable, has not
been fully explored.
To complete this study, the following questions will be addressed:
1. What is the relationship between variables such as content load, workload, type of
learning (shallow verses deep), self-directed learning, degree of stress, degree of
resilience/coping, readiness for change, and instructional styles/types, that affect the
Canadian medical student learning environment?
2. What impact does each of these variables have on Canadian medical student learning
outcomes?
3. What is the meaning that students ascribe to these factors and their learning environment?
CJSTL Analysis
3
While it is possible to combine the data at either the collection, analysis or interpretation
stages of a study, most Mixed Methods studies analyse the data separately and then triangulate
the results during the interpretation stage and in the discussion. When analyzing the data, the
most common qualitative method is through identification of themes and relationships whereas
the most common quantitative data analysis method is through descriptive or exploratory
procedures followed by inferential or confirmatory procedures. Using both approaches will allow
for a more thorough exploration of the questions.
Given the questions posed in this narrative, together with the pragmatic worldview, and
the advantages of using competing paradigms in the same study, the proposed study will use a
Mixed Methods approach utilizing the Concurrent Triangulation design that involves both
quantitative and qualitative methods being carried out at the same time on all 400 students within
the program (100 students in each of 4 years) and the data combined at the results level. The
preferred quantitative analysis method is that of Instructional Equation Modeling (SEM), using
one of many possible software packages that will utilize quantitative data collected from wellestablished questionnaires regarding the learning environment and student behavior, and learning
outcomes from course assessments mentioned earlier. The preferred qualitative methods will
include the use of focus groups and observations of medical students to determine the student’s
perceptions of all three aspects explored in this study to include their learning environment, their
coping mechanisms (behaviors) and their learning outcomes.
For the purposes of the proposed study, the Path Model SEM will be utilized since
various factors and variables will be tested to produce quantitative data (such as established
questionnaires and student grades on various class assessments) and a method that works with
both these observed and any possible latent variables needs to be included. SEM analysis goes
through the steps of model specification, data collection, model estimation, model evaluation,
and if needed, model modification (Lei & Wu, 2007).
When looking at Focus Groups, open-ended questions relating to serious life events and
their subjective impact can be explored in a controlled setting. The focus groups proposed in this
study would consist of six to 10 participants who encompass the full range of possible
observations.
Direct observation can also be carried out in this proposed study. Direct observation is an
Ethnographic qualitative approach that involves the observer immerses him/herself in the
experience for prolonged periods in a single or a small number of settings (Cousin, 2009). For
the purposes of this proposed study, specific settings involving the medical student learning
environment can be chosen and detailed field notes will be taken with the goal of trying to
understand the meanings underlying human behavior. In this way, the observer can obtain both
an ‘insider’ view of what is going on as well as a more detached ‘outsider’ view of a setting.
Attention will be needed to determine exactly which settings, length of observation, and analysis
of field notes. As with any qualitative method, it is important for the observer to remain
cognizant that their own positioning can influence their interpretations and ability to observe.
CJSTL Analysis
4
References
Adler, N. E., Boyce, T., Chesney, M. A., Cohen, S., Folkman, S., Kahn, R. L., & Syme, S. L.
(1994). Socioeconomic Status and Health: The Challenge of the Gradient. American
Psychologist, 49(1), 15–24.
Cousin, G. (2009). Researching Learning in Higher Education: An Introduction to
Contemporary Methods and Approaches. (J. Wisdom, Ed.). New York, New York:
Routledge, Taylor & Francis Group.
Creswell, J. W. (2009). Research Design: Qualitative, Quantitative,and Mixed Methods
Approaches. (V. Knight & S. Connelly, Eds.) (third.). Thousand Oaks, California: Sage
Publications Inc.
Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2005). Medical Student Distress: Causes,
Consequences, and Proposed Solutions. Mayo Clinic Proceedings, 80(12), 1613–1622.
Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic Review of Depression,
Anxiety, and Other Indicators of Psychological Distress Among U.S. and Canadian Medical
Students. Academic Medicine, 81(4), 354–372.
Entwistle, N. J., & Ramsden, P. (1983). Understanding Student Learning. London: Croom Helm.
Entwistle, N. J., & Tait, H. (1990). Approaches to learning, evalutions of teaching, and
perferences for contrasting academic environments. Higher Eduation, 19, 291–299.
Gunstone, R. F., & White, R. T. (1981). Undersanding Gravity. Science Education, 65, 291–299.
Hanson, W. E., Creswell, J. W., Plano Clark, V. L., Petska, K. S., & Creswell, J. D. (2005).
Mixed Methods Research Designs in Counseling Psychology. Faculty Publications,
Department of Pscyhology, 224–235. Retrieved from
http://digitalcommons.unl.edu/pschfacpub
Jacobs, S. R., & Dodd, D. (2003). Student Burnout as a Function of Personality, Social Support,
and Workload. Journal of College Student Development, 44(3), 291–303.
Lei, P.-W., & Wu, Q. (2007). Introduction to Structural Equation Modeling: Issues and Practical
Considerations. In Instructional Topics in Educational Measurement (pp. 33–43).
Edmonton, Alberta: the National Council on Measurement in Education.
Meyer, J. H. ., & Muller, M. W. (1990). Evaluating the quality of student learning I - An
unfolding analysis of the assocation between perceptions of learning context and
approaches to studying at an individual level. Studies in Higher Education, 15, 131–154.
CJSTL Analysis
5
Niemi, P. M., & Vainiomaki, P. T. (2006). Medical Students’ Distress - Quality, Continuity, and
Gender Differences During a Six-Year Medical Programme. Medical Teacher, 28(2), 136–
141.
Pena, L., & Reis, D. (1997). Student Stress and Quality of Education. Revista de Aministracao de
Empresas, 37(4), 16–27.
Powell, R. A., & Single, M. (1996). Methodology Matters-V: Focus Groups. International
Journal for Quality in Health Care, 8(5), 499–504.
Stewart, S. M., Lam, T. H., Betson, C. L., Wong, C. M., & Wong, A. M. . (1999). A Prospective
Analysis of Stress and Academic Performance in the Frist Two Years of Medical School.
Medical Education, 33, 243–250.
Toews, J. A., Lockyer, J. M., Dobson, D. J. G., Simpson, E., Brownell, A. K., Brenneis, F., …
Cohen, G. S. (1997). Analysis of Stress Levels among Medical Students, Residents, and
Graduate Students at Four Canadian Schools of Medicine. Academic Medicine, 72(11),
997–1002.
Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: the influence of
learning context and student approaches to learning on learning outcomes. Higher
Education, 22(3), 251–266. doi:10.1007/BF00132290