ICSEI 2014, Yogyakarta
Paper Proposal
“Using an Integrative Mixed-Methods Research Design for the
Analysis of the School Effectiveness Improvement in Chile.”
Abstract
This paper reports the initial stages of the study titled “Using an Integrative MixedMethods Research Design for the Analysis of the School Effectiveness Improvement in
Chile”. This research is part of a more extensive study that looks at the School Effectiveness
Improvement trajectories of some Chilean schools and the sustainability of that
improvement. Its main aim is to contribute to the understanding of the factors and processes
that take place in schools that demonstrated a sustainable improvement over the last decade.
For this purpose, an integrative mixed-methods approach for the analysis of findings is
employed. The proposed methodological approach consists of different stages of data
analysis and integration. It includes quantitative and qualitative sequential analysis of data
sets; use of analytical matrices for the integration of the data; use of data transformation
techniques and creation of school portraits that include integrated data to illustrate
improvement dimensions. In a further analytical stage, all the previous integration phases
are illustrated through some integrative analytical models, which are assessed in
methodological terms. Finally, the research insights coming from the integrated data are
discussed and some synergistic conclusions aiming to contribute to the holistic
understanding of the School Improvement phenomenon in Chile are drawn.
1. Objectives or purposes
To gain a deep and holistic understanding of the School Effectiveness Improvement
phenomenon in Chile, through the employment of an integrative mixed-methods research
design.
To gain a better understanding of the factors and processes characterizing schools
that have shown sustainable improvement trajectories in the last decade.
2. Perspective(s) or theoretical framework
The research described in this proposal considers the exploration of two fundamental
conceptual areas: School Effectiveness and School Improvement fields (Ávalos, 2007; Bellei,
1
2009, 2010, 2013; Chapman et al., 2012; González & Bellei, 2013; Teddlie & Reynolds,
2000; Townsend, 2007). Nevertheless, for the specific purposes of this study, which main
aim emphasises the use of an integrative mixed-methods research design for the
understanding of the School Improvement phenomenon in Chile, the exploration of the
mixed-methods literature and theoretical development is given special importance.
In the last decade, the use of mixed methodologies in social sciences and educational
research has increased in popularity and sophistication (Tashakkori & Teddlie, 2010; Teddlie
& Tashakkori, 2009; Creswell, 2009; Gorard & Taylor, 2004; Sammons, 2010). Mixed
methodologists argue that this approach improves the quality of the research findings,
strengthening the validity, reliability and trustworthiness of the research inferences and
contributing to the better understanding of a phenomenon by giving a more holistic and
complete picture of it (Day et al., 2006, 2008; Jang et al., 2008; Kington et al., 2011;
Sammons et al. 2005, 2007).
The exploration of the theory produced in the area provides a solid framework to
situate the research. It also contributes to the development of conceptual elaborations or new
models through the revision of the existent models and the connection of them with the
findings of this research. In summary, the theoretical exploration is fundamental for the
elaboration of integrative mixed-methods models that contribute to the understanding of the
School Effectiveness Improvement in Chile.
3. Methods and techniques
As already stated, this research employs an integrated mixed methodology to
address the School Effectiveness Improvement in Chile. The methodological design
includes many stages of data analysis and integration. Firstly, a separate analysis of the
quantitative and qualitative data is performed. The quantitative part includes the creation of
longitudinal databases of students’ academic outcomes, as well as, data sets on students and
schools’ characteristics. In the analysis of data, some complex statistical techniques such as
multilevel multiple regression are employed. In the qualitative part, school case-studies are
undertaken. The main sources of qualitative data are semi-structured interviews and
observation records. In this specific research, the analysis of both strands is not parallel, but
sequential and the integration starts in the qualitative part, since new research questions
have been raised after the quantitative analysis, which has implied a more in-depth
exploration.
The next phase of integration includes the use of analytical matrices or tables to
analyse quantitative and qualitative data (Balbontín, 2012; Day et al., 2006). In order to
integrate both types of data, data transformation strategies are used (Creswell, 2009;
Creswell & Plano Clark, 2007; Teddlie & Tashakkori, 2009).
A third integration stage includes the creation of school portraits which integrate
quantitative and qualitative data. In this phase the quantitative data is mostly transformed,
since the schools’ stories are presented in a narrative way.
A further analytical stage consists on the presentation of integrative analytical
models that illustrate the previous integration phases.
2
Finally, an in-depth discussion about the findings coming from the integrated data
and some synergistic interpretations are presented.
4. Data sources or evidence
The quantitative sources of evidence are national databases that provide longitudinal
information about students’ academic outcomes and about schools and students
characteristics. The qualitative sources of evidence are semi-structured interviews and
observation records, which are analysed as part of the case-studies.
5. Expected Results
It is expected that the employment of a mixed-methods research design in this study
may contribute to a more effective understanding of the School Effectiveness Improvement
phenomenon in Chile.
6. Educational importance for theory, practice, and/or policy
This research represents a theoretical and methodological advance in the Chilean
context because the use of integrative mixed-methods has been almost unexplored in
educational research. The findings of this research will inform policy makers and
practitioners, since the improvement of the educational quality is one of the most important
issues in the current political agendas.
7. Connection to the themes of the congress
This particular study can be connected with the following conference theme:
School Effectiveness, School Improvement and School Transformation–What
do they Mean in Different and Different Paradigms.
References
Ávalos, B. (2007) School Improvement in Latin America: Innovations Over 25
Years (1980-2006). En: Townsend, T. (ed.) (2007) The International Handbook of School
Effectiveness and Improvement. Dordrecht, The Netherlands, Springer, pp. 183-201.
3
Balbontin, R. (2012) Exploring Effective Secondary Schools in Challenging
Contexts: A Study in Two Chilean Regions. Tesis de PhD, Universidad de Nottingham,
Nottingham.
Bellei, C. (2009) Does Lengthening The School Day Increase Students’ Academic
Achievement? Results from a Natural Experiment in Chile. En: Economics of Education
Review, 28, pp. 629-640.
Bellei, C. (editor) (2010) Asistencia Educativa en Chile: ¿Aporte al mejoramiento
Escolar? Editorial Ocholibros, Santiago.
Bellei, C. (2013) Supporting Instructional Improvement in Low-Performing Schools
to Increase Students’ Academic Achievement. The Journal of Educational Research.
Volumen 106, isssue 3.
Chapman C., Armstrong, P., Harris, A., Muijs, D., Reynolds, D. & Sammons,
P. (eds.) (2012) School Effectiveness and Improvement Research, Policy and Practice:
Challenging the Orthodoxy? London/New York, Routledge.
Creswell, J. W. (2009) Research Design: Qualitative, Quantitative and Mixed
Methods Approaches, 3rd edition. Thousand Oaks, CA, Sage.
Creswell, J. W. & Plano Clark, V. (2007) Designing and Conducting Mixed
Methods Research. Thousand Oaks, CA, Sage.
Day, C., Sammons, P. & Gu, Q. (2008) Combining Qualitative and Quantitative
Methodologies in Research on Teachers’ Lives, Work, and Effectiveness: From Integration to
Synergy. Educational Researcher, 37 (6) pp. 330-342.
Day, C., Sammons, P., Kington, A., Gu, Q. & Stobart, G. (2006) Methodological
Synergy in a National Project: The VITAE Story. Evaluation and Research in Education,
19 (2), pp. 1-24.
González, C & Bellei, C. (2013) Sostenibilidad del Mejoramiento Escolar
Impulsado por Programas de Asistencia Técnica Educativa. Revista Perspectiva
Educacional, Vol 52 (1).
Gorard, S. & Taylor, C. (2004) Combining Methods in Educational and Social
Research. Maidenhead, Open University Press.
Jang, E., McDougall, D. E., Pollon, D., Herbert, M. & Russell, P. (2008)
Integrative Mixed Methods Data Analytic Strategies in Research on School Success in
Challenging Circumstances. Journal of Mixed Methods Research, 2 (3), pp. 221-247. 364
Kington, A., Sammons, P., Day, C. & Regan, E. (2011) Stories and Statistics:
Describing a Mixed Methods Study of Effective Classroom Practice. Journal of Mixed
Methods Research, 5 (2), pp. 103-125.
4
Sammons, P. (2010) The Contribution of Mixed Methods to Recent Research on
Educational Effectiveness. En: Tashakkori, A. & Teddlie, C. (eds.) Handbook of Mixed
Methods in Social & Behavioural Research, 2nd edition. Thousand Oaks, CA, Sage.
Sammons, P., Day, C., Kington, A., Gu, Q., Stobart, G. & Smees, R. (2007)
Exploring Variations in Teachers’ Work, Lives and their Effects on Pupils: Key Findings and
Implications from a Longitudinal Mixed-Method Study. British Educational Research
Journal, 33 (5), pp. 681-701.
Sammons, P., Siraj-Blatchford, I., Sylva, K., Melhuish, E., Taggart, B. & Elliot,
K. (2005) Investigating the Effects of Pre-School Provision: Using Mixed Methods in the
EPPE Research. International Journal of Social Research Methodology, 8 (3), pp. 207-224.
Tashakkori, A. & Teddlie, C. (2010) (eds) Handbook of Mixed Methods in Social
& Behavioural Research. Thousand Oaks, CA, Sage.
Teddlie C. & Reynolds, D. (2000) The International Handbook of School
Effectiveness Research. London, Falmer Press.
Teddlie C. & Tashakkori, A. (2009) Foundations of Mixed Methods Research:
Integrating Quantitative and Qualitative Approaches in the Social and Behavioural
Sciences. Thousand Oaks, CA., Sage.
Townsend, T. (ed.) (2007) The International Handbook of School Effectiveness
and Improvement. Dordrecht, The Netherlands, Springer.
5