Computers & Education 126 (2018) 334–345
Contents lists available at ScienceDirect
Computers & Education
journal homepage: www.elsevier.com/locate/compedu
The flipped classroom: A review of its advantages and challenges
Gökçe Akçayıra,∗, Murat Akçayırb
a
b
T
Gazi University, Faculty of Gazi Education, Department of Computer Education and Instructional Technologies, Ankara, Turkey
Kırıkkale University, Faculty of Education, Department of Computer Education and Instructional Technologies, Kırıkkale, Turkey
A R T IC LE I N F O
ABS TRA CT
Keywords:
Teaching/learning strategies
Improving classroom teaching
This study presents a large-scale systematic review of the literature on the flipped classroom, with
the goals of examining its reported advantages and challenges for both students and instructors,
and to note potentially useful areas of future research on the flipped model's in and out-of-class
activities. The full range of Social Sciences Citation Indexed journals was surveyed through the
Web of Science site, and a total of 71 research articles were selected for the review. The findings
reveal that the most frequently reported advantage of the flipped classroom is the improvement
of student learning performance. We also found a number of challenges in this model. The majority of these are related to out-of-class activities, such as much reported inadequate student
preparation prior to class. Several other challenges and the numerous advantages of the flipped
classroom are discussed in detail. We then offer suggestions for future research on flipped model
activities.
1. Introduction
The flipped or inverted classroom is a new and popular instructional model, in which activities traditionally conducted in the
classroom (e.g., content presentation) become home activities, and activities normally constituting homework become classroom
activities (Bergmann & Sams, 2012; Sohrabi & Iraj, 2016). In the flipped classroom, the teacher helps the students instead of merely
delivering information, while the students become responsible for their own learning process and must govern their own learning
pace (Lai & Hwang, 2016). Since classroom time is not used to transmit knowledge to students by means of lectures, the teacher is
able to engage with students by means of other learning activities such as discussion, solving problems proposed by the students,
hands-on activities, and guidance. Today, the concept of the flipped classroom has been implemented in many different disciplines
(math, social sciences, humanities, etc.), and in schools and universities around the world (Hao, 2016).
While some studies indicate that flipped classrooms offer many positive educational outcomes, other studies draw attention to
limitations associated with flipped classroom. For example, in the flipped model student learning achievement and satisfaction may
be enhanced (Missildine, Fountain, Summers, & Gosselin, 2013); students may be more satisfied with the flipped method; and it can
be more economical than traditional instruction (O'Flaherty & Phillips, 2015). However, challenges can include more required time to
redesign the course as a flipped classroom (Schlairet, Green, & Benton, 2014), low self-regulated behaviors by some students (Sun,
Wu, & Lee, 2017), and the resulting failure of some students to properly schedule their time to comprehend the out-of-class learning
content (Lai & Hwang, 2016). The literature indicates that the flipped model of instruction entails both opportunities and challenges.
However, to date only a few studies have reviewed the educational outcomes and challenges of flipped classrooms (e.g., Betihavas,
Bridgman, Kornhaber, & Cross, 2016; Zainuddin & Halili, 2016). This new review of the advantages and challenges reported in
∗
Corresponding author.
E-mail address: gokceakcayir@gmail.com (G. Akçayır).
https://doi.org/10.1016/j.compedu.2018.07.021
Received 6 January 2018; Received in revised form 25 July 2018; Accepted 31 July 2018
Available online 01 August 2018
0360-1315/ © 2018 Elsevier Ltd. All rights reserved.
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
research studies associated on this approach can more comprehensively indicate best practices, and areas for future research and
development. In the present study, 71 research studies examining flipped classroom, published to the end of 2016, were identified
and systematically analyzed.
2. Theoretical background
Unlike traditional teacher-centered instruction, in which students are treated as empty vessels that passively absorb information
(Betihavas et al., 2016), flipped classrooms are centered around the students – not the teacher (Bergmann & Sams, 2012). Therefore,
many researchers agree that student-centered learning theories (e.g., active learning, collaborative learning) can be more fully
utilized in the flipped classroom (Betihavas et al., 2016; Lai & Hwang, 2016; Sohrabi & Iraj, 2016). According to Bishop and Verleger
(2013), student-centered learning embodies a set of theories that include active learning, peer-assisted learning, and collaborative
learning.
Active learning can be simply defined as “any instructional method that engages students in the learning process” (Prince, 2004, p.
223). Active learning requires that students engage in meaningful learning activities (Sohrabi & Iraj, 2016) and that they be accountable for their own learning (Blaschke, 2012). In flipped classrooms, students experience active learning (Lai & Hwang, 2016)
and have opportunities to perform higher order thinking activities (Roehl, Reddy, & Shannon, 2013). As Davies, Dean, and Ball
(2013) stated, in flipped classrooms students are transformed from passive listeners into active learners.
Peer-assisted learning is “the acquisition of knowledge and skills through active helping and supporting among status equals or
matched companions” (Topping & Ehly, 1998, p. 1). According to Nederveld and Berge (2015), several opportunities for peer-assisted
learning exist in flipped classrooms, in both class activities (e.g., collaboratively solving problems, cooperating to complete projects)
and out-of-class activities by means of technology (e.g., discussion boards, social network sites).
Collaborative learning broadly “is a situation in which learners interact in a collaborative way” (Dillenbourg, 1999, p. 8). Collaborative learning leads to deeper learning and shared understanding (Kreijns, Kirschner, & Jochems, 2003), and it provides opportunities for students to develop social skills (Johnson & Johnson, 1999). Flipped classrooms incorporate collaborative learning.
Students are responsible for their own learning; they participate in small-group activities; they learn in an active mode; and the
teacher maintains a facilitator role. Since class-time is not used to deliver knowledge by means of lectures in flipped classrooms,
teachers can conduct collaborative small group activities to interact with the students (Bergmann & Sams, 2012). Tucker (2012) also
noted that in flipped classrooms students can use their class time to work together and engage in collaborative learning.
3. Purpose of the study
The flipped classroom has attracted scholarly attention especially due to its inclusion of improved technologies (Davies et al.,
2013). Hence, research on the flipped classroom has increased in recent years (Lo & Hew, 2017). But to date, only a few studies have
reviewed the literature on this evolving method. These few review studies are typically characterized by short publication range
coverage, focus on only one learner type, or focus on only one academic discipline. For example, Betihavas et al. (2016) included only
five studies in their review and only focused on nursing education. O'Flaherty and Phillips (2015) included studies that were only
conducted at the level of higher education. Zainuddin and Halili's (2016) study was limited in its publication-date coverage (only
three years were examined) and reviewed only 20 published articles. Though several advantages of the flipped model have been
reported by researchers such as increased learning motivation (e.g., Bhagat, Chang, & Chang, 2016) and course grade (e.g., Mason,
Shuman, & Cook, 2013), it is not clear whether these are common contributions of all flipped classrooms, or only particular findings
from individual applications. A more systematic review will help to map areas of uncertainty and to depict the big picture (Petticrew
& Roberts, 2006). Therefore, this study initially reviews the advantages and challenges reported in a larger number of research
studies on the flipped classroom, covering a longer range of publication dates than earlier review studies.
Because in-class and out-of-class activities are amongst the most important elements of the flipped classroom (Bergmann & Sams,
2012), this study secondly provides an analysis of flipped model activities, both in and out-of-class. This will detail the current state of
activities in flipped classrooms and should provide useful guidance to researchers for further exploration.
To achieve these purposes, we identified all of the articles that investigated the flipped classroom model published in all of the
Social Sciences Citation Indexed (SSCI) journals to 31 December 2016. The following research questions (RQs) specifically guided this
analysis:
RQ1. What advantages of the flipped classroom are indicated in the studies published in the SSCI-indexed journals?
RQ2. What challenges imposed by the flipped classroom are indicated in the studies published in the SSCI-indexed journals?
RQ3. Which activities (in-class and out-of-class) were used in flipped classrooms in the studies published in the SSCI-indexed
journals?
4. Method
4.1. The manuscript selection process
As it is impossible to include all relevant studies (Heitink, Van der Kleij, Veldkamp, Schildkamp, & Kippers, 2016), several specific
criteria are used by researchers in the selection of manuscripts. Examples include the importance of the journals in the field (Hwang &
335
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Fig. 1. Procedure for the selection of articles.
Tsai, 2011), the selection of all articles published in several leading journals in a field (Shih, Feng, & Tsai, 2008), and the use of
databases in which studies are indexed, such as ProQuest (Heitink et al., 2016), Education Resources Information Center (ERIC), and
SSCI (Kucuk, Aydemir, Yildirim, Arpacik, & Goktas, 2013).
For this study, we selected scientific articles on the flipped classroom which were published in journals indexed in the SSCI
database. The SSCI database was selected as a tool to access the field tags of SSCI indexed articles, such as by topic and research area
(Luor, Johanson, Lu, & Wu, 2008). It is typically used as the source for many review studies (e.g., Kucuk et al., 2013). The Web of
Science (WOS) site was the point of access because it provides a search engine for all of the SSCI indexed journals.
In WOS, we used the advanced search function and input the search terms “flip” and “invert.” Because some authors use the term
“inverted classroom” interchangeably with the term “flipped classroom,” the search was conducted using multiple terms with the
Boolean operator “OR.” In order to locate education related studies, we selected all educational categories (“education & educational
research,” “education special,” “education scientific discipline,” and “psychology educational”). While conducting the search, no time
period was specified (1980–2016 is the full range of the search engine). However, the document type “article” and the language
“English” were selected as the search parameters. The last search was conducted on 31 December 2016.
In the first search, 206 articles were discovered. These articles were downloaded as full texts to a computer. Each article then was
examined by two researchers to check whether it was suitable for the purposes of the study. During this examination, inclusion and
exclusion criteria were adopted. Articles were included when they covered the flipped classroom as a primary component. More
specifically, the researchers attempted to select articles in which the flipped classroom was the focus, and not simply mentioned.
Editorials and review articles were excluded. Following our application of the criteria, 71 articles were found to be compliant with
the purposes of the study. A summary of the search results is presented in a flow chart in Fig. 1.
4.2. The data coding and analysis processes
The data from all 71 articles were analyzed by two researchers. To assist the researchers in this analysis, we used the software
program ATLAS.ti 7, which makes it easier to organize the data and to handle large datasets (Paulus, Woods, Atkins, & Macklin,
2017). Firstly, to calculate coding reliability, 18 (25.35%) articles were picked randomly and blind coded by the two researchers.
Then the coding results were compared to determine inter-rater reliability, as measured by Cohen's kappa. This was done with the
University of Pittsburgh's Coding Analysis Toolkit (CAT), a web-based software program that measures inter-rater reliability from
ATLAS.ti-coded datasets (Lu & Shulman, 2008). The Cohen's kappa was found to be 0.75 for advantages (RQ1), 0.78 for challenges
(RQ2), 0.92 for in-class, and 0.83 for out-of-class activities (RQ3). According to Landis and Koch (1977), any value between 0.61 and
0.80 is considered substantial, and between 0.81 and 1.00 is regarded as perfect agreement. As Heitink et al. (2016) suggested, initial
disparities were discussed by the two researchers to further align their methods. After ensuring the coding reliability, the two
researchers independently coded the rest of the articles.
The data collected from the articles were analyzed using the qualitative content analysis method, which enables systematic and
objective categorization of the qualitative data (Berelson, 1952). No coding scheme (pre-developed form or template) was used
because pre-developed forms can unduly direct researchers while coding, which might distract from the intended purpose/s (Şimşek
& Yıldırım, 2011). Therefore, an analytical three-stage open coding process (established by Emerson, Fretz, & Shaw, 2011) was
utilized to analyze the data. To code the advantages and challenges of the flipped classrooms (addressing RQ1 and RQ2), all of the
findings/results, discussions, and conclusion sections from the 71 articles were read, as was done by Akçayır and Akçayır (2017).
While reading these sections we paid attention to determinant words to identify advantages and challenges reported in the studies. In
other words, our selection of advantages and challenges was based upon the actual empirical results reported in each study. Likewise,
to code in and out-of-class activities (addressing RQ3), the methods sections from the articles were read. At the end of this stage, all of
the data were coded and collected using the ATLAS.ti 7 software.
The codes then were grouped into more comprehensive categories based on their similarities. To name the categories, the most
descriptive wording was determined. In the final stage, the codes were rearranged according to emergent categories.
336
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Fig. 2. Number of articles published by year.
5. Results and discussion
5.1. General characteristics of the studies
The general characteristics of the studies, such as distribution over time and learner types (types of participants), are briefly
described in this section.
5.1.1. The year distribution of the studies published in the SSCI-indexed journal articles that examined the flipped classroom
Within our reviewed studies, the first study that examined the flipped classroom was published in 2000 (see Fig. 2). Even this
early study reported that advancements in learning technologies offer new opportunities for students to learn and for instructors to
deliver the learning content (Lage, Platt, & Treglia, 2000). The authors also found a positive result in that the majority of the students
and the instructor perceived the flipped model positively. Compared to traditional classrooms, the only problem with the flipped
classroom was the technology cost in these early years (Lage et al., 2000). However, the financial cost problem may not affect flipped
classrooms today because it is less expensive to access technology now than in the past.
After more than a decade, flipped classroom studies became popular among scholars; the numbers of such studies began to
steadily increase after 2012. One possible explanation for this increase is the growing availability of Internet technologies.
Interestingly, the “flipped classroom” was first addressed in a Horizon Report in 2012 (Johnson, Adams, & Cummins, 2012). In that
report, the flipped classroom was described as a promising model that can open doors to new approaches in education. The importance of reliable Internet connection, and of preserving class time for collaboration and discussion, were also highlighted.
Fig. 2 shows that 79% of all the flipped classroom studies were conducted between 2015 and 2016. Particularly in developing
countries, the availability of technology and the Internet are increasing. Thus, it seems likely that interest in the flipped classroom
will continue after 2016, and that more research will be devoted to it.
5.1.2. Commonly selected learner types (types of participants) in the studies published in the SSCI-indexed journal articles that examined the
flipped classroom
The majority of the reviewed studies (80%) were conducted at the higher education level. Though this model is seen as useful for
courses in all curriculum areas (from elementary to high school, as well as higher education; see Bergmann & Sams, 2012), only 16%
of the studies focused on K-12 learners (see Fig. 3). While some researchers found positive outcomes within K-12 environments
Fig. 3. Learner type.
337
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Fig. 4. Distribution of the codes.
(Bhagat et al., 2016; McEvoy et al., 2016), others reported some troublesome issues related to the flipped model in K-12 environments. For example, young students may have very limited time to use technology for learning at home (for out-of-class activities)
because their parents do not like them to use mobile phones too often (Wang, 2016). Also, if young students are not sufficiently techsavvy, they may face problems when attempting to access online content (Leo & Puzio, 2016). Our results further revealed that there
was a notable absence of teachers (1%) and adults (3%) as subjects in flipped classroom research. The flipped classroom model could
be potentially effective for teaching elderly people because it is similar to the andragogy approach in some respects. In both, the
learner accepts responsibility for his or her own learning; there is flexibility in the instruction; and both allow greater self-determination (Betihavas et al., 2016). For this reason, we see great growth potential in the area of adult-student centered flipped
learning research.
5.1.3. Distribution of our emergent codes
Providing substantial details of qualitative data (including the distribution of all created codes) helps to achieve analytical objectives (Paulus et al., 2017). In this section, we provided an overview of the codes that we created for the 71 articles during the
analysis process (see Fig. 4). A total of 627 codes were created, of which 219 (35%) relate to the advantages, 102 (16%) relate to the
challenges, 162 (26%) relate to the in-class activities, and 144 (23%) relate to the out-of-class activities in the flipped classroom
studies. We wish to highlight that the codes relating to the advantages of flipped classrooms (35%) are nearly double the number of
those relating to the challenges (16%). In other words, in this set of reviewed studies, the authors reported more advantages derived
from flipped classrooms than challenges.
5.2. RQ1: What advantages of the flipped classroom are indicated in the studies published in the SSCI-indexed journals?
As a result of open coding, six inductive categories were created for the advantages (learner outcomes, pedagogical contributions,
dispositions, interaction, time efficiency, and other). We then divided each inductive category into detailed sub-categories, based on
where the reported advantages naturally fit (see Table 1). It is important to note that some studies reported more than one advantage.
For such studies, more than one code (more than one sub-category) was applied for the advantages. Therefore, it is possible for
percentages to be more than 100%. The findings for each inductive category are discussed in detail in the following sections.
5.2.1. Learner outcomes
The advantages in terms of students' learning outcomes (such as satisfaction, engagement, and motivation) are gathered under
this category. More than half of the studies (52%) reported that using the flipped model improves the learning performance of
students, when measured by GPAs, standardized test scores, and course grades. According to these results, one of the most significant
advantages of this model is that it helps to improve learning performance, which is one of the key elements of quality education.
According to the Leo and Puzio (2016), active learning strategies provide potential explanations of how flipped learning may improve
learning performance among students. However, Jensen, Kummer, and Godoy (2015) conducted a quasi-experimental design study to
compare two groups (composed of flipped classroom and active non-flipped classroom students) to examine the effects of this model
on student learning performance. They concluded that the learning performances were equal between the two groups, but that
flipped classrooms do incorporate active learning strategies. Therefore, some researchers (e.g., Kay & MacDonald, 2016) argue that
rather than devoting considerable time and resources to develop online videos and other out-of-class materials for the flipped model,
it may better to focus on carefully selecting in-class instruction methods and designing better active learning strategies to utilize
within traditional instruction.
According to our reviewed studies, the flipped model also enhances student satisfaction (18%) and their level of engagement
(14%), both of which are important elements in educational environments. On this point, Al-Zahrani (2015) drew attention to the
quality of flipped course tools and materials, especially video-recorded lectures, and argued that these tools should be carefully
prepared to promote a higher level of student engagement and satisfaction. Another study by Ryan and Reid (2016) concluded that
the length of videos should be selected to match the students' attention span to ensure more effective engagement.
338
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Table 1
Advantages of the flipped classroom.
Inductive categories
Sub-categories
f
%
Sample research
Learner Outcomes
Improves learning performance
Satisfaction
Engagement
Motivation
Increases knowledge
Improves critical thinking skills
Feeling more confident
Promotes creativity
Focus on Problem solving skills
Better retention
Improves application skills
Improves ICT skills
37
13
10
7
7
6
5
2
1
1
1
1
52.11
18.31
14.08
9.86
9.86
8.45
7.04
2.82
1.41
1.41
1.41
1.41
Bhagat et al. (2016)
Bösner, Pickert, and Stibane (2015)
Khanova et al. (2015)
Huang and Hong (2016)
Galway et al. (2014)
van Vliet, Winnips, and Brouwer (2015)
Sockalingam et al. (2016)
Al-Zahrani (2015)
L. Chen, Chen, and Chen (2015)
Liou, Bhagat, and Chang (2016)
Liou et al. (2016)
Huang and Hong (2016)
Pedagogical Contributions
Flexible learning
Enables individualized learning
Enhances enjoyment
Better preparation before class
Fosters autonomy
Offers collaboration opportunities
Enables more feedback
Fosters higher self-efficacy
Provides peer-based learning
Increases study effort
Supports interest in the course
Improves attendance
Suitable for large group teaching
Decreases withdrawals
16
8
8
6
6
4
4
2
2
2
2
1
1
1
22.54
11.27
11.27
8.45
8.45
5.63
5.63
2.82
2.82
2.82
2.82
1.41
1.41
1.41
Nguyen, Yu, Japutra, and Chen (2016)
González-Gómez et al. (2016)
Wanner and Palmer (2015)
Sahin et al. (2015)
Huang and Hong (2016)
Foldnes (2016)
Zawilinski et al. (2016)
Aşıksoy and Özdamlı (2016)
Sage and Sele (2015)
Chen et al. (2014)
Leo and Puzio (2016)
Chen et al. (2014)
Balaban, Gilleskie, and Tran (2016)
Flynn (2015)
Time Efficiency
More efficient class time
More time for practice
9
5
12.68
7.04
Davies et al. (2013)
Moraros et al. (2015)
Dispositions
Positive
Positive
Positive
Positive
13
10
6
2
18.31
14.08
8.45
2.82
Mylott, Kutschera, Dunlap, Christensen, and Widenhorn (2016)
Kong (2014)
Chao, Chen, and Chuang (2015)
Lage et al. (2000)
Interaction
Interaction (Students-Instructor)
Interaction (General)
Interaction (Students-Students)
7
4
3
9.86
5.63
4.23
Ryan and Reid (2016)
Chen et al. (2014)
Galway et al. (2014)
Other
Less anxiety
Cost effective
Students adapt quickly
3
2
1
4.23
2.82
1.41
Teo, Tan, Yan, Teo, and Yeo (2014)
Hardin and Koppenhaver (2016)
Mason et al. (2013)
feedback from students
perceptions (students)
attitudes
perceptions (teachers)
Advantages such as enhancing confidence (7%), promoting creativity (3%), and increasing problem solving skills (1%) were
reported by a few researchers (see Table 1). Al-Zahrani (2015) conducted a quasi-experimental study which examined the flipped
classroom's impact on student creativity, and showed that the flipped classroom may stimulate student creativity, despite fact that
this advantage is not commonly reported in the reviewed studies. Many of the reviewed studies also address learner performance.
However, learning outcomes encompass more than merely learner performance. To provide more evidence on the educational effectiveness of flipped classrooms, future researchers may wish to focus more on other learning outcomes, such as motivation.
5.2.2. Pedagogical contributions
According to our findings (see Table 1), the most prominent pedagogical contribution of the flipped classroom is its flexibility
(23%). Since many flipped classrooms use online materials to deliver instruction before a class (He, Holton, Farkas, & Warschauer,
2016), students are theoretically able to learn anytime and anywhere (McDonald & Smith, 2013). The flexibility of this model allows
students to study at their own pace, which appeals especially to busy students who are overscheduled and may miss lectures
(Bergmann & Sams, 2012). Similarly, González-Gómez, Jeong, Airado Rodríguez, and Cañada-Cañada (2016) reported that students
can pause, rewind, and review lectures using technology that is typically available in the flipped model. Flexibility is a main reason
for students' (Nguyen, Yu, Japutra, & Chen, 2016; Wanner & Palmer, 2015) and instructors' (Hardin & Koppenhaver, 2016) reported
satisfaction with the flipped model.
Enabling individualized learning (11%) and enhancing student enjoyment of the lectures (11%) are other pedagogical contributions of the flipped classroom. A few studies (8%) reported that due to this model students were better prepared before class.
According to Sahin, Cavlazoglu, and Zeytuncu (2015), students found the viewing of videos easier than reading the course book. This
339
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
may explain why students might be better prepared before attending class than in traditional instruction settings. There is only sparse
data in these reviewed studies concerning increases in study effort (3%), improved attendance (1%), or decreased withdrawals (1%).
It is not clear whether these are real pedagogical contributions of flipped model or not. Further investigation of these topics could
resolve these questions.
5.2.3. Dispositions, interaction, time efficiency & other
Several studies reported that students' feedback (18%) and perceptions (14%) regarding flipped courses are positive. For example,
Zainuddin and Attaran (2016) found that most university students had positive perceptions towards flipped classrooms and wanted to
continue using this model instead of conventional teaching methods. A few studies (8%) demonstrated that the flipped classroom
helped the students to develop better attitudes toward learning experiences (e.g., Fautch, 2015; Hung, 2015).
Another positive feature is students-students and students-instructor interaction opportunities (20% because flipped classrooms
allow extra time in-class for this. We found greater students-instructor (10%) and students-students (4%) interaction reported in these
studies. One possible explanation for increases in students-instructor interaction is that the role of the instructor in this model is
altered from presenter of content to learning coach (Bergmann & Sams, 2012). Accordingly, Gilboy, Heinerichs, and Pazzaglia (2015)
stated that flipped model student-centered activities in class might increase students-instructor interaction.
A final reported advantage of the flipped classroom is ability to use class time more efficiently (13%). Since students are provided
all lectures as homework (to be studied outside of class), valuable class time is freed up and may be dedicated to student-centered
learning activities such as discussion, feedback, hands-on activities, etc. Therefore, class time can be used more efficiently than in
traditional classrooms. One of the reviewed studies (Mason et al., 2013) reported that engineering students adapted to the flipped
model fairly quickly. However, further research is needed to confirm whether the flipped model is easy to implement among other
types of students.
5.3. RQ2: What challenges imposed by the flipped classroom are indicated in the studies published in the SSCI-indexed journals?
Though flipped classrooms provide many advantages in educational environments, this model imposes some challenges as well
(see Table 2). We found a number of reported challenges and divided them into five inductive categories (pedagogical, students' and
teachers' perspectives, technical & technological, and other). The majority of flipped classroom challenges are related to out-of-class
activities, such as inadequate student preparation prior to class and the students' need for guidance at home.
Table 2
Challenges of the flipped classroom.
Inductive categories
Sub-categories
f
%
Sample research
Pedagogical
Limited student preparation before class
Students need guidelines at home
Unable to get help while out of class
Inability of instructors to know if students watched videos or
not
Effects of the flipped method not long lasting
Implementation issues
9
7
7
2
12.68
9.86
9.86
2.82
Al-Zahrani (2015)
Wanner and Palmer (2015)
Chen et al. (2015)
Fautch (2015)
1
1
1.41
1.41
van Vliet et al. (2015)
Khanova et al. (2015)
Time consuming
Workload increase
Students do not prefer it
8
7
6
11.27
9.86
8.45
Adoption problems
Anxious about the new method
4
4
5.63
5.63
Resistance to change
Students find the method unfair/unreasonable
2
1
2.82
1.41
Smith (2013)
Khanova et al. (2015)
Porcaro, Jackson, McLaughlin, and O'Malley
(2016)
Chen et al. (2014)
Porcaro, Jackson, McLaughlin, and O’Malley
(2016)
Chen et al. (2014)
Wilson (2013)
Teachers' Perspective
Time consuming
Higher workload
Difficult to manage tasks
Planning the sequence of activities
10
5
1
1
14.08
7.04
1.41
1.41
Wanner and Palmer (2015)
Sage and Sele (2015)
Chen et al. (2015)
Schneider and Blikstein (2016)
Technical and Technological
Quality of videos
Inequality of technology accessibility
Need for technology competency (students)
Cost
Need for technology competency (teachers)
Requirements of specific infrastructure
9
6
5
2
1
1
12.68
8.45
7.04
2.82
1.41
1.41
Moraros et al. (2015)
Chen et al. (2015)
Jensen et al. (2015)
Jensen et al. (2015)
Leo and Puzio (2016)
Porcaro et al. (2016)
Other
Parental bias
Lack of institutional support
1
1
1.41
1.41
Wang (2016)
Wanner and Palmer (2015)
Students' Perspective
340
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
The most commonly reported problem is students' limited preparation before class time (13%). If a student does not take time to
study at home, s/he may not perform well in the classroom activities, and this may diminish the advantages of the flipped classroom
(Sayeski, Hamilton-Jones, & Oh, 2015). Hwang, Lai, and Wang (2015) also stated that engaging students in self-learning at home is
one of the key factors of seamless flipped learning. Moreover, since students may not be accustomed to this model, they might lose
their bearings (i.e., they won't know what to do) in the flipped model. In order to avoid this situation, they need clear guidelines
(10%) regarding how they should use their pre-class time and course materials. Another pedagogical issue is students' inability to get
immediate help/feedback while they study at home (10%). Students who need help during out-of-class activities normally must take
notes, write down questions, and wait for in-class discussions to obtain answers. To eliminate this problem, some researchers (e.g.,
Cummins, Beresford, & Rice, 2016; Fautch, 2015; Hardin & Koppenhaver, 2016) utilized text messages or discussion boards to offer
their students instant feedback during out-of-class activities. These systems also help to reduce transactional distance in the flipped
model (Chen, Wang, Kinshuk, & Chen, 2014). Additionally, there were some concerns about the difficulty of ensuring that students
would study each lecture entirely before class (3%).
From the students' perspective, the flipped model requires more time (11%) and work (10%) compared to a traditionally structured
course (see Table 2). One possible reason for this is the nature of this model, which prompts students to preview the learning materials for
better in-class participation (Hung, 2015). On this point, a study by Smith (2013) found that students generally considered studying
lectures outside the classroom to be an extra time burden. According to (Chen et al., 2014), another possible reason is that some of the
students acquired passive learning habits from the traditional classroom, where learning requires less time and work. In addition to these
problems, students sometimes did not prefer this new model nor view it as useful compared to traditional instruction (8%). Since the
flipped model is a relatively new approach for students, there was some uncertainty about what the class might entail, and this uncertainty
can cause problems such as anxiety (6%), adoption problems (6%), and resistance to change (3%).
From the teachers' side, we found two commonly reported challenges. Similar to the students, some teachers reported that this
model might require more time (14%) and workload (7%). Pre-recording video lectures and preparing other flipped model materials
is time consuming for teachers. Designing appropriate accompanying quiz questions and other out-of-class activities requires further
time commitment (Howitt & Pegrum, 2015). The actual time needed to prepare flipped course materials can be nearly six times more
than traditional course preparation (Wanner & Palmer, 2015). That is why some teachers questioned themselves many times as to
why they were doing this (Howitt & Pegrum, 2015). This result is a top challenge for the flipped model (14%), which might prevent
future instructors from implementing it in their courses.
Technical and technological challenges, which include developing high quality of out-of-class materials, technology accessibility,
and technology competency also may cause problems in the flipped classroom. A considerable number of studies (13%) reported that
video lectures which are limited in their technical and pedagogical features (e.g., poor audio quality, tedious, dry) can negatively
affect student learning experiences in the flipped classroom. For example, poor audio quality in videos (He et al., 2016) has negatively influenced students' learning. In another example, Giuliano and Moser (2016) found that the length of videos is inversely
correlated with the percentage of videos viewed by the students. On this point, Battaglia and Kaya (2015) and Mason et al. (2013)
recommended that considering most students' attention span, the length of videos should not be longer than 20 min. Researchers have
drawn attention to this video problem (Zainuddin & Attaran, 2016; Zainuddin & Halili, 2016) and warned that poor quality video
lectures may result in negative learning outcomes (e.g., He et al., 2016; Moraros, Islam, Yu, Banow, & Schindelka, 2015).
Since the flipped model requires all students to have access to technology (hardware/software), whether provided by a university,
a school, or by themselves (Jensen et al., 2015), the accessibility of technology cannot be ignored. Some studies reported that when a
student has no access to technology (8%), this model can be daunting. Moreover, though the current generation of students is
assumed to experience fewer difficulties when using technological products (Akçayır et al. 2016), some researchers (7%) reported
that insufficient technological competency (e.g., username/password management, navigating online resources) might be a problem
that affects flipped classroom efficacy.
5.4. RQ3: Which activities (in-class and out-of-class) were used in flipped classrooms in the studies published in the SSCI-indexed journals?
Since the flipped classroom can embody many forms, there is a large variety of activities to use both in-class and out-of-class. It is
important to use appropriate learning activities for the flipped classroom to be truly effective (Leo & Puzio, 2016; Porcaro, Jackson,
McLaughlin, & O'Malley, 2016). Therefore, to provide insights for teachers who are wary of the use of the flipped model in their
classrooms and to offer guidance to future researchers, we have attempted to provide an analysis of the flipped classroom activities
featured in the reviewed studies.
Regarding in-class activities, the researchers preferred mainly active learning activities, such as discussion (38%), small group
activities (31%), feedback (28%), problem solving (28%), and collaborative group work (11%). Active learning has long been
considered a more effective teaching strategy than the traditional lecture approach (Jonassen, 1995). According to Jensen et al.
(2015), active learning in-class activities are the main contributions of the flipped model and might be the most influential elements
for positive learning outcomes.
It is evident from our review that most of the researchers preferred to use more than one in-class activity during class time (see
Table 3). Regarding this point, Gilboy et al. (2015) recommended that it is important to choose only a few in-class activities to use
throughout the course rather than a different one for each class. According to them, this allows students to become familiar with the
class activities and helps to avoid the risk of students focusing on the process of the strategy rather than the learning content.
When we looked at out-of-class activities utilized in the reviewed flipped model studies (see Table 4), we found that information
transmission occurred outside the classroom in the form of lectures mainly delivered through videos (79%). There are several reasons
341
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Table 3
In-class activities in flipped classrooms.
Categories
f
%
Sample research
Discussion
Small group activities
Feedback
Problem solving
Questions and answers
Group discussions
Collaborative group work
Case studies
Hands-on experiments
Quizzes
Students' presentations
Audience responses (Clicker)
Assignments with teacher assistance
Gaming
Micro lectures
Group projects
Writing
Guest speakers
Word cloud
Concept mapping
Brain storming
27
22
20
20
12
9
8
8
8
7
5
3
3
2
2
1
1
1
1
1
1
38.03
30.99
28.17
28.17
16.90
12.68
11.27
11.27
11.27
9.86
7.04
4.23
4.23
2.82
2.82
1.41
1.41
1.41
1.41
1.41
1.41
Kiviniemi (2014)
Al-Zahrani (2015)
Kong (2014)
Khanova et al. (2015)
Galway et al. (2014)
González-Gómez et al. (2016)
Eichler and Peeples (2016)
Jeong, González-Gómez, and Cañada-Cañada (2016)
Battaglia and Kaya (2015)
Zainuddin and Attaran (2016)
McLaughlin et al. (2013)
Khanova et al. (2015)
Nguyen et al. (2016)
Battaglia and Kaya (2015)
Khanova et al. (2015)
McLean, Attardi, Faden, and Goldszmidt (2016)
Zawilinski et al. (2016)
McLean et al. (2016)
Porcaro et al. (2016)
Porcaro et al. (2016)
Kong (2014)
Table 4
Out-of-class activities in flipped classrooms.
Categories
f
%
Sample research
Videos
Readings
Quizzes
Discussion
PowerPoint presentations
Homework
Reflection
Online modules
Web Quest
Further research
Audio lectures
Interactive tutorials
56
35
30
8
3
3
2
2
2
1
1
1
78.87
49.30
42.25
11.27
4.23
4.23
2.82
2.82
2.82
1.41
1.41
1.41
Teo et al. (2014)
Kiviniemi (2014)
Wanner and Palmer (2015)
Weaver and Sturtevant (2015)
Howitt and Pegrum (2015)
Smith (2013)
Sage and Sele (2015)
McLean et al. (2016)
Hung (2015)
Nguyen et al. (2016)
Bösner et al. (2015)
Eichler and Peeples (2016)
for this preference. First, Bishop and Verleger (2013) stated that to meet the flipped classroom criterion, out-of-class activities must
include video lectures. According to Smith (2013), it is better to use video lectures, since students prefer streaming content as an outof-class activity. Similarly, Battaglia and Kaya (2015) reported that students mainly focus on videos rather than text readings during
out-of-class study. Another reason for using video lectures is that they allow students to take advantage of the pause, rewind, and fastforward features (Battaglia & Kaya, 2015).
Aside from videos, the researchers preferred to use readings (49%) and quizzes (42%) to help students prepare for in-class
participation. Readings include assigned textbook readings and other supplemental reading materials (e.g., Gilboy et al., 2015;
Hibbard, Sung, & Wells, 2016; Zawilinski, Richard, & Henry, 2016). Khanova, Roth, Rodgers, and McLaughlin (2015) noted that
reading assignments should not take many, many hours to complete because students may not complete overly long reading assignments. In addition to readings, nearly half of researchers also used online quizzes to provide data on materials accessibility and
the students' completion of learning materials; this tool also measures students' learning while out-of-class. For example, Wilson
(2013) implemented (graded) quizzes on a regular basis, not only to provide students an explicit reason to complete tasks but also to
provide incentive for participation in out-of-class learning activities. In another study by Fautch (2015), quizzes were used as a way to
check on their viewing of the lecture videos. Students also found that online quizzes encouraged them to complete their study of
online materials (Galway, Corbett, Takaro, Tairyan, & Frank, 2014).
Fautch (2015) drew attention to a main problem that persists in the course format: the inability of students to ask questions
immediately as they see and hear content for the first time. Though some researchers felt the need for discussion/interaction tools to
provide feedback/help to students while out-of-class (aforementioned as a pedagogical challenge), our review of the literature has
shown that only a few (11%) researchers have addressed this. Few of the included activities helped to solve the inability of students to
ask questions/interact immediately. Therefore, teachers need to more effectively integrate discussion/interaction tools to make this
model more useful in education. According to Fautch (2015), one way to circumvent the inability to ask immediate questions is to
offer students instant feedback in the form of a text message.
342
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
There are some additional points which should be noted here. Previous studies (e.g., Al-Zahrani, 2015; He et al., 2016; Zainuddin
& Attaran, 2016) highlighted the importance of using high-quality out-of-class tools/materials to increase the effectiveness of the
flipped model. Aside from the quality of the materials, the quantity and level of the loading should also be handled carefully because
these can negatively affect the students' efforts for preparation (Lin & Hwang, 2018). Finally, apart from these four activities, it can be
said that there are no commonly used out-of-class activities in our reviewed studies (see Table 4).
6. Conclusion and future research
In this study, we conducted a systematic review of the literature on uses of the flipped model in education. The number of studies
on this method has been increasing each year. As the flipped model continues to grow in popularity, probably more research will be
conducted on it. Our examination of extant studies through 2016 also indicates that there has been little interest in implementing this
method in the K-12 setting. Thus, this model's pedagogical potential has not yet reached far into K-12 schools. We recommend
increased research on K-12 flipped model learning.
In general, the flipped model in education yields positive academic outcomes. The majority of reviewed studies reported that the
flipped model promotes improvements in student learning performance. Additionally, researchers have reported numerous advantages of this model (e.g., enhanced learning motivation, students' positive attitudes), which is encouraging. But to date, there is
insufficient evidence to warrant generalization. Additional research is needed to examine this model's advantages, and whether its
broader use could lead to greater student confidence, engagement, and more positive attitudes. Though claimed by some researchers
(e.g., Jensen et al., 2015), it is not clear whether the benefits of the flipped model are due to active learning. Another question is, “If a
researcher uses active learning strategies in a traditional course instead of flipping the classroom, would s/he gain the same positive
academic outcomes?” If the answer is “yes,” then maybe there is no need to devote considerable time to designing and implementing
the flipped classroom (developing video lectures, quizzes, etc.) or to subjecting students to large changes in their instructional format.
However, it is a fact that flipped learning is not only conducted for positive academic outcomes but also to assist students individually
and to create more opportunities to practice during class time, and so on. Therefore, future researchers should look for evidence to
determine differences between the flipped model and non-flipped active learning models in terms of advantages, challenges, and
time-effectiveness (which is one of the most reported challenges of the flipped model).
Another point which should be considered is that most of the reviewed studies on the flipped classroom focused on only one
course. Therefore, generalizability is an issue concerning the conclusions of these studies. If all courses were to be redesigned as a
flipped model, would both the instructors and the students be able to allocate enough time to prepare adequately? As was mentioned
earlier, time commitment and increased workload (heavy demands of the flipped model) are two of the main reported challenges by
both the students and the instructors. There is a need for more flipped classroom research which focuses on numerous course
implementations, to see whether the model is suitable for large-scale implementation. It is also evident from this review that the
effects of the flipped classroom might not long-lasting. Future studies are also required to determine whether the effects of the flipped
classroom are sustained over longer term applications.
This study highlights some potential challenges that need to be addressed for effective implementation of flipped classrooms.
First, since the quality of videos (found to be a top technical challenge) directly/indirectly impacts flipped learning, instructors
should pay more attention to the quality of instructional videos (e.g., not too long and keep them interesting) while designing the
flipped classroom. There is a need for studies that explore strategies and technologies to produce high quality videos when one has
less technical ability and time. Second, it would be better if instructors could provide more interaction/communication tools (these
are apparently not commonly used at present) to help students to obtain feedback/help when they are doing tasks/homework outside
the class. Finally, since technology competency emerged as a challenge, instructors need to examine the technology availability and
competency of students before implementing the flipped model.
Our review also revealed some conflicting results which require further investigation by future researchers. For instance, while
some studies reported that the flipped model encourages better student preparation before class, others observed limited student
preparation before class. Similarly, when student perspectives were examined, findings of positive student perceptions have been
reported as an advantage, but negative factors such as low student preference also have been reported as challenges. Thus, there is a
need for further research which explores why some studies conclude that these aspects are advantages and others label them as
challenges.
7. Limitations
In this review, specific criteria were used to search the SSCI database for relevant studies to be examined. This scientific approach
allowed us to capture a strongly representative selection of studies on the flipped classroom model. However, different search criteria
might yield slightly different selection/data results. For example, if conference papers were to be analyzed, the reported advantages
and challenges might be different. Thus, further research is needed to review the flipped classroom model using different selection
criteria.
References
Al-Zahrani, A. M. (2015). From passive to active: The impact of the flipped classroom through social learning platforms on higher education students' creative thinking.
British Journal of Educational Technology, 46(6), 1133–1148.
343
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Aşıksoy, G., & Özdamlı, F. (2016). Flipped classroom adapted to the ARCS model of motivation and applied to a physics course. Eurasia Journal of Mathematics, Science
and Technology Education, 12(6), 1589–1603.
Akçayır, M., Dündar, H., & Akçayır, G. (2016). What makes you a digital native? Is it enough to be born after 1980? Computers in Human Behavior, 60, 435–440.
Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational
Research Review, 20, 1–11.
Balaban, R. A., Gilleskie, D. B., & Tran, U. (2016). A quantitative evaluation of the flipped classroom in a large lecture principles of economics course. The Journal of
Economic Education, 47(4), 269–287.
Battaglia, D. M., & Kaya, T. (2015). How flipping your first-year digital circuits course positively affects student perceptions and learning. International Journal of
Engineering Education, 31(4), 1126–1138.
Berelson, B. (1952). Content analysis in communications research. Free Press.
Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. USA: International Society for Technology in Education.
Betihavas, V., Bridgman, H., Kornhaber, R., & Cross, M. (2016). The evidence for ‘flipping out’: A systematic review of the flipped classroom in nursing education.
Nurse Education Today, 38, 15–21.
Bhagat, K. K., Chang, C.-N., & Chang, C.-Y. (2016). The impact of the flipped classroom on mathematics concept learning in high school. Journal of Educational
Technology & Society, 19(3), 134–142.
Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the research. Paper presented at the 120th ASEE annual & exposition, Atlanta, USA.
Blaschke, L. M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. International Review of Research in Open and
Distance Learning, 13(1), 56–71.
Bösner, S., Pickert, J., & Stibane, T. (2015). Teaching differential diagnosis in primary care using an inverted classroom approach: Student satisfaction and gain in skills
and knowledge. BMC Medical Education, 15, 1–7.
Chao, C. Y., Chen, Y. T., & Chuang, K. Y. (2015). Exploring students' learning attitude and achievement in flipped learning supported computer aided design
curriculum: A study in high school engineering education. Computer Applications in Engineering Education, 23, 514–526.
Chen, L., Chen, T.-L., & Chen, N.-S. (2015). Students' perspectives of using cooperative learning in a flipped statistics classroom. Australasian Journal of Educational
Technology, 31(6), 621–640.
Chen, Y., Wang, Y., Kinshuk, & Chen, N.-S. (2014). Is FLIP enough? Or should we use the FLIPPED model instead? Computers & Education, 79, 16–27.
Cummins, S., Beresford, A. R., & Rice, A. (2016). Investigating engagement with in-video quiz questions in a programming course. IEEE Transactions on Learning
Technologies, 9(1), 57–66.
Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course.
Educational Technology Research & Development, 61(4), 563–580.
Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.). Cognitive and computational approaches (pp. 1–19). Oxford, UK: Elsevier.
Eichler, J. F., & Peeples, J. (2016). Flipped classroom modules for large enrollment general chemistry courses: A low barrier approach to increase active learning and
improve student grades. Chemistry Education: Research and Practice, 17(1), 197–208.
Emerson, R. M., Fretz, R. I., & Shaw, L. L. (2011). Writing ethnographic fieldnotes. Chicago, IL: The University of Chicago Press.
Fautch, J. M. (2015). The flipped classroom for teaching organic chemistry in small classes: Is it effective? Chemistry Education: Research and Practice, 16(1), 179–186.
Flynn, A. B. (2015). Structure and evaluation of flipped chemistry courses: Organic & spectroscopy, large and small, first to third year, English and French. Chemistry
Education: Research and Practice, 16(2), 198–211.
Foldnes, N. (2016). The flipped classroom and cooperative learning: Evidence from a randomised experiment. Active Learning in Higher Education, 17(1), 39–49.
Galway, L. P., Corbett, K. K., Takaro, T. K., Tairyan, K., & Frank, E. (2014). A novel integration of online and flipped classroom instructional models in public health
higher education. BMC Medical Education, 14, 1–9.
Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47(1),
109–114.
Giuliano, C. A., & Moser, L. R. (2016). Evaluation of a flipped drug literature evaluation course. American Journal of Pharmaceutical Education, 80(4), 66.
González-Gómez, D., Jeong, J. S., Airado Rodríguez, D., & Cañada-Cañada, F. (2016). Performance and perception in the flipped learning model: An initial approach to
evaluate the effectiveness of a new teaching methodology in a general science classroom. Journal of Science Education and Technology, 25(3), 450–459.
Hao, Y. (2016). Exploring undergraduates' perspectives and flipped learning readiness in their flipped classrooms. Computers in Human Behavior, 59, 82–92.
Hardin, B. L., & Koppenhaver, D. A. (2016). Flipped professional development: An innovation in response to teacher insights. Journal of Adolescent & Adult Literacy,
60(1), 45–54.
He, W., Holton, A., Farkas, G., & Warschauer, M. (2016). The effects of flipped instruction on out-of-class study time, exam performance, and student perceptions.
Learning and Instruction, 45, 61–71.
Heitink, M. C., Van der Kleij, F. M., Veldkamp, B. P., Schildkamp, K., & Kippers, W. B. (2016). A systematic review of prerequisites for implementing assessment for
learning in classroom practice. Educational Research Review, 17, 50–62.
Hibbard, L., Sung, S., & Wells, B. (2016). Examining the effectiveness of a semi-self-paced flipped learning format in a college general chemistry sequence. Journal of
Chemical Education, 93(1), 24–30.
Howitt, C., & Pegrum, M. (2015). Implementing a flipped classroom approach in postgraduate education: An unexpected journey into pedagogical redesign.
Australasian Journal of Educational Technology, 31(4), 458–469.
Huang, Y. N., & Hong, Z. R. (2016). The effects of a flipped English classroom intervention on students' information and communication technology and English
reading comprehension. Educational Technology Research & Development, 64, 175–193.
Hung, H.-T. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28(1), 81–96.
Hwang, G.-J., Lai, C.-L., & Wang, S.-Y. (2015). Seamless flipped learning: A mobile technology-enhanced flipped classroom with effective learning strategies. Journal of
Computers in Education, 2(4), 449–473.
Hwang, G.-J., & Tsai, C.-C. (2011). Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010. British Journal
of Educational Technology, 42(4), E65–E70.
Jensen, J. L., Kummer, T. A., & Godoy, P. D. D. M. (2015). Improvements from a flipped classroom may simply be the fruits of active learning. CBE-Life Sciences
Education, 14, 1–12.
Jeong, J. S., González-Gómez, D., & Cañada-Cañada, F. (2016). Students' perceptions and emotions toward learning in a flipped general science classroom. Journal of
Science Education and Technology, 25(5), 747–758.
Johnson, L., Adams, S., & Cummins, M. (2012). NMC Horizon report: 2012 K–12 educationAustin, TX: The New Media Consortium.
Johnson, D. W., & Johnson, R. T. (1999). Learning together and alone: Cooperative, competitive, and individualistic learning (5th ed.). Boston: Allyn & Bacon.
Jonassen, D. H. (1995). Supporting communities of learners with technology: A vision for integrating technology with learning in schools. Educational Technology,
35(4), 60–63.
Kay, R., & MacDonald, T. (2016). Comparing flipped, active, and lecture-based teaching approaches in higher education. Paper presented at the EdMedia: World
conference on educational media and technology 2016, Vancouver, BC, Canada.
Khanova, J., Roth, M. T., Rodgers, J. E., & McLaughlin, J. E. (2015). Student experiences across multiple flipped courses in a single curriculum. Medical Education,
49(10), 1038–1048.
Kiviniemi, M. T. (2014). Effects of a blended learning approach on student outcomes in a graduate-level public health course. BMC Medical Education, 14(1), 1–7.
Kong, S. C. (2014). Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing
flipped classroom strategy. Computers & Education, 78, 160–173.
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review
of the research. Computers in Human Behavior, 19(3), 335–353.
Kucuk, S., Aydemir, M., Yildirim, G., Arpacik, O., & Goktas, Y. (2013). Educational technology research trends in Turkey from 1990 to 2011. Computers & Education,
68, 42–50.
Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education,
31(1), 30–43.
344
Computers & Education 126 (2018) 334–345
G. Akçayır, M. Akçayır
Lai, C.-L., & Hwang, G.-J. (2016). A self-regulated flipped classroom approach to improving students' learning performance in a mathematics course. Computers &
Education, 100, 126–140.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
Leo, J., & Puzio, K. (2016). Flipped instruction in a high school science classroom. Journal of Science Education and Technology, 25(5), 775–781.
Lin, H.-C., & Hwang, G.-J. (2018). Research trends of flipped classroom studies for medical courses: A review of journal publications from 2008 to 2017 based on the
technology-enhanced learning model. Interactive Learning Environments, 1–17. https://doi.org/10.1080/10494820.2018.1467462.
Liou, W.-K., Bhagat, K. K., & Chang, C.-Y. (2016). Beyond the flipped classroom: A highly interactive cloud-classroom (hic) embedded into basic materials science
courses. Journal of Science Education and Technology, 25(3), 460–473.
Lo, C. K., & Hew, K. F. (2017). A critical review of flipped classroom challenges in K-12 education: Possible solutions and recommendations for future research.
Research and Practice in Technology Enhanced Learning, 12(4), 1–22.
Luor, T., Johanson, R. E., Lu, H.-P., & Wu, L.-l. (2008). Trends and lacunae for future computer assisted learning (CAL) research: An assessment of the literature in SSCI
journals from 1998–2006. Journal of the American Society for Information Science and Technology, 59(8), 1313–1320.
Lu, C.-J., & Shulman, S. W. (2008). Rigor and flexibility in computer-based qualitative research: Introducing the coding analysis toolkit. International Journal of Multiple
Research Approaches, 2(1), 105–117.
Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering
course. IEEE Transactions on Education, 56(4), 430–435.
McDonald, K., & Smith, C. M. (2013). The flipped classroom for professional development: Part I. Benefits and strategies. The Journal of Continuing Education in Nursing,
44(10), 437–438.
McEvoy, C. S., Cantore, K. M., Denlinger, L. N., Schleich, M. A., Stevens, N. M., Swavely, S. C., ... Novick, M. B. (2016). Use of medical students in a flipped classroom
programme in nutrition education for fourth-grade school students. Health Education Journal, 75(1), 38–46.
McLaughlin, J. E., Griffin, L. M., Esserman, D. A., Davidson, C. A., Glatt, D. M., Roth, M. T., ... Mumper, R. J. (2013). Pharmacy student engagement, performance, and
perception in a flipped satellite classroom. American Journal of Pharmaceutical Education, 77(9), 1–8.
McLean, S., Attardi, S. M., Faden, L., & Goldszmidt, M. (2016). Flipped classrooms and student learning: Not just surface gains. Advances in Physiology Education, 40(1),
47–55.
Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education,
52(10), 597–599.
Moraros, J., Islam, A., Yu, S., Banow, R., & Schindelka, B. (2015). Flipping for success: Evaluating the effectiveness of a novel teaching approach in a graduate level
setting. BMC Medical Education, 15(27), 1–10.
Mylott, E., Kutschera, E., Dunlap, J. C., Christensen, W., & Widenhorn, R. (2016). Using biomedically relevant multimedia content in an introductory physics course for
life science and pre-health students. Journal of Science Education and Technology, 25(2), 222–231.
Nederveld, A., & Berge, Z. L. (2015). Flipped learning in the workplace. Journal of Workplace Learning, 27(2), 162–172.
Nguyen, B., Yu, X., Japutra, A., & Chen, C.-H. S. (2016). Reverse teaching: Exploring student perceptions of “flip teaching”. Active Learning in Higher Education, 17(1),
51–61.
O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85–95.
Paulus, T., Woods, M., Atkins, D. P., & Macklin, R. (2017). The discourse of QDAS: Reporting practices of ATLAS.ti and NVivo users with implications for best practices.
International Journal of Social Research Methodology, 20(1), 35–47.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences. Oxford, UK: Blackwell Publishing Ltd.
Porcaro, P. A., Jackson, D. E., McLaughlin, P. M., & O'Malley, C. J. (2016). Curriculum design of a flipped classroom to enhance haematology learning. Journal of
Science Education and Technology, 25(3), 345–357.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231.
Roehl, A., Reddy, S. L., & Shannon, G. J. (2013). The flipped classroom: An opportunity to engage millennial students through active learning. Journal of Family and
Consumer Sciences, 105(2), 44–49.
Ryan, M. D., & Reid, S. A. (2016). Impact of the flipped classroom on student performance and retention: A parallel controlled study in general chemistry. Journal of
Chemical Education, 93(1), 13–23.
Sage, M., & Sele, P. (2015). Reflective journaling as a flipped classroom technique to increase reading and participation with social work students. Journal of Social
Work Education, 51(4), 668–681.
Sahin, A., Cavlazoglu, B., & Zeytuncu, Y.,E. (2015). Flipping a college calculus course: A case study. Journal of Educational Technology & Society, 18(3), 142–152.
Sayeski, K. L., Hamilton-Jones, B., & Oh, S. (2015). The efficacy of IRIS STAR legacy modules under different instructional conditions. Teacher Education and Special
Education, 38(4), 291–305.
Schlairet, M. C., Green, R., & Benton, M. J. (2014). The flipped classroom: Strategies for an undergraduate nursing course. Nurse Educator, 39(6), 321–325.
Schneider, B., & Blikstein, P. (2016). Flipping the flipped classroom: A study of the effectiveness of video lectures versus constructivist exploration using tangible user
interfaces. IEEE Transactions on Learning Technologies, 9(1), 5–17.
Shih, M., Feng, J., & Tsai, C.-C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals.
Computers & Education, 51(2), 955–967.
Şimşek, H., & Yıldırım, A. (2011). Qualitative research methods in social sciences. Ankara: Seçkin Publishing.
Smith, J. D. (2013). Student attitudes toward flipping the general chemistry classroom. Chemistry Education: Research and Practice, 14(4), 607–614.
Sockalingam, S., James, S.-L., Sinyi, R., Carroll, A., Laidlaw, J., Yanofsky, R., et al. (2016). A Flipped Classroom approach to improving the quality of delirium care
using an interprofessional train-the-trainer program. Journal of Continuing Education in the Health Professions, 36(1), 17–23.
Sohrabi, B., & Iraj, H. (2016). Implementing flipped classroom using digital media: A comparison of two demographically different groups perceptions. Computers in
Human Behavior, 60, 514–524.
Sun, J. C.-Y., Wu, Y.-T., & Lee, W.-I. (2017). The effect of the flipped classroom approach to OpenCourseWare instruction on students' self-regulation. British Journal of
Educational Technology, 48(3), 713–729.
Teo, T. W., Tan, K. C. D., Yan, Y. K., Teo, Y. C., & Yeo, L. W. (2014). How flip teaching supports undergraduate chemistry laboratory learning. Chemistry Education:
Research and Practice, 15(4), 550–567.
Topping, K., & Ehly, S. (1998). Peer-assisted learning. New York, USA: Routledge.
Tucker, B. (2012). The flipped classroom. Education Next, 12(1), 82–83.
van Vliet, E. A., Winnips, J. C., & Brouwer, N. (2015). Flipped-class pedagogy enhances student metacognition and collaborative-learning strategies in higher education but effect does not persist. CBE-Life Sciences Education, 14(3), 1–10.
Wang, Y. H. (2016). Could a mobile-assisted learning system support flipped classrooms for classical Chinese learning? Journal of Computer Assisted Learning, 32(5),
391–415.
Wanner, T., & Palmer, E. (2015). Personalising learning: Exploring student and teacher perceptions about flexible learning and assessment in a flipped university
course. Computers & Education, 88, 354–369.
Weaver, G. C., & Sturtevant, H. G. (2015). Design, implementation, and evaluation of a flipped format general chemistry course. Journal of Chemical Education, 92(9),
1437–1448.
Wilson, S. G. (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. Teaching of Psychology, 40(3), 193–199.
Zainuddin, Z., & Attaran, M. (2016). Malaysian students' perceptions of flipped classroom: A case study. Innovations in Education & Teaching International, 53(6),
660–670.
Zainuddin, Z., & Halili, S. H. (2016). Flipped classroom research and trends from different fields of study. International Review of Research in Open and Distance Learning,
17(3), 313–340.
Zawilinski, L. M., Richard, K. A., & Henry, L. A. (2016). Inverting instruction in literacy methods courses. Journal of Adolescent & Adult Literacy, 59(6), 695–708.
345