The 15th International Scientific Conference
eLearning and Software for Education
Bucharest, April 11-12, 2019
10.12753/2066-026X-19-029
On Using ADDIE/SAMR Methodology to Improve the Performance
in Blended Learning
Grigore ALBEANU, PhD.
Scientific Research Centre in Mathematics and Computer Science,
"Spiru Haret" University, 13 Ion Ghica Str., Bucharest, Romania
g.albeanu.mi@spiruharet.ro
Florin POPENTIU-VLADICESCU, PhD.
Politehnica University, Bucharest &
Academy of Romanian Scientists, Bucharest, Romania
popentiu@imm.dtu.dk
Abstract: In large, blended learning combines the usage of online educational resources with
traditional classroom approaches. However, blended learning exists on a continuum between 100%
face-to-face and 100% online for training, student participation, and course delivering/accessing.
Establishing the percent of blend is compulsory. The performance of blended learning classroom can be
improved not only by technology mediation, but also developing well suited educational resources to be
integrated during learning tasks. ADDIE/SAMR methodology deals with the development strategy of
blended learning-oriented courses according to the life-cycle steps (Analysis, Design, Development,
Implementation and Evaluation) taking into account SAMR transformation model by the following
operations: Substitution, Augmentation, Modification, and Redefinition. The methodology was used to
develop a computational intelligence course for a master program in modern technologies in
information system engineering. The learning environment is supported by Blackboard platform, where
communication is mediated by e-mail, and forum. The perceptions and satisfaction of students enrolled
on platform to participate in the blended manner to the course were registered and analysed. The
following data were collected: Educational resources format; Educational resources usability;
Educational resources clarity; Educational resources accessibility; Calendar of meetings in class;
Interaction maturity in class/online; Teams development; The adequacy level of assignments;
Comparison of work in class / assignments; Preference for traditional courses. Computer-based
assessment practices proved value in increasing the student participation and motivation and to provide
feedback to students in a timely manner to help them to identify weaknesses, reflect on their strong
behaviour, and improve their learning strategy. Results are presented through a comparison against
traditional version of the course. Moreover, details on student performance, tutor rating, student self
evaluation are obtained and discussed.
Keywords: ADDIE; SAMR; blended learning; performance; computational intelligence.
I.
INTRODUCTION
Recently, education started new models for teaching and learning [2, 3, 4, 6, 10]. The
continuum "Traditional classes - Augmented traditional classes - Augmented Virtual Classes - Virtual
classes" is used, with different degrees of augmentation by training companies, schools, and
universities. In large, blended learning combines the usage of online educational resources with
traditional classroom approaches. Blended learning exists on such a continuum between 100% face-toface and 100% online for training, student participation, and course delivering/accessing. Establishing
215
the percent of blend is not only recommended by experts in education, but also can be established
during the analysis step when one organization starts a training program.
The paper discusses the usage of an integrated model of instructional design based on ADDIE
[2, 5, 9] and SAMR existing models [18, 19]. Details on implementation and evaluation of a course in
computational intelligence for software engineering developed using ADDIE/SAMR approach are
given after a short description of the mentioned model.
II.
ON ADDIE/SAMR APPROACH
ADDIE/SAMR methodology deals with the development strategy of blended learning oriented
courses according to the life-cycle steps (Analysis, Design, Development, Implementation and
Evaluation) taking into account SAMR transformation model by the following operations:
Substitution, Augmentation, Modification, and Redefinition [18, 19, 23, 29].
The ADDIE model is based on the original Instructional System Design [29], and helps the
instructional designer to learn about the future students, to understand the ecosystem (generation type:
X, Y, Z, ALPHA; job orientation; organizational objectives), to better define the learning objectives,
to identify a suitable set of methods, techniques and technologies that fit the skills, ecosystem and
objectives, to identify the best infrastructure, software and platform to assure the most exciting
experience to learners, and to monitor and evaluate professional development activities and, if
necessary, a decision to redesign will be considered [2].
The SAMR aim is the transformation both of educational materials and training methods.
Moreover, the trainer will be transformed by professional development actions [18]. The integration of
ADDIE and SAMR models will help the instructional designer to produce modern study curriculum,
syllabus, educational content, and evaluation items according to the needs of current generation of
learners.
2.1
The ADDIE instructional design model
Scientists and practitioners agreed that ADDIE is the most used instructional model which
integrates pedagogy, learning theories, and other instructional design principles [2, 9, 29].
The objectives of the first ADDIE' s phase are directed to describe requirements concerning
the front-end units, the state of the art in the field, data collection and analysis methods for needs
assessment, data collection and analysis methods for performance analysis, milestones identification
and reliability solutions.
Once the analysis is complete, the design phase starts by developing program mission and
objectives, and concludes by establishing specific teaching objectives, assessment strategy and the
societal trends influencing education and training. The design has been described what will be
accomplished, but the development phase will describe how to fulfill the mission and educational
objectives. Mostly, the development is a detailed design process describing the basis for selection of
methods and media, and estimate development time for various instructional options taking into
account the trends in educational technology.
During implementation, the instructional developer will work closely with one or more
content/subject matter expert and with production team members (digital photographers, hypertext
writers, web programmers, video specialists, web designer, social communicators and marketers). The
main implementation objectives are related to project planning and management, Gantt charts and
PERT networks creation, monitoring and analysis, keeping budget in accepted limits and selection of
suitable implementation strategies, including communication protocols between actors involved during
project implementation.
The evaluation step should provide information on the fulfilling level of learning objectives
according to some model, like revised Bloom pyramid, or the contract-based quality indicators of the
training program.
In a Blended learning context [20, 24, 27], a number of different learning modes that are
important to be considered: Face-to-face (F2F) workshops, typically facilitated by the trainer/expert,
that include a range of different tasks, such as lectures, group activities, and individual work; Virtual
216
learning (VL), such as individual e-learning (single-channel system) or participation in virtual groups
(multi-channel system, groups on Facebook, WhatsApp, Google etc.), including by mobile-learning;
Work-based learning (WbL), during practical tasks, project development, and graduation/dissertation
thesis elaboration through face-to-face dialogue between student and supervisor, or participating to
communities of practice. The design and development phases of ADDIE model are responsible with
the tailoring of F2F+VL+WbL according to the program goals and objectives. F2F should be based
not only classical lectures or digital presentations, but also on digital simulations, audio or video
demonstrations in order to increase the teaching value and the learner knowledge and skills. VL is
planned in such a way that small units of information will be offered to learner, and formative
evaluation will be used after a small number of such units. WbL is planned whenever a project-based
teaching is most suited to be used as a learning model.
The course on Computational Intelligence has been designed to provide fundamental theory,
concepts and applications of computational intelligence methods in software engineering, in particular
neural networks (from basic neuron to spike neural networks and deep learning), fuzzy computing
based systems, evolutionary algorithms, and nature-inspired collaborative learning.
2.2
The SAMR transformation model
The SAMR model was created by Dr. Ruben Puentedura [18] and provides a way of seeing
how computerized technology has an impact on improving the teaching and learning process. The
model name comes from S - Substitution (Computer technology is used to perform the same task as
before using computers), A - Addition (Computer technology provides an effective tool for performing
common tasks), M - Modification (This is the first step over the line between classroom improvement
and classroom transformation), R - Redefinition (Computer technology allows new tasks that were
unthinkable so far).
The first two levels are called Improvement, and technology helps to achieve traditional tasks.
However, sometimes these levels may not be necessary when we look at all the possibilities offered by
technology. The other two levels are called Transformation. Here is the real metamorphosis of the
classroom, and the technology allows creative tasks that are very different from those found in
traditional classrooms/lectures [19].
Hence, the SAMR model provides instructors with a continuous review of the practice to
make the best use of technology [23]. According to the four letters, there are four ways to integrate
technology with learning. Technology enters the student world and installs lifelong learning habits.
For example, if the task is to find a place in an atlas, then the four modes of action allow us
the following developments [18, 19]. Substitution: We use Google Earth to identify locations. As a
result, technology helps us without any functional change. Addition: We use Google Earth tools to
measure the distance between two places. Result: The technology acts as a direct replacement of the
instrument, but with a functional improvement. Modification: Use Google Earth layers like Panoramio
or 360 cities to identify locations. Result: The technology allows a significant reconfiguration of the
task. Redefinition: We create a guide using Google Earth and publish it online. Result: Technology
allows for tasks that were previously unthinkable.
Moreover, we can imagine the usage of technology with artificial intelligence algorithms to
optimize various tasks, like logistic activities, tourism, and transportation.
2.3
The integrated ADDIE/SAMR instructional design model
Developing a training program can benefit of ADDIE model, but when a reengineering of an
existing training program is necessary, the integration of ADDIE and SAMR is a good solution.
Continuously, SAMR can be used in all steps of ADDIE. The final product is a high quality training
program, high quality training materials, high quality F2F lectures, high quality VL, and highly
improved WbL. This kind of integration was used to upgrade an already existing course on "Soft
Computing Methodologies (SCM)" to an updated and extended course "Computational Intelligence for
Software Engineering (CI4SE)". Moving for the theoretical approach SCM to the more applied
oriented CI4SE, was possible as a result of existing technology. The VL part of the course is supported
by the Blackboard platform, while F2F and WbL make use of classic approaches mediated by
technology.
217
III.
EXPERIMENTAL RESULTS
3.1
On the content of the CI course
Based on ADDIE, after the first three steps and taking into account the SAMR model, the
training needs was identified, designed new parts, and redesigned already existing parts, and
developed the front-end of every item program. Short details are presented, in the following, about the
trained subject.
According to [7, 11, 12, 22, 32], the main fields of Artificial Intelligence considered as special
topics for CI are: Artificial Neural Networks (ANN) [12, 30, 31], Fuzzy Systems [12, 35],
Evolutionary Computation [12, 34, 36], and Nature-Inspired Computation [37].
Inspired by the biological network of neurons, ANNs can be used as nonlinear models to
classify data or to solve input-output relations [12, 31]. The mathematical model of ANN is
represented by a weighted directed graph, having three types of vertices (neurons) - input, hidden and
output. The ANN results depend on three functions for every processing unit: network input fin, neuron
activation fact, and network output fout. If the associated graph is acyclic then ANN is called feed
forward network, otherwise is mentioned as recurrent network. The weights of the network are
obtained after a training process. The first kind of learning task, called fixed, uses a set of pairs training patterns - (x, y), where x is an input vector, and y is the output vector produced by ANN when
received as input the vector x. Both x and y can be multivariate with different dimensions. The
learning process is evaluated by some metric, like square root of deviations of actual results from
desired output. Another kind of learning task, called free, uses only input vectors, the objective of
ANN being to solve a clustering / classification problem. A similarity measure is necessary to identify
the prototypes. The power of ANN depends on the activation model of neurons, and the number of
hidden layers.
When the inputs are fuzzy [31, 35], intuitionistic fuzzy [1], or of neutrosophic type [21], the
activation process is based on defuzzification/deneutrofication procedures. Fuzzy systems make use of
fuzzy sets, fuzzy numbers, and fuzzy logic. An intelligent FS is a knowledge based system used to
answer to questions/queries formulated by a user according to a linguistic variables language. The
natural language processing based interface is responsible on fuzzification/neutrofication procedure.
Neutrosophic thinking for engineering applications is based on three indicators: one for
truth/membership degree (T), one for the degree of indeterminacy (I), and one for false/nonmembership degree (F). If F +T = 1, and I=0, then the fuzzy framework [31] is considered. If F+T<1,
and I = 1 - (F + T), then the intuitionistic-fuzzy theory of Atanassov is obtained [1]. The neutrosophic
framework considers 0 ≤ T + I + F ≤ 3, with 0 ≤ T, I, F ≤ 1. Defuzzification can be obtained easy by
the centroid method. Converting an intuitionistic-fuzzy entity or a neutrosophic entity to a crisp value
is not so easy. Firstly, an indicator function, denoted by H, used to estimate the overall degree of
truth/membership should be computed (as in fuzzy representation), and this function will be used to
compute a crisp value. The indicator function can be computed by H = F for
every item from universe of discourse. The parameters and  are positive numbers, in
decreasing order, with their sum is 1 (unity). The method was proposed by Wang et al. [25], and the
parameters should be determined based on the available information about the problem under
treatment.
Evolutionary computation is an area of research covering genetic algorithms, evolutionary
strategies, and genetic programming. The techniques are based on a population of individuals and the
following operations [12]: reproduction (crossover/recombination), random variation (mutation,
hypermutation), competition, and selection. The objective of any evolutionary algorithms is to
optimize the searching process in a robust and intelligent manner, as inspired by biological
reproduction schemes.
3.2
On assessment method and results
The learning environment is supported by Blackboard platform, where communication is
mediated by e-mail, and forum. The perceptions and satisfaction of students enrolled on platform to
participate in the blended manner to the course were registered and analyzed.
218
The following data were collected and processed:
a) the format of educational resources (text, pdf, presentation, audio, video, demonstrative
software);
b) the usability level of educational resources (low, medium, high);
c) the clarity level of educational resources (very low, low, medium, high, excellent);
d) the accessibility of educational resources (only from the campus, accessible from anywhere
as private user, accessible from everywhere as public user);
e) the calendar of meetings in class (F2F lessons, virtual classroom); deadlines for online
assignments (note review, essay, multi-choice etc.) are suitable, underestimated, overestimated.
f) the maturity of interaction in class/online (how active/interested is someone in a subject:
F2F debates, e-mail messages, forum threads);
g) the teams' size (for group oriented applications/projects);
h) the adequacy level of assignments (easy, medium, difficult, very difficult, research);
i) comparison of work in class / online assignments (better, similar);
j) the preference for traditional courses (a degree of truth in [0, 100]).
Computer-based assessment practices proved value in increasing the student participation and
motivation and to provide feedback to students in a timely manner to help them to identify
weaknesses, reflect on their strong behavior, and improve their learning strategy. Results are checked
through a comparison against the traditional version of the course. The number of students enrolled for
both MSC (version 2017-2018) and CI4SE (version 2018-2019) was 20. The students of CI4SE found
a more interesting course due to the following reasons: CI4SE is an applied course, and CI4SE provide
more multimedia material, demonstrative software, and real life applications.
Also, both the trainer and tutor described an improved experience during F2F and VL classes,
with the new CI4SE. The confidence in CI4SE was 80% (students), respective 90% (trainers). Such
results are realistic when thinking that CI4SE was designed for adult education.
IV.
CONCLUSIONS
A positive experience was reported by students enrolled to CI4SE, a course on computational
intelligence for software engineering developed according to ADDIE/SAMR approach by
reengineering SCM - an old theoretical course on soft computing methodologies. The integrated
approach will be used to reengineered more courses trained in the framework of the program study
"Modern Technologies in Information System Engineering".
Reference Text and Citations
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
Atanassov, K. T., 1988. Review and New Results on Intuitionistic Fuzzy Sets, Mathematical Foundations of
Artificial Intelligence Seminar, Sofia, 1988, Preprint IM-MFAIS- 1-88. Reprinted: Int. J. Bioautomation, 20(S1),
S7-S16 (2016).
Bates, Anthony William (Tony), 2016. Teaching in a Digital Age. Guidelines for designing teaching and learning,
https://opentextbc.ca/teachinginadigitalage/.
Cabero, Julio; Llorente, Carmen; Puentes, Angel, 2010. Online Students' Satisfaction with Blended Learning, C35,
v. XVIII, 2010, Scientific Journal of Media Literacy, pp. 149-156, DOI:10.3916/C35-2010-03-08.
Dahinden, Markus; Faessler, Lukas, 2011. Monitoring blended learning environments based on performance data,
in Proceedings of the IADIS International Conference e-Learning 2011, Rome, Italy 20-26 July 2011, pp. 401-408.
Durak, G.; Ataizi, M.; The ABC’s of Online Course Design According to Addie Model, Universal Journal of
Educational Research, 4/2016, 9, pp. 2084-2091 (online: DOI: 10.13189/ujer.2016.040920).
El-Din, Ashraf Salah; Adascalitei, Adrian; Temneanu, Marinel; Aradoaiei, Sebastian, 2018. Blended Learning
Methodologies and ePedagogical Approaches Used in an Electrical and Computer Engineering Education Program
Leading to International Accreditation, Proceedings of ICVL, pp. 63-77.
Engelbrecht, Andries P., 2007. Computational intelligence: an introduction (2nd edition), John Wiley & Sons Ltd.
Ghirardini, Beatrice, 2011. E-learning methodologies. A guide for designing and developing e-learning courses.
FAO, Viale delle Terme di Caracalla, Rome, Italy.
Holden, Jolly T., 2015. An introduction to the ADDIE instructional systems design model, WP,
https://static.secure.website/wscfus/4899501/uploads/White_Paper--Introduction_to_the_ADDIE_ISD_Model.pdf.
IAL, Blending classroom with work and technology: how to design a blended curriculum, https://www.ial.
edu.sg/content/dam/projects/tms/ial/Find-resources/Learning-resource-and-tools/blended-learning-guide/Blended
%20classroom%20with%20work%20and%20technology%20(IAL)%201st%20edition%20(1.2).pdf.
219
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
Konar, Amit. 2005. Computational Intelligence. Principles, Techniques and Applications, Springer-Verlag Berlin
Heidelberg.
Kruse, Rudolf; Borgelt, Christian; Braune, Christian; Mostaghim, Sanaz; Steinbrecher, Matthias, 2016.
Computational Intelligence. A methodological Introduction (2nd edition), Springer.
Means, Barbara; Toyama, Yukie; Murphy, Robert; Baki. Marianne, 2013. The Effectiveness of Online and Blended
Learning: A Meta-Analysis of the Empirical Literature, Teachers College Record, Vol. 115, 47 pages, Columbia
University.
Moskal, Patsy; Dziuban, Charles; Hartman, Joel; Blended learning: A dangerous idea?, Internet and Higher
Education, 18 /2013, pp. 15–23.
Nauta, Jan M.; Platenkamp, Jo; Hettinga, Marike, 2016. Development and Evaluation of a Blended Learning
Course, Proceedings of eLmL 2016: The Eighth International Conference on Mobile, Hybrid, and Online Learning,
IARIA, pp. 17-19.
Palade, Vasile; Bocaniala, Cosmin Danut; Jain, Lakhmi (Eds), 2006. Computational Intelligence in Fault
Diagnosis, Springer-Verlag London.
Palit, Ajoy K.; Popovic, Dobrivoje. 2005. Computational intelligence in time series forecasting: theory and
engineering applications, Springer-Verlag London.
Puentedura,
Ruben
R.,
2006.
Transformation,
Technology,
and
Education,
Hippasus,
http://hippasus.com/resources/tte/ puentedura_tte.pdf [available March 4, 2019].
Romrell, Danae; Kidder, Lisa C.; Wood, Emma, 2014. The SAMR Model as a Framework for Evaluating
mLearning, https://files.eric.ed.gov/fulltext/EJ1036281.pdf [available March 4, 2019].
Saliba, Gina; Rankine, Lynnae; Cortez, Hermy, 2013. Fundamentals of blended learning, University of Western
Sidney.
Smarandache, Florentin, 2007. A unifying field in logics: neutrosophic logic. Neutrosophy, neutrosophic set,
neutrosophic probability and statistics (6th edition), Ann Arbor: ProQuest Information & Learning (2007).
Sumathi, S.; Surekha, Paneerselvam. 2010. Computational intelligence paradigms: theory & applications using
MATLAB, CRC, Taylor and Francis Group, LLC.
Tsybulsky, Dina; Levin, Illya, 2016. SAMR Framework for Study Technology Integration in Science Education,
New Perspectives in Science Education, 5th edition, pp. 599-604.
Vaughan, Norman, 2014. Student Engagement and Blended Learning: Making the Assessment Connection, Educ.
Sci. 2014, 4, pp. 247–264; doi:10.3390/educsci4040247.
Wang, H., Smarandache, F., Zhang, Y.-Q., Sunderraman, R., 2005. Interval Neutrosophic Sets and Logic: Theory
and Applications in Computing, Hexis.
Wu, Shelly Xiaonan; Banzhaf, Wolfgang. The use of computational intelligence in intrusion detection systems: A
review, Applied Soft Computing, 10/2010, pp. 1–35.
Yigit, Tuncay; Koyun, Arif; Yksel, Asim Sinan; Cankaya, Ibrahim Arda, Evaluation of Blended Learning
Approach in Computer Engineering Education, Procedia - Social and Behavioral Sciences, 141/ 2014, pp. 807 –
812.
Zadeh, Lotfi A., 1994. Fuzzy logic, neural networks, and soft computing, Communications of the ACM, 37(3), pp.
77-84.
***, online: AFMAN 36-2234, http://www.au.af.mil/au/awc/awcgate/edref/afman36-2234.pdf, 1993 (available
4/03/2019)
***, online: Neural Networks, IEEE TV - An introduction /2013 [https://ieeetv.ieee.org/conferencehighlights/artificial_neural_networks_intro?rf=channels|56|Selected_Videos&].
***, online: Neural Networks, Welch Labs - Part1/2014 [https://www.youtube.com/watch?v=bxe2T-V8XRs],
Part2/2014 [https://www.youtube.com/watch?v=UJwK6jAStmg], Part3/2014 [https://www.youtube.com/watch?
v=5u0jaA3qAGk], Part4/2014 [https://www.youtube.com/watch?v=GlcnxUlrtek],
Part5/2014 [https://www.
youtube.com/watch?v=pHMzNW8Agq4], Part6/2015 [https://www.youtube.com/watch?v=9KM9Td6RVgQ],
Part7/2015 [https://www.youtube.com/watch?v=S4ZUwgesjS8].
***, online: Computational Intelligence Lab (Hofmann, Thomas), https://www.video.ethz.ch/lectures/dinfk/2018/spring/263-0008-00L.html, 2018.
***, online: Computational Intelligence Applications, Tesla Autopilot Full Self Driving Demonstration demo,
https://www.youtube.com/watch?v=aKed5FHzDTw, 2016.
***, online: Evolutionary computation, TEDx Talks, https://www.youtube.com/watch?v=D3zUmfDd79s, 2013.
***, online: Fuzzy Logic. An introduction, Cis IEEE: https://www.youtube.com/watch?v=P8wY6mi1vV8, 2011.
***, online: Genetic Algorithm Tutorial, Full Stack Academy, https://www.youtube.com/watch?v=XP8R0yzAbdo,
2017.
***, online: Nature Inspired Computation, GMU Television, https://www.youtube.com/watch?v=RSXpu_ceWDg,
2014.
220
Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.