Vocational learners: testing readiness for online learning
Authors:
Dr Peter J Smith
Faculty of Education
DEAKIN UNIVERSITY VIC 3216
AUSTRALIA
Email:
pjbs@deakin.edu.au
Telephone:
+61 3 5227 1452
Fax:
+61 3 5227 2014
Professor Eugene Sadler-Smith
University of Plymouth Business School
University of Plymouth
PLYMOUTH PL4 8AA
UNITED KINGDOM
Email:
Eugene.Sadler-Smith@pbs.plym.ac.uk
Telephone:
+44 1752 232 870
Keywords: online learning, self-directed learning, collaborative learning, vocational
learners, readiness.
Abstract
McVay’s (2000) Readiness for Online Learning questionnaire is tested for its potential
value as an instrument to assess the readiness for online learning of technical level and of
university level vocational learners. The research indicates that the instrument yields a
reliable factor structure that is replicable across the two learner groups, and is
interpretable in a context of broader research and theory on flexible learning. Although
the reliability of the instrument and its items is acceptable for the university level sample,
reliability was less than acceptable for the technical level learners. The utility of the
instrument is discussed for each of these samples of learners.
1
Introduction
Research Questions
This exploratory paper sets out to investigate the potential value of the McVay (2000)
Readiness for Online Learning questionnaire as a tool to measure vocational student
readiness for online learning sequences that form a component of more broadly based
flexible delivery training programs. On the basis of data from a sample of vocational
learners at a technical and further education (TAFE)1 institute, and a sample of university
students enrolled in vocationally oriented courses, we investigate the following research
questions:

Does the instrument provide acceptable levels of scale reliability?

Does the McVay instrument provide for factor-analytic treatment that yields
results interpretable in a framework of other research with vocational learners?

Are resultant factor solutions reliable and replicable across different learner
groups?
These questions are important ones both for theory and practice for a number of reasons.
First. It is important to have reliable and valid measures of learners’ readiness; second,
the measurement of learners’ readiness must be interpretable in terms of extant and
evolving theories of online and workplace learning; third, the assessment of readiness
should be possible across groups in order to facilitate comparison. Finally, the learner
characteristic of readiness is likely to be a crucial factor in determining levels of
motivation and performance in learning from online sources, especially among groups
2
such as vocational learners who may not be readily predisposed to independent, or even
classroom learning.
Readiness for flexible delivery of training
In the Australian setting flexible learning methods generally require learners at TAFE or
at university to use independent learning materials in verbal form (print, screen-based,
audiotapes, audio-CD) to construct knowledge independently of the interaction and
collaboration that can be exercised when learning in group settings. Along with those
more independent learning methods, flexible delivery can also include workplace
experience and practice, and the opportunity for interaction with fellow learners, and with
instructors. Online technologies now available provide opportunity for independent
learning to be supplemented and enhanced through collaborative learning and discussion
between learners, and between learners and their instructors.
In a large-scale research project focussing on student readiness for flexible
delivery of training, Smith (2000) tested the learning preferences of 1252 technical and
further education (TAFE) vocational learners using the Canfield Learning Styles
Inventory. The research identified two major underlying dimensions of readiness. The
first of these was a dimension associated with the degree to which TAFE learners were
able to be self-directed in their use of flexible delivery materials for learning; and the
second was associated with their preferences for learning through verbal means such as
text or listening, or non-verbal means such as hands-on experience and practice.
In a context of the research indicating the presence of differences between individuals,
and differences between client groups, the investigation showed that vocational learners
3
were largely characterised by low preferences towards self-directedness; and low
preferences towards verbally presented material, but a high preference for learning in
socio-cultural settings where there can be discussion with their colleagues.
That large-scale research was provided with some confirmatory evidence through
a study by Warner, Christie and Choy (1998) who, similarly, showed that TAFE
vocational learners were not generally self-directed. Additionally, TAFE vocational
learners were generally uncomfortable with training delivery methods that required
engagement with large amounts of text learning, preferring instead hands-on experience
and practice. Finally, that research also showed a preference among vocational learners
for learning in settings that enabled collaboration with instructors and with fellow
learners. In her study of the metacognitive strategies used by these vocational learners,
Boote also showed that the strategies needed for effective self-directed learning were
largely underdeveloped among TAFE students.
At a university level, similar dimensions of learning preferences have also been
noted by Sadler-Smith and Riding (1999) when they identified that the learning
preferences of UK business students vary in terms of dependent versus independent
learning, collaborative versus non-collaborative learning, and preferences for particular
instructional media. In an Australian study comparing the learning preferences of
University students with those of TAFE students, Smith (2001) identified the same two
dimensions of self-direction and verbal-non-verbal preferences for learning with the two
groups of learners. The TAFE students, however, showed a significantly higher
preference for learning in a collegial environment than did the university students. Even
among some professional groups (for example, HR practitioners) it appears that
4
traditional methods may sometimes be preferred over self-directed methods (SadlerSmith, Allinson & Hayes, 2000).
The findings suggest that any measurement of the readiness for flexible learning
of TAFE and university learners and managers needs to provide information on readiness
to be self-directed in learning, and readiness to engage with the learning materials and the
learning environment provided by the particular form of flexible delivery being proposed.
Readiness in online environments
While online delivery of flexible learning sequences can also be similarly challenged by
the lack of readiness for self-directedness, by a low preference for verbally presented
materials, and by a need to learn collaboratively, there are also opportunities for online
learning to overcome some of these potential challenges, most particularly where the
online delivery provides opportunity for collaborative learning through devices such as
computer-mediated communication (CMC).
There has been an amount of research in online learning that has focussed on
learner willingness to be self-directed and to take control and self-manage their learning.
For example, Wang and Newlin (2000), working with undergraduate students in a course
on psychological statistics showed that success among their students in a virtual
classroom was predicted by 'intellectual inquisitiveness, and an internal locus of control'
(p.142). Research by Valenta, Therriault, Dieter and Mrtek (2001), using the Canfield
Learning Styles Inventory, has also identified a strong independent learning factor as a
possible predictor of success with technology-mediated distance education.
5
However, it is not always made clear in the literature what is actually being
referred to when the term 'online' is applied to education and training. At one end of a
spectrum it can mean that learning materials are available to learners electronically for
them to read on screen or download, but without any two-way communication between
instructor and learner. At a similarly basic level it can mean merely the occasional
exchange of emails between learner and instructor, but with no other learning resources
or experiences provided in electronic form. At a much more sophisticated level it can
mean that learning resources are available electronically, and supported by a groupware
system where learners can interact together and interact with their instructor.
Given that the research indicates a preference among vocational learners for
learning environments that provide opportunity for socio-cultural construction of
knowledge we suggest that any measurement of readiness for online learning needs to be
inclusive of collaborative learning. That collaborative learning in an online setting is
provided by a conceptualisation of CMC that is characterised by groupware systems (eg
First Class) that enable learners to interact with other learners, with their instructors, and
with other experts who may be external to the teaching-learning environment (Stacey,
1998). The interaction may be synchronous or asynchronous, but the important thing
here is the opportunity to participate in a social construction of knowledge that has been
researched by the constructivists (eg von Glasersfeld, 1987). There is considerable
evidence in the online literature (eg Harasim et al, 1995; McKavanagh et al, 2002) that
‘active participation, encouraged by the atmosphere and course design and in other ways,
is central to a successful online course’ (Harasim et al, 1995: 177). Finally, Rainbow and
Sadler-Smith (forthcoming), in their research with business students in the UK, also
6
argued that to be effective computer aided learning needs to be integrated with
opportunities for dialogue.
We suggest that any instrument that has potential to test readiness for online
learning there needs to be inclusion of items that specifically test for readiness to engage
in collaborative learning and discussion online.
The Readiness for Online Learning questionnaire
A Readiness for Online Learning questionnaire was developed by McVay (2000) for use
in conjunction with an orientation program for baccalaureate students within a university
in the United States. Although McVay's own work indicated the instrument provided
predictive validity, there was work to be done on establishing reliability through
statistical procedures, and in exploring the value of the instrument for research purposes.
In a paper about to be published, Smith, Murphy and Mahoney (2003) have established
the reliability of the instrument at university student level using a sample of US and
Australian undergraduate university students. That same research also showed that the
data was readily factorable using factor analytic techniques, and yielded two factors. One
of these factors measured the degree of self-management of learning engaged in by
respondents; the second factor measured the degree to which respondents were
comfortable with learning and communicating online. Further research (Smith, 2003)
with a sample of 314 Australian undergraduate university students also established the
reliability of the instrument, and identified the same two factors.
These research findings using the McVay (2000) questionnaire with university
students were interesting to the current study with vocational learners both at TAFE and
7
university levels, since the factors identified in the research with university respondents
to the questionnaire are similar to those identified among TAFE vocational students.
Because validity had been at least partially established through McVay's (2000) work,
and the instrument showed good reliability and a stable factor structure, the McVay
Readiness for Online Learning instrument was seen by the authors as potentially useful in
the measurement of vocational student readiness for online learning. It needs to be
emphasised here that the questionnaire measures dispositional factors among students,
and does not seek to measure their ability to engage with the technical software and
hardware skills of effective online learning.
Other attractions for the McVay instrument include the fact that it contains
specific questions relating to willingness to collaborate and communicate online; and that
it is quite quick to administer and to score.
Method
McVay's (2000) Readiness for Online Learning questionnaire was administered during a
scheduled classroom training sessions to 81 TAFE vocational learners who were training
through a registered training provider. Learners attended some classes at the training
provider's premises, and pursued other learning experiences in their workplace
environment. Learners were asked to respond to the questionnaire in the context of their
current training program. The Readiness for Online Learning questionnaire comprises
thirteen questions, shown in Table 1. Respondents to the questionnaire rate their
agreement with each statement on a four-point Likert scale.
8
Occupations the learners were training in were business (35), animal husbandry
and welfare (11), and hospitality (35). Ages ranged from 18 years to 34 years, with a
mean age of 21.2 years, and a standard deviation of 4.1. There were 22 males and 59
females in the sample. The levels of training were at Certificate II to Certificate IV in the
Australian Qualifications Framework.
The university student group with which the TAFE comparisons were made also
comprised students from undergraduate vocational programs at university level, with 314
students drawn from journalism, social work, law, business, teacher education, nursing
and health sciences. The age range of the students was from 18 to 54 years with ninety
percent of the sample being 25 years or younger. The mean age was 21.4 years with a
standard deviation of 5.3. The university student data is the same as that used in the
Smith (2003) research.
Results
Reliability
For the university sample, the reliability of the questionnaire is satisfactory, with a
Cronbach alpha of 0.79. Both Coakes and Steed (1997) and Pallant (2001) suggest alpha
values above 0.7 are sufficient for reliability to be assumed. No items showed a
Corrected Item-Total correlation of less than 0.3, and no item showed an Alpha if Deleted
score less than the Cronbach. These are the criteria suggested by Pallant (2001, p.87) as
necessary for retention of an item in a questionnaire.
9
For the TAFE sample, the Cronbach alpha was 0.76 which is acceptable.
However, a number of items showed Corrected Item-Total correlations of less than 0.3,
and Alpha if Deleted scores less than the Cronbach.
Factor Analysis
Because the research was designed to compare the vocational learners with the previously
researched (Smith, 2003) sample of undergraduate students, it was necessary first to
establish if the factor structure for the two groups was the same. An exploratory
principal components factor analysis with varimax rotation was conducted on the data
from the sample of TAFE vocational learners. It is acknowledged here that the sample of
81 is comparatively small for a factor analysis, but it did meet the minimum criterion set
by Coakes and Steed (1997) of five subjects per variable. Factorability of the matrix was
established through a sizeable number (41) of significant inter-item correlations, a
Kaiser-Meyer-Olkin Measure of Sampling Adequacy of 0.68, and a highly significant
result for Bartlett's test of sphericity.
The results of the combined sample factor analyses for both samples of learners are
shown below as Table 1.
10
Table 1 Questionnaire items and factor loadings
Factor
Learner Group
Eigenvalue
% of Variance
Questionnaire item
1. I am able to easily access the
Internet as needed for my
studies.
2. I am comfortable
communicating electronically.
3. I am willing to actively
communicate with my
classmates and instructors
electronically.
4. I am willing to dedicate 8 to 10
hours per week for my studies.
5. I feel that online learning is of
at least equal quality to
traditional classroom learning.
1
2
TAFE
3.80
26.92
University
3.21
24.71
TAFE
2.10
16.13
University
2.28
17.53
.04
.03
.74
.62
-.10
.02
.82
.87
.11
.09
.77
.78
.63
.52
.18
.16
.17
.07
.30
.56
.46
.38
.42
.33
.01
.36
.49
.25
.60
.75
.00
.00
.41
.42
.00
.13
10. In my studies, I am selfdisciplined and find it easy to
set aside reading and homework
time.
.82
.78
.01
-.02
11. I am able to manage my study
time effectively and easily
complete assignments on time.
12. As a student, I enjoy working
independently.
13. In my studies, I set goals and
have a high degree of initiative.
.80
.68
.19
.09
.46
.52
.38
-.03
.56
.75
.10
.02
6.
7.
8.
9.
I feel that my background and
experience will be beneficial to
my studies.
I am comfortable with written
communication.
When it comes to learning and
studying, I am a self-directed
person.
I believe looking back on what
I have learned in a course will
help me to remember it better.
Factor structure
Two questions that had shown distinctive loadings on a factor in the undergraduate
sample failed to show distinctive loadings in the vocational sample. Apart from that, the
11
factor structure for the vocational learner sample was identical to that of the
undergraduate group, and yielded the same two factors of 'self management of learning'
and 'comfort with e-learning'. Accordingly, the researchers have felt justified in
combining the undergraduate and vocational learner samples for the purpose of the factor
analysis, and factor score comparisons.
We adopted a factor loading criterion of 0.40 for inclusion of the item in the
interpretation, more stringent than Tabachnik and Fidell (1996), who suggest 0.32, and
consistent with Comrey and Lee (1992) who suggest the criterion should be set a little
higher than 0.32. Additionally, where an item loaded on more than one factor, we have
followed the advice of Arrindell, Emmelkamp, Brilman and Monsma (1983) and have
included the item in the factor on which it scored highest, provided the difference
between the two-factor loadings was at least 0.2.
The factor analysis for both samples has resulted in two factors after application
of Cattell's (1966) scree test. The first factor we have interpreted as 'Self-management of
learning', and the second factor as 'Comfort with e-learning'. These two factors and their
interpretation are the same as identified by Smith, Mahoney and Murphy (2003).
Similarity in Factor Structures
The factors identified for the two samples of learners were compared for their similarity
using the Cattell et al (1969) test of salient similarity. The two identified factors were
shown to be similar for both learner samples, at a level beyond p<0.05.
Discussion
12
The McVay instrument has yielded reliable factor structures that have now been
identified in two studies with university students (Smith, Murphy & Mahoney, 2003;
Smith 2003) and the current study with TAFE level vocational students. The
interpretation of the two factors is also reliable, with the first factor being associated with
readiness for self-management of learning, and the second factor with comfort with elearning.
Identifying these two factors as replicable and reliable across different learner
groups is encouraging, and even more encouraging that the factor interpretations are
consistent with other research and theory completed on the broader issue of readiness for
flexible learning. As argued by Smith et al (2003), the 'Comfort with e-learning' factor is
recognisable as an e-learning focussed dimension not unlike the Verbal-Nonverbal
dimension identified by Smith (2000, 2001) in his work with Australian university and
TAFE students in a broader context of resource-based flexible learning materials. The
McVay questionnaire describes a readiness for engagement with the particular form of
resource- based learning delivery that is online, and that incudes accessing learning
materials through the Internet, as well as collaborating with a group online. As discussed
earlier in this paper, there is considerable evidence (McKavanagh et al., 2002; Harasim et
al, 1995; Stacey, 1998) that willingness to participate in online discussion groups is
crucial to the effectiveness of online learning in contexts that require students to construct
meaning and understandings.
Similarly, the identification of a 'self-management of learning' factor in the
McVay instrument is clearly evident also in the broader research by Smith (2000, 2001),
by Sadler-Smith and Riding (1999), and Warner, Christie and Choy (1998). Indeed, the
13
need for self-direction, or self-management of learning, runs clearly throughout the
distance education and resource-based flexible learning literature. Several other writers
(eg Boote, 1998; Evans, 2000; Kember, 1995) have commented that self-direction is a
necessary pre-requisite for effective resource-based learning in distance education and
flexible delivery.
The identification through the Readiness questionnaire of these two factors with
congruence in the broader resource-based, distance education and flexible delivery
indicates that the instrument is measuring learner preferences that have been identified in
broader studies that have been designed to investigate preferences or styles. Accordingly,
the instrument may have potential value in the measurement of readiness for the online
components of flexible delivery, and could form a useful adjunct to the assessment of
learner readiness for more broadly based flexible learning.
However, although the instrument has shown promise in its identification of
factors that are consistent with other research and theory in flexible learning, the item
reliability for the TAFE level vocational students was unsatisfactory, and calls the value
of the instrument in its current form into some question, at least for that group of learners.
At the same time, the reliability of the test and its items was satisfactory for the university
learners. The previous research by Smith, Murphy and Mahoney (2003) and Smith (2003)
with different samples of university students also showed acceptable item and test
reliability. At this stage it appears that the lower reliability of the instrument is associated
with its application to the TAFE learners. However, it needs to be acknowledged here
that the instrument has only been researched with TAFE and university students, such
that reliability when applied to other potential learner groups is as yet untested.
14
There are a number of possible reasons for that difference in reliability between
the two samples. First, the sample of TAFE level learners was considerably smaller than
the university sample. It would be a prudent next step in the assessment of the McVay
instrument to repeat the analysis with a larger sample of TAFE level students. Second, it
is also possible that the questions in the instrument are more meaningful to university
students than to TAFE students, such that the variability of question interpretation may be
smaller among the university students. It needs to be recalled here that the instrument
was originally designed for university students such that migration of its application to
other groups could be problematic. If the instrument is to be useful with other than
university students, it appears necessary that several items be reworked to yield better
reliability.
Conclusion
In conclusion, although at this stage the McVay (2000) instrument has potential value in
assessing readiness for online learning, particularly in the context of the assessment of
readiness for more broadly based flexible learning, it is not recommended for use with
other than university level students. Further research on the instrument, similar to the
current study, is necessary to determine its value for non-university groups; and further
work is required to improve the reliability of several of the test items. The instrument has
potential to be valuable if assisting practitioners in identifying those aspects of online
learning delivery that may have a debilitating effect upon learner motivation and
performance, and hence may facilitate intervention to ensure that the effectiveness of
15
online learning is maximised or at least optimised across and within different groups of
learners.
Footnote
In Australia a Technical and Further Education (TAFE) institution provides vocational
education and training courses at a post-school, sub-university level. TAFE institutions
are roughly equivalent to Further Education Colleges in the United Kingdom.
References
Arrindell, W.A., Emmelkamp, P.M.G., Brilman, E., and Monsma, A. (1983)
'Psychometric evaluation of an inventory for assessment of parental rearing
practices', Acta Psychiatrica Scandinavica, 67: 163-177.
Boote, J. (1998) 'Learning to learn in vocational education and training: are students and
teachers ready for it?', Australian and New Zealand Journal of Vocational
Education Research, 6: 59-86.
Cattell, R.B. (1966) 'The scree test for the number of factors', Multivariate Behavioral
Research, 1: 245-276.
Cattell, R.B., Balcar, K.R., Horn, J.L. & Nesselroade, J.R. (1969) ‘Factor matching
procedures: An improvement of the s index; with tables’, Educational and
Psychological Measurement, 29: 781-792.
Coakes, S.J. & Steed, L.G. (1997) SPSS Analysis Without Anguish. Brisbane: John Wiley
& Sons.
16
Comrey, A.L. & Lee, H.B. (1992). A First Course in Factor Analysis. (2nd. Ed.),
Hillsdale NJ: Erlbaum.
Evans, T. (2000) ‘Flexible delivery and flexible learning: developing flexible learners?’,
in V. Jakupec & J. Garrick (eds), Flexible Learning, Human Resource and
Organisational Development. London, Routledge, pp. 211-224.
Kember, D. (1995) Open Learning Courses for Adults: A Model of Student Progress.
Educational Technology Publications, Englewood Cliffs NJ.
McKavanagh, C., Kanes, C., Beven, F., Cunningham, A, & Choy, S. (2002) Evaluation
of web-based flexible learning, NCVER, Adelaide.
McVay, M. (2000). Developing a Web-based distance student orientation to enhance
student success in an online Bachelor's degree completion program. Unpublished
practicum report presented to the Ed.D. Program, Nova Southeastern University,
Florida.
Rainbow, S.W. & Sadler-Smith, E. (forthcoming) 'Attitudes to computer assisted learning
amongst business and management students', British Journal of Educational
Technology.
Sadler-Smith, E., Allinson, C.W. & Hayes, J. (2000) ‘Cognitive style and learning
preferences: some implications for CPD’, Management Learning, 31: 239-256.
Sadler-Smith, E. & Riding, R. (1999) 'Cognitive style and instructional preferences',
Instructional Science, 27: 355-371.
Smith, P. J. (2000) 'Preparedness for flexible delivery among vocational learners',
Distance Education, 21(1): 29-48.
17
Smith, P. J. (2001) ‘Learning preferences of TAFE and university students’, Australian
and New Zealand Journal of Vocational Education Research, 9: 87-109.
Smith, P.J., Murphy, K.L. & Mahoney, S.E. (2003) 'Identifying factors underlying
readiness for online learning: An exploratory study', Distance Education, 24(1):
in press.
Smith, P.J. (2003) 'Learning preferences and readiness for online learning’, Paper to be
presented to the 5th annual conference of the European Learning Styles
Information Network, University of Bradford, June, 2003.
Stacey, E. (1998) 'Study of the enhancement of learning through group interaction by
computer mediated communication.' Unpublished doctoral dissertation, Monash
University.
Tabachnik, B.S. & Fidell, L.S. (1996) Using multivariate statistics. New York: Harper
Collins.
Warner, D., Christie, G., & Choy, S. (1998) The readiness of the VET sector for flexible
delivery including on-line learning. Brisbane: Australian National Training
Authority.
Acknowledgement: The Readiness for Online Courses survey is taken from How to be a
successful distance learning student: Learning on the Internet written by Marguerita
McVay, and used in this study with the kind permission of the author.
18