A method and case of studying teachers' professional development
Edgar Krull
University of Tartu, Estonia
E-mail: ekrull@ut.ee
Paper presented at the European Conference on Educational Research,
Edinburgh, 20-23 September 2000
Introduction
In spite of its importance for teacher education the research on teacher
professional expertise and thinking as a field of study was established less than twenty
years ago. In general, the scientific study of people who were exemplary in some
specific field of occupations was not undertaken in no systematic way by
psychologists, and others until the deGroot’s (1965) study of chess experts was
published in English.
The need for a better knowledge how teachers develop from a beginner to a
professional stems directly from the necessity of increasing the quality of teacher
education. On the one hand, years long debates about the questionable effectiveness
of teacher education have brought teacher educators to a conclusion that the
evaluation of teacher education, on initial as well as on in-service levels, should be
based on the assessment of graduates’ capabilities of practical teaching and
professional development (Wise & Leibbrand, 1996). On the other hand, in order to
evaluate teachers' pedagogical competence objectively at the graduation from initial
teacher education, from induction year, or from in-service training courses the experts
should know what typical expertise could be expected at different levels of their
professional development. For example, they need to know what specific professional
qualities distinguish successful teachers with different teaching experience. Yet, it is
especially important to know what professional expertise should teachers normally
achieve by the end of certain periods of time worked at school when the certification
decisions must be taken.
On former research in the field
The research on development of teachers' professional skills has thus far as
stated by D. Berliner (1994) been of small scale, generally qualitative, and highly
2
interpretative; rather descriptive than experimental. In few experimental settings
reported differences between beginning and expert teachers behaviour in routine
teaching; in designing and delivering of instruction in a new condition; in interpretation of pictures representing classroom events, and instructional tasks; in perception
of classroom events when watching simultaneously three videotaped lessons; in
attention paid to slides representing different pedagogical situations; in evaluation of
specific aspects of videotaped lessons; in writing scenarios for teaching lessons; etc
have been studied. All these methods are valuable as they allow digging profoundly
into subtleties of teacher job but at the same time they are expensive and require a lot
of work. Furthermore, these methods do not allow studying teachers' professional
development in large groups. For these purposes different inquiry methods should be
more appropriate. Nevertheless, these methods have not been very popular in the
research of teacher professional development thus far as they allow studying only
specific aspects of this phenomenon.
Methodology of study
The present study is an attempt to use and to prove the practicability of an
inquiry approach to learn changes in teachers' professional thinking and attitudes
depending on their school experience. As a first step in this work, a questionnaire for
exploring different aspects of teachers' pedagogical reasoning was designed. This
design was based on three major principles. First, we decided to use in its
development as many projective questions as possible. This idea was drawn from our
former research on differences in student and practising teachers' pedagogical
thinking. For example, in this study, instead of asking teachers to explain what is the
difference between notions of student leaning motivation and abilities we asked them
to describe their pupils by motivation and abilities.
Secondly, in order to cover all vital aspects of teacher professional thinking by
questionnaire items, all sides of the teacher work were taken under consideration.
Then these aspects were checked against major categories of a teacher evaluation
instrument developed on the basis of Texas Teacher Appraisal Instrument and
University of Washington Teacher Appraisal System (Krull, 1998a, p. 159–162).
Finally, the main subtopics of the questionnaire were fixed as follows: general
teaching strategies, classroom management, presentation of subject matter, creating of
a positive classroom atmosphere, activities beyond classroom and school, and
3
activities for in-service training and development of subject teaching methodology
were stated.
Thirdly, multiple reply options for questionnaire items were constructed
congruously with Berliner's model of teacher professional development which
discriminates in teacher professional development five stages: novice, advanced
beginner, competent, proficient, and expert teacher (Baron, Berliner, et al., 1996):
Novice (0–2 years of teaching experience). As in other fields of activities the first steps of a
novice teacher are relatively context free and inflexible. S/he is often unable to take into
consideration the big picture of educational events, and needs simple, ready-to-apply rules of
action. This is a stage at which the real practice and coping with every-day problems are
more important for teachers than the theory taught in pre-service teacher education.
Advanced beginner (2–3 years of teaching experience). At this level experience starts to be
melded with formal knowledge and episodic and case knowledge are built up and
complement the theoretical knowledge learned in the programs of teacher education. The
strategic knowledge of when to ignore and when to follow rules is intensively developed in
this stage, and the context begins to guide the decision-making. Although the novice and the
advanced beginner are intensively involved in the instructional process, they often fail to
take full responsibility for their actions. As Baron a/o. pointedly state: “ This occurs because
they are labelling and describing events, following rules, recognising and classifying
contexts, but not yet actively determining through personal agency what is happening”
(Baron a/o., 1996, pp. 1131-1132).
Competent teacher (3–4 years of teaching experience). Two major characteristics
distinguish competent performers from a beginner. First, they make conscious choices about
what they are going to do. They have priorities, schedules of activities, and sensible means
for achieving the aims they have in their minds. Second, through enacting their knowledge
and skills they are able to determine what is important and what is unimportant. They know
while dealing with classroom management when is the right time to stop misbehaviour, and
who is behind it, and they can decide, depending on the circumstances, on what part of a
program it is reasonable to spend more time for teaching and where to move on more
quickly. Teaching at this level calls for deep devotion from the teachers to their jobs. Not all
advanced beginners are likely to reach this level of professional development.
Proficient teacher (5 and more years of teaching experience). About the fifth year, a
minority of the teachers, in addition to their competence, reaches a remarkable rapidity and
flexibility in their professional activities. At this stage intuitive knowledge becomes
prominent in decision making. Thanks to the wealth of experience they accumulated, and to
their reflectivity, teachers perceive the events and situations they encounter in a holistic way.
They intuitively notice analogy in instructional or educational settings, which the beginners
fail to see. For instance, they perceive without conscious effort that the progress of a current
lesson is endangered by the same circumstances that caused the failure of another lesson
with other pupils a month ago. Nevertheless, as stated by Baron, a/o.”… the proficient
performer…while intuitive in pattern recognition and in ways of knowing, is still likely to be
analytic and deliberative in deciding what to do” (1996, p. 1132).
Expert teacher. Expert teachers have an intuitive grasp of situations and the ability to find
correct solutions apparently without any analytical actions. This high performance of expert
teachers is mainly enabled by two factors: by their rich supply of specific knowledge and
skill ready for immediate application, and by their capability to identify operatively
situations were the application of these skills is appropriate. The ability of expert teachers to
deal with a classroom’s basic routines in a non-reflective or automatic way allows them to
take more time for a deeper analysis of untypical problems. The accumulating experience in
the analysis of problem situations provides the expert teacher with the ability to navigate
skilfully from one level of generalisation to another, and to find rapidly appropriate
solutions. Expert teachers act flexibly without any observable effort. They always seem to
know always where to be and what to do at the right time. They often appear to grasp nonanalytically the right solutions in problem situations, and to respond to their environment in
an effortless manner.
4
In other words, the reply options were designed to imitate teachers' possible reactions
to questions depending on their level of professional development. So, the whole
design of the questionnaire can be imagined as a specification table with one
dimension describing different teaching activities, and with other dimension
describing five levels of probable reactions as suggested by Berliner's model. All
multiple choice questionnaire items developed on the basis of described principles
had five reply options describing teachers' probable responses at the specific levels of
professional development and representing sets of ordinal scales.
Examples:
Item 43. On what basis do you decide that it is necessary to inform pupils about the learning
outcomes?
( ) Educational courses always emphasised this need.
( ) Basing on my theoretical knowledge and teaching experience.
( ) My teaching experience convinced me in this need.
( ) I see that considered information on outcomes increases pupils' learning motivation.
( ) This problem is not worth to be thought about as I feel unintentionally when it is
necessary to inform pupils about their learning outcomes.
Item 60. To what degree do you pay attention to your pupils' involvement in learning
activities?
( ) I try to watch on their involvement as much as I can.
( ) I do not always watch intentionally as I perceive when certain students are not involved.
( ) There is no need to watch all of them as I know pupils who are not learning for different
reasons.
( ) I do not watch their involvement intentionally as I always recognise pupils who are not
involved.
( ) Without any reflection I perceive immediately that a pupil is not involved, and I handle
the situation in an appropriate way.
The validity of the questionnaire (Krull, 1999) was increased by its multiple
adjustments on the basis of tryouts by teacher educators and practising teachers.
Purpose of study and sample of respondents
The main objectives of this survey were: (1) to study differences in Estonian
teachers’ pedagogical thinking and decision making depending on their school
experience; (2) to check the validity of Berliner’s model in Estonian conditions; and
(3) to specify limits of an inquiry method for tracing changes in teachers' professional
development.
5
In order to achieve these objectives a sample of potential respondents
representing teachers by groups of subjects taught and school experience was
compiled on the basis of Estonian teachers' database. The questionnaires were
delivered to the target schools and potential respondents and collected by the
Members of Estonian Association of Retired School Principals. The survey was
carried out in the autumn, 1999. From 1400 distributed questionnaires 781 were filled
in and returned.
In the actual sample 82,1 percent of respondents were women and 17,9 percent
men. This ratio is in good harmony with the general distribution of sexes among
Estonian teachers. The distribution of respondents by their school experience as full
time teachers can be seen in table 1.
Table 1. Distribution of respondents by school experience.
Number of Cumul.
Percentage Cumul.perc
respondents number of of
entage
of
respondents respondents respondents
School experience
in years
1. 0 – 3
2. 4 – 6
3. 7 – 8
4. 9 – 11
5. 12 – 15
6. 16 – 19
7. 20 – 25
8. over 25
Missing
53
95
56
89
116
58
89
197
28
53
148
204
293
409
467
556
753
781
6,8
12,2
7,2
11,4
14,9
7,4
11,4
25,2
3,6
6,8
19,0
26,1
37,5
52,4
59,8
71,2
96,4
100,0
The distribution of respondents by groups of school experiences is extremely uneven
as beginning teachers are underrepresented and experience teachers over represented.
For unknown reasons younger teachers were less co-operative with researchers than
their older colleagues. This condition almost balked achievement of the second
objective of this study as major changes in teachers' professional development take
place in first five years of their career. The average school experience of our
respondents was 17.4 years, and the most experienced teacher has worked 51 years.
The respondents' distribution by subject and school level is shown in table 3.
6
Table 2. Distribution of respondents by subject and school level
Teachers of…
Number Cumul. Percenta Cumul.
of
number ge
of percenta
teachers of
teachers ge
of
teachers
teachers
(1) mother tongue and foreign 242
242
31,0
31,0
languages
(2) mathematics
127
369
16,3
47,2
(3) sciences, biology and geography 69
438
8,8
56,1
(4) physics and chemistry
64
502
8,2
64,3
(5) physical and musical education, 94
596
12,0
76,3
and handicraft
(6) history and social studies
56
652
7,2
83,5
(7) elementary classes (teach all 45
697
5,8
89,2
subjects)
(8) different other subjects not 57
754
7,3
96,5
belonging to the first seven groups.
Missing
27
781
3,5
100,0
As it can be seen the teachers of mother tongue and foreign languages are clearly over
represented and teachers since subject group three underrepresented.
Results of study
Currently the statistical processing of data is under way. Therefore it is
possible to talk about preliminary findings only. In this report the main focus will be
on the results reflecting differences and changes in respondents replies depending on
their school experiences.
Our analysis revealed a very disappointing fact that there was no clear
relationship between selected reply options (designed according to stages of Berliner's
theory) and teachers' school experience in our case. All correlation coefficients
expressing relationship between teachers' school experience groups and replies to
questionnaire items related to professional activities were less than 0.20. This means
that the selection of all reply options did not depend on respondents teaching
experience and the lists of reply options of questionnaire items did not represent
ordinal scales as it was intended. Nevertheless, a more careful analysis revealed that
respondents' preferences for certain reply option in many items depended heavily on
7
the respondents' school experience. Quite impressive relationships between
preferences of reply options and teachers' school experience appeared in questions on
pupil characteristics, classroom management and discipline, classroom instruction and
teaching style, on curricular and methodological work, and on relationship with
parents.
Characterization of students
In our former study (Krull, 1998b) we discovered that there are significant
differences in student and practising teachers’ perception of pupils' learning
motivation and abilities. Student teachers tended to confuse these notions but the
majority of teachers discriminated between them. If a strong and statistically
significant correlation r = 0.60 (p< 0.05) was found between student teachers’
perceptions of their pupils’ leaning motivation and capabilities, then the
Figure 1.Correlations between descriptions of pupils' learning
motivation and abilities depending on teachers' school experience
0.6
Correlations
0.5
0.4
0.3
0.2
0.1
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
same indices for teachers with one to five years of school experience was only 0.28
(and statistically insignificant). This means that the majority of teachers do not
consider pupils with a high learning motivation as having unconditionally high
abilities and pupils with low motivation as low in abilities. Our present study
confirmed that understanding of these notions and discrimination of them really
depends on teachers' school experience. But the study also proved that this
relationship is more complicated than was initially expected. So, we discovered that
for beginning teachers (with school experience up to two years) the correlation
8
between descriptions of pupil learning motivation and abilities is quite low, then it
gradually increases for more experienced teachers, and afterwards abruptly drops for
teachers whose teaching experience is over 20 years (see Figure 1).
Classroom management and discipline
Our survey revealed that experienced teachers admit having less classroom
management problems than their younger colleagues do. This can be seen from their
replies to many questions. For example a very strong relationship was discovered
between teachers replies to the question asking how often they have discipline
Question 20. How often do you experience discipline problems in your classes
Answer (1): I have almost no discipline problems (r=0.94)1.
Answer (2): I have discipline problems from time to time (r=-0.87).
Question 98. Are you capable to cope with a student whose behavior is the most problematic or who
is least motivated for learning in your classroom?
Answer (2): Yes, but with some reservations (r =0.63).
Figure 2. Distribution of replies to questions on discipline problems
depending on teaching experience
80
Percentage of replies
70
60
50
40
30
20(1)
20
20(2)
10
98(2)
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
1
Figures in parenthesis present correlation between teacher school experience groups and preferences
of reply options.
9
problems in their classes and indices of their school experience. Correlation between
teachers' school experience and preferences for the first reply option were r = 0,94,
and r = -087 for the second reply option. Similar regularities were discovered in
teachers' replies to many other items pertaining to discipline and classroom
management. The distributions of replies to some questions can be seen in figures 2,
3, and 4.
As it can be seen in Figure 2 the preferences for these three reply options
regarding discipline problems level off for teachers with school experience from 16 to
nineteen years, and afterwards do not change significantly any more.
Question 51. Do you establish rules and procedures for your classes in the beginning of school year?
Answer (2): No, my students realise my requirements without saying (r = 0.87).
Question 53. How do you insure the implementation of established rules and procedures in your
classes?
Answer (5): I presume that my students generally know what expectation I have for their behaviour and
they behave in expected way (r = 0.75).
Figure 3. Distributions of replies to questions on classroom management by teaching
experience
Percentage of replies
60
50
40
51(2)
53(5)
30
20
10
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
In the case of questions 51 and 53 preferences for reply options 2 and 5 seem level off
for teachers with school experience 7–8 years, and then they start to change again
when school experience is over 15 years.
Question 56. Does it happen that you have time left over or are in shortage of time at the end of your
classes?
Answer (2): Usually not (r=0.85)
Answer (3): From time to time it happens (r=-0.87)
Question 61. To what extent depends discipline in classes on well-considered instructional design and
scenario?
10
Answer (1): Avoid most discipline problems (r=0.86).
Figure 4. Distributions of replies to questions on classroom management
by school experience
70
Percentage of replies
65
60
55
50
56(2)
45
56(3)
40
61(1)
35
30
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
The distributions of replies to the question 56 prove (see Figure 4) that teachers with
school experience less than eight years feel that they have more often time left over or
are in shortage of time at the end of their classes than their more experienced
colleagues. Teachers' perceptions and understanding of classroom management start
to change again when their school experience is over 19 years.
The results exposed in figures 2, 3, and 4 prove that depending on their school
experience teachers become gradually more confident in their ability of classroom
management and handling discipline problems. Also, more experienced teachers
understand better than their younger colleagues do the importance of well-considered
instructional design and scenario for successful classroom management. Nevertheless,
the changes in teachers' perceptions and understanding of discipline and classroom
management problems are not improving linearly: quick changes usually take place in
first ten years of teaching, then there is typically a stabilisation period and when the
teaching experience exceeds 19 years their perception and understanding of these
phenomena start to change, sometime even improve again.
Classroom instruction and personal teaching style
Analysis of items studying this aspect of teachers' professional activities
proved that with increasing school experience teachers find typically more time for
dealing with their students, are more able to get all students involved in learning
11
activities (with an exception for teacher with school experience over 26 years), pay
more attention to the generalisation and transfer of subject material, and they reflect
on and analyse their lessons when they have problems rather than after each lesson.
Instead, the less the teachers' experience is the more they claim to analyse each lesson
taught. The distributions of corresponding replies can be seen in more details in
Figures 5 and 6.
Question 25. Do you find time for dealing with all students in your classes?
Answer (1): Usually I don't (r=-0.79)
Question 26. Can you get all students involved in learning activities?
Answer (2) : Sometimes I can, sometimes not (r=-93)
Answer: (4) Yes, I can practically always
Percentage of replies
Figure 5. Distributions of replies to questions on classroom instruction by
teaching experience
50
45
40
35
30
25
20
15
10
5
0
25(1)
26(2)
26(4)
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
Figure 5 shows that with increasing teaching experience less teachers admit that they
cannot find time for dealing with all students, or that they cannot always get all
students in learning activities. The percentage of teachers who believe that they are
able to get students involved in learning increases quite quickly up to 25 years of
teaching experience and then drops abruptly for unknown reasons. Instead, teachers
with school experience over 25 years preferred third reply option "I almost can" in
61.2 percents of cases.
Question 70. Do you make generalisations and emphasise relationships between phenomena and issues
when presenting new material.
Answer (2): No, because my students do not understand this kind of things (r = -0,89).
Question 82. How often do you reflect on and analyse taught lessons?
Answer (1): After each class (r =0.86).
12
Answer (2): When I have problems in my classes (r=-0.74).
Figure 6. Distributions of replies to questions on classroom instruction
by teaching experience
90
Percentage of replies
80
70
60
70(2)
50
82(1)
40
82(2)
30
20
10
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
Relationship with parents
The analysis of replies to questions on this aspect of teachers' job proved that
generally more experienced teachers pay more attention to working with parents (see
Figure 7). They have conferences more often with them, and there are less highly
experienced than younger teachers who do not have regular conferences with parents.
Also, more experienced teachers seem to be more flexible in handling confidential
information on their pupils.
Question 85. When and how often you have individual conferences with parents?
Answer (1): Very often (r=0.89).
Answer (4): Usually I do not meet them (r=-0.84
Question 87. Do you keep confidential information on students' transgressions, problems and living
conditions for yourself if it suits into limits established by legislation?
Answer: (1) Yes
Answer: (2) Generally, yes
13
Figure 7. Distributions of replies to questions on relationship with parents
by teaching experience
Percentage of replies
60
50
40
85(1)
85(4)
30
87(1)
20
87(2)
10
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
Curricular and methodological work
In this aspect of teachers' job the results of our study did not reveal any clear
relationship between teachers school experience and their respect for the instructional
requirements established by the state curriculum. At the same time, more experienced
teachers claim to be better informed about new pedagogical approaches, and they
participate more frequently in curriculum design than their younger colleagues.
Details on these aspects can be seen in Figures 8, 9, and 10.
Question 101. Are you familiar with new approaches in general pedagogy and in your subject
methodology?
Answers (1+2): Yes; Yes, in main lines.
Answer ( 3): Yes, in some way (r=-0.83).
As it can be seen in Figure 8 the distribution of replies for teachers with school
experiences up to ten years is quite stable, then the teachers' self-confidence starts to
grow, and levels off for teachers with school experiences over 20 years.
14
Figure 8. Distributions of replies to questions on curricular and
methodological work by teaching experience
90
80
Percentage of replies
70
60
50
101(1+2)
40
101(3)
30
20
10
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
Question 102. Do you participate in design of general curriculum principles at your school?
Answer (1): No (r=-0.89).
Answer (2): Sometimes (r=-0.72)
Answer (4): Usually I do (r=0.89).
Figure 9. Distributions of replies to questions on curricular work by
teachers' school experience
Percentage of replies
60
50
102(1)
102(2)
102(4)
40
30
20
10
0
0-3
4-6
7-8
9-11
12-15
16-19
Teaching experience in years
Question 103. Do you participate in design of your subject syllabus at the school?
Answer (1): Yes, always (r=0.87).
Answer (4): Usually not (r=-85)
20-25
>25
15
Figure 10. Distributions of replies to questions on curricular and
methodological work by teachers' school experience
70
Percentage of replies
60
50
40
30
103(1)
103(4)
20
10
0
0-3
4-6
7-8
9-11
12-15
16-19
20-25
>25
Teaching experience in years
The data represented in figures 9 and 10 prove that teacher involvement in curriculum
design gradually increases with the growth of their experience.
Conclusions
Preliminary analysis of data collected by our survey proves that teacher
professional thinking and understanding of pedagogical phenomena develop all the
long of their professional career. Yet, the nature and speed of changes in teacher
professional thinking and decision-making depend on the specific aspects of their job.
In the case of discipline and classroom management problems a quite quick
development of reasoning in this field takes place in first ten years of teaching, then
the understanding of these phenomena stabilises for about ten years, and again starts
to change. In the fields of classroom instruction and working with parents a gradual
growth of teachers' pedagogical expertise was documented. Also, along with
increasing of their school experience teaches tend to claim that they are better familiar
16
with educational trends than their younger colleagues, and they are gradually more
and more getting involved in curricular work. The most complicated seemed to be
changes in teachers' understanding of their students' personal characteristics.
Nevertheless, all these conclusions are based on the analysis of teacher replies that
reflect how they feel and maybe want to be seen by other people. That means that
their real behaviour might be quite different in many cases. In spite of this weak side
the study proved that certain teacher professional qualities are improving all along of
their professional career. The documented changes in teachers' attitudes, convictions
and pedagogical thinking by this study are important for understanding many nuances
in the development of teachers as of a professional group.
The study proved that possibilities of our research method are limited. The
analysis of data revealed that a written inquiry as a data collection method does not
guarantee the interpretation of questionnaire items and of their reply options by all
respondents in the same way. This was probably the main reason why the correlation
between teacher experience groups and preferences of reply options was unexpectedly
low and why so many respondents, instead of choosing one reply option per item
chose two or three. Therefore, the results of this study should be supplemented and
specified by data collected with other methods. For example by semi-structured
interviews carried out by trained interviewers, and by classroom observations.
The results of this study did not prove or disapprove the validity of Berliner's
model of teachers' professional development at least for three major reasons. First, the
actual sample of respondents involved too small number of beginning teachers whose
behaviour this model describes. Secondly, the method of data collection used was not
quite appropriate for achieving this goal. And thirdly, the percentage of teachers
achieving forth or fifth levels of professional expertise, as it is unknown, could be too
small for statistical analysis of data. Nevertheless, our study demonstrated that
changes in teachers' professional thinking take place far longer than it would be
predicted by the Berliner's model, and that many of them do not improve in a linear
way with increasing teaching experience. Also, this study gave us a lot of useful
information on Estonian teachers' average attitudes towards different pedagogical
problems, and on their perception of phenomena related to their job. The research
provided us with a good basis for further and more specified studies on teachers'
professional development.
17
Bibliography
Baron, T., Berliner, D.C., Blanchard, J., et al. (1996). A future for teacher
education. Developing a strong sense of professionalism. In John Sikula (Ed.).
Handbook of research on teacher education. Second edition. New York: Simon &
Schuster Macmillan. Pp. 1108 - 1149.
Berliner, D. (1994). Expertise: The wonders of exemplary performance. In J. N.
Mangiery & C. C. Block (Eds.), Creating powerful thinking in teachers and students.
Fort Worth, TX: Holt, Rinehart & Winston. Pp. 161 – 186.
DeGroot, A. (1965). Thought and choice in chess. The Hague: Mouton.
Krull, E. (1998a). Evaluation of teaching (Õpetamise tõhususe hindamine).
Õpetajakoolitus II. Tartu: Tartu Ülikool (in Estonian)
Krull, E. (1998b). Some features of development in teacher professional thinking. In
the book Õpetajakoolitus (Teacher Education) III. Tartu: Tartu University Press. P 29
– 50. (In Estonian, abstract in English.)
Krull, E. (1999). Classroom instruction for teachers' eyes. A questionnaire ( Töö
klassis õpetaja pilguga. Küsimustik.) Tartu: TÜK trükikoda. (In Estonian).
Wise, A. E. and Leibbrand, J. (1996). Profession-based accreditation: A foundation
for high-quality teaching. Phi Delta Kappan, Vol 78. N 3. P 202-205.