Spring 2016 Teacher Survey
January 16, 2016
Thank you for taking this survey. The purpose of this survey is to learn more about your experiences with several educational reforms currently underway in Massachusetts. ESE would like your feedback on your district and the state’s programs to support reform efforts related to the implementation of the Massachusetts Curriculum Frameworks, classroom practices, and educator professional growth and development. The survey is divided into four main parts:
A: Teacher Background
B: Classroom Practices
C: State Curriculum Frameworks
D: Educator Growth and Development
In order to reduce the burden on teacher s’ time, all teachers will answer some core questions from each module plus a selection of questions from each topic. The survey should take approximately
25 to 35 minutes to complete.
If you are unsure about a question, please respond in a manner that most closely reflects your experiences.
DRAFT, for review only. Do not circulate without the permission of the authors.
1

Teacher Survey Spring 2016
1. This school year (2015-16), what grade(s) do you teach?
PLEASE SELECT all that apply
 Kindergarten  5 th  10 th
 1 st
 2 nd
 3 rd
 4 th
 6 th
 7 th
 8 th
 9 th
 11 th
 12 th
 13 th (special education students aged 18-22)
 Ungraded
2. This school year (2015-16), what subject(s) do you teach?
PLEASE SELECT all that apply
1
 English/Language arts/reading/writing
2
 Mathematics
3
 Social Studies
4
 Science, Technology and/or Engineering
5
 World language(s)
6
 Visual or performing arts (art, music, dance, theater)
7
 Health/physical education
8
 Career/technical education
9
 Special education
10
 English Language Development
11
 Specialized Instructional Support Personnel (SISP)
12
 Other ( specify ) _____________________________________________
3. This school year (2015-16), are you the teacher of record for students in any of the following subjects? a. English/ Language Arts/Reading/Writing b. Mathematics
Yes
□
□
No
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
2

Teacher Survey Spring 2016
4. Including the current school year (2015-16), how long have you worked as a teacher?
Please do not include student teaching.
YEARS MONTHS a. Total amount of time teaching | | | | | | b. Total amount of time teaching in current school c. Total amount of time teaching in current district
| | |
| | |
| | |
| | | d. Total amount of time teaching in Massachusetts | | | | | |
5. During the 2015-16 school year, approximately what percent of students that you teach in this school are considered:
Write the percent of students or check one box in each row.
Percent of students None Don’t know a. b. c.
Low-achieving students (i.e., students who are at least one year below state grade-level expectations or who perform below proficiency on the state assessment in one or more subjects)
High-achieving students (i.e., students who are at least one year above state grade-level expectations or who perform above proficiency on the state assessment in one or more subjects)
Students with individualized education programs
(IEPs)
_ _ _%
_ _ _%
_ _ _%
□
□
□
□
□
□ d. English language learners (ELLs) _ _ _%
□ □
DRAFT, for review only. Do not circulate without the permission of the authors.
3

Teacher Survey Spring 2016
6. Which option below best describes your familiarity with the Mathematics Curriculum
Framework for the grade(s) you teach?
[Respondent should only be able to select one]

I am unfamiliar or only slightly familiar with the Mathematics Curriculum Framework.

I have a general understanding of the Mathematics Curriculum Framework but I am not familiar enough with the details to use them in the development of lessons and assessments.

I have a detailed understanding of the Mathematics Curriculum Framework.
7. How prepared do you feel to teach the Mathematics Curriculum Framework to each of the following groups of students ?
a. Low-achieving students b. c.
Students with individualized education programs (IEPs)
English language learners
(ELLs)
I do not feel prepared
□
□
□
I feel slightly prepared
□
□
□
I feel moderately prepared
□
□
□
I feel wellprepared
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
4

Teacher Survey Spring 2016 a. b. c. d. e.
8. During the 2015-16 school year, how often do you engage in the following practices in the class(es) you teach?
Never or hardly ever
Sometimes
(1-3 times per month)
Often
(1-3 times per week)
Daily or almost daily
Build students’ mathematical content knowledge by closely connecting it to their learning from the previous grade
Build students’ mathematical content knowledge by covering concepts in more depth when compared to the previous standards
Build students’ use of mathematical language when presenting their ideas
Connect students’ mathematical content knowledge across mathematical topics
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
Provide opportunities for students to apply mathematics to every-day problems
□ □ □ □ f. g. h. i. j. k.
Provide a balanced curriculum that includes all of the following: conceptual understanding, procedural skills and fluency, and application of mathematical skills
Integrate the Standards for
Mathematical Practice into my curriculum and lesson planning
Use hands-on activities that foster student participation in the Standards of Mathematical Practice
Provide structured opportunities for students to develop arguments and have conversations to support their mathematical reasoning
Assess students in their mathematical practices skills (as opposed to content skills)
Use model curriculum units developed by the state
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
5

Teacher Survey Spring 2016
This section asks you to answer questions about your target class .

Please focus only on your instruction in mathematics to students in 4 th grade.

If that is a split class (i.e., the class contains more than one group for mathematics instruction and each group is taught separately), respond for only one group.
9. (a) How many students are in the target class ?
Number of students: _ _ _
(b) Approximately how many students in the target class would you describe as highachieving, and low-achieving?
High-achieving students (i.e., students who are at least one year above state grade-level expectations or who perform above proficiency on the state assessment in mathematics)
_ _ _
Low-achieving students (i.e., students who are at least one year below state grade-level expectations or who perform below proficiency on the state assessment in mathematics)
_ _ _
(c) How many students in the target class are English language learners, and how many have IEPs?
Number of students who are English language learners: _ _ _
Number of students who have IEPs: _ _ _
10. Did you teach the same subject to students in the same grade as the target class* before your state started implementing the Mathematics Curriculum Framework (i.e., before the
2012-13 school year)?

Yes  [Conditional logic: if selected, respondent will get both parts of the next question]

No, I taught different subjects/grades  [Conditional logic: if selected, respondent will not get part 2 of the next question]

No, I wasn’t a teacher during or before the 2012-13 school year 
[Conditional logic: if selected, respondent will not get part 2 of the next question]
DRAFT, for review only. Do not circulate without the permission of the authors.
6

Teacher Survey Spring 2016
11. In your target class, what level of coverage did you give or do you plan to give each of the following topics, and has your level of coverage decreased, stayed the same, or increased, compared to your level of coverage in similar classes before Massachusetts started implementing the Mathematics Curriculum Framework (i.e., before the 2012-13 school year)? a. Calculate simple probabilities b. Demonstrate understanding of angle measurement c. Demonstrate understanding of data in tables or graphs d. Demonstrate understanding of fraction multiplication e. Demonstrate understanding of geometric or arithmetic patterns f. Demonstrate understanding of rate of change/slope g. Perform measurement conversions h. Perform the procedures of adding and subtracting fractions i. Represent fractions j. Solve one step equations (e.g., 5x =
12)
No coverage
0
□
□
□
□
□
□
□
□
□
Number of lessons
Minor coverage
1
Moderate coverage
2
□ □
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
Major coverage
3
Change in Coverage Relative to
Similar Classes Before Your State
Started Implementing the
Mathematics Curriculum
Framework
Decreased Stayed the same
Increased
1 2 3
□ □ □ □
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
7

Teacher Survey Spring 2016
Instruction of special populations
[Conditional logic: If a teacher indicated that they did not teach ELL students and/or SWD, they will not receive the related questions in this section.]
English Language Learners (ELL) refers to the subset children who have been identified as not fully
English proficient on the (* insert name of ELP assessment used in the state* ) and receive ESL or bilingual education support provided by you or another teacher. When we ask questions about ELLs, answer only for these students. Please do not answer for students who have never been or are no longer identified as ELLs as per the (* insert name of ELP assessment used in the state *).
Students with disabilities ( SWD ) refers to students with IEPs who participate in the regular state assessment (with or without accommodations). Please do not answer for students who participate in the alternate assessment.
12. When you provide ELA to ELLs in your target class , how often do you consult the
Mathematics Curriculum Framework or the English Language Proficiency (ELP) standards?
Standards Never Rarely Sometimes Often a. b.
Mathematics Curriculum
Framework
English Language Proficiency
Standards
□
□
□
□
□
□
□
□
13. Thinking about your target class, when you provide mathematics instruction to English
Language Learners, to what extent do you provide the following instructional supports (i.e., adaptations or scaffolding)? a. Additional time to work
I do not provide this
□
Occasionally provide
□
Often provide
□
Always provide
□ b.
Multiple modalities (writing, reading, listening, and speaking)
□ □ □ □ c. Sentence or paragraph frames
□ □ □ □ d.
Connections to prior knowledge and experiences
□ □ □ □ e. Home language support f. Visuals, realia, or gestures g. Glossaries and other reference materials h. Graphic organizers
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□ i.
Teacher modeling language learning strategies (e.g., understanding the multiple meanings of mathematics vocabulary)
□ □ □ □
DRAFT, for review only. Do not circulate without the permission of the authors.
8

Teacher Survey Spring 2016
14. Thinking about your target class, to what extent do you coordinate your instruction with an
ELL teacher who provides supports for the English Language Learners ? a.
Joint lesson planning with an
ELL teacher b. Co-teaching with an ELL teacher c. Consultation with an ELL teacher d.
Monitoring individual student’s progress with an ELL teacher
Not
Applicable
□
□
□
□
Never
□
□
□
□
Rarely
□
□
□
□
Sometimes
□
□
□
□
15. Thinking about your target class , when you provide instruction to students with disabilities, what percentage of time are you teaching to the grade-level standards (as opposed to teaching to standards for a higher or lower grade level)?
Often
□
□
□
□
Amount of time spent teaching to grade-level standards: _ _ _%
DRAFT, for review only. Do not circulate without the permission of the authors.
9

Teacher Survey Spring 2016
16. Thinking about your target class, when you provide mathematics instruction to students with disabilities to what extent do you provide the following instructional supports (i.e., adaptations or scaffolding)? a. b. c. d. e. f. g.
Teach the same content using simpler numbers
Teach the foundational-level skills that students with disabilities have not yet mastered; then have students with disabilities apply newly learned skills in the context of the grade-level content standards
Model explanations and have the students with disabilities practice those explanations to communicate (verbally and in writing) about mathematics
Simplify text that contextualizes mathematics problems
Read to the student(s) text that contextualizes mathematics problems
Explicitly connect hands-on materials to pictorial representations and to symbolic representations of problems
When multiple strategies or algorithms apply, focus primarily on the one that's easiest for the students with disabilities to use
I do not provide this
□
□
□
□
□
□
□
Occasionally provide
□
□
□
□
□
□
□
Often provide
□
□
□
□
□
□
□
Always provide
□
17. Thinking about your target class*, to what extent do you coordinate your instruction with a special education teacher who provides supports for the students with disabilities ?
□
□
□
□
□
□ a. b. c. d.
Joint lesson planning with a special education teacher
Co-teaching with a special education teacher
Consultation with a special education teacher
Monitoring individual student’s progress with a special education teacher
Not
Applicable
□
□
□
□
Never
□
□
□
□
Rarely
□
□
□
□
Sometimes
□
□
□
□
Often
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
10

Teacher Survey Spring 2016
18. Thinking about your target class*, to what extent is the mathematics instruction you provide to each of the following groups of students similar to the mathematics instruction you provide to non-ELL or non-SWDs students? a. b.
How different is the instruction you provide to English
Language Learners, compared to the instruction you provide to non-ELL and non-SWDs?
How different is the instruction you provide to Students with Disabilities, compared to the instruction you provide to non-ELL and non-SWDs?
Very different
Somewhat different
□ □
□ □
The same
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
11

Teacher Survey Spring 2016
19. Please indicate your level of agreement with the following statements about the
Massachusetts Mathematics Curriculum Framework.
Don’t
SELECT ONE RESPONSE PER ROW know
Disagree
Somewhat disagree
Somewhat agree a. The Mathematics Curriculum Framework sets appropriate expectations for student learning at each grade level.
□ □ □ □ b. The Mathematics Curriculum Framework positively affects the degree to which students are prepared for secondary school.
□ □ □ c. The Mathematics Curriculum Framework makes learning relevant to students' everyday lives. d. I have made instructional shifts over the last year to ensure students meet the expectations of the state’s current Mathematics Curriculum
Framework.
□
□
□
□
□
□ e. Mathematics state assessments provide useful information about how well my students are mastering the state standards.
□ □
Disagree
□ f. Students’ results from Mathematics state assessments are useful for improving my practice. g. The implementation of the Standards for
Mathematical Practice enables students to have a deeper understanding of mathematical concepts.
□
□
□
□
□
□
20. Please indicate your level of agreement with the following statements.
Somewhat disagree a. b. c. d.
The Mathematics Curriculum Framework clearly indicates the content (topics and cognitive demands) I should teach.
The Mathematics Curriculum Framework clearly indicates the content (topics and cognitive demands) I should not teach.
I have received guidance from my district or state that clearly indicates the order in which I should teach each content area in the Mathematics Curriculum Framework.
I have received guidance from my district or state that clearly indicates how much time I should spend on each content area in the Mathematics Curriculum Framework.
□
□
□
□
□
□
□
□
□
□
□
□
□
□
Somewhat agree
□
□
□
□
Agree
□
□
□
□
□
□
□
□
□
□
Agree
□
DRAFT, for review only. Do not circulate without the permission of the authors.
12

Teacher Survey Spring 2016
21. Please indicate your level of agreement with the following statements. a. b. c. d. e. f.
My district has made teaching to the
Mathematics Curriculum Framework a major priority.
My principal has made teaching to the
Mathematics Curriculum Framework a major priority.
Teachers who poorly implement the
Mathematics Curriculum Framework will have a lower summative evaluation rating.
There are negative repercussions for teachers at this school whose students perform poorly on the state standardized assessment.
Teachers at this school are recognized for using exemplary classroom practices that support the Mathematics Curriculum
Framework.
Teachers at this school are recognized for their student s’ achievement gains on state standardized assessments.
Disagree
□
Somewhat disagree
Somewhat agree
□ □
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
Agree
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
13

Teacher Survey Spring 2016
22. Please answer each of the three questions on the following supports you receive for the
MATHEMATICS Curriculum Framework. a) Which of the following supports for implementing the Mathematics Curriculum Framework have you used? b) How helpful were these supports? c) Do you need more support?
SELECT
ONE
RESPONSE
PER ROW
A24a.
Implementation
Have you used this support?
SELECT ONE RESPONSE PER ROW
A24b.
Implementation
How helpful was the support?
SELECT
ONE
RESPONSE
PER ROW
A24c.
Do you need more of this type of support?
Yes No
Not helpful
Somewhat helpful
Helpful
Very
Helpful Yes No a. Textbooks aligned to the state standards
□ □ □ □ □ □ □ □ b. Curriculum resources aligned to the state standards
□ □ □ □ □ □ □ □ c. Formative or diagnostic assessments aligned to the state standards
□ □ □ □ □ □ □ □ d. Digital tools (on-line textbooks, webinars, videos, on-line communities, applications, etc.) e. Information about how the state standards change what is expected of students
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□ f. Information about how the state standards change what is expected of my instructional practice g. Professional development on the state standards h. Training on how to integrate the
Standards of Mathematical Practice into my lessons i. Other (specify):
___________________
□ □
□ □
□ □
□ □
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□ □
□ □
DRAFT, for review only. Do not circulate without the permission of the authors.
14

Teacher Survey Spring 2016
23. Please indicate your opinion on the degree to which the following are aligned to the
Mathematics Curriculum Framework.
Not at all aligned
Somewhat aligned
Aligned
Strongly aligned
Not applicable a. b. c. d. e. f. g. h. i. j.
State-mandated assessments used by your district
District-mandated summative assessments
School-created summative assessments
Teacher-created summative assessments
Formative or diagnostic assessments used district-wide
Mathematics textbooks used in your school
Mathematics curriculum selected or developed by your district
State-developed or organized professional development activities that I’ve participated in this year
District-developed or organized professional development activities that I’ve participated in this year
Administrator feedback provided to you from classroom observations (i.e. walk-throughs, formal observations, etc.)
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
24. Including this current school year, how long do you believe each of the following will remain in effect? a.
The current Mathematics Curriculum
Framework
1-2 Years
□
3 Years
□
4 Years
□
5+ Years
□
□
□
□
□
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
15

Teacher Survey Spring 2016
25. During the 2015-16 school year (including summer 2015), which of the following types of professional development activities have been available to you, and, if available, what impact have they had on strengthening your instructional practice?
SELECT ONE RESPONSE
PER ROW
SELECT ONE RESPONSE
PER ROW
IMPACT ON STRENGTHENING YOUR PRACTICE
NOT
AVAILABLE
AVAILABILITY
AVAILABLE
BUT DID NOT
PARTICIPATE PARTICIPATED
NO
IMPACT
LIMITED
IMPACT
MODERATE
IMPACT
STRONG
IMPACT a. Workshops, seminars, courses, institutes or conferences organized by my school or district b. Online professional development resources
(such as webinars, videos, online articles, online courses, etc.) offered by or through my district c. Conferences organized by local, regional, or national organizations d. Graduate courses in education and/or my subject area e. Teacher-led professional development opportunities (such as workshops, webinars, seminars, etc.) f. Peer coaching at my school g. Peer observations at my school h. Professional Learning
Communities or other peer-led collaborative opportunities) i. Other (specify)
___________________
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
16

Teacher Survey Spring 2016
26. To what extent did the information from your previous year’s evaluation influence which professional development activities you are participating in during the 2015-16 school year
(including summer 2015).
SELECT ONLY ONE
1
2
□
Not at all
□
Small extent
3
4
0
d
□
Moderate extent
□
Large extent
□
I was not evaluated in 2014-2015
□
Don’t know
DRAFT, for review only. Do not circulate without the permission of the authors.
17

Teacher Survey Spring 2016
27. Please indicate your level of agreement with the following statements about all the professional development activities that you are participating in during the 2015-16 school year (including summer 2015). My professional development activities are …
SELECT ONE RESPONSE PER ROW
NOT
APPLICABLE DISAGREE
SOMEWHAT
DISAGREE
SOMEWHAT
AGREE AGREE NO OPINION a. improving my content knowledge of the academic subjects I teach.
□ □ □ □ □ □ b. advancing my understanding of effective instructional strategies. c. having a positive impact on my classroom instruction. d. helping me improve my teaching of the state curriculum frameworks. e. helping me use the analyses of student performance data to identify the needs of struggling students. f. helping me address the socioemotional needs of my students. g. well-aligned to the MA Standards and Indicators of Effective
Teaching. h. helping me teach content aligned to the curriculum frameworks to students with disabilities. i. helping me teach content aligned to the curriculum frameworks to
English language learners. j. Other (please specify)
____________________________
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
18

Teacher Survey Spring 2016
28. Please indicate the degree to which you agree with the following statements about collaboration in your school.
(Collaboration includes all structured or planned activities when you meet with colleagues to discuss curriculum, instruction, assessments, students and/or any other aspect of your job.)
SELECT ONE RESPONSE PER ROW
DISAGREE
SOMEWHAT
DISAGREE
SOMEWHAT
AGREE AGREE a. My administration provides adequate guidance (e.g., expectations, protocols, etc.) to support peer collaboration. b. My administration provides adequate training to support peer collaboration. c. My administration provides materials, tools, or equipment that helps teachers work together more effectively. d. Collaborating with other teachers in my school is a productive use of my time. e. Teachers participate in professional learning communities in my school. f. Peer collaboration is embedded into the school schedule on a regular basis.
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
29. Which of the following sentences best describes the impact of your collaboration with other teachers? As a result of my col laboration with other teachers…
SELECT ONLY ONE
1
□
I did not change my instructional practice.
2
3
4
□
I made some changes to improve my instructional practice.
□
I made major changes to improve my instructional practice.
□
I did not collaborate with other teachers.
DRAFT, for review only. Do not circulate without the permission of the authors.
19

Teacher Survey Spring 2016
30. We are interested in learning about all feedback you receive about your instruction, including informal feedback and feedback received as part of the evaluation process. During the 2015-
16 school year (including summer 2015), who provided you with feedback related to your instruction?
2
3
4
5
6
7
8
1
SELECT ALL THAT APPLY
□
Colleague/Peer
□
Mentor
□
Coach
□
Administrator
□
Students
□
Parents/Families
□
External Experts
□
Other; please specify_____________________________
31. How would you characterize the frequency and impact of the feedback provided on strengthening your instructional practice?
FREQUENCY OF FEEDBACK IMPACT OF FEEDBACK
From: a. Colleague/Peer b. Mentor c. Coach d. Administrator e. Students f. Parents/Families g. External Experts h. Other
____________
LITTLE
FEEDBACK
PROVIDED
□
□
□
□
□
□
□
□
OCCASIONAL
FEEDBACK
PROVIDED
□
□
□
□
□
□
□
□
REGULAR
FEEDBACK
PROVIDED
□
□
□
□
□
□
□
□
NO
IMPACT
□
□
□
□
□
□
□
□
LIMITED
IMPACT
□
□
□
□
□
□
□
□
MODERATE
IMPACT
□
□
□
□
□
□
□
□
STRONG
IMPACT
□
□
□
□
□
□
□
□
DRAFT, for review only. Do not circulate without the permission of the authors.
20

Teacher Survey Spring 2016
32. To what extent do you receive ongoing support from coaches, peer-based professional learning communities, or administrators following professional development (PD) activities in your school? Does this support help you to effectively implement what you have learned from the PD in your classroom?
NOT AT
ALL
34a. ONGOING SUPPORT
SMALL
EXTENT
MODERATE
EXTENT
LARGE
EXTENT
34b. EFFECTIVELY IMPLEMENT
NOT
HELPFUL HELPFUL
VERY
HELPFUL a. Coaching support and feedback focused on implementation of PD content b. Peer-based professional learning communities focused on implementation of
PD content
□
□
□
□
□
□
□
□
□
□
□
□ c. Administrator support and feedback focused on implementation of
PD content
□ □ □ □ □ □
33. During the 2015-16 school year, to what extent did the feedback (oral or written) you received based on a classroom observation focus on the following topic areas? a. Pedagogical content knowledge b. Classroom organization c. Emotional support for students
Not at All
□
□
□
Minimal
Extent
□
□
□
Moderate
Extent
□
□
□
Great
Extent
□
□
□ d.
Instructional strategies to help special education students (including those with mild learning disabilities but excluding those with severe or profound disabilities)
□ □ □ □
□
□
□ e.
Instructional strategies to help English language learners
□ □ □ □ f.
How to align my teaching to the Mathematics
Curriculum Framework g. Behavior management
□
□
□
□
□
□
□
□
□
□
□
□ h. Other (please specify): ______________
DRAFT, for review only. Do not circulate without the permission of the authors.
21