MEMORANDUM
TO:
STANDARDS REVIEW COMMITTEE
FROM:
INSTITUTE FOR LAW TEACHING AND LEARNING
SUBJECT:
DRAFT OUTCOMES STANDARDS
DATE:
FEBRUARY 6, 2016
This memorandum constitutes the response of the Institute for Law Teaching and Learning
to the ABA’s draft outcomes standards. The Institute wishes to thank the ABA and the
Standards Review Committee in particular for the opportunity to present these comments.
These comments focus primarily on the questions posed by the Subcomittee in its draft.
Background Regarding the Institute
Established in 1991, the Institute for Law Teaching and Learning’s mission is to help law
schools meet the obligations they owe “to their students and to society to provide a learning
environment that helps students achieve the highest academic standards and prepares
students to assume their responsibilities as effective, moral attorneys.” Among other goals,
the institute seeks “to serve as a clearinghouse for ideas to improve the quality of education
in law school” and “to support student-centered curriculum reform.”
Since its founding in 1991, the Institute has:




Published two books on teaching, learning and assessment topics, OUTCOMES
ASSESSMENT FOR LAW SCHOOLS (2000) by Professor Greg Munro and GETTING
GRAPHIC 2 (1994) by Professor Corrine Cooper;
Published 30 issues of THE LAW TEACHER, a twice yearly periodical that provides a
forum for ideas for improving teaching and learning in law schools and informs law
teachers of the activities of the Institute;
Published three videotapes regarding teaching and learning topics; and
Hosted 14 conferences on a variety of teaching, learning and curriculum topics
attended by hundreds of law professors.
The two directors of the Institute, Professor Gerald F. Hess of Gonzaga and Professor
Michael Hunter Schwartz of Washburn, and the Institute Consultant, Professor Sophie
Sparrow of Franklin Pierce, have published and spoken extensively on a wide variety of law
teaching and learning topics. Together, they have published 6 books and 22 law review
articles, essays and other shorter works on law teaching and learning topics and have
delivered more than 175 presentations on law teaching and learning topics at conferences
and as invited speakers at national and foreign law schools. More on the Institute can be
found at http://lawteaching.org.
Overarching Comments
The Institute believes that the draft outcomes standards, absent any revision, represent a
monumental step forward for legal education and the Institute endorses them. If adopted,
the standards will serve as a powerful tool for helping law schools serve their students and
the public by engaging law schools in a process of defining themselves by adopting
meaningful mission statements, elaborating the student learning implications of their
missions, evaluating their effectiveness in producing practice-ready lawyers, and using those
evaluations to inform curricular and teaching improvement. This process will ensure that
law schools act with integrity and work to continuously improve their efforts to deliver on
their obligations to their students and the general public.
The Institute therefore not only endorses the adoption of the draft outcomes standards, but
also stands ready to support the standards within the larger legal education community, and
to work with the ABA, if requested, to develop programs designed to assist law schools in
adapting to the new standards.
Because the proposed outcomes standards will be a significant change for most law schools,
the Institute recommends that the new standards be implemented gradually. Two or three
years after adoption, law schools can be expected to have defined skills, knowledge and
values outcomes. During this same time frame, the ABA also can expect law schools to do a
preliminary, inputs-based assessment of their efforts to produce graduates who possess the
desired outcomes by creating a curriculum map in which the law school identifies, for each
outcome, where in the law school’s curriculum students first encounter instruction aimed at
that outcome, where in the curriculum the students practice performing the skills or practice
demonstrating the knowledge, and by where in the curriculum the students must
demonstrate mastery of the skills and knowledge or adoption of the values.
Each year over the five years that follow this initial two or three-year period, law schools can
be expected to assess, systematically, reflectively, and by multiple means, whether their
students graduate having attained two or three of the law school’s outcomes. For all law
schools, the goal should be to assess all outcomes over the course of the ABA’s seven-year
review process. Law schools also can be expected to evaluate their own assessment efforts
and to use data from those assessments to inform curricular reforms, to make changes to
teaching practices, and to evaluate their outcomes.
At least during this seven- or eight-year initial time frame, the Institute does not recommend
that law schools be required to demonstrate that a specified percentage of their graduates
(80% or 100%) have attained all of their identified outcomes. Assessment methods, in the
context of law, are insufficiently developed at this time to predict whether a reliable and valid
set of measures will exist so many years down the road. Sometime in the future, after the
ABA and law schools have had experience with the new standards, a further discussion of
2
whether to create a list of required outcomes and outcomes measures may become
appropriate.
Outcomes the ABA Should Specify
The Standards should leave it to law schools to determine which learning outcomes
they seek to measure.
As part of the process of determining learning outcomes, a law school should identify the
types of practice its students enter upon graduation. Then the school is positioned to
identify the knowledge, skills, and values critical to success in those areas of practice.
Lawyering Skills and the Degree of Mastery
The Standards should leave it to law schools to determine which skills to assess and
what degree of mastery is appropriate given the schools’ mission and goals.
Given the complexity of lawyering knowledge, skills, and values, the attributes of students,
and law school resources, law schools should identify, define, and publish the skills
important for their students to learn, the degree of mastery expected upon graduation, and
the assessment methods used to assess their graduates. Law schools should engage in a
systemic, integrated, and ongoing plan to identify learning outcomes, gather data, and use
that data to improve law students’ learning. In identifying the skills and level of mastery, law
schools can rely upon the knowledge, skills and values identified as important to practice in
published studies such as the MacCrate Report, Schultz & Zedeck, and Best Practices.
Guidance from the ABA on Measurement of Outcomes - Specified Curricular
Requirements
In addition to the requirements in 302 (a), proposed Standard 302(a)(3) should be expanded.
The current version of 302(a)(3) requires law schools to include as learning outcomes:
(3) knowledge and understanding of a lawyer's ethical responsibilities as representatives of
clients, officers of the courts, and public citizens responsible for the quality and availability
of justice.
Knowledge of a lawyer’s ethical responsibilities is an important part of professionalism and
law schools should articulate learning outcomes regarding professional ethics. Schools
should identify learning outcomes related to other critical aspects of professionalism as well.
For example, the MacCrate Report (pp. 140-141) articulates four fundamental values of the
profession and the Best Practices Report (pp. 82-83) lists professional values and attributes
central to success in the practice of law. Empirical studies of lawyers have identified a set of
professional attributes and values that contribute significantly to success in the practice of
law. These include honesty, reliability, judgment, respect, diligence, commitment to life-long
3
learning, and self-motivation. In fact, many lawyers rate these values and attributes as more
important to success in practice than skills and knowledge. (Stephen Gerst and Gerald Hess,
Professional Skills and Values in Legal Education, 43 Valparaiso L. Rev. (2009). Consequently,
law schools should identify and measure learning outcomes related to professional values
and attributes.
Proposed Standard 302 (c) should require law schools to offer live-clinics, externships or
simulations so that students can develop the kinds of skills, values and self-reflection
required in the practice of law.
Guidance on Validation and Selection of Outcomes Measures
Proposed Standard 303(a) or, even better, an entirely separate standard, should include an
expectation that all law school courses provide formative feedback to students throughout
courses and should evaluate student performance with multiple, valid and reliable assessment
measures.
The interpretations should provide guidance on a variety of methods and kinds of evidence
law schools can use to demonstrate that they are engaged in valid assessment.
Law schools should rely upon a variety of methods and evidence to demonstrate that their
students are meeting their schools’ learning outcomes. Law schools should base their
assessment methods on their curriculum, identified learning outcomes and resources, with
the goal of using the evidence gathered to improve student learning. It is helpful for the
interpretations to include specific examples of evidence and assessment methods.
.
Respective Roles of Student Assessment and Institutional Assessment
There may come a time when law schools will be able to require attainment of their
outcomes as a requirement for graduation. At this juncture, however, given the lack of valid,
reliable and scalable methods of assessment and the absence of research demonstrating that
law school curricula are effective and efficient in producing graduates with the desired skills,
knowledge and values, assessing institutional effectiveness is more important and more
justifiable than assessing students for purposes of gate-keeping their graduation and
admission to the bar. Until legal education has developed such methods of assessment, law
schools cannot justify imposing gate-keeping assessments on their students.
This fact does not justify allowing law schools to eschew their duty to assess their
institutional effectiveness. The process of creating, interpreting and trying to use
assessments will be invaluable to law schools. For example, the process would encourage
law schools to develop formative assessments, to improve and increase their existing
summative assessments, to design their curricula based on student outcomes rather than on
professorial preferences, and to reconsider their teaching methods.
4