2012-13 Public Sector Structural Reform in K

Public Sector Structural Change in K-12 Education
2012-13 Syllabus, FALL Semester
Session #1: September 5, 2012
Introduction, Course Overview, and Expectations; ESEA Flexibility (Exercise)
1. Lyndsey Layton, Obama to Issue No Child Left Behind Waivers to States, Washington Post,
Sept. 22, 2011
2. Kimberly Heffling & Ben Feller, No Child Left Behind: 10 States Receive Waivers from
Education Law’s Sweeping Requirements, AP, Feb. 9, 2012
3. Motoko Rich, States with Education Waivers Offer Varied Goals, N.Y. Times, July 26, 2012
4. Jeremy Ayers & Isabel Owen, No Child Left Behind Waivers: Promising Ideas from Second
Round Applications, Center for American Progress, July 27, 2012, at 4-7, Apps. A and B,
website graphic
5. State of New Jersey Department of Education, ESEA Waiver Request (filed Nov. 14, 2011;
approved by U.S. Dep’t of Ed. Feb. 9, 2012), at 15-19, 37-41, 53-58; App. 2
Discussion Questions
1. How do the NCLB and ESEA Flexibility requirements differ? Why do you suppose the U.S.
Education Department offered ESEA Flexibility option? Is ESEA Flexibility, including as it will
be implemented in New Jersey, a weakening or a strengthening of prior requirements? Do
the differences between NCBL and ESEA Flexibility reflect divergent understandings of how
government can work to influence behavior and solve public problems? How comfortable
are you with the U.S. Education Department’s decision to offer this option instead of
waiting for Congress to act?
2. New Jersey’s approved application for ESEA Flexibility says that its “central goal . . . is to
ensure that all children, regardless of life circumstances, graduate from high school ready
for college and career.” The application acknowledges that “[c]urrently, New Jersey is far
from accomplishing this mission.” How likely is it that the steps New Jersey proposes in its
application will succeed in this mission? As far as you can tell, what is the New Jersey
Department of Education’s understanding of how government can work to influence
behavior and solve public problems?
3. Seven field-based Regional Achievement Centers (RACs) are an important feature of New
Jersey’s plan for turning around the state’s lowest performing schools, called “Priority
Schools.” Suppose that you are a member of a cross-functional team of leaders and
managers from the New Jersey Department of Education and from some of the RACs
themselves, and that the team is charged, among other things, with developing a mission
statement for the RACs’ interactions with Priority Schools. The mission statement is
supposed to be congruent with the Department’s more general mission quoted above. The
task is important, and New Jersey Commissioner of Education Christopher Cerf has asked to
review the statement himself. Initial discussions have indicated that the task is not
straightforward. For example, various suggestions have been made to frame the statement
around the following or cognate words, many of which appear in the approved request for
ESEA Flexibility, but may reveal conflicting understandings of the tenor of the RACs’ work
with Priority Schools: supervise, facilitate, collaborate, serve, direct, recommend,
compromise, not compromise, negotiate, empower, achieve, teach (educate, instruct),
learn, adults (principals, teachers), children. What process would you propose the team use
to develop the mission statement, given the many directions it could take? Although the
task at hand is to come up with a process for developing the statement, not to develop the
statement itself, you might give a moment’s thought to which of the listed words, or which
unlisted alternatives, you would use to describe the RACs’ mission as to Priority Schools.
Exercise in Class: New Jersey Regional Achievement Centers
 You are a member of a cross-functional team with leaders and managers from the New
Jersey Department of Education and from some of the state’s seven field-based
Regional Achievement Centers (RACs).
 Your team is charged, among other things, with developing a mission statement for the
RACs’ interactions with failing schools, designated “Priority Schools.”
 The mission statement is to be consistent with the Department’s general mission “to
ensure that all children, regardless of life circumstances, graduate from high school
ready for college and career.”
 There seem to be divergent views about the RACs’ mission, as indicated by the following
word cloud derived from the state’s ESEA Flexibility application and initial discussions
about the RACs’ role vis-à-vis Priority Schools: supervise, facilitate, collaborate, serve,
direct, recommend, compromise, not compromise, negotiate, empower, achieve, teach
(educate, instruct), learn, adults (principals, teachers), children.
 Your team has allocated 15 minutes to coming up with an agreed upon process for
developing the mission statement? Please do so now.
Writing Assignment (after class):
In one page or less, please reflect on the New Jersey Regional Achievement Center exercise:
 What plan did your team come up with (bullets/summary)?
 What went well for your group?
 What could have gone better?
 What was your role in the team’s deliberations?
 How did your role contribute to the team’s accomplishment of its goals or any
challenges it encountered?
Due as a MS Word attachment emailed to CPRL@law.columbia.edu by Monday
September 10 at 9:00 a.m.
Sessions #2: September 12, 2012
When Bureaucracy (Technology-Based Regulation) Works and Fails, and What that Suggests
about What Ails U.S. K-12 Public Education
1. Council on Foreign Relations, U.S. Education Reform and National Security (Joel I. Klein
& Condoleeza Rice, Chairs 2012), at 14-35
2. Rudi R. Volti, An Introduction to the Sociology of Work and Occupations (2007), at 83
(1st full paragraph: “Modern bureaucracy . . .”)-93
3. Cary Coglianese & David Lazer, Management-Based Regulation: Prescribing Private
Management to Achieve Public Goals, 37 L. & Soc. Rev. 691 (2003), at 691-706 (read
through the preamble to Part III only; the rest of Part III is optional)
4. Examples from the New York City and other school systems, 1965-2012:
a. Diane Ravitch, The Great School Wars: A History of the New York City Public Schools
(2d ed. 2000), at 329-37
[Note: Professor Ravitch describes competing 1960s proposals for decentralizing the
NYC schools. Shortly after the period shed discusses, the state legislature adopted a
decentralization plan similar to the one supported by the UFT (see p. 335). A quarter
century later, the following op-ed was published giving a contentiously stated but
widely shared view of the result of the new arrangement.]
b. Johnny Ray Youngblood, Draining the School Swamp, N.Y. Daily News (1992)
[Note: Doubts whether large public school systems have solved these problems in
the ensuing decades are addressed or suggested by the next three readings.]
c. Eva Moskowitz, Charter-School Envy, N.Y. Post, July 23, 2012
d. Motoko Rich, Enrollment Off in Big Districts, Forcing Layoffs, N.Y. Times, July 24,
e. Collective Bargaining Agreement between the New York City Dep’t of Education and
the United Federation of Teachers (UFT) (2007)
5. James S. Liebman & Charles F. Sabel, A Public Laboratory Dewey Barely Imagined, 28
N.Y.U. Rev. L. & Soc. Change 183 (2003), at 184-89, 195-204 (excerpt #1)
Discussion Questions
1. What is bureaucracy, understood as a form of organizing concerted activity? How does
bureaucracy manifest itself in public agencies devoted to delivering services? Agencies
devoted to regulating private behavior?
2. What are some examples of bureaucracy working well? What are examples of
bureaucracy working poorly? What accounts for the differences?
3. How would you characterize the current state of public education in the US? To what
extent is bureaucracy responsible for the problems facing US public education? What
other causes are important?
4. Focusing on the Ravitch excerpt, how was the decentralization of the New York City
school system proposed in the 1960s expected to improve conditions for school
children? Why else was decentralization proposed? How did the decentralization
proposals differ, and what accounts for the differences?
5. Based on the Youngblood and later pieces, did decentralization in New York City schools
alleviate bureaucracy and its harmful effects? Did decentralization benefit poor and
minority children? Who did it benefit?
6. What relationship is there, if any, between bureaucracy and
a. the New York City-UFT Collective Bargaining Agreement?
b. difficulties big city systems are having today in addressing declining enrollments?
c. failings of court-ordered school desegregation, finance equity and “adequacy”?
Session #3: September 19, 2012
A Menu of Alternatives to Bureaucracy, Including “Experimentalist” or “Public-Learning”
1. Review from session 1: Cary Coglianese & David Lazer, Management-Based
Regulation: Prescribing Private Management to Achieve Public Goals, supra
2. Charles F. Sabel & William H. Simon, Minimalism and Experimentalism in the
Administrative State, 100 Geo. L.J. 53 (2011) [Note: in this and other articles,
footnotes are for your information only; you are not required to read them.]
3. Diane Ravitch, The Death and Life of the Great American School System (2010), Ch.
9 through the flag for Footnote 11
4. Jody Freeman, Extending Public Law Norms through Privatization, 116 Harv. L. Rev.
1285 (2003) (excerpts)
5. New York City’s Chancellor’s District:
a. Randi Weingarten & Michael Mulgrew, Mayor Bloomberg: Stop Closing
Schools, There’s a Better Way, http://www.nydailynews.com/opinion/mayorbloomberg-stop-closing-schools-a-better-article-1.1118928#ixzz21TBru12L
b. MassInsight, Meeting the Turnaround Challenge: New York City’s Chancellor’s
District (2010), http://www.massinsight.org/publications/stgresources/99/file/4/pubs/2010/04/20/III_District_Profile_NYC_CD.pdf
6. Optional: For a simplified version of the Sabel & Simon piece, see James S.
Liebman, Public-Sector Reform through Learning Organizations and the Supporting
Role of Universities, at 1-11 (through the 1st ¶ of Part V) – or read the whole thing if
you’re interested in the overall logic of the Center for Public Research and
Leadership of which this course is a part
Discussion Questions
1. What variations on or alternatives to bureaucratic organization can you identify
being advocated, discussed or criticized in the readings?
2. As to each variation or alternative, how if at all does it seek to mitigate common
problems with bureaucracy?
3. In what ways are the variations and alternatives similar to each other, and in what
ways do they differ?
4. With the caveats that many of the readings and excerpts only allude to solutions
and do not present full-fledged defenses of them, and that we will spend more
time addressing each of them later on in the semester, what are your preliminary
thoughts about -a. how effectively each proposal avoids the difficulties with bureaucracy?
b. other difficulties the proposals are likely to encounter?
c. which seems most appealing to you and why?
Session #4: September 26, 2012
Some Classic Learning Organizations in Action
1. Steven J. Spear, The High Velocity Edge: How Market Leaders Leverage Operational
Excellence to Beat the Competition (2010), pp. 1-32, 36-55, 83-91 (skip 56-82 for now)
(excerpts from Chapters 4-6 and 8: Alcoa, Nuclear Navy, Pratt-Whitney, Ave. A, Toyota)
Discussion Questions
1. Based on Spear’s description of the organizations at the center of each of his examples,
what would you say is the organization’s “secret sauce?”
2. In what ways are the organizations and their strategies that Spear describes similar? In
what ways do they differ?
3. What attributes of the organizations and their strategies if any justify calling them
“experimentalist” or “learning organizations” in the senses introduced in Session #3?
Are any of them more or less appropriately described as examples of that approach to
organizing concerted action? Are any of them more or less useful as models for
organizing concerted action in other contexts, including in public-sector contexts? Why?
4. As to each example Spear gives (including each of the different strategies described for
some of the entities), how would you define the actual “organization” as issue that
might be thought to be experimenting or learning? How is that organization embedded
within others described; how do they all relate to and interact with each other?
Assuming Spear’s examples they are relevant at all, do they reinforce your sense of the
value of “learning” or “experimentalist” organizations as alternatives to bureaucracies,
especially public-sector bureaucracies? Do they allay any of your concerns about other
difficulties those kinds of organizations may encounter? Or, do they underscore or add
new concerns about those organizations?
Session #5: October 3, 2012
More Public-Learning Organizations in Action
1. Baltimore and Maryland/O’Malley: Tina Rosenberg, Armed with Data, Fighting More
than Crime, N.Y. Times Opinionator, May 2, 2012
2. New York City School System under Bloomberg/Klein:
a. Stacey Childress, et al., Managing for Results at the New York City Department of
Education, in Jennifer O’Day, et al., Education Reform in New York City: Ambitious
Change in the Nation’s Most Complex School System (2011), at pp. 1-15 of draft
b. Alliance for Excellent Education, New York City’s Strategy for Improving High
Schools: An Overview (Jan. 2010), at pp. 1-11 (first half of p. 11 only)
c. Joel Klein, Yes, New York City Schools Are Improving, NY Daily News Opinion, Sept.
10, 2012
d. Optional:
i. For more detailed reviews of the student results in New York City between 2003
and 2010, see
 James Kemple, Children First and Student Outcomes: 2003-2010, in O’Day, et
al., supra
 James S. Liebman & Jonah Rockoff, Moving Mountains in New York City: Joel
Klein’s Legacy by the Numbers, Ed. Week (Nov. 30, 2010)
ii. For discussion of how the NYC system plays itself out for two very different
schools in the same building, see Daniel Lautzenheiser, A Tale of Two Schools:
What New York Dep’t of Education is Getting Right, Huffington Post (July 24,
3. Case Study (one of the following three, as indicated)
a. Students with last name beginning with A-G: Environmental Regulatory Innovations
i. Charles Sabel, Archon Fung & Bradley Karkkainen, Beyond Backyard
Experimentalism, Boston Rev. Oct. 1999 (excerpts)
ii. Theordore Lowi, Frontyard Propoganda, A Response to Fung et al., Boston Rev.
Oct. 1999
iii. Matt Wilson & Eric Weltman, Government’s Job, A Response to Fung et al.,
Boston Rev. Oct. 1999
b. Students with last names beginning with H-R: Texas School Reform
i. Liebman & Sabel, A Laboratory Dewey Barely Imagined, supra, at 231-50 (except
ii. Richard F. Elmore, Details, Details, Details, 28 N.Y.U. Rev. L. & Soc. Change
315 (2003) (excerpts)
iii. Martha Minow, School Reform Outside Laboratory Conditions: A Response to
Liebman & Sabel, 28 N.Y.U. Rev. L. & Soc. Change 333 (2003) (excerpts)
iv. James S. Liebman & Charles F. Sabel, The Fragile Promise of Provisionality: A
Reply to Responses, 28 N.Y.U. Rev. L. & Soc. Change 369 (2003) (excerpts)
c. Students with last names beginning with S-Z: Drug Courts
i. Michael Dorf & Charles Sabel, Drug Treatment Courts and Emergent
Experimentalist Government 53 Vand. L. Rev. 831 (2000), at 1-5, 10 (1st full ¶) –
12 (run-over ¶ only), 16-33 (Part I), 52-56
ii. Martha Minow, School Reform Outside Laboratory Conditions: A Response to
Liebman & Sabel, 28 N.Y.U. Rev. L. & Soc. Change 333 (2003) (excerpts) [NOTE: It
is important for each group of students to consider criticisms of so-called
“experimentalist” reforms. Although Dean Minow’s piece responds to an article
about education reform, not drug courts, consider how her three criticisms (lack
of capacity, resources and consensus) also apply to the drug court context.]
iii. James S. Liebman & Charles F. Sabel, The Fragile Promise of Provisionality: A
Reply to Responses, 28 N.Y.U. Rev. L. & Soc. Change 369 (2003) (excerpts)
iv. Optional:
 Gov’t Accounting Office, ADULT DRUG COURTS: Studies Show Courts Reduce
Recidivism, but DOJ Could Enhance Future Performance Measure Revision
Efforts (2011), http://www.gao.gov/assets/590/586793.pdf
 Shelli B. Rossman et al., National Institute of Justice Study, The Multi-Site
Adult Drug Court Evaluation (Dec. 2011),
Discussion Questions: All members of the class are assigned readings about Baltimore Mayor,
now Maryland Governor, Martin O’Malley’s approach to organizing government’s response to
social problems and about school reform in New York City ca. 2003-2010. Each of you also is
assigned to read about one of three other sets of public-sector innovations in the 1990s and
early 2000s: new forms of environmental regulation, Texas school reform, drug courts. For each
of the domains about which you read, consider the following questions:
1. What public problem was the arrangement described designed to solve?
2. In what sense is each of the arrangements “experimentalist” or focused on “public
3. In what ways is the arrangement similar to and how does it differ from other ways of
addressing public problems such as bureaucracy, performance-based regulation or
managerialism, privatization, or craft-based activity?
4. How would you define the “organization(s)” that are said to have emerged from each of
the arrangements? Who are the key actors? To what extent are those actors public or
private? In what sense if any do they together constitute a real “organization,” as
opposed to a fleeting confederation of people who happen to be focusing on the same
problem at the same time and place?
5. What criticisms would you, or do the critics whom you read, apply to the arrangements
as a whole? How serious are the criticisms? Are there convincing responses?
Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to
CPRL@law.columbia.edu by Tuesday October 4, 2012 at 2:00 p.m., addressing any one of
Discussion Questions 2-5 above. Feel free to focus on only a single: trait of the arrangement,
alternative method of addressing problems, similarity or difference between the arrangement
and one or more of the alternative methods of addressing problems, organizational or disorganizational trait, criticism, etc.
Session #6: October 10, 2012
A Conversation with Garth Harries, Assistant Superintendent, New Haven Public Schools, and
Shael Polakow-Suransky, Chief Academic Officer and Senior Deputy Chancellor of the New York
City Department of Education
1. Garth Harries Bio
2. New Haven School Change, PowerPoint Presentation
3. Emily Bazelon, Can Public Schools Really Change? Why New Haven’s Ambitious New
Education Strategy Might Actually Succeed, Slate, Sept. 27, 2012,
4. Shael Polakow-Suransky Bio
5. NY City Dep’t of Education, Citywide Instructional Expectations for 2011-12
6. Optional:
a. NY City Dep’t of Education, Citywide Instructional Expectations for 2012-13
b. The Broad Center, Who’s the Boss Now? Shifting the Balance of Power in New
York City Schools (2011)
Discussion Questions: Please come to class with three or four questions you would like to pose
to the speakers. Some of the questions should address issues you have encountered in, or
concerns triggered by, our readings and discussions about the most effective ways to organize
classrooms, schools, and school systems to accelerate student learning and improve student
results and life chances.
Session #7: October 17, 2012
Philosophical Pragmatism and Learning Organizations
1. Christopher K. Ansell, Pragmatist Democracy: Evolutionary Learning as Public Philosophy
(2011), Chapters 1, 5 (pp. 3-19, 84-101)
Discussion Questions:
1. What is the relationship between bureaucracy and partisan politics? How do they
interact to undermine the capacity of public agencies to solve public problems? How
does Ansell move from the conclusion that this interaction is responsible for the failure
of “governance” to the solution of expanding the role of agencies and the breadth of
public participation in them? How convincing is that solution? Does the structure of
Ansell’s argument as laid out in Chapter 1’s summary of the rest of the book (pp.15-19)
provide a convincing case for putting administrative agencies at the heart of solutions to
paralysis in governance and obdurate partisanship?
2. How do you interpret and how useful is Ansell’s idea of “evolutionary learning,” which
occurs when a “problem-driven perspective,” “reflexivity” and “deliberation” work
together “recursively”?
3. Ansell distinguishes a form of evidence-based evaluation and decision making based on
experimental and quasi-experimental research (which he associates with the work of
Donald Campbell) from “experimentalism” or “evolutionary learning” (p.12). What is the
difference between the two strategies? Do you agree with Ansell’s preference for the
latter over the former – or, more accurately, his preference for treating the latter as
encompassing but going beyond the former? What difficulties do so-called “wicked,”
“compound,” or “high-interaction” problems present for experimental and quasiexperimental research? Chapter 5, footnote 30 quotes a description of Dewey’s
approach to inquiry as follows: “[T]he creative formation of conjectures in the absence
of data is not the key to successful inquiry. Rather the interweaving of observation,
conjecture, verification, and additional observation are the key ingredients of successful
problem solving.” Can inquiry of this sort provide a substitute for classic experimental
and quasi-experimental study methods in situations where too little is known, too much
is changing too quickly, or the problems are too many and too ‘small’ to permit or justify
classic study methods?
4. How enlightening did you find Ansell’s dualisms: “progressive conservatism,”
“cosmopolitan localism,” “analytical holism,” and “processual structuralism”? How
descriptive are they of experimentalism as you understand it, and how convincing are
they as attributes of a “third way” to solving public problems that sits in between
bureaucracy and interest-group politics on the one hand and market-based solutions
and pure electoral politics on the other hand?
5. How well does Ansell’s description of “Pragmatist problem-solving” capture the
processes you have seen at work in examples we have considered: Alcoa, Pratt &
Whitney, the Nuclear Navy, Toyota, drug courts, 1990s Texas education reform, Habitat
Conservation Plans, the Chesapeake Bay Plan, TRI/TURA regulation of toxic releases, the
New York City and New Haven school reforms, etc.? What is missing from Ansell’s
discussion to make it more descriptive of these examples? What is missing from the
steps taken in these examples that keeps them from accomplishing the kind of problem
solving that Ansell promotes? In what ways, if any, does Ansell understate the difficulty
of his conception of public problem-solving?
Exercise in Preparation for Class
Imagine a public agency with which you have some familiarity. Before class jot down
some notes for use in class identifying (1) features of the operation or outcomes of that
agency that strike you as harmful to the agency’s effectiveness or evidence of its
ineffectiveness; (2) which of the McKinsey & Co. “7S’s” (see framework below) those
features involve; (3) steps, consistent with the Pragmatist Learning organizations Ansell
discusses, that you can imagine taking to improve, or even transform, the agency; and
(4) difficulties you imagine those steps and that transformation would encounter
Sessions #8 and #9: October 24 and 31, 2012
Addressing Core Difficulties of Government by Bureaucracy: Too Little Central Expertise, Too
Much Street-Level Discretion; the Craft and Performance-Management Alternatives
1. Michael Lipsky, Street-Level Bureaucracy: Dilemmas of the Individual in Public Services
(1979), at 18-19, 48-53
2. James Q. Wilson, Varieties of Police Behavior (1968), at 1-11, 279-97 (end of paragraph
running over from p.296)
a. Optional: To get a richer sense of the diversity of problems patrol officers and their
supervisors face when bureaucratic rules are used to govern street-level activity in
the order-maintenance context, read or skim pp.16-82.
3. Diane Ravitch: Why Finland’s Schools Are Great (by Doing What We Don’t),
4. Liebman & Sabel, A Laboratory Dewey Barely Imagined, supra, at 213-31 (excerpt #3)
5. Eric Hanushek, Teacher Deselection, in Creating a New Teaching Profession 165-78 (D.
Goldhaber & J. Hannaway eds. 2010)
6. Emma Brown, 98 D.C. Teachers Fired for Poor Performance, Washington Post, Aug. 1,
7. Simon Head, The Grim Threat to British Universities, N.Y. Rev. Books, Jan. 13, 2011
8. Rick Hess & Linda Darling-Hammond, How to Rescue Education Reform, N.Y. Times, Dec.
5, 2011
9. Arthur E. Wise & Michael D. Usdan, The Political Future of the Teaching Profession, Ed.
Week, March 13, 2013
Discussion Questions:
1. In private- and public-sector organizations, why is shop-floor and street-level discretion
2. How do bureaucracies try to solve the problem of shop-floor or street-level discretion?
What problems does the bureaucratic solution pose for local- level employees, their
supervisors, the organization as a whole? How (if at all) are those problems manifested
in police departments, K-12 school systems, welfare agencies, environmental regulatory
agencies, criminal courts, universities?
3. What solutions to the problem of shop-floor and street-level discretion are proposed or
implied by James Q. Wilson, Diane Ravitch, the first and second iterations of the 1990s
reforms in New York City’s Community School District (CSD) 2 (see Liebman & Sabel),
Hanushek, Great Britain’s system for distributing public money to universities based on
faculty productivity, Head’s critique of the British system, and Hess & DarlingHammond? How is each solution similar to and different from the others – a question
with particular salience when applied to the two phases of the New York City CSD 2
reforms? In what ways does each of the proposed solutions avoid problems otherwise
encountered in regulating the actions of street-level government officials and in what
ways does each fail to solve those problems? What additional problems does each of
the proposed solutions create?
4. Simon Head attributes the travails of British university academics to “command and
control bureaucracy,” the “industrial” model, and “business school” ideas such as “Total
Quality Management” and “benchmarking” – as if these are the same thing? Are they?
How would you characterize the driving force behind the evaluation of the productivity
of British academics? What do you think of the British system? Of Head’s critique of the
5. Rick Hess is a conservative advocate of privatization and performance-management
approaches to public education. Linda Darling-Hammond is a progressive advocate of
teacher autonomy and professionalization. They argue that their ability to agree on the
wrongheadedness of current federal education policy is proof that it must certainly be
wrongheaded. Are there other reasons why people with their bents on school reform
issues might find common cause? What do you think of their critique of federal policy
and proposed alternative?
Session #10: November 7, 2012
Using Street-Level Public Learning and Problem Solving to Increase Organizational Expertise and
Harness Street-Level Discretion
1. Spear, High Velocity Edge, supra, at 56-71 and review pp. 88-91 (excerpts from Chapters
7 and 8 re: Toyota)
2. Joan E. Talbert, Collaborative Inquiry to Expand Student Success in New York City
Schools, in in O’Day, et al., supra, at 1-6 (to 2 paragraphs), 12-17 & table 2
3. Elizabeth Chu et al., Getting Big to Go Small: Case Studies of Collaborative Inquiry Teams
in New York City, Nov. 2012, 2-10, 19-34 (DRAFT: do not circulate)
4. Peg Tyre, The Writing Revolution, Atlantic Monthly (Oct. 2012),
5. Collaborative Inquiry in Teacher Teams Data (NYC – PPT)
6. Jennifer Morrison, Why Teachers Must be Data Experts, 66 Ed. Leadership No. 4 (20082009)
7. Roxanna Elden, Data-Driven and Off Course, Ed. Next (Winter 2011)
8. Multidisciplinary Rounds How-to Guide. Cambridge, MA: Institute for Healthcare
Improvement; 2010, www.IHI.org
9. Optional:
a. Jim Frederickson, Are We Learning the Right Lessons from New Dorp, Atlantic blog,
Sept. 28, 2012, www.theatlantic.com/national/archive/2012/09/are-we-learningthe-right-lessons-from-new-dorp-high-school/263010/]
b. Carnegie Foundation Network Initiative
i. Carnegie Foundation for the Improvement of Teaching, Improvement Research,
http://www.carnegiefoundation.org/improvement-research/approach (last
visited Sept. 2, 2012)
ii. Jal D. Mehta, Louis M. Gomez, & Anthony S. Bryk, Schooling as a Knowledge
Profession, Educ. Week, March 30, 2011
Discussion Questions:
1. A key feature of bureaucratic organization is the concentration of knowledge and
expertise at the center of the organization, using the central imposition of and local
adherence to rules to diffuse that expertise to sites where it is applied to solve
problems. What is the problem with this approach to the amassing and use of
expertise? How does the New Dorp High School example bear on the question of the
best likely sources of expertise, and do you agree with Tyre (or Frederickson, if you read
it) on the answer to that question?
2. As described in the readings, how do Toyota problem-solving teams, inquiry and data
teams, and multidisciplinary rounds in hospitals (and the Carnegie network initiative, if
you read about it) generate and use knowledge and expertise? What similarities and
differences do you see in these various approaches to the problem of knowledge? How
do they differ from bureaucratic approaches to expertise and knowledge? (Relatedly,
how do “multidisciplinary rounds” differ from traditional hospital rounds?”) How likely
are these newer approaches to avoid the problems of knowledge and expertise
encountered in bureaucracies?
3. What is the difference between “getting small in order to go big” (as described in
Talbert’s discussion of New York City Inquiry Teams prior to 2010) and “Getting Big to
Go Small” (as described in Chu et al.’s discussion of those teams since 2010)? How
important is that difference? What accounts for it?
4. What is the relationship between these new approaches and “professionalism”? Do
they make street-level actors more or less professional – a question, you may
remember, that was addressed in last week’s Liebman & Sabel reading about the
Community School District 2 experience in New York City in the 1990s? Or put
differently, do the new approaches change the meaning of professionalism in any way?
Or put yet another way, to what extent do these new approaches to the generation of
knowledge and expertise help solve, or exacerbate, the problem of enabling while
controlling the discretion of “street-level bureaucrats?”
5. What would you expect to be the biggest challenges to the success of these new
approaches to generating expertise and knowledge, and how might organizations
respond to those challenges? How do the new approaches affect the usual routines and
allocation of time at the sites where they occur? How do they keep team-based
activities from degenerating into diffuse “bull sessions?” How do they assure that
learning that occurs at one site is usefully diffused to other sites – and how
straightforward do you imagine the diffusion process to be?
Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to
CPRL@law.columbia.edu by Tuesday November 6, 2012 at 2:00 p.m., addressing one of
Discussion Questions 2-5 with regard to at least one (or in the case of question 3, with regard to
both) of the knowledge-generating mechanisms referenced in the relevant question.
Inquiry Team Exercise: To be described in class
Session 11: November 14, 2012
The Skilled Generalist: The Role and Skillset of Professionals in Learning Organizations
1. Lane Wallace, Multicultural Critical Theory at B-School?, N.Y. Times, Jan. 10, 2010, at
Bu.1, http://www.nytimes.com/2010/01/10/business/10mba.html?pagewanted=all
2. Karen Sloan, Reality’s Knocking: The Ivory Tower Gives Way to the Real World’s
Demands, Nat’l L.J., Sept. 7, 2009,
3. Howard B. Miller, Structural or Cyclical?, Calif. Bar J. Feb. 2010, at 8,
4. Lincoln Caplan, An Existential Crisis for Law Schools, N.Y. Times, July 15, 2012,
5. Arne Duncan, Teacher Preparation: Reforming the Uncertain Profession—Remarks of
Secretary Duncan at Teachers College, Columbia University, Oct. 22, 2009 (text, video)
6. Laura Pappano, Skills to Fix Failing Schools, N.Y. Times, Dec. 29, 2009, Educ. Life 1,
7. Skilled Generalist Handout
8. James S. Liebman, Public-Sector Reform through Learning Organizations and the
Supporting Role of Universities, at 11-17 (or read the whole thing if you haven’t already
and you are interested in the overall logic of the Center for Public Research and
Leadership of which this course is a part)
9. Optional:
a. Karen Sloan, Think of it as a Residency for Lawyers, Nat’l L.J., June 5, 2012
b. Jamie Alter & Jane G. Cogshall, Teaching as a Clinical-Practice Profession:
Implications for Teacher Preparation and State Policy (2009),
c. Nat’l Council for Accreditation of Teacher Education, Transforming Teacher
Education Through Clinical Practice (2010),
d. NYU Revamps Third Year [Law] Curriculum, The Nat’l Jurist, Nov. 2, 2012,
Discussion Questions:
1. Each of you is in a school defined by a profession. What major changes if any do you
think are in store for that profession over the next several years or decades? What allied
changes if any do you think are in store for professional schools of the sort you attend?
2. How does your professional school seem to understand the definition of a profession or
a professional? How would you define a professional? If your and your school’s
definitions differ, what accounts for your differing view?
3. Based on the answers to Questions 1 and 2 and also on the readings and discussion thus
far in the semester, your project work, and your other professional experiences, what
skills would you add to the preliminary list on the Skilled Generalist Handout, which you
think reforming public-sector organizations require of their leaders and managers?
Would you omit or revise any of the skills listed on that handout?
(Informal) Writing Assignment: Come to class with a list of skills you would add to the Skilled
Generalist handout (as many or few as you think are appropriate). Your list may be handwritten or
typed, but please have it in a form you can hand in or email in after class.
Session #12: November 21, 2012
Part 1 (55 minutes): Student Project Presentation (Louisiana “Course Choice” Project): What is
the Project? To What Extent Does it or its Institutional Context Fit Public Learning or Another
Model? What Are the Team’s Initial Thoughts on What the Project or its Context Suggest About
the Possibility of Public Learning and Problem-Solving, and What Public Learning and ProblemSolving Suggest about the Project or its Context?
Part 2 (55 minutes): The Finland (Public Elementary and Secondary Education) Rorschach
Reading for Part 1:
2. Martha Minow, School Reform Outside Laboratory Conditions, 28 N.Y.U. Rev. L. Soc.
Change 333-38 (2003) (many of you have already read this; please review)
3. Liebman & Sabel, A Laboratory Dewey Barely Imagined, supra, at 384 (starting TAN 102)
– 386 (ending TAN 108) (many of you have already read this; please review)
4. Additional reading to be identified by the presenting team at least a week ahead of time
Reading for Part 2:
1. Review: Diane Ravitch: Why Finland’s Schools Are Great (by Doing What We Don’t),
2. Anu Partanen, What Americans Keep Ignoring About Finland’s School Success, The
Atlantic, Dec. 29, 2011
3. Charles Sabel, AnnaLee Saxenian, Reijo Miettinen, Peer Hull Kristensen & Jarkko
Hautamäki, Individualized Service Provision as the Key to the New Welfare State:
Lessons from Special Education in Finland (Sitra Studies 62 Dec. 2011), at pp. 4-15, skim
26-30, 30 (“The next national core curriculum”)-53, 57-64
Discussion Questions for Part 1:
1. Does Louisiana Course Choice fit into an inquiry learning or experimentalist model? If so,
how? If not, why not?
2. As you read about the Course Choice program and how it is expected to change public
education in Louisiana, what are some of the main management challenges you
identify? Particularly, think about how this new program will affect the public school
system and its operation going forward. What is likely to change about the delivery and
management of public education in the state of Louisiana? Does this program offer
solutions for improving the Louisiana school system? Why or why not?
Discussion Questions for Part 2:
1. What is Finland’s secret sauce vis-à-vis elementary and secondary education?
2. How does your answer to Question 1 bear on the problem of street-level discretion? On
what it does or should mean for a teacher to be considered a professional? On the
problem of building organizational knowledge and expertise?
3. How transferrable is the Finland success mechanism to the U.S.? What are the main
challenges to such a transfer? What is the relevance to that question of differences
between Finland and the U.S. in terms of social homogeneity, how much the population
values public education, the commitment to “equality,” the extent to which
responsibility for public education resides at the national or a more local level of
government, the disposition to develop policy in this area top-down vs. bottom-up?
4. What lessons, if any, can Finland learn from current U.S. education reforms?
5. When two nations achieve different outcomes on international benchmark measures,
what is the best method of explaining the different outcomes?
Sessions #13 and 14: November 28 and December 5, 2012
Student Project Presentations: What is the Project? To What Extent Does it or its Institutional
Context Fit Public Learning or Another Model? What Are the Team’s Initial Thoughts on What the
Project or its Context Suggest About the Possibility of Public Learning and Problem-Solving, and
What Public Learning and Problem-Solving Suggest about the Project or its Context?
Part 2 (55 minutes): The Finland (Public Elementary and Secondary Education) Rorschach
Order of Presentations:
3. November 28, 2012, 1st 55 minutes: Teacher Evaluation Pilot/Networks:
4. November 28, 2012, 2nd 55 minutes: Teacher Evaluation Communication
5. December 5, 2012, 1st 55 minutes: Connecticut Dep’t of Education: Quality Review
6. December 5, 2012, 2nd 55 minutes: Connecticut Reform Accelerator Accountability
1. Readings for Teacher Evaluation Pilot/Networks (Nov. 28, 2012)
a. NYC Department of Education, Building Network Capacity around Teacher
b. School Book, A Last-Minute Deal on Teacher Evaluations
c. New York Times Schools for Tomorrow Conference
d. School Book, Test Driving a Pilot Teacher Evaluation System
e. Organization Chart for Network ABC
2. Reading for Teacher Evaluation Communication (Nov. 28, 2012)
a. To be provided by student team
3. Readings for Connecticut Dep’t of Education: Quality Review (Dec. 5, 2012)
a. To be provided by student team
3. Readings for Connecticut Reform Accelerator Accountability (Dec. 5, 2012)
a. To be provided by student team
Discussion Questions
1. Teacher Evaluation Pilot/Networks (Nov. 28, 2012)
a. Is it possible to implement a large-scale system to evaluate and improve individual
teachers? What lessons can be learned from the Toyota model about successfully
implementing a large-scale system for evaluation and improvement of front-line
b. How does the Teacher Effectiveness project map to various alternative models of
governance, such as learning organizations, professionalism, etc.? How does the
Danielson Framework serve as a tool for implementing that model? Does teacher
evaluation help or hinder front-line professionalism and discretion in classroom?
c. What does it mean to “build capacity?” Is the New York City building capacity in the
right places for the right people?
d. What are the strengths and shortcomings of New York City’s approach to teacher
2. Teacher Evaluation Communication (Nov. 28, 2012)
a. How do the communication strategies and recommendations in the article compare
with your own experience of being evaluated in an educational or related context?
b. Are the key lessons learned in Colorado regarding effective communication (detailed
on pages 2-3) relevant to broader discussions of learning organizations? How do
they compare with the emphasis of other learning organizational models we have
talked about?
c. How would you improve the usefulness of the Colorado report to make it more
useful for other organizations seeking to learn from Colorado’s experience?
3. Connecticut Dep’t of Education: Quality Review (Dec. 5, 2012)
a. To be provided by student team
4. Connecticut Reform Accelerator Accountability (Dec. 5, 2012)
a. To be provided by student team
SPRING Semester
Session #15: January 28, 2013
Learning “Regimes” and the Elastic Boundaries of Private-Sector Learning “Organizations”: Disintegrated Firms, Inter-firm Collaboration, and Private Associations
1. Private-Sector Examples: Ronald J. Gilson et. al, Contracting for Innovation: Vertical
Disintegration and Interfirm Collaboration, 109 Colum. L. Rev. 431, 458-71 (2009) (Part
III: pages 12-18 of the article print)
2. Joseph Rees, Hostages of Each Other: The Transformation of Nuclear Power Safety After
Three Mile Island 1-7, 91-150 (1998)
3. Optional: National Commission on the BP Deepwater Horizon Oil Spill and Offshore
Drilling, Deepwater: The Gulf Oil Disaster and the Future of Offshore Drilling 235-41
(Final Report, May 22, 2010),
Discussion Questions
1. In what ways can organizations blur or reach beyond their boundaries with other
entities – and in what ways can public organizations blur or reach outside the
public/private boundary – in search of better pathways to success? In this context,
reconsider Ansell’s claim that institutions can be collectively controlled and revised but
that, in the process, “institutional design must be closely attentive to ways that
institutions are complexly ‘scaffolded’ by webs of related institutions” (14)
2. How would you define a “regime” of the sort(s) described in the reading? What is the
value of thinking about organizational and governance arrangements as “regimes” of
this sort?
3. In the examples in Gilson, et al.:
a. Why did the companies involved vertically disintegrate? What did they gain and lose
by taking that step?
b. What’s in it (or what’s lost) for the satellite organizations that operate within the
larger organization’s orbit?
c. To what extent does “cooperation” as opposed to “cooptation” or “subjugation”
describe the interaction between the larger and satellite organizations?
4. In your experience and based on prior readings, how do simulated markets attempt to
engage regulated entities in regulating themselves? How do “learning” regimes try to do
the same? Which works better when and why?
5. In the INPO example offered by Rees:
a. How much or little does the INPO arrangement for regulating the safety of nuclear
power plants add to or detract from your confidence in the ability to generate
power safely through nuclear fission? What aspects of the arrangement influence
your conclusion the most?
b. What role in the arrangement is played by the Nuclear Regulatory Agency?
c. What role is played by secrecy in the disclosure of results?
d. What role is played by organizational or industry “culture” and what accounts for
changes in that culture?
e. What else is the secret for success – assuming you see any evidence of success?
6. How promising is the INPO model for regulating other kinds of environmental harm,
e.g., from deep-water oil drilling? Would technology-based regulation or market
simulation work better?
Session #16: February 4, 2013
Learning “Regimes” and the Elastic Boundaries of Public-Sector Learning “Organizations”: From
Rotten Cantaloupes to Equality Directives to the Role of Charter Schools in Systematic Education
1. Public Regulatory Regimes:
a. Charles F. Sabel & William H. Simon, Contextualizing Regimes: Institutionalization as
a Response to the Limits of Interpretation and Policy Engineering, 110 Mich. L. Rev.
1265, 1274-97 (2012)
b. Olatunde Johnson, Beyond the Private Attorney General: Equality Directives in
American Law (2012) (excerpts) (full article, with footnotes, is available at
c. Richard Thompson Ford, Moving Beyond Civil Rights, N.Y. Times, Oct. 27, 2011
2. Charter Schools – the Evidence Thus Far in New York City
a. Margaret Raymond, Multiple Choice: Charter School Study in 16 States (CREDO at
Stanford University 2009), at 1-4 (Executive Summary excerpts),
b. Margaret Raymond, Charter School Performance in New York City (CREDO at
Stanford University 2010), at 2 (Summary),
c. Caroline M. Hoxby et al., How New York City’s Charter Schools Affect Achievement
(Nat’l Bureau of Economic Research 2010), at v (first two ¶s), vii-viii (Executive
Summary excerpts),
d. New York City Charter School Center, The State of the New York City Charter Sector
2012, http://c4258751.r51.cf2.rackcdn.com/state-of-the-sector-2012.pdf (pp.7-11,
1. Charter Schools as Components of Regimes
a. Paul Hill & Christine Campbell, Growing Number of Districts Seek Bold Change with
Portfolio Strategy (Center on Reinventing Public Education June 2011), at 1-3,
b. Neerav Kingsland, An Open Letter to Urban Superintendents in the United States of
America, on Ed Week, Rick Hess Straight Up, Jan. 23-27, 2012
c. The School District of Philadelphia, A Blueprint for Transforming Philadelphia’s Public
Schools (May 2012), at 1-20
[Note: Apart from a plan to close approximately one-third of the district’s schools as
a money-saving strategy, the reforms subsequently proposed by the new
Superintendent of Philadelphia Schools in January 2013 are less radical than those
proposed in the May 2012 Blueprint. If you are interested in learning more, see The
School District of Philadelphia, Action Plan v.1.0 (Jan. 7. 2013),
http://www.philasd.org/announcements/actionplan/APv1.0.pdf ]
d. Eric Nadelstern, Schools Closed = Kids Saved, NY Daily News Opinion, Jan. 9, 2013
2. Criticism of Charter Schools
a. Naomi Klein, The Shock Doctrine (introduction- excerpts) (2010)
b. Diane Ravitch, Privatizing Public Education in Philadelphia?, EdWeek Blogs (May 15,
c. Adam B. Schaeffer, Tough Truths About Charters, N.Y. Daily News, Sept. 6, 2012,
Discussion Questions
1. Do the Hart & Sacks and other examples of legal, governance, or organizational
“regimes” described in the reading clarify, or confuse, that concept, or give you a
different perspective on the value of thinking about organizational and governance
arrangements as “regimes” of this sort?
2. As a consumer of leafy greens, meat, and other foodstuffs, do the changes in regulation
in those contexts described by Sabel & Simon give you more or less confidence that
what you eat is safe? What aspects of the old and new forms of regulation account for
your answer?
3. What difficulties has enforcement of civil rights norms encountered over the last several
decades? In what ways, if any, do the experimentalist approaches and/or the “equality
directives” described in the readings change those dynamics? How, if at all, do
Professors Sabel and Simon’s and Professor Johnson’s accounts of these new
approaches overlap or differ?
4. Given the evidence in the reading about how charter schools work, what do they have
to offer as solutions, or contributions to solving, the problems of public K-12 education?
a. Do charter schools harm or improve student life chances? What accounts for the
different results in the two Raymond studies and the Hoxby study?
b. Even if charter schools can improve the results of students who attend them, are
they bad for “school system(s)” they affect? In what ways? What do you make of
suggestions in the reading that charter schools are a right wing conspiracy or a left
wing conspiracy?
c. Can charter schools be scaled to reach 100% of urban school children as Kingsland
suggests? How would you expect a “100% solution” to affect the results the
Raymond and Hoxby studies reach?
d. As far as you can tell from the reading, is the role of charter schools the same or
different in Cleveland, New Orleans, New York, the Philadelphia plan as originally
conceived (May 2012), the “portfolio districts” that Hill & Campbell discuss?
To what extent do any of those roles qualify as “privatization”? To what extent
do any of them qualify as components of a K-12 learning “regime” – and, if any
do qualify, how do they contribute to such regimes?
Apart from the answer to the question just above, in what ways could charter
schools contribute to a K-12 learning regime?
In what ways, if any, would you expect “privatizing” and/or “learning” uses of
charter schools to maximize their positive impact on student outcomes?
Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to
CPRL@law.columbia.edu by Monday February 4, 2013 at 8:00 a.m., on the most effective role,
if any, for charter schools within a school-reform “regime.” In doing so, try to use the themes
and concepts of the course to extract yourself from, or to find a better way to resolve or
defuse, the standard debates about charter schools. For example: Are charter schools the
reform itself or only a part of it, and if the latter, what part? Are they typically designed to be
used (or, from a slightly different perspective, are they best used) as a tool for performance
management, institutional learning, freeing up professionals to work their magic, or
marketizing school system? From an institutional learning perspective, is a charter schools best
thought of as the experimentalist institution itself, as a piece of a wider experimentalist
structure or regime, or both—and how so? Or are charter schools anathema to institutional
Session #17: February 11, 2013
Tools Supporting Structured Public Learning: The “Balanced Scorecard” Approach to Strategy and
1. Robert Kaplan & David Norton, The Balanced Scorecard: Translating Strategy into Action
1-19, 147-66 (1996)
2. Jody Zall Kusek & Ray C. Rist, Ten Steps to a Results-Based Monitoring and Evaluation
System (World Bank 2004), http://www.oecd.org/dataoecd/23/27/35281194.pdf, at 1127, 65-79
Discussion Questions
1. What is the objective of the Balanced Score Card (“BSC”) and of allied measurement
strategies? What role do measurement and accountability play? What role do leading
and lagging indicators play?
2. How are the Kaplan & Norton and Kuzek & Rist approaches to using measurement to
improve success similar and different? Do you see any basis for preferring one over the
other in general or in particular circumstances?
3. The title of this session suggests that there is an affinity between the BSC (and allied
measurement strategies) and institutional learning. What affinities (or discontinuities)
do you see? Why or why not might these approaches be equally or more clearly
understood as management-based or performance-based tools? Market-mimicking
tools? Bureaucratic oversight tools?
4. What critique of these tools might adherents of “craft” or “professionalism” make?
5. Imagine an institution with which you are familiar and how it might go about designing a
BSC to advance its objectives—and exercise you will soon be asked to undertake for a
state or local school system. What value do you see to the exercise? What difficulties?
How would your answers differ if the institution in question had only recently been
created or reorganized versus one that has been in existence and relatively unchanged
for a long time?
Session #18: February 18, 2013
The Role of Qualitative Review in Strategically Structured Public Learning: Using Leading
Indicators to Build Expertise and Harness Street-Level Discretion
1. John Braithwaite and Valerie Braithwaite, The Politics of Legalism: Rules versus
Standards in Nursing-Home Regulation, 4 Soc. & Leg. Stud. 307 (1995)
2. Kathleen Noonan, et al., Legal Accountability in the Service-Based Welfare System:
Lessons from Child Welfare Reform, 34 L. & Soc. Inq. 523, 523-26, 533-64 (2009)
3. Daniel E. Ho, Fudging the Nudge: Information Disclosure and Restaurant Grading, 122
Yale L.J. 574 (2012) (excerpts)
4. Anemona Hartocollis, New York City Ties Doctors’ Income to Quality of Care, N.Y. Times,
Jan. 11, 2013
5. 2013 Medicare Nursing Home evaluation rubric
Discussion Questions
1. What differences are there in the qualitative review procedures used in support of:
nursing home regulation in the US as opposed to Australia; child welfare services in Utah
and Alabama; restaurant health inspections in San Diego versus New York City; and
quality of health care provided by doctors in public hospitals in New York?
2. To what extent do these differences map to differences in the governance model being
implemented in each case, be it, for example, a model driven by professionalism,
experimentalism, performance-management, or bureaucracy? To what extent do they
map to the dichotomy between “rules” and “standards” that is discussed in the Noonan
3. Is restaurant grading as practiced in San Diego and/or New York appropriately thought
of as “qualitative” review at all? Why or why not? Does the use of a numerical scoring
system automatically deny an evaluation scheme the advantages of being qualitative?
What are those advantages?
4. Are the outcomes of these various kinds of inspections and reviews lagging or leading
indicators—or something else entirely? Do the authors make any assumptions about
which type of indicator each category of review or inspection should provide under the
circumstances? What type of indicator do you believe each category or review or
inspection should strive to be, and why?
5. In what ways do the different approaches to review and inspection serve or disserve the
values and objectives promoted by Balanced Score Cards and allied measurement
6. A chief worry of doctors contemplating pay differences based on quality of care is that
“they could be penalized for conditions they cannot control, including how clean the
hospital floors are, the attentiveness of nurses and the availability of beds.” Are these
conditions over which doctors have no control? Consider differences in background
health conditions of the patients of hospitals in different neighborhoods. Might doctors
in hospitals in impoverished neighborhoods where health conditions are generally
worse fear that they will be penalized because they have chosen or been assigned to
work with more challenging patients who are sicker when they are admitted and thus
likely to have less favorable outcomes on average? Might this have the perverse effect
of encouraging “better” doctors to prefer jobs in hospitals in more affluent
neighborhoods, increasing inequities already faced by patients in poor neighborhoods?
(This, of course, is a problem also faced in reviewing nursing homes, child welfare
services, and schools, if perhaps not so clearly restaurants.) Are there any solutions for
this problem other than not evaluating doctors based on quality?
7. Relatedly, continue, as you began to do in the last session, to imagine an institution with
which you are familiar and for which you have been asked to designing a BSC. What
model of qualitative review, if any, might you adopt?
Session #19: February 25, 2013
Qualitative Review of Schools and Educators
1. Schools:
a. Helen F. Ladd, Education Inspectorate Systems in New Zealand and Netherlands: A
Policy Note, 5 Ed. Fin. & Pol’y 378-92 (2010)
b. Quality Review Rubric 1: NYC Quality Review Rubric, Composite of 2007-2010
c. Quality Review Rubric 2: NYC Quality Review Rubric, 2012-13
2. Educators:
a. Robert Pianta & Hamre, Conceptualization, Measurement, and Improvement of
Classroom Processes: Standardized Observation Can Leverage Capacity, 38
Educational Researcher 109 (2009), at pp. 109 to very top of 112; very bottom of
113 to very top of 116
b. Charlotte Danielson, Evaluations that Help Teachers Learn, 68 The Effective Educator
35-39 (Dec. 2010/Jan. 2011)
c. Full Danielson Rubric, http://usny.nysed.gov/rttt/teachersleaders/practicerubrics/Docs/Teachscape_Rubric.pdf (skim only)
d. NYC Abbreviated Danielson Rubric
e. Teresa Watanabe & Stephen Ceasar, L.A.'s Revamped Teacher Evaluation System
Getting Mixed Grades, LA Times, Nov. 26, 2012
Discussion Questions
1. What similarities and differences do you see when you compare:
a. Quality review of schools in New Zealand, the Netherlands, and New York
b. The two school quality review rubrics included in the reading
c. The Pianta/Hamre and Danielson approaches to qualitative review of teachers
d. The two Danielson rubrics included in the reading
e. Quality review of schools and of teachers?
When you make the same comparisons, what links to different governance models (the
usual: bureaucracy, performance management, professionalism, institutional learning,
etc.) do you see?
Continuing the mental exercise from the last two classes, in thinking about building a
balanced scorecard for an institution with which you are familiar, how valuable (or not)
is each of these approaches to qualitative review as a model for a key component of
your balanced scorecard? As a leading or lagging indicator?
Next week, we will discuss the empirical evidence on the relative value and correlation
of test scores, student surveys, and observations of teachers as components of teacherevaluation schemes. Based on what you already know, what uses, if any, would you
make of qualitative review in the process of evaluating teachers?
How do the four rubrics in the reading fare on the criteria proposed by last week’s
readings (Braithwaites, Noonan et al., Ho) for predicting the usefulness of qualitative
review schemes?
“Validity” refers to whether measures used reflect the conditions or outcomes that are
actually of interest. (E.g., thermometers provide a valid reflection of body temperature,
but do not provide as valid a reflection of health; in the latter regard, that is, they are an
imperfect “proxy” for health.) In your view, how “valid” (in this sense) are the measures
proposed in the two school-review and two teacher-evaluation rubrics in the reading?
What measures would be more valid? Can those measures be encapsulated in a quality
review rubric for schools or teachers?
“Reliability” refers to how effectively a measurement scheme determines whether a
particular condition (which condition may or may not be “valid” in the sense used
above) is present. (E.g., thermometers may, in theory, provide a valid reflection of body
temperature, but differences in the manufacture and use of thermometers may lead to
unreliability –i.e., to different readings of the body temperature of the same person at
the same time.) How “reliable” in this sense do you think the various methods of
qualitative review described discussed are? This is the issue of “inter-rater reliability.” Is
the likelihood of inter-rater unreliability” the death knell of qualitative review, or does it
present some an opportunity of some sort for making particularly good use of such
observations? How do the readings from last week (Braithwaites, Noonan et al., Ho)
bear on this question?
Teacher-Observation Exercise: To be described in class. In preparation for class, please
familiarize yourself with the NYC Abbreviated Danielson Rubric listed above.
Session #20: March 4, 2013
Building a Better State or Local Balanced Scorecard for Public Education
1. Example 1: New York City’s Accountability System
a. Stacey Childress, et al., Managing for Results at the New York City Department of
Education, in Jennifer O’Day, et al., Education Reform in New York City: Ambitious
Change in the Nation’s Most Complex School System (2011), at pp. 7 (“Implementing
Systems …”) - 15 of draft (reprise from Session 5)
b. Ledyard McFadden, New York’s Quest for Excellence, Educational Leadership (Oct.
c. New York City Accountability and Results, 2002-2012
d. NY Times Room for Debate, Making the Grade in NYC (Oct. 9, 2012)
e. Optional: Educator Guide to New York City High School Progress Report (2011-2012)
2. Examples 2-5: Differentiated Accountability and Support Systems included in ESEA
Waiver applications for Louisiana, Massachusetts, New Mexico, and South Dakota, as
described in Center for Education Policy, What Impact Will NCLB Waivers Have on the
Consistency, Complexity and Transparency of State Accountability Systems? (2012), at 113, 15-18, 20-21, 24-27
3. Example 6: The MET Project, Ensuring Fair and Reliable Measures of Effective Teaching:
Culimating Findings from the MET Project’s Three-Year Study (Jan. 2013),
oner_Brief.pdf, at page-before-p. 1 (“About the MET Project”) and pp. 6-20
4. Other perspectives:
a. The Pew Forum on Education Reform, Excerpts from “A Tribute to Al Shanker”
(reprinted from Ed Week, May 14, 1997), pp. 35-37 only (“Al Shanker Speaks …”)
b. Eva L. Baker, Paul E. Barton, Linda Darling-Hammond, Edward Haertel, Helen F. Ladd,
Robert Linn, Diane Ravitch, Richard Rothstein, Richard Shavelson & Lorrie A.
Sheppard, Problems with the Use of Student Test Scores to Evaluate Teachers
(Economic Policy Institute 2010), at pp. 1-4
c. Deborah Kenney, Want to Ruin Teaching? Give Ratings, N.Y. Times, Oct. 14, 2012,
d. Motoko Rich, Beleaguered? Not Teachers, a Poll on ‘Well-Being’ Finds, NY Times,
March 28, 2013
Writing Assignment
As an MS Word or Excel attachment, prepare and email to CPRL@law.columbia.edu by
Sunday March 3, 2013 at 1:00 p.m., an outline or sketch of a balanced scorecard for use by a
state education department, school district, or school to hold school(s) and/or educators
accountable for satisfying the relevant jurisdiction’s goals for promoting student learning and
development. In your paper, please indicate:
a. The school or set of schools, and/or the set of educators to which your balanced scored
card applies
b. The kinds of measures you consider to be “valid” for this purpose, as that term is
defined in the Discussion Questions for Session 19.
Your outline or sketch need not address all of the following matters but should consider some
of them, among other topics that you believe are important:
c. The appropriate mix of quantitative and qualitative (e.g., test score and other)
measures, in any, and of leading and lagging indicators, if any
d. The tools you would use to determine how well schools and/or educators are
performing on those measures
e. The weights to be accorded to different measures
f. Whether outcomes would or would not be publicly disclosed
g. The stakes or consequences, if any, attached to particular outcomes
Discussion Questions:
(Your writing assignment is not expected to address these questions explicitly, but you should
be prepared to discuss them orally in class.)
1. What considerations drove the design of your balanced score card?
2. What role, if any, did different governance models play in your design?
3. Did you follow any particular model or example (e.g., Kaplan & Norton; Kuzek & Rist; any of
those in the readings for this or earlier weeks), and why?
4. How did you solve for the problem of student- and school-level differences that arise
because students and teachers are not randomly assigned to schools, leaving some schools
and teachers with much more challenging student populations than others?
5. How do you imagine Albert Shanker would have gone about holding schools accountable?
Which governance model(s) does Shanker appear to favor?
6. How would Baker et al. hold teachers accountable?
7. To what extent do the recent findings of the MET Project answer the concerns of Baker et
al. and resolve the major issues in the long-running teacher-evaluation debate?
Session #21: March 11, 2013
A Conversation with Experts in the Mechanics of Teacher Evaluation
1. Raj Chetty, John N. Friedman & Jonah Rockoff, The Long Term Impacts of Teachers,
Executive Summary and video (40 minutes):
2. Teresa Wantanabe & Stephen Caesar, L.A. Times, Nov. 25, 2013, LA’s Revamped Teacher
Evaluation System Getting Mixed Grades, http://articles.latimes.com/2012/nov/25/local/lame-teacher-eval-20121125
3. Nicholas Kristof, The New Haven Experiment, N.Y. Times, February 15, 2012,
4. Arne Duncan, The Tennessee Story, Huffington Post/Education (July 23, 2012),
5. Dave Levin & Tim Daly, Better Teaching, Beyond Evaluations, NY Daily News Opinion, Jan. 4,
6. Susan Keyock, Please Help Me Be a Better Teacher, NY Daily News Opinion, Jan. 2, 2013
7. Speaker BIOs
Assignment: This conversation will be partly about whether it is worthwhile, but mainly about
how best, to implement a teacher-evaluation system that aims to benefit students and
teachers. Please come to class with three or four questions you would like to pose to experts on
the choices they have made among the different approaches to teacher evaluation addressed
in last week’s and this week’s reading, the outcomes they are seeing or at least are hoping to
achieve the obstacles they face in implementing teacher-evaluation systems, and the solutions
they advocate.
Spring Break: Week of March 18
Session #22-23: March 25, 2013 (extended session)
Public Problem Solving and Democracy
1. The case for a problem-solving democracy:
a. Christopher K. Ansell, Pragmatist Democracy, supra at 134-40, 166-83
b. Archon Fung, Deliberation and Social Conflict, in Empowered Participation:
Reinventing Urban Democracy pp. 173-97 (2004)
2. Doubts:
a. Jeffrey Henig et al., Parent and Community Engagement in NYC and the
Sustainability Challenge for Urban Education Reform, in O’Day et al., supra, at 33-38,
43-45 (ending with first paragraph of “The Three Groups” section), 46, 48-54
b. Gordon Whitman, Making Accountability Work, 28 N.Y.U. Rev. L. & Soc. Change 361
(2003), at 361-67
3. Examples:
a. Emma Brown, D.C. Parents Develop Alternatives to Chancellor’s School-Closure Plan,
Wash. Post, Jan. 1, 2013
b. Andrew Ujifusa, More States Consider 'Parent Trigger' Laws, Educ. Week, March 21,
c. Bridgeport:
i. Bridgeport Team’s 2011 Student Paper, The Theoretical Model pp. 42-53
ii. Michele Monar, Parents Take Lead in District’s Engagement Policy, Ed. Week
(Aug. 14, 2012)
iii. Optional: CPRL, Parent Problem-Solving Groups Manual: A New Approach to
Parent Engagement (2012)
c. Optional: Oliver C. Moles, Jr. & Arnold F. Fege, New Directions for Title 1 Family
Engagement: Lessons from the Past, Handbook on Family and Community
Engagement, at 3 (title page), 5 (introductory § only), 8 (very end of page) – 12
Discussion Questions
1. In a portion of Ansell we read earlier, he stated that, “Organizational transformation of
public agencies cannot easily occur without fundamental change in the relationship
between agencies and democratic publics.” (p.17) Thus far, we have spent a lot of time
talking about organizational transformation of public regimes, but not so much on how
those regimes interact with their “democratic publics.” What kinds of changes in politics
and democracy does Ansell have in mind? How do they differ from “normal politics”
that are familiar at the local, state, and federal levels in contexts such as public
education, health care, deficit reduction, etc?
Ansell’s point suggests that for every governance model of how public agencies should
be organized and administered, there is a corresponding approach to politics and
democratic interaction. Taking that claim at face value for a moment, what form of
politics and democracy would seem to fit best with bureaucracies? With performance
management (New Public Management)? With professionalism/craft? With
experimentalist regimes? In each case, what is the role of representation? Direct
democracy? Interest or intermediary groups? Experts vs non-experts? Elections versus
other forms of deliberation and participation?
Suppose, on the other hand, that, given the interplay of unevenly distributed resources,
other economic realities, and divergent individual preferences, interest-group politics is
the only stable or “natural” form of democratic interaction between the public and
public actors and agencies. If that were the case—in other words, if “fundamental
change in the relationship between agencies and democratic publics”—is not possible
because one (the existing) form naturally predominates, would that mean that
“organizational transformation of public agencies” of the sort Ansell contemplates is
unlikely or impossible? Do you think interest-group politics are “natural” and
What form of politics and democracy do Professor Henig and colleagues suggest were
necessary or best suited to the period during which the Bloomberg/Klein education
reforms were taking in New York City (2003 to 2010)? What form of politics and
democracy did Mayor Bloomberg and Chancellor Klein instead prefer? Do you agree
with Henig et al. or Bloomberg-Klein about the best approach to engaging the public? Is
there an alternative to those two competing approaches?
What forms of politics and democracy do you see being implemented in the “Traxton”
example in Professor Fung’s article on community policing? In the Bridgeport, “parent
working group” example? In the challenge issued by Washington, D.C. Schools
Chancellor Kaya Henderson to parents opposing the closure of their schools? How are
these approaches to politics and democracy similar or different? How promising are
these approaches (and how effective were they in the particular contexts described) and
what challenges and obstacles do the approaches face? What do these examples
suggest about the possibility of overcoming the obstacles Whitman discusses to
authentic stake-holder participation and problem-solving?
Sessions #24, 25 and 26 (first half): April 1, 8, and 15 (initial 50 minutes), 2013
Student Project Presentations: After providing the class with enough of a description of your
project and its broader context to establish or reestablish a common understanding, please do
one or both of the following:
1. Critically consider your project or its broader context from the perspective of
experimentalism or, if you would prefer, from the perspective of one or more of the
other governance models we've discussed. If you chose to focus on the institutional
"context," not just the project, you may broaden it to include anything in between your
project and the outer boundary of the overall reform strategy your client NYCDOE,
CSDE, or Zoom is pursuing. In what ways (if any) does the project or reform strategy
proceed from, or is it grounded in, one or more of the governance model(s) we have
discussed in class? How could the project or the broader reform strategy succeed more
fully or efficiently if it were modified in some fashion along the lines suggested by one or
another or a combination of the governance models? How does the project or the
broader reform strategy deviate from the relevant governance model(s), and how do
those deviations impede the project's or reform strategy's effectiveness? Overall, what
could the leaders of your project or the broader reform strategy learn from your
understanding of the theory underlying, and the best means of implementing, the
relevant governance model(s)?
2. Critically consider experimentalism, or if you would prefer, one or more of the other
governance models we've discussed from the perspective of your project or its broader
context. Again, in what way does the project or strategy proceed from, or is it grounded
in, one or more of those governance model(s)? What weaknesses, difficulties or
challenges does your project or its broader context reveal about the theory of change
and improvement that underlies the relevant governance model(s)? What light does the
project or broader strategy shed on how easy or hard it is to implement the relevant
governance model(s) effectively? How might those challenges lead you to modify the
relevant governance model(s)? What do the challenges lead you to conclude about
which model(s) would best facilitate the relevant reform strategy -- or about the validity
and usefulness of one or more, or all, of the governance models in general? Overall,
what could the proponents of the relevant governance model(s) learn from your project
or its context?
April 1 - NYC DOE teacher evaluation (first 50 minutes); communications (last 50 minutes)
April 8th - CSDE quality review (first 55 minutes); school turnaround (last 45 minutes)
April 15th - Zoom (first 50 minutes)
Readings and Discussion Questions; Electronic Copy of Presentation:
1. One week prior to your presentation, please email to Professor Liebman and Margaret
Symuleski a short set of background readings (no more than 20 pages) and a set of
discussion questions for your presentation.
2. By 1 pm on the Sunday preceding your Monday session, please email to Professor
Liebman an electronic version of your presentation.
Session #26: April 15, 2013 (final 55 minutes)
Transitioning from Bureaucracy to the New Age Organization
1. Ansell, supra, ch. 3, pp. 43-55, 61-62
2. Examples:
a. Aldine, Texas: Heather Zavadsky, Bringing School Reform to Scale (2009), Ch. 2,
District Approach #1 (Aldine School District)
b. Norfolk, Virginia: Zavadsky, supra, Ch.6, District Approach #3 (Norfolk School District)
c. New York City: Eric Nadelstern, The Evolution of School Support Networks in New
York City, Center on Reinventing Public Education Working Paper #2012-2 (2012),
http://www.crpe.org/sites/default/files/pub_psdp_NYCnetworks_jun12.pdf [MAX,
Note from Prof. Liebman: I agree with Professor Nadelstern’s retrospective judgments about how
the New York City reform could have been improved, but I would add one failing of the Networks
that has kept them from becoming the key intermediary Nadelstern imagines: The networks had
two somewhat competing objectives that had to be kept in balance, self-consciously, in a
structured way: (1) support schools operationally (fending off the bureaucracy) and (2) promote
and facilitate, without dictating steps schools were to take to assure, improvement of student
achievement. Because the Networks were always palpably accountable for the former objective,
because schools only opted into networks they trusted to serve them operationally, but
Networks were not effectively held accountable for student improvement, the Networks took
their service obligations more seriously than their achievement-facilitation role.
Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to
CPRL@law.columbia.edu by Monday April 15, 2013 at 8:00 a.m., on your understanding of the
“transformation strategy” (as defined below) that was implemented in Aldine, Norfolk, or New
York City; what its strengths and weaknesses are (including compared to the transformation
strategies used in the other cities); how you would have improved on that strategy; how useful
you think the transformation strategy would be in different K-12 contexts (e.g., federal vs. state
vs. local; large vs. small; urban vs. rural; collegial vs. conflictual; ones with strong or weak
unions, etc.); how do Ansell’s insights bear on your judgments; and what does the example you
focus on suggest about the validity of, or weaknesses in, Ansell’s analysis. By “transformation
strategy,” I do not mean the governance or instructional strategy itself that each district
adopted (e.g., “empowerment” in NYC), but instead the strategy the district adopted for
moving the institution and its people to give up the old ways and embrace and adopt the new
ways. For example, two districts that both decide to adopt a performance-management
system focused on teacher-evaluation outcomes could make the transition to that common
approach in very different ways, depending upon, for example, how abrupt or gradual the
change is; how the change is introduced, explained and phased-in; who participated in deciding
on a strategy and on the details of its implementation; etc. It is the latter set of choices that
you should address in this paper.
Discussion Questions
1. How would you characterize the transition strategy implemented in Aldine? Norfolk?
New York City? What is the appropriate scale for comparing transition strategies (e.g.,
evolution vs. revolution; top-down vs. bottom-up)?
2. Which transition strategy seems to be most promising or to have worked best? Why?
3. One claim that is sometimes made in support of strategies based on newly created
charter schools or to justify more comprehensive (“start from scratch”), less incremental
approaches to school turnaround is that it is easier to build learning institutions from
scratch than to transform bureaucracies into them? Do you agree with that claim? What
examples from the reading over the course of the year support or contradict this view?
What is an example of a successful transformation of a previously bureaucratic
4. Even if it is more difficult to transform bureaucracies than to replace them wholesale,
there sometimes are no alternatives to transformation. In those cases, what steps are
most likely to facilitate effective transformation? Are there kinds of organizations or
organizational conditions that support or impede transformation?
5. How important are charismatic or determined leaders? Can existing organizations
transform themselves, and sustain the changes, without charismatic leadership? What
are the implications of your answer for the likely success of transformation efforts?
Given example from the readings for this week and for the year that support your views
on these questions.
Exercise: To be described in class.
Session #27: April 22, 2013
Re-envisioning Schools and the Technology they Need: A Conversation with Sharren Bates, Chief
Product Officer, inBloom; Larry Berger, Chief Executive Officer, Wireless Generation; Doug Jaffe,
Senior Fellow, Regents Research Fund, New York State Education Department; and Jon Schnur,
Executive Chairman, America Achieves
1. Panelist Bios
a. Sharren Bates
b. Larry Berger
c. Doug Jaffe
d. Jon Schnur
2. Joel Rose, How to Break Free of Our 19th-Century Factory-Model Education System, The
Atlantic (May 9, 2012)
3. Jonathan Schorr & Deborah McGriff, Future Schools, Education Next, Summer 2011
4. Joanne Weiss, The Innovation Mismatch: “Smart Capital” and Education Innovation,
Harv. Bus. Rev. Blog, March 31, 2011, http://blogs.hbr.org/innovations-ineducation/2011/03/the-innovation-mismatch-smart.html
5. Frank Catalano, How Will Student Data Be Used [in the Gates-Funded Shared Learning
Infrasctuture], Mindshift, KQED, July 3, 2012,
6. Bill Tucker, The Country’s Most Ambitious Digital Learning Project, Education Next, Jan.
31, 2012
7. Alan Schwarz, Moorsville’s Shining Example (It’s Not Just About Laptops), N.Y. Times,
Feb. 12, 2012
8. Matt Richtel, In Classroom of the Future, Stagnant Scores, N.Y. Times, September 3,
9. Optional:
a. Alison Bailey et al, “Unleashing the Potential of Technology in Education.” Boston
Consulting Group, August 2011, at 8-19
b. Stacey Childress, Rethinking School, Harv. Bus. Rev., March 2012,
c. Lawrence S. Bacow et al., Barriers to Adoption of Online Learning in U.S. Higher
Education, Ithaka (May 2012), http://www.sr.ithaka.org/researchpublications/barriers-adoption-online-learning-systems-us-higher-education
d. Cisco, Case Study, New York City School of One,
e. Steve Rowley, Christensen’s Curve & The Digital Learning Revolution, Getting
Smart, July 11, 2012,
Discussion Questions: Come to class with four or five questions for the panelists based on the
readings and on your aspirations and concerns about changes in the design of schools, the role
of educators, and the expected contributions of technology.
Session #28: April 29, 2013
Summing Up; Looking Forward
Reading (tentative)
1. http://opinionator.blogs.nytimes.com/2013/02/02/the-boys-at-theback/?src=me&ref=general (Boys at the Back)
2. David Whitman, An Appeal to Authority, Education Next, Fall 2008
3. Reform School on the Bayou, Wall St. J., April 16, 2012
4. Catherine Gewertz, Districts Gird for Added Use of Nonfiction, Education Weekly, March
14, 2012
5. Review of “ All Together Now: Creating Middle-Class Schools through Public School
Choice” by Richard D. Kahlenberg. Harvard Educational Review
6. Students First Policy