The Interplay of Teaching,
Learning, and Research:
Graduate Student
Collaborations in Law and
Psychology
A. Holtzworth-Munroe, Ph.D. & A.G. Applegate, J.D.
Annakarina Bortner
B.M. D’Onofrio, Ph.D. & J.E. Bates, Ph.D.
Indiana University Bloomington
March 30, 2012
Outline
I. Introduction
A. Interdisciplinary Team
B. Child Informed Mediation Study (CIMS) –
Interdisciplinary Research Program (improving outcomes
for families with children)
C. CIMS – Opportunity for Interdisciplinary Training (in law,
psychology research, and program evaluation)
II. SOTL Study – Interdisciplinary Training Program
(improving student learning outcomes)
A. Methods
B. Knowledge Tests: Methods and Results
C. Focus Group Discussions: Methods and Results
1. Focus Group Coding
2. Focus Group Student Quotes
D. Coding Student Behavior During Mediations
E. Student Involvement in Program Evaluation Research
III. Discussion
I. Introduction
A. Interdisciplinary Team
Interdisciplinary Collaborations Grow out of
Overlapping Interests and Goals:
Our Team
• Amy G. Applegate (IU School of Law):
• Family and Children Mediation Clinic
• Training clinic for law students
• Interested in effectiveness of her training and program
• Amy Holtzworth-Munroe (IU Psychology)
• Relationship distress
• Couples therapy
• Intimate Partner Violence
• Brian D’Onofrio (IU Psychology)
• Behavioral genetics; divorce and mediation
• Jack Bates (IU Psychology)
• Developmental psychopathology; Longitudinal research with children
• Graduate and Law Students
Goals
• Does interdisciplinary training improve
student learning outcomes (SOTL)?
• Does interdisciplinary training improve
outcomes for families and children in
mediation (CIMS)?
Interdisciplinary Training
• Law and Psychology intersect in real world
• Family law: area of extensive overlap
• Psychologists conduct child custody evaluations
• Family mediators may be lawyers or psychologists
• Family Law Education Reform Project
(FLER) urged more interdisciplinary training
in family law
• But little social science research on family
law issues or interventions
Mediation in Parental Divorce or
Separation Cases
• Litigated cases
– Adversarial process
– Many parents without lawyers
– Burden on the court system
• Mediation
– Confidential settlement process
– Party self-determination – voluntary, informed decisionmaking
– Impartial, neutral mediator assists parties in
•
•
•
•
•
identifying the issues
reducing misunderstanding
clarifying priorities
exploring areas of compromise
finding points of agreement as well as legitimate points of
disagreement
Mediation in Parental Divorce or
Separation Cases
• Mediation as an alternative to litigation
– Mediation assumed better:
• Parents know what is best for their own children
• Parents invested in the process
– One study (Emery et al, 2001) found mediation:
• Increased involvement of nonresidential parent
• Not associated with increases in co-parenting conflict
• Increased cooperation and flexibility
Setting for Our Interdisciplinary Training Effort:
IU Maurer School of Law Viola J. Taliaferro
Family and Child Mediation Clinic
• Existing law course/clinic, before SOTL study of
interdisciplinary training:
• Intensive Course in Domestic Relations Mediation (40
hours)
– Law students become registered (licensed) mediators
– Course had state-required interdisciplinary component
• Transition into Clinic
–
–
–
–
–
Law students co-mediate 5-8 cases during semester
Weekly classes
Supervision, reflective journals
Law student mentors
Clinic did not have interdisciplinary component
IU Clinical Science Program
• Existing psychology clinics and practica
before SOTL study of interdisciplinary
training:
• Evidence Based Practice
– Basic Research on Families and Children
– Research on Interventions
– Evidence Based Therapy Practica
• Became interested in possibility of
interdisciplinary approaches to mediation
I. Introduction
B. Child Informed Mediation
Study (CIMS) – Interdisciplinary
Research Program
(improving outcomes for families with
children)
Interdisciplinary Mediation Approaches
• McIntosh: two new interdisciplinary approaches
– Both involve a “child consultant”
• psychology or mental health professional
– 1) Child Focused Mediation (CF)
– 2) Child Inclusive Mediation (CI)
• In both CF and CI, legal (mediator) and
psychological (child consultant) professionals work
together to prepare case and work with parents.
• Goal:
– Bring child’s “voice” or perspective into the mediation
– Make agreements that best meet needs of the children
Child Information Mediation Study
(CIMS)
• Use CF and CI
– interdisciplinary approaches to mediation
• Mediators = law students
• Child consultants = psychology graduate
students
• Working together for better outcomes for
families with children.
Child Focused Mediation (CF)
• For parents with children of any age
• Intake: Mediators (law students) meet and screen
parents
• Child consultant (psych student) does not meet
child or interview parents but gets basic
information about children
• Parent feedback session (at start of mediation):
Child consultant provides parents with
developmental information about children and
impact of parental separation and conflict on
children; tries to individualize to their children
• Mediation negotiations: After parent feedback
session, parties enter negotiations with
mediators
CF: Parent Feedback Session
• Child Consultants (psych students):
• Ask Questions about Children
– Can you tell me about your child? Picture of them?
– What activities do you enjoy with your child?
– In 20 years, what do you want your child to say about how
you handled this situation?
• Give research-based information on issues to be
considered in mediation:
–
–
–
–
development
impact of parental separation
impact of inter-parental conflict
other issues (e.g., parenting time)
• Often use educational materials to make points
Child Inclusive Mediation (CI)
• For parents of children ages 5 - 17
• Intake: Mediators (law students) meet and screen
parents
• CI Assessment: Child consultant (psych student):
– developmental history interview (regarding children)
with each parent
– Interviews child
• Parent feedback session (at start of mediation):
Child consultant provides parents with feedback
about their child
• Mediation negotiations: After parent feedback
session, parties enter negotiations with mediators.
Child Interview in CI Mediation
• Explanation
– Understand what its been like for you
– Not asked to make decisions
– Will share messages with parents (confidentiality)
• Age appropriate assessment of:
– Experience of separation and conflict
– Attachment to each parent and others
– Needs and options
Child Inclusive Example
Altered to Maintain Confidentiality
Case 1: Will
• Will was 10 years old
• Parents recently separated
• Interview with Will
– Quiet, shy, reserved, careful (detail re picture)
– Cooperative
– Bright, doing well in school, activities, friends
Will
• Magic Wand Question:
– Want them to stop fighting and get along
Case of Will
• When child consultant met with parents:
• Focus on inter-parental conflict
– Impact on Will
– Need to decrease conflict
• in general and in front of Will
– Need for him to maintain good relationships with
both parents (get re-connected), but their conflict
was interfering with this
Why CF and CI May Work
• Wakeup Call to Parents
– Emotional information changes their behaviors
– Focus away from “winning” against each other to
doing what is best for their children
• Developmentally Appropriate Agreements
– Favors stability and stronger parent-child
relationships
• Interdisciplinary Consideration of Issues**
– Helps change mediation agenda:
• During mediation negotiations, law student mediators refer
to messages from the psychology student child
consultants
McIntosh
, 2007
McIntosh’s (2008) Research:
Comparing CF and CI Mediation
• Compared CF and CI mediation:
–
–
–
–
In Australia
~150 families, children ages 5 - 17
No random assignment: CF first, then CI
No Mediation As Usual (MAU) comparison group
• 2 year follow-up data:
– Both lowered inter-parental conflict
– Both improved child mental health
• CI fared better:
– Father satisfaction with parenting arrangement
– Father involvement with children’s lives
– Less re-litigation
• Improved outcomes for families important to us.
McIntosh & Long, 2006; McIntosh, 2007
Child Informed Mediation Study (CIMS)
• Our study of new, interdisciplinary mediation
methods
• In U.S.
• Random assignment to types of mediation
• MAU as a comparison condition
• Children ages 5 – 17:
– Randomly assigned to 1 of 3 conditions:
• MAU
• CF
• CI
• Children all under age 5:
– Randomly assigned to 1 of 2 conditions:
• MAU
• CF
Child Informed Mediation Study (CIMS)
• Most cases referred to Law School Clinic by court
• Mediation intake
– law student mediators
• Parties invited to participate in study
– psych student child consultants
• Random assignment to CI, CF, or MAU
• Research assessment conducted
– Parents and children age 5 or older
• If CI: child interview conducted
– psych student child consultants
• Mediation
– IF CI or CF: parent feedback session
• child consultants and mediators present
– CI/CF/MAU: negotiation and try to reach agreement
• mediators
• Post-mediation research forms
– parents and students
• One and two year follow-up research assessments
– Parents and children age 5 or older
I. Introduction
C. CIMS– Opportunity for
Interdisciplinary Training
(in law, psychology research, and
program evaluation)
Child Informed Mediation Study
(CIMS)
• Interdisciplinary teaching/training opportunity
• New practicum opportunity for psychology
students:
– No previously existing practicum on mediation or
child consultation
• Law school mediation course and clinic:
– Significant change to curriculum and practice
• Separate mediation intake and negotiation sessions
• Have child consultants present in CI and CF
• Affect on mediation agenda
Interdisciplinary Training in CIMS Study:
Learning Family Law and
Psychology Research Findings
• Law students learn about psychology research findings
on effects of divorce and child development:
– Guest lectures (and readings) from psychology professors
– Involved in CI and CF parent feedback sessions conducted
by psychology child consultants (hear information
presented to parents)
• Psych students learn about law and mediation:
– Guest lecture (and readings) from law professor
– Feedback from law professor on parent feedback sessions
– Planning cases with mediators, and mediators in room for
the parent feedback sessions
• Both: Joint case rounds
Interdisciplinary Training in CIMS Study:
Learning Program Evaluation Research Methods
• CIMS is a program evaluation study:
– uses psychology research methodology to
compare 3 types of mediation
• Evidence Based Practice movement:
– Established in psychology
– New but gaining recognition in law and family law
• Opportunity for interdisciplinary training of
students in program evaluation research
Interdisciplinary Training in CIMS Study:
Learning Program Evaluation Research Methods
• Law Students:
– Learn importance of evidence based practice and learn program
evaluation methods
– Guest lecture from psych professors
– See research being conducted in mediation clinic
• Psych students:
– Learn about program evaluation in real world setting
– Learn to disseminate program evaluation to other disciplines
– Apply knowledge from other classes to this study
• Both involved as:
– Research participants:
• Complete Informed Consent Form
• Student outcome form (their perceptions of the case)
• Had behavior during mediation observed and coded (discussed later)
– Researchers:
• Planning and implementing study
• Interpreting findings and generating questions for the future
Interdisciplinary Work:
Potential Misunderstandings
in Program Evaluation Research
• Terminology:
– “Experiment”
– “Issues”
– “Psycho-”
• Study participant recruitment
– Coercion and Consent (legal and research)
• Confidentiality
– Legal concerns
• Privileged communications
• Self-incrimination
• Prevent possible use in court cases
– Research frustration
• Changing/impeding the process
Interdisciplinary Work:
Potential Misunderstandings
in Program Evaluation Research
• Random assignment
– Lack of equal access vs “Do No Harm”
• Treatment manuals
– Manuals vs individualized plans for each case
• Ease of implementation and affordability
– Added work for students/more professionals/higher cost
• Support for involvement in research
– Need approval and support from centralized authority (judges)
• Students were involved in discussion of these issues, when
designing and implementing the CIMS study
– Interdisciplinary training re program evaluation research methods
II. SOTL Study –
Interdisciplinary Training
Program
SOTL Study of
Interdisciplinary Training of Students
• CIMS study designed to:
– Examine whether interdisciplinary training and
practice make divorce mediation more effective
• Better for families and children
• CIMS study provided opportunity for SOTL
study:
– Evaluate interdisciplinary training
• Effect on students’ learning and attitudes
SOTL Study Hypotheses
• Effects of the interdisciplinary training will:
– Increase students’ knowledge of family law,
psychology research, and program evaluation
– Increase students’ positive attitudes towards
interdisciplinary work and research
– Students will become involved in program
evaluation research
II. SOTL Study
A. Methods
Measures: Overview
• Student Knowledge Tests
• Knowledge acquired
• Student Focus Group Discussions
• Experience and attitudes
• Attempt to Code Student Behavior During
Mediation
– discuss later
• Student Involvement in the CIMS Study
– Information on whether they are doing program
evaluation research?
Timing of Measures: Knowledge Tests
and Focus Group Discussions
• To examine change during semesters
– Measures given at:
• Start of semester (pre-training)
• End of semester (post-training)
• To examine effects of interdisciplinary
training relative to a “no-interdisciplinary
training” control condition
– Measures given during:
• Baseline (pre- interdisciplinary training) semester
• Interdisciplinary training semesters
Baseline semester
• Pre-interdisciplinary training comparison
group (Spring 2009)
• 8 law students
– Mediation As Usual (MAU) training only
• no CI or CF training
• no systematic interdisciplinary training
– Note: Psych professors did give other guest lectures that
semester (horse already out of the barn?)
• 11 psychology students
– No relevant training that semester
– No post-semester baseline group data for
psychology
Interdisciplinary Training Semesters
• Interdisciplinary semesters (Fall 2009, Spring
2010, Fall 2010)
– MAU, CF, and CI mediations conducted
– CIMS program evaluation study conducted
– Guest lectures (psych and law)
• Law Students:
– ~8 new law students each semester (N = 25)
• Psychology Students:
– Same 8 psychology students in all three semesters of
training (N = 8)
– Only examined psychology students’ data for the first
semester (Fall 2009)
• Data comparable to that of law students (one semester)
Students’ Previous Relevant Training
• No random assignment of students to baseline
versus interdisciplinary training semesters
– so don’t know if they are comparable or not
• Assessed their previous training
– background questionnaire
• Baseline semester > interdisciplinary semesters
students in:
• Relevant previous courses
• Other relevant experiences (e.g., research)
– Doesn’t end up being a problem for data analyses
• Potential Problem: Ideally would have more
baseline semesters
– But faculty and students eager to get going on
interdisciplinary training
Outcome Measures
Legal
Questions
Knowledge
Tests
Psychology
Questions
Psychology
Research on
Divorce
Outcome
Measures
Focus
Groups
Program
Evaluation
Research
Methods
Law Students Only
Psychology Students Only
Both
II. SOTL Study
B. Knowledge Tests:
Methods and Results
Knowledge Test Questions
60 Questions Total
Law:
• 31 Questions
• Relevant laws and legal statutes about divorce & mediation
Program Evaluation Research Methods (PERM):
• 12 Questions
• Program evaluation methodology
Psychology Research on Divorce (PRD):
• 17 Questions
• Research findings on relevant issues
– psychological consequences of divorce for children
– effectiveness of interventions for divorcing families
Knowledge Test Example Questions
• Law
– In Indiana, the court may not grant the divorce of a couple
until 60 days after one spouse files for divorce.
• True
• Program Evaluation Research Methods (PERM)
– In an attempt to keep subjects in your experimental and
control groups comparable on the variables you think
might affect the study findings, you can use:
• Random assignment
• Psychology Research on Divorce (PRD)
– Reviews of the role of noncustodial fathers after divorce
suggest:
• The quality of a relationship between a noncustodial father and his
children is an important predictor of child adjustment after
separation.
Knowledge Test:
Methods and Results
• Students completed tests anonymously
– Not worry about evaluation by professors
– Not used in course grade
• Issues:
– Students didn’t study for test
• Not a peak performance measure
– Can’t match pre- and post-semester tests of individual
students, to see how individual students changed
• Limits statistics
• Limits ability to look for predictors of outcome among
students
Pre-Training Knowledge Test Scores of
Law vs Psychology Students
• Both Baseline and Interdisciplinary Semesters,
at start of semesters:
– Law students had more knowledge of law
– Psych students had more knowledge or program
evaluation research
– No difference in knowledge of psychology
research findings on divorce
Changes in Law Student
Knowledge Tests: Baseline Semester
Question Type
(total points possible)
Pre or Post
Mean
t(df)
p
Law Knowledge
(31)
Pre Training
18.13
-3.95(14)
.001*
Post Training
22.69
Pre Training
4.75
Post Training
4.75
Pre Training
11.06
Post Training
10.75
PERM
(12)
PRD
(17)
.000(14)
n.s.
.286(14)
n.s.
PERM (Program Evaluation Research Methods);
PRD (Psychology Research on Divorce); *p<.05
Baseline Semester:
-Law Students increased knowledge of law, but not
psychology
Changes in Law Student Knowledge Tests:
Interdisciplinary Training Semesters
Question Type
(total points possible)
Pre or Post
Mean
t(df)
p
Law Knowledge
(31)
Pre Training
16.12
-7.15(48)
.000*
Post Training
21.9
Pre Training
4.76
Post Training
5.52
Pre Training
9.80
Post Training
11.02
PERM
(12)
PRD
(17)
-1.80(48)
.08
-2.31(48)
.025*
PERM (Program Evaluation Research Methods);
PRD (Psychology Research on Divorce); *p<.05
Interdisciplinary Semesters:
-Law Students increased knowledge of law and
psychology
Changes in Psych Student Knowledge Tests:
Interdisciplinary Training Semester
Question Type
(total points possible)
Pre or Post
Mean
t(df)
p
Law Knowledge
(31)
Pre Training
13.25
-1.72(13)
.10
Post Training
16.14
Pre Training
8.25
Post Training
9.57
Pre Training
10.5
Post Training
11.07
PERM
(12)
PRD
(17)
-1.262(13)
n.s.
-.627(13)
n.s.
PERM (Program Evaluation Research Methods);
PRD (Psychology Research on Divorce); *p<.05
Interdisciplinary Semester:
-Psych Students increased knowledge of law and
psychology, but not statistically significant
(small sample size)
Law vs Psychology Students:
End of Interdisciplinary Semester
Knowledge Tests
• At end of interdisciplinary semesters:
– Law students still know more law
– Psychology students still know more about
program evaluation research
– Still no difference in knowledge of psychology
research findings on divorce
Summary of Knowledge Test Results
Baseline Semester:
-Law Students increased knowledge of law, not
psychology
Interdisciplinary Semesters:
-Law Students increased knowledge of law and
psychology (statistically significant)
-Psych Students increased knowledge of law
and psychology (not statistically significant)
II. SOTL Study
C. Focus Group Discussions:
Methods and Results
Focus Group Discussions
• Law students interviewed by psych professors
• Psych students interviewed by law professor
• Combined group interviewed by both
– Not presenting combined group data
1. Focus Group Coding
Coding Focus Group Discussions
• Listened to all discussions
• Developed coding system
– Capture major themes being discussed
• Coding manual
– 47 codes
– 4 point scale
Focus Group Discussion Coding Manual
4 point scale
• 0: did not engage in the
coded behavior
• 1: slightly/briefly
• 2: to a noticeable or
moderate degree, but
not extensively.
• 3: a lot or extensively.
Coding Focus Group Discussions
• Trained undergraduate coders
• Independent coding
• Weekly coding meetings
– discuss questions, disagreements
– calculate inter-coder reliability
• Selected best coders; averaged their ratings
• Made 5 summary codes
Example of Summary Code
• Confidence/Enjoy Code
– Students appeared to
have confidence in their
understanding of:
•
•
•
•
divorce (D1),
mediation (M1)
research (R1)
Enjoyed interdisciplinary
training (IT2)
• Five such summary
codes
CONFIDENCE/
ENJOY CODES
Alpha
0.611
Code
D1
M1
R1
IT2
Alpha if
item
deleted
0.816
0.615
0.285
0.265
Psychology Students:
Changes on Focus Group Summary Codes
Baseline
Semester
(N=1)
Interdisciplinary Semester
(N=1)
Subscale
Pre Training
Post Training
Training Goals
Pre Training
2.18
0.30
---
Confidence/Enjoy
2.00
2.33
2.55
Information from Current
Course
0.50
0.60
2.05
Positives
1.87
1.53
1.50
Info/Enjoy IT
1.00
---
2.20
Note: Baseline semester-- No End of Semester Data (Psychology students hadn’t received
any relevant training that semester).
Only one group (or “subject”) in interdisciplinary semester, so couldn’t do statistical
analyses.
Law Students:
Changes on Focus Group Summary Codes
Baseline Semester
(N=1)
Subscale
Interdisciplinary Semesters
(N=3)
Training Goals
Post
Pre Training
Training
2.10
0.60
2.10
Post
Training
1.25
Confidence/Enjoy
1.73
2.80
1.35
2.57
Information from
Current Course
Positives
0.55
2.65
0.67
2.30
1.53
2.40
1.30
1.95
Info/Enjoy IT
0.60
2.40
1.25
2.65
Pre Training
t(df)
p
17.00(1)
.037*
-10.12(2)
.010*
-9.63(2)
.011*
-2.17(2)
.163
-7.00(1)
.090
Note: Only one group (or “study subject”) in baseline semester, so couldn’t do statistical
analyses.
Summary of
Focus Group Discussion Coding Findings
• Law students:
– Fewer negative attitudes towards research
– More confidence in knowledge of research on divorce
and mediation
– Learned a lot from, and enjoyed, interdisciplinary
training
– Ideas for more interdisciplinary training in the future
• Psychology students:
– More confidence in their understanding of mediation
– Increased knowledge of relevant research
– Enjoyed the interdisciplinary training
2. Focus Group Student Quotes
Law Students:
Similarities Across Baseline and
Interdisciplinary Training Semesters
Law Students,
Start of Baseline Semester
• Want experiential learning
– “The thought of being able to be so hands on
while a law student was really appealing. . .”
– “I’m excited to deal with putting two people who
are having problems in a room and learning how
to deal with this and remain neutral in this
situation . . .”
Law Students,
End of Baseline Semester
• Got experiential learning
– “When you learn how to listen to people the way
you do in mediation . . . the skill applies to other
settings [like] when you are talking to your clients
or when you are preparing for trial.”
– “The clinic is so experiential . . .”
Law Students,
Start of Interdisciplinary Training Semester
• Want experiential learning
– “Practically, this experience will be helpful for
dealing with clients in the future. . .”
– “I would really love to be able to listen to people
and understand where they’re coming from. . .”
Law Students,
End of Interdisciplinary Training Semester
• Got experiential learning
– “I feel like this definitely made me think of my
future career profession differently.”
– “I learned how to transition from advocate to
neutral. It’s a different approach where you shift
your mindset to stay neutral. It was very helpful.”
Law Students,
Start of Baseline Semester
• Concerned about the court system making
custody decisions
– “I think there are a lot of cases where it’s clear,
but in middle road cases I’m not really confident
how the court system can accurately decide which
parent should have the child or primary custody.”
Law Students,
End of Baseline Semester
• Continued concerns about the court system
making custody decisions (time available)
– “I’m more skeptical of decisions made by judges
after the twenty minute hearing and reading the
case file… There have been mediations where I
was thinking one thing for the first two or even
four hours, but after several hours of conversation
you discover these game changing facts. . .”
Law Students,
Start of Interdisciplinary Semester
• Concerns about court system making custody
decisions
– “It seems like mediation would be better than
having some judge say what to do. You’re working
together and you feel like you have more control
over the outcome. It seems to me that they would
be happier. They’re divorced but it’s better than
walking into a combative court proceeding.”
Law Students,
End of Interdisciplinary Semester
• Continued concerns about court system
making custody decisions (mediation better)
– “…When mediation works for a couple, it really
works and the outcomes are really good and you
can tell that’s really going to work for them.”
– “Every couple I had seemed to really like the
process a lot. It seemed really empowering to
them to have some control over decision making.”
Law Students:
Differences Across Baseline and
Interdisciplinary Training Semester
Law Students,
Start of Baseline Semester
• Research: Desire for information (possible
interest in research)
– “I’d like to know how it plays out over time if a
couple who divorces has high or low conflict . . .
How does conflict change over time?”
Law Students,
End of Baseline Semester
• Research: Continued desire for information
(possible interest in research)
– “I want to know more about the child inclusive
mediation process . . . .”
– “I’d like to see a study that compares the long
term durability of agreements reached through a
very directive mediation style versus a more
facilitative mediation style . . . .”
Law Students,
Start of Interdisciplinary Training Semester
• Research: Desire for information (interest in
research)
– “I don’t know anything about research. I only
know stereotypes probably because I’ve heard it
on the news or Maury.”
– “I know almost nothing about research…”
– “Learning how to evaluate interventions based on
psychological research would be helpful...”
Law Students,
End of Interdisciplinary Training Semester
• Frustrations of conducting research
– “I think the most frustrating thing for me was the
random assignment. I just kept on getting divorce
mediation as usual . . .”
– “I never realized how reluctant people are to
participate in research. I was surprised by parents’
resistance.”
Law Students,
End of Interdisciplinary Training Semester
• Positive experiences with research
– “I liked that the psychology department came in and
told us . . . the way the study works and how it is
designed . . . and this is what we’re hoping to
accomplish and yes there are flaws. . . I like that
transparency and that honesty in what we were
doing.”
– “I learned a lot about research on divorce and
mediation. . I find myself referring to information
given by the psychology faculty when I’m talking to my
parents. I tell people about mediation and random
assignment. . .”
– “I learned all my research knowledge from the child
feedback session.”
Law Students,
End of Baseline Semester
• No discussion of interdisciplinary training at
start of semester
• Want interdisciplinary training (didn’t get much)
– “I’d like to hear from psychology faculty a little bit
more . . . .”
– “I think interdisciplinary training was a really good
thing. The day we just sat down with psychology
students (focus group discussion) it was good to see
the difference in views between people.”
– “I’d like to see psychology students mediating with
law students.. . . ”
Law Students,
Start of Interdisciplinary Training Semester
• Excited about upcoming interdisciplinary
training
– “I’m psyched to be working with the
psychology department. It’s very different
from what we do on a day to day basis. We
do custody battles. . .”
– “I’m excited to work together. . .”
Law Students,
End of Interdisciplinary Training Semester
• Liked interdisciplinary training
– “Interdisciplinary training was really helpful. Sometimes
we couldn’t tell what it was because it went hand in hand
so well. . . It was so intertwined that it just flowed. I’m
appreciative to have had it. It definitely helped.”
– “It’s nice to have the child feedback session so that what
you’re thinking in your head comes out through the child
consultants.”
– “Child consultants could get parents focused on the best
interests of [the child]. Recently we did a child inclusive
mediation and the parents . . . were very emotional when
[they learned about] their daughter’s reaction to the
divorce and how their actions [were impacting] their
daughter . . .”
Psychology Students:
Interdisciplinary Training Semesters
(no comparable baseline semester data)
Psychology Students,
Start of Interdisciplinary Training Semester
• Desire to understand the divorce process
– “I want to have a better grasp of the whole
divorce process and how it could affect
children and families. I’m coming in with little
knowledge of this process.”
Psychology Students,
End of Interdisciplinary Training Semester
• Understand divorce process (it is an extended and
complicated process)
– “Something I realized over the course of the
semester… is how complicated it is, how much parents
have on their minds, how much they have to think
about, decisions that have to be made, and how many
things they have to take into account.”
– “ . . . This can be an extended process. It’s not just that
you do mediation and you’re divorced; it can be
months or years. It’s an ongoing developmental
process that changes as the needs of the children and
parents and their willingness or ability to manage all
of the complexities change over that time.”
Psychology Students,
Start of Interdisciplinary Training Semester
• Want to incorporate psychological
information into legal process
– “I want to know practically, what it looks like to
incorporate psychological information into this
process. This is a different approach and different
rules than what we’re used to working with.”
Psychology Students,
End of Interdisciplinary Training Semester
• Want to incorporate psychological
information into legal process continued
(want to do even more and see impact of it)
– “It would be nice to have a sense of our impact.
You present this information with the idea of
motivating them to make good decisions for their
children… but you don’t quite know how that’s
going to play out later . . .”
– “It would be nice to know or see how the parents
are talking about it after having heard the
feedback from the child consultants.”
– “I’m wondering how much the parties will actually
adhere to those agreements...”
Psychology Students,
End of Interdisciplinary Training Semester
• Liked Interdisciplinary training
– “It’s been fun working with people with different
strengths and I think it’s pretty complimentary
honestly. It’s been a nice change.”
– “It’s been really interesting to be involved in
thinking about the practical concerns of how to
implement this intervention. . .”
Focus Group Discussion:
Summary of Student Quotes
• Some similarities in law student experience in
baseline and interdisciplinary training
semesters
• During interdisciplinary training semesters,
law students learned about research and
enjoyed interdisciplinary work
• And the psychology students enjoyed
interdisciplinary work and wanted even more
involvement
II. SOTL Study
D. Coding Student Behavior
During Mediations
Another Attempted Measure:
Student Behavior During Mediations
• In psychology program evaluation research:
– Code what happens during therapy sessions
• Measure of:
– Adherence (fidelity) to treatment manual
• Are therapists doing what they are supposed to do?
• Do different treatment approaches (being compared)
actually differ as they are supposed to?
– Competence
• How well are therapists implementing the intervention?
– For us, also a measure of the effects of teaching and
training
• What are the students actually doing in mediation?
Another Attempted Measure:
Student Behavior During Mediations
• Thus, we planned to code law and psych
student behaviors during:
– CI
– CF
– MAU
– During baseline and interdisciplinary semesters
• Developed coding system of mediator and
child consultant behavior
– Mix of adherence and competence coding
Another Attempted Measure:
Student Behavior During Mediations
• Coding system examples:
– General skills
• listened empathically
• provided examples
– Skills specific to each type of mediation
• CI: Brought in materials completed by child
• CF: Used general developmental research findings to
discuss the children
• Coders:
– Law and psych students
• Didn’t work
Another Attempted Measure:
Student Behavior During Mediations
• Methodological problem:
– In psychology research: record sessions, for later
coding:
• complex human interactions
• can watch and re-watch tape
• can stop tape and discuss codes assigned
• Not able to record mediation sessions
– confidential process
• Tried to code from online, live observation
• Coding system: too complex, too many codes
• Future plans:
– Micro-analytic coding: Fewer codes coded frequently
Student Behavior During Mediation
•
•
•
•
Coder problem:
Failure of interdisciplinary training
Interdisciplinary issue: Judging competence
Law students did not like coding
– Didn’t understand the importance
– Time consuming and not part of what they expected
to do for the course
– Weren’t comfortable judging each other
• Future plans:
– Use psychology student coders and/or train law
students better
II. SOTL Study
E. Student Involvement in Program
Evaluation Research
Outcome: Student Involvement in
Program Evaluation Research
• Psychology students:
– Helping to run all aspects of the study
– Co-authors on multiple conference presentations
and publications
• Law students:
– Involved in parts of the study
• Attempt to code student behavior during mediation
• Coding content of mediation agreements
– Co-authors on conference presentations and
publications (3 law students to date)
Recap of Results Presented
• Various measures/outcomes:
• Student Knowledge Tests
– Law
– Psychology research
– Program evaluation research methods
• Student Focus Group Discussions:
– Coded
– Quotes
• Attempt to Code Student Behavior During Mediation
• Student Involvement in Program Evaluation Research
• Across these measures:
– Convergence of findings
– Generally positive impact of interdisciplinary training
III. Discussion
Discussion: Study Limitations
• Baseline Semester Problems:
– Only one baseline semester
– Baseline and interdisciplinary semester students differed
in previous experience
– Had already begun some interdisciplinary work and
teaching in baseline semester:
• IU conference on law and social science in divorce
• Psychology professors gave guest lectures in law
student course (not on CI and CF, but other topics)
• Started interdisciplinary research on the detection of
violence in mediation cases
• Dilemma: Must establish interdisciplinary team and
collaborative working relationship to plan project but
doing so meant we’d already begun before SOTL
project began
Discussion: Study Limitations
• “Horse already out of the barn” problem:
– Baseline versus interdisciplinary training
semesters:
• Fewer statistically significant differences than expected
– On knowledge test scores
– Focus groups: Even baseline semester students discussed how
much they learned from interdisciplinary training!
• Dilemma:
– Developing interdisciplinary efforts and including
student in exciting new developments versus
conducting a methodologically stronger study
Discussion: Study Limitations
• Related problem: Small number of psychology
students
– Baseline semester:
• No post-semester data from psych students
• Inability to directly compare baseline and interdisciplinary
semesters
– Interdisciplinary training semesters:
• Only one group of psychology students (versus 3 groups of law
students)
• N = 8 psychology students in interdisciplinary semester
• Lack of statistical power to detect changes in psych students
• Due to psychology training program (small size)
• Future– Need more psych students:
– Continue study in future, with new psych students
– Conduct study at other universities
– Conduct study with other types of social service students
(e.g., social work or counseling psychology graduate
students)
Discussion: Findings
• Despite these problems…
• On knowledge test:
– Law students:
• Baseline semester: increased only scores on law section
• Interdisciplinary semesters: also increased scores on both
psychology sections
– Psych students:
• Interdisciplinary semesters: Increased scores on both law
and psychology sections of knowledge test
• During Focus Group Discussions:
– Both groups reported enjoying interdisciplinary
training, etc.
• Coding Data
• Student Quotes
• Both groups involved in CIMS program evaluation
research study and resulting publications
Discussion: Law vs Psych Students
• Overall, appears to have been bigger impact
on law students than psych students.
• Possible reasons:
– Small sample size of psych students (no stat
power)
– Law students more motivated
• More likely to go into practice (more directly relevant)
– One law professor (less time in psych class) vs two
psych professors (more time in law class)
• But law professor on-site and provided feedback to
psych student child consultants
– Most interactions at law school clinic (not psych
building), so may have seemed more integrated
into law training
Discussion: Law vs Psych Students
• Overall, appears to have been bigger impact on
law students than psych students. Possible
reasons (cont’d):
– Law students attended CI and CF parent feedback
sessions (held by psych students)
• Continued exposure to discussion of psychology research on
effects of divorce
– Psych students did not attend mediation negotiations
and agreements
• Did not have continued exposure to discussion of legal issues
• Not integrated into entire process
Challenges of
Interdisciplinary Training and Research
• General interdisciplinary issues (discussed before):
– Speaking different languages
• Interdisciplinary issues with conducting program
evaluation research methods (discussed before):
– Protect confidentiality of mediation process
• Law students’ perception of burden on mediation process:
– Research process slows down mediation process
– Extra time and effort for law students
• Psychology students perceptions:
– Didn’t always feel equal “ownership” in process (conducted
at law school)
– Carried more of the “burden” of conducting the research
Benefits
of Interdisciplinary Training and Research for
Both Groups of Students
• Changes in clinical practice at the IU Mediation Clinic –
substance and process
• Increased focus on process for parents and outcomes for
parents and children
• Teacher/student/practitioner development – experience
through another lens
• Research informing the teaching and practice
• Teaching and practice informing the research
• Opportunity to do research in the real world
• Overall: The Interplay of Teaching, Learning, and
Research: Graduate Student Collaborations in Law and
Psychology
– Worth it
– Continuing with more new interdisciplinary projects
Thank you!
• Questions?
• More information or future questions:
– holtzwor@indiana.edu (Amy Holtzworth-Munroe)
– aga@indiana.edu (Amy G. Applegate)
Additional tables, if needed
PRE-TRAINING (START OF SEMESTER):
Knowledge Test Scores of Law vs Psychology Students
Question Type
(total points
possible)
Baseline
Law
Baseline
Psych
Interdisciplinary
Law
Pre
Training
Interdisciplinary
Psych
Pre
Training
16.12
13.25
2.14(31)
.040*
Pre
Training
Pre
Training
t(df)
p
Law (31)
18.13
10.82
4.19(17)
.001*
PERM (12)
4.75
8.36
-3.21(17) 4.76
.005*
8.25
-3.85(31)
.004*
PRD (17)
11.06
8.86
1.69(17)
.109
10.50
-.998(31)
.326
9.80
t(df)
p
PERM (Program Evaluation Research Methods); PRD (Psychology Research on Divorce); *p < 0.05
Both Baseline and Interdisciplinary Semesters, at start of semesters:
-Law Students had more knowledge of Law
-Psych Students had more knowledge of Program Evaluation Research
Methods
-No statistically significant difference between Baseline and Interdisciplinary
Semesters (despite differences in previous experiences)
Interdisciplinary Semesters
Knowledge Tests: End of Semester
Question Type
(total points possible)
Student Type
Mean
t(df)
p
Law Knowledge
(31)
Law
21.9
.532(30)
.000*
Psychology
16.14
Law
5.52
Psychology
9.57
Law
11.02
Psychology
11.07
PERM
(12)
PRD
(17)
PERM (Program Evaluation Research Methods); PRD (Psychology Research on Divorce); *p<.05
Law Students still know more law.
Psych Students still know more program evaluation.
No differences in psychology research on divorce.
-6.20(30)
.000*
-.062(30)
.951
Individual Codes: Law Student Changes (3 Interdisciplinary Training Semesters)
Code
Pre or Post
Mean
t(df)
p
M1- Confidence in
understanding of mediation
Pre
1.27
-11.50(2)
.007*
Post
2.80
Pre
0.20
Post
2.53
Pre
2.47
Post
1.13
Pre
0.80
Post
2.00
Pre
0.60
Post
1.73
Pre
0.70
Post
2.60
M5- Current course as a source
of knowledge about mediation
M14- Expressed negative feelings
about mediation
R1- Confidence in understanding
of research
R5- Current course as a source of
knowledge about research
IT2- Enjoyed interdisciplinary
training
6 codes significant; Could not calculate for psychology (n=1 group)
-13.23(2)
.006*
7.56(2)
.017*
-10.40(2)
.009*
-6.43(2)
.023*
-19.00(1)
.033*