National   Poverty   Center
Gerald   R.
Ford   School   of   Public   Policy,   University   of   Michigan   www.npc.umich.edu
This   paper   was   delivered   at   a   National   Poverty   Center   conference.
Any   opinions,   findings,   conclusions,   or   recommendations   expressed   in   this   material   are   those   of   the   author(s)   and   do   not   necessarily   reflect   the   view   of   the   National   Poverty   Center   or   any   sponsoring   agency.

DRAFT – DO NOT CITE OR DISTRIBUTE
By Matthew Stagner, Jane Reardon-Anderson, Jennifer Ehrle,
Katherine Kortenkamp, and Sara Bernstein
The Urban Institute, 2100 M Street, NW, Washington, DC 20037
Contact: mstagner@ui.urban.org
Presented at: Marriage and Family Formation Among Low-Income
Couples: What Do We Know From Research?
Sponsored by the National Poverty Center
University of Michigan
Gerald R. Ford School of Public Policy
September 5, 2003
Washington, DC
Funding for this research provided by the U.S. Department of Health and Human Services,
Administration for Children and Families, Office of Planning, Research and Evaluation.
1

DRAFT – DO NOT CITE OR DISTRIBUTE
Background
This paper presents progress to date on a review examining how marriage and relationship programs—defined as programs to improve the relationship between two people involved romantically—affect the quality of couples’ relationships. This is an important issue given changes in the past several decades in marriage and divorce rates, the rise of single parenting, and the effects these changes may have on children and adults. Below, we briefly review these changes and their effects, describe several interventions that address the problem, and discuss the past research on these interventions as well as the work we propose for this systematic review.
Family structure in the United States changed rapidly in the second half of the twentieth century. The two-parent family norm has been increasingly replaced by a wide variety of family forms. In 2001, 69 percent of children lived in two-parent families, down from 77 percent in 1980 (Federal Interagency Forum on Child and Family Statistics 2002).
Divorce is common. About half of all recent first marriages are expected to end in divorce (Ooms 2002). Of children born into two-parent families, 34 percent will experience a disruption of their parents’ union by age 16. One-third of all births are outof-wedlock. And couples opting to cohabitate rather than marry is becoming an increasingly common phenomenon. Forty percent of non- marital births occur within cohabiting unions (Bumpass and Lu 2000).
Numerous studies indicate that children growing up in single-parent families experience worse outcomes than children growing up in two-parent families (Acs and Nelson 2001;
Amato and Keith 1991; McLanahan and Sandefeur 1994; Wu and Martinson 1993). And many studies show that divorce, specifically, can have negative effects on children’s well-being (Amato 1993; Amato and Keith 1991; Chase-Lansdale, Cherlin, and Kiernan
1995; Chase-Lansdale and Hetherington 1990). Even when parents remarry, an analysis of the research suggests that this does not necessarily appear to improve outcomes
(Amato 1993).
Recent research also suggests a possible relationship between marriage and positive outcomes for adults. Married couples build more wealth on average than singles or cohabiting couples, while divorce and unmarried childbearing increase the risk of poverty for children and mothers (Lupton and Smith 2002). Individuals who are married have better health and longer life expectancies than similar singles (Lillard and Waite 1995).
Married mothers have lower rates of depression than cohabiting or single mothers
(Brown 2000). Research also shows that, on average, unhappily married adults who divorce or separate are no happier than unhappily married adults who stay together
(Waite et al. 2002). Given this research, policymakers want to understand what is known about the effectiveness of interventions to improve marriage and prevent divorce, particularly for low- income families who may not be able to afford such services.
2

DRAFT – DO NOT CITE OR DISTRIBUTE
Interventions and Past Research
A variety of programs aimed at improving relationships and marriages currently exist.
They frequently develop from government, research, or faith-based initiatives, and may operate privately for profit. They are often housed in health centers, community centers, public agencies, churches, or education settings. They include topics in their curriculum such as communication, parenting, or finances. The formats utilized may be instructive or group-oriented, and programs may operate with different group sizes and treatment dosage amounts. The programs reach many populations, including individuals (youth, fathers, mothers), couples (premarital, married), and families.
Experts suggest that while the interpretation of research on the outcomes of these programs is complex, generally there is promising evidence that they can provide significant benefits to their clientele (Stanley, Markman, and Jenkins 2002). According to Stanley, Markman, and Jenkins, couples can learn to reduce patterns of negative interaction and maintain higher levels of relationship satisfaction. They note that in some studies, higher risk couples show the strongest effects, and some research has found that beneficial effects appear to last up to five years after training.
Many reviews of program evaluations have examined the effects of marriage programs.
Some use a narrative format that presents an author’s summary of a group of studies.
One such narrative review examining relationship satisfaction rates concluded that marital and family enrichment programs produce positive results, leading to significant improvements in premarital, marital, and family capabilities (Guerney and Maxson
1990).
Several reviews have examined the effects of marriage programs using statistical metaanalysis to synthesize findings. Most recently Carroll and Doherty (2003) reviewed 25 studies of premarital programs and found them to be generally effective in producing immediate and short term gains in interpersonal skills and relationship quality. Another review examined the effectiveness of two types of therapy programs—behavioral marital therapy and behavioral premarital intervention programs—and found that both were more effective than no treatment (Hahlweg and& Markman 1988). Another meta-analysis examined 85 studies of premarital, ma rital, and family enrichment programs, and also concluded that these programs were effective (Giblin et al. 1985). A meta-analysis of 16 studies examining the Couples Communication program concluded that positive benefits for couples could be anticipated from the program (Butler 1999). Finally, one metaanalysis examined 71 studies on marital psychotherapy and concluded that couples who took marital counseling were better off than 70% of those that did not (Shadish 1993).
The Need for a Systematic Review
The current review is a systematic review of evaluations of all types of marriage and relationship programs, drawing on research since 1960. It will determine an average impact of the studies on such measures as satisfaction, relationship quality, and
3

DRAFT – DO NOT CITE OR DISTRIBUTE communication by using meta-analytic techniques. It differs from the other reviews in three main ways.
First, it summarizes the studies using meta-analytic techniques. In a meta-analytic review, each study is represented by a data point and the review synthesizes all of the data points to produce an average impact of the programs on selected outcome measures.
This ensures that the outcome of the review is based purely on quantitative results extracted from each study.
The second way that this review differs from others is that it has a very broad focus in terms of the intervention. It includes evaluations of many different types of marriage programs: therapy programs, enrichment programs, education programs, and so on.
Third, this review is a systematic review. What distinguishes a systematic review is that the author must state exactly how the review will be conducted in a protocol before beginning the work. The author must draw a very specific plan for how he or she will locate the studies that will be synthesized, and this search must be comprehensive. He must also state how he intends to extract data and synthesize the results, and how he will adjust for potential complications arising from any of these three stages in the process.
The reasons fo r doing a systematic review are to eliminate the possibility of any bias in the outcome of the review and to ensure the review is completely transparent. It should be possible for another researcher to perfectly replicate the review by following the author's protocol. This process is intended to minimize both "publication bias" as well as the reviewer’s possible personal biases.
The current review, because it is systematic, comprehensive in scope (covers all types of programs), and uses meta-analytic techniques, will present an unbiased synthesis of the results of all high quality evaluations of interventions covering the broadest spectrum of marriage and relationship programs.
Our study is currently under review by the Campell Collaboration for approval as a registered review. A group of researchers founded the Campbell Collaboration in
February 2000 to manage the production of systematic reviews conducted on policy topics in education, crime and justice, and social welfare. It is based on the model of the
Cochrane Collaboration, which prepares and maintains systematic reviews of the effects of interventions in health care. The main goal of the Campbell Collaboration is to help people make well- informed decisions about the effects of interventions by promoting high-quality research and minimizing bias in reviews of research. It also aims to foster communication among reviewers and promote access to reviews. Campbell’s systematic reviews are published electronically so that they can be updated continuous ly as the research changes.
Search Process
4

DRAFT – DO NOT CITE OR DISTRIBUTE
The search strategy for this review identifies experimental and quasi-experimental evaluations of marriage and relationship programs conducted since 1960. All studies included in the current review must meet three main criteria:
•
Studies must have a control group that receives no treatment.
•
The control group must be created in one of two ways: o by random assignment, or o by using a statistical method to correct for differences between the control and treatment groups if the groups were not randomly assigned.
•
The program must focus primarily on relationship improvement.
The search strategy includes academic databases of published literature, internet searches for published and unpublished research, manual searches of journals and books, and professional contacts (experts in the field of marriage and relationship programs). The database and internet searches used the following key words: sample words—marriage, marital, premarital, relationship, couple, premarriage, newlywed; and program words— program, satisfaction, quality, stability, enrichment, education, therapy, counseling, learning, outcome, communication, treatment. The sample words and program words were each listed using a series of "OR" statements, with an "AND" statement in the middle linking the two groups of words. If there were more than 2,000 abstracts retrieved from a search, reviewers added four "study" words to the search string: evaluation, impact, experiment, and intervention. Two reviewers retrieved abstracts using this method, screened them, and dropped only those that clearly did not meet the three inclusion criteria mentioned above. Concerns about the quality of studies were not addressed at this stage.
Results of the Search Process
The search yielded 12,832 abstracts: 6,037 from academic databases, and 6,795 from internet sites. Most of these articles and reports did not include empirical impact findings on an intervention of interest. Reviewers retained 394 studies from the databases and 228 studies from the internet, for a combined total of 622 studies. All duplicates were then deleted and some studies were added from either the manual searches of journals or previous reviews of marriage programs. Four hundred seventy- five studies were left at the end of this phase of the review.
Results of the Coding Process So Far
Reviewers pulled or put in orders through inter- library loan for all 475 studies. Upon screening the full text versions of these studies, many were found to not meet the inclusion criteria and some were found to be duplicates. Also, all 475 studies have not
5

DRAFT – DO NOT CITE OR DISTRIBUTE yet been screened, since some must still be retrieved through inter- library loan. Of the
337 studies retrieved at this point, reviewers have dropped 267 studies. The remaining 70 studies have been coded for very basic information. The results of this initial coding phase for these 70 studies are described below.
Of the 70 studies, 51 are from the United States, 14 from Canada, 2 from Germany, 2 from the Netherlands, and 1 from Australia. The participants in all but one of the studies are couples. The non-couple program was designed for women only. The average number of participants per study was 90 individuals (or 45 couples).
Forty-four studies targeted married couples specifically. Six studies included a mix of married and cohabiting couples, and eleven studies focused on premarital couples (i.e., couples intending to marry). One study included couples that were dating and eight studies did not specify what types of couple their program targeted.
The types of programs evaluated all had relationship improvement as a primary component of the intervention. Yet given the variety of programs, reviewers classified programs into the following intervention types (see figure 1). These classifications may be modified once the reviewers code all of the studies and assess the different types of interventions.
•
Therapy programs (23 studies) use a therapist with either one-on-one sessions for couples or a group couple format. Couples utilizing therapy are usually distressed and the therapy is tailored to address the problems couples are having.
•
Marriage preparation programs (6 studies) provide knowledge and skill-based training to help couples sustain and improve their relationship once they are married (Senediak, 1990 in Carroll and Doherty, 2003). These programs utilize a prevention perspective and seek to help nondistressed couples maintain their relatively high levels of functioning (Markman & Hahlweg, 1993).
•
Enhancement, enrichment, or encounter programs (14 studies) tend to be of shorter lengths and use a variety of formats that vary by provider. Religious groups offer many of these programs. They are often held as weekend retreats and are usually for nondistressed couples.
•
Communication skills building or education classes (24 studies) emphasize skillbuilding in areas like communication or parenting. Couples practice role-playing and learn communication techniques. Couples using these programs can be distressed or non-distressed.
•
Other (3 studies): there was also an alcohol treatment program, one self- initiated intervention program (couples were instructed to do “fun things” together as a way to help their relationship), and one program that focused on “community building” through relationship enhancement.
6

DRAFT – DO NOT CITE OR DISTRIBUTE
The average number of sessions per program was 8.7, and the average length of each program was 7.6 weeks. Studies gathered data on many different types of outcomes, but most frequent were relationship satisfaction, communication, and adjustment. In most studies, standardized scales were used to measure these outcomes. Some examples of the scales most frequently used include the Dyadic Adjustment Scale (DAS) and the Locke-
Wallace Marital Adjustment Test (MAT) for marital adjustment. Sixty- five studies used at least one of these types of scales to record outcomes. Twenty- five of these created their own scales or method of collecting relationship measures in addition to the scales.
Five studies used only their own measures.
Figure 1: Program types for 70 studies
Other
4%
Communication Skills Building or Education
34% Therapy
33%
Enhancement, Enrichment, or
Encounter
20%
Marriage Preparation
9%
Most of the studies collected (57) used random assignment and none of these appear to have major quality concerns relating to their design. Only one study used a quasiexperimental design. Twelve studies attempted to use random assignment but did not fully succeed. These studies were thus coded as non-random assignment. If reviewers find during the detailed coding phase (see next steps) that the researchers controlled for differences between the treatment and control groups, they will record the study as having a higher quality non-random design. If they find, however, that researchers did not control for differences, the study will be excluded from the review.
Next Steps
Reviewers will continue screening the remaining studies from the group of 475 as they become available. They will also begin coding studies that meet inclusion criteria in greater detail. They will record various characteristics of each study, including information on the sample participants (sex, age, race, education level, income, and geography), intervention (program type, duration, frequency, and size) and study (time from intervention to follow-up, mode of follow-up, and year of evaluation). Reviewers
7

DRAFT – DO NOT CITE OR DISTRIBUTE will also assess quality at this stage. Quality criteria will include randomization procedures, allocation concealment, sample selection bias, sample size, treatment and control group differences at baseline, control group contamination, attrition rates, and evaluator conflict of interest. For each of these potential quality concerns reviewers will code studies as either having a problem, not having a problem, or not presenting enough information to judge. Reviewers will use these criteria to develop a system for deciding what studies to eliminate based on quality concerns.
Once detailed coding is complete, reviewers will have to decide how to synthesize the findings. Because these studies vary in terms of intervention type, outcome measures, and design reviewers will probably not group them into one analysis. Rather, they will likely synthesize results by group. Ideally, reviewers will group all similar interventions together (e.g., therapy, marriage preparation, enrichment, and education), and within each group combine studies with similar outcome measures. Reviewers will also group studies by their design (e.g., all random assignment studies will be analyzed together).
Yet once all studies are coded, reviewers might discover additional ways of grouping studies. For example, several interventions of different types may use the same curriculum and reviewers may choose to estimate an effect size for all interventions using that curriculum.
Reviewers will create an average effect size for each group of studies on selected outcome measures. An effect size is a measure that is created around the number "1" that indicates whether or not the average program is associated with a higher incidence of an outcome or a lower incidence of an outcome. An effect size of greater than 1 indicates that at follow- up, the treatment group was more likely to exhibit the outcome than the control group. If the studies permit (i.e., if they report both pre- and post-test outcomes), reviewers will also assess the impact of the programs on behavior changes over time. If studies in a group vary significantly in the types of outcomes measures used, reviewers will have to address whether and how to best group these studies.
Reviewers will assign a weight to each study in the review. The weight will correspond to the study’s sample size so that studies with large sample sizes will carry more weight than studies with small sample sizes.
In addition to the general impact analyses, reviewers will also examine the effect of different moderator variables on the size and direction of each impact. In other words, reviewers will test to see how the presence of different characteristics in the studies affect the impact. Reviewers will likely consider characteristics collected in the detailed coding phase of the project, like demographic characteristics of the sample participants, as moderators. Reviewers will determine the influence of these moderators using regression models. The number of moderators that can be tested will depend on the sample size, i.e., the number of studies obtained. Which moderators reviewers ultimately choose will depend on their variability across studies.
Conclusion
8

DRAFT – DO NOT CITE OR DISTRIBUTE
A systematic review of this nature will inform policy audiences of the effectiveness of marriage and relationship programs. This review is different from other reviews in that it involves a comprehensive search on a wide variety of program types, and it will use meta-analytic techniques to synthesize all the high-quality evaluations that are available on this topic. Readers of the review are thus assured that all evaluations that meet the inclusion criteria are presented and synthesized in a way that is data-driven and transparent to minimize any possible bias in the outcome. For these reasons, readers will be getting the "best" possible answer to the question posed.
Yet there are several challenges worth noting related to conducting a review and using the results to answer policy questions. For one, reviewers confront various complications when extracting data and preparing data for analysis. For example, reviewers have to decide how to handle missing data. For this review, reviewers will contact primary authors to retrieve any unpublished, but available, results and that can then be added to the analysis. If the data that are critical to the analysis cannot be retrieved (e.g., an outcome measure) then the study will be dropped from the analysis of that outcome or from the entire review. If the missing data are not as critical (i.e., a certain moderator variable), the study will simply be excluded from the analysis of that moderator.
Reviewers will also confront challenges when deciding how to group interventions into distinct program categories. Descriptions of programs can be very unclear, and not all researchers and practitioners use the same terminology. Because of this, programs with different-sounding titles may have program components that appear very similar.
Likewise, programs with similar-sounding titles may have very different program components.
It is also difficult to compare outcome measures across studies. Most of the outcomes analyzed in this review will be in scale form (e.g., couples answer questions on a topic like communication, and they are each given a number according to their level of satisfaction or skill). It may be difficult for reviewers to compare these scales across studies. Reviewers may have to standardize these measures across studies so that the studies can be compared in a single analysis. For similar reasons, researchers may have a difficult time coding moderator variables for each study.
Another difficulty may arise from the fact that primary researchers differ in the type of data they present and in the way that they present it. For instance, studies may differ in the types of demographic data that they present for their sample. Or they may record certain characteristics of the sample differently (e.g., "married" versus "married and living with spouse").
Assessing quality is also a significant and rather complex task. Evaluations of marriage and relationship programs tend to be fairly small in scale, and vary greatly in the types of researchers who evaluate them, the practitioners who run them, and the levels of funding available to support them. For these reasons, it is likely that quality will not be consistent across studies. Reviewers will have to make careful choices when deciding what
9

DRAFT – DO NOT CITE OR DISTRIBUTE constitutes a "high quality" study, as those decisions determine what studies will be included in the analysis.
Aside from these methodological concerns, the reviewers must consider the more fundamental challenge of matching the review to the policy question. For example, policymakers currently want to know the impact of marriage and relationship programs, especially on low- income populations. There are two obstacles to answering this question using this type of review. First, this review will assess the average impact of all evaluations of interventions, not interventions themselves. This means that the impact found by the review will be biased towards the effect found by interventions that were evaluated versus those that were not evaluated. Only programs that were evaluated—and evaluated well—will be included in the analysis. Programs that are rigorously evaluated may be more likely to have more funding available or be led by more experienced practitioners when compared to programs that are evaluated poorly or never evaluated at all.
The second obstacle is that most of these programs (certainly those that have been evaluated and probably also programs that exist in general) do not target a low- income population. Most of them do not specifically include low- income participants. This will make it difficult for policymakers to generalize the findings from this review to the lowincome population.
Despite the challenges that are likely to be posed by difficulties synthesizing primary data and matching the review to the question posed by policymakers, the review itself will be a presentation of an unbiased, data-driven analysis of the impact of marriage and relationship programs on the relationship satisfaction of study participants.
10

DRAFT – DO NOT CITE OR DISTRIBUTE
References
Acs, G. and Nelson, S. (2001). “Honey, I’m home.” Changes in living arrangements in the late 1990s (Brief No. B-38). Washington, D.C.: The Urban Institute.
Amato, P.R. (1993). “Children’s adjustment to divorce: theories, hypotheses, and empirical support.” Journal of Marriage and the Family 55: 23-38.
Amato, P.R. and Keith, B. (1991). “Parental Divorce and the Well-being of Children: A
Meta-analysis.” Psychological Bulletin 111(1): 26-46.
Brown, S.L. (2000). "The Effect of Union Type on Psychological Well- Being:
Depression Among Cohabitors versus Marrieds." Journal of Health and Social Behavior
41: 241-255.
Bumpass, L. and Lu, H. (2000). “Trends in cohabitation and implications for children’s family contexts in the United States” Population Studies 54: 29-41.
Butler, M.H. and Wampler, K.S. (1999). “A Meta-Analytic Update of Research on the
Couple Communication Program.” American Journal of Family Therapy 27(3): 223.
Carroll, J.S. and Doherty, W.J. (2003). “Evaluating the Effectiveness of Premarital
Prevention Programs: A Meta-Analytic Review of Outcome Research.” Family
Relations , 52: 105-118.
Chase-Lansdale, L.P., Cherlin, A.J., and Kiernan, K.E. (1995). “The Long-Term Effects of Parental Divorce on the Mental Health of Young Adults: A Developmental
Perspective.” Child Development 66: 1614-1634.
Chase-Lansdale, L.P. and Hetherington, E.M. (1990). “The Impact of Divorce on Life-
Span Development: Short and Long term Effects.” In Life-Span Development and
Behavior , Volume 10, edited by Paul B. Bates, David L. Featherman, and Richard M.
Lerner. Hillsdale, N.J.
Federal Interagency Forum on Child and Family Statistics. (2002). “America’s Children:
Key National Indicators of Well-Being, 2002.” Washington, D.C.: U.S. Government
Printing Office.
Ford, P. (2002). “In Europe, Marriage is Back.” The Christian Science Monitor .
Giblin, P., Sprenkle, D.H., and Sheehan, R. (1985). “Enrichment
Outcome Research: A Meta-Analysis of Premarital, Marital and Family Interventions.”
Journal of Marital & Family Therapy 11(3): 257-71.
11

DRAFT – DO NOT CITE OR DISTRIBUTE
Guerney, B. and Maxson, P. (1990). “Marital and Family Enrichment Research: A
Decade Review and Look Ahead.” Journal of Marriage and the Family 52(4), Family
Research in the 1980s: The Decade in Review.
Hahlweg, K. and Markman, H.J. (1988). “Effectiveness of Behavioral Marital Therapy:
Empirical Studies of Behavioral Techniques in Preventing and Alleviating Marital
Distress.” Journal of Consulting and Clinical Psychology 56: 4400-477.
Hao, L. (1996). “Family Structure, Private Transfers, and the Economic Well- Being of
Families with Children.” Social Forces 75: 269-22.
Lillard, L.A. and Waite, L.J. (1995). “Til Death Do Us Part: Marital Disruption and
Mortality.” American Journal of Sociology 100: 1131-56
Lupton, J. and Smith, J.P. (2002). “Marriage, Assets and Savings.” In Shoshana
Grossbard- Schedtman (ed.) Marriage and the Economy (Cambridge: Cambridge
University Press)
McLanahan, S. and Sandefeur, G. (1994). Growing Up with a Single Parent, What Hurts,
What Helps. Cambridge, Mass.
Ooms, T. (2002). “Marriage and Government: Strange Bedfellows?” Center for Law and
Social Policy, Couples and Marriage Series, Policy Brief 1. Washington D.C.
Ross, C.E. et al. (1990). “The Impact of the Family on Health: Decade in Review.”
Journal of Marriage and the Family 52: 1059-1078.
Shadish, W.R. (1993). “Effects of Family and Marital Psychotherapies: A
Meta-Analysis.” Journal of Consulting & Clinical Psychology 61(6): 992-
1002.
Stanley, S.M., Markman, H.J., and Jenkins, N.H. (2002). “Marriage Education and
Government Policy: Helping Couples Who Choose Marriage Achieve Success.” From www.smartmarriages.com on 8/30/02.
Waite, L., Browning, D., Doherty, W.J., Gallagher, M., Luo, Y., & Stanley, S.M. (2002).
“Does Divorce Make People Happy? Findings from a Study of Unhappy Marriages”
Institute for American Values, New York.
Wilmoth, J. (1988). “The Timing of Marital Events Over the Life-Course and Pre-
Retirement Wealth Outcomes.” Paper presented at the annual meeting of the Population
Association of America, Chicago, Ill. (April).
Wu, L.L. and Martinson, B.C. (1993). “Family structure and the risk of premarital birth.”
American Sociological Review 58: 210-232.
12