Eileen Gambrill
School of Social Welfare
University of California at Berkeley
Presented July 14, 2005 Leadership Symposium on Evidence-based practice in the human services, Sacramento, California

1.
To describe the philosophy and process of evidence based practice and policy as presented in original sources (e.g. Sackett et al, 1997, 2000; Gray, 1997,
2001).
This is a new educational and practice paradigm for closing the gaps between research and practice in order to maximize opportunities to help clients and avoid harm. It attends to ethical, evidentiary and application concerns. It has not “been around for decades” as suggested in Guide for Child Welfare
Administrators on Evidence Based Practice.(2005)
Chadwick Center, Children’s Hospital of San Diego, p. 5.
2.
Describe the origins of evidence-based practice.
Gambrill (2005)

3. Suggest choices ahead for the state of California regarding what view of evidence-based practice and policy to use.
• Will we take advantage of the new and evolving paradigm of
EBP described by its originators that:
– is systemic; it has implications for educators and researchers as well as for administrators, line staff and clients
– attends to ethical issues such as informed consent
– attends to application issues
– offers a way to handle uncertainty in making decisions in an honest way (sharing ignorance as well as knowledge; there is a willingness to say “I don’t know)
Gambrill (2005)

• Will we define this narrowly as basing decisions on practice related research; using practice guidelines.
This view leaves out the role of clinical expertise, attention to client values and preferences, and application problems.
• Will we define this as business as usual: use the name but not the substance.
Gambrill (2005)

• Beneficence
• Avoid Harm
• Informed Consent
• Autonomy; Self-determination
Gambrill (2005)

EBP “is the integration of best research evidence with clinical expertise and [client] values (Sackett, Straus, Richardson,
Rosenberg & Haynes, 2000, p.1).
It is the conscientious, explicit and judicious use of current best evidence in making decision about the care of individual
[clients]” (Sackett, et al.,1997, p.2).
Evidence-based health care refers to “use of best current knowledge in decision making about groups and populations”
(Gray, 2001).
The origin of the philosophy and process of EBP is not the
Institute of Medicine (IOM). (See for example Sackett et al.,
1997, 2002; Gray, 1997, 2001).
Gambrill (2005)

Client Characteristics and Circumstances
Clinical Expertise
Client preferences and actions Research Evidence
Haynes, R., Devereaux, P. & Guyatt, G. (2002). Editorial: Clinical expertise in the era of evidence-
Based medicine and patient choice. ACP Journal Club, 136: A-11-14.
Gambrill (2005)

Clinical expertise includes use of effective relationship skills and the experience of individual helpers to rapidly identify each client’s unique circumstances, characteristics and “their individual risks and benefits of potential interventions and their personal values and expectations” (p.1). It is drawn on to integrate information from these varied sources (Haynes,
Devereaux & Guyatt, 2002).
Client values refer to “unique preferences, concerns and expectations each [client] brings to an…encounter and which must be integrated into…decisions if they are to serve the
[client]” (p.1).
Gambrill (2005)

1.
Convert information needs related to practice decisions into answerable questions
2.
Track down, with maximum efficiency, the best evidence with which to answer them
3.
Critically appraise that evidence for its validity, impact
(size of effect) and applicability (usefulness in practice)
4.
Apply the results of this appraisal to practice/policy decisions. This involves deciding whether evidence found (if any) applies to the decision at hand (e.g., is a client similar to those studied, is there access to services described) and considering client values and preferences in making decisions as well as other application concerns
5.
Evaluate our effectiveness and efficiency in carrying out steps 1-4 and seek ways to improve them in the future
(Sackett et al., 2000, pp.3-4).

• Effectiveness: In elderly clients who are depressed, what method is most effective in reducing depression?
• Prevention: In poor, inner-city children, are head start programs effective in decreasing school drop out?
• Risk/prognosis: In children who are abused, are actuarial or consensus risk assessment measures most predictive of future abuse?
• Description/Assessment: In families in which there is parent-child conflict, is self report or observation most accurate in describing interaction patterns?
Gambrill (2005)

• Other kinds of questions include harm, cost and selfdevelopment.
• Different questions require different kinds of research methods to critically appraise proposed assumptions.
These differences are reflected in the use of different
“quality filters” to search for and appraise related research (e.g. see Gibbs, 2003; Sackett et al, 2000).
Gambrill (2005)

Critical Thinking Values: Integral to EBP
• Courage: Critically appraise claims regardless of negative reactions
• Curiosity: An interest in deep understanding and learning
• Intellectual empathy: Accurately understanding and presenting the views of others
• Humility: Awareness of the limits of knowledge including our own; lack of arrogance (e.g. promoting bogus claims of effectiveness)
• Integrity: Honoring the same standards of evidence to which we hold others
• Persistence: Willingness to struggle with confusion and unsettled questions
Based on R. Paul (1993). Critical Thinking: What every person needs to survive in a
Rapidly changing world (3 rd Ed.) Foundation for Critical Thinking.www.critical thinking.org
Gambrill (2005)

• Evidence cart (Strauss and Sackett, 1998)
• Knowledge manager (resource person)
(Gray, 1998)
• Decision aids (e.g. computer based, client informed choice form)
• Systematic Reviews: Cochrane and
Campbell databases
Gambrill (2005)

Evidence-based practice:
• Is an alternative to authority-based practice.
Appeals to:
A famous person
Popularity
Tradition
Consensus
Gambrill (2005)
“If Freud said it,it must be true.”
“Eighty percent of social workers use…I’m going to use it too.”
“That’s the way we have always done it.”
“We all believe that ____”

• It is as much about the ethical obligations of educators and researchers to be honest brokers of knowledge and ignorance as it is about the obligations of practitioners and administrators.
• Many kinds of “evidence” may be considered in deciding what outcomes to pursue and what services to offer including:1) client preferences, 2) evidentiary status of different services, 3) resources available, 4) cost, 5) legal regulations, 6) codes of ethics
Gambrill (2005)

Implications of Evidence-Based Practice
• Move away from authority-based decision making
(appeal to tradition, consensus, popularity, status).
• Honor ethical obligations to clients such as informed consent.
• Make practices and policies and their outcomes transparent.
• Attend to application problems: Encourage a systemic approach to improving services.
• Maximize knowledge flow: Honest brokering of knowledge: Sharing ignorance and uncertainty as well as knowledge.
Gambrill (2005)

Origins of Evidence-Based Decision Making*
1.
Study of variations in service delivery and clinical practice.
2.
Identification of gaps that show that people were not acting systematically or promptly on research findings.
•
Failure to start services that did more good than harm at reasonable cost.
•
Failure to stop services shown to be of little value.
3. Economic pressures.
4. Key individuals (e.g., David Sackett)
5. The knowledge revolution:
• Increased recognition of harmful side-effects of health care
•
Increased recognition of misplaced faith in adequacy of peer review
•
Flaws in books and editorials
•
The evolution of the systematic review and web based updates
Gambrill (2005)

Origins of Evidence-Based Decision
Making*, cont.
6. The web revolution.
7. Appeal to clinicians (breaking down the division between research and practice-emphasizing that clinicians have to use their judgment and scientific training to interpret and integrate guidelines).
8. The value of EBP to clients.
*Source: Gray, J.A.M.(2001). Evidence-based medicine for professionals. In A. Edwards and G. Elwyn (Eds.), Evidence-based patient choice: Inevitable or impossible?
(pp. 19-33). New York: Oxford.
Gambrill (2005)

•
Whether to draw on the broad philosophy and evolving process of EBP described by its originators
(e.g. Sackett et al, 1997, 2000) as a way to handle the uncertainty in making decisions in an informed, honest manner (sharing ignorance as well as knowledge).
•
This choice has implications for researchers and educators as well as for clients, clinicians and administrators. It will affect all of the following choices
Gambrill (2005)

1. Who will select the questions on which research efforts are focused?
–
An elite? (a state or national board)
–
Administrators?
–
Clients and line staff (They know what questions arise most often in everyday practice; lets gather this information).
2. What criteria will be used to select services?
–
Consensus (e.g. opinions of a state advisory body)
–
Popularity
–
What is available
– Evidentiary (shown via critical tests to do more good than harm)
–
Cost/benefit ratio
–
Ethical concerns
Gambrill (2005)

3. How transparent (honest) to be regarding the evidentiary status of services, (for example acknowledging lack of evidence for services offered), with clients, staff, funding bodies, consumer advocates.
Whether to involve clients as informed participants; clearly describe the evidentiary status of services provided including those offered by agencies from which services are purchased.
Gambrill (2005)

The one suggested in the Guide for Child Welfare
Administrators: (GCWA) l. Well-supported, proven efficacious practice
2 Supported and probable efficacious practice.
3. Supported and acceptable practice.
4. Promising and acceptable practice.
5. Innovative or novel practice.
6. Experimental or concerning practice.
Gambrill (2005)

Concerns with the GCWA hierarchy
1. Use of justificationary language that encourages confirmation biases and wishful thinking:
• Use of the word ‘proven’
• Repeated use of words such as ‘support’
• The word ‘harm’ is not mentioned.
2. Hides the fact that most services are of unknown effectiveness.
3. Use of vague terms such as ‘probable’, ‘efficacious’,
4. In Appendix A we find that ‘clinical-anecdotal literature’ and ‘generally accepted in clinical practice’ are included as indicators of level 1 evidence.
Gambrill (2005)

1.
Services that have been critically tested and found to help clients.
2.
Services that have been critically tested and found to harm people.
3.
Service methods are of unknown effectiveness and are not in a research study or are in poor quality research.
4.
Services of unknown effect in a good quality research program.
Gambrill (2005)

4. How rigorous to be in assessing the accuracy of claims.
Consider differences between systematic and narrative
(traditional) reviews. In a systematic review:
• There is a search for all literature related to a question, in all languages, in both published and unpublished sources, including hand searches of journals.
• The search process is clearly described including the databases reviewed. Authors describe where they searched, how they searched.
• Rigorous criteria are used to appraise what is found and these are clearly described. (See Cochrane and Campbell
Collaboration reviews).
Gambrill (2005)

Systematic reviews differ greatly from incomplete, nonrigorous, non-transparent reviews. Traditional narrative reviews do not control for as many biases and thus overestimate positive effects of services. They are misleading in their conclusions. Compare claims of effectiveness made by the developers of Multisystemic
Therapy and conclusions of Littell’s (2005) in Children and
Youth Services Review.
Gambrill (2005)

5. Whether to blow the whistle on pseudoscience, quackery, fraud and related propaganda tactics such as bogus claims of what is done and ‘what is known’
Consider this: “The practice of child welfare has long been based on a strong…”
(Guide for Child Welfare
Administrators on Evidence-based Practice ,2005. p. 5)
This is highly misleading if by ‘strong’ we mean based on high quality research and quality of services offered.
Consider the poor quality of parent training offered
(Gambrill and Goldman, 2005). Bogus claims are not benign. They have resulted in harming in the name of helping and interfere with further exploration; they
Gambrill (2005) mislead rather than inform.

6. Whether to purchase services from agencies that offer ineffective or harmful services. Whether to:
• Stop using ineffective and harmful services
• Start using effective services
7. How systemic to be.
• Whether to provide the resources and training needed for staff to maximize the quality of services offered such as access to needed databases/knowledge manager
• Whether to help all staff to acquire skills in evidence-based practice as well as effective self development skills drawing on evidence informed educational formats (See critiques of traditional continuing education formats such as Thomson O’Brien et al, 2003, Cochrane Library)
Gambrill (2005)

What style of EBP to use:Will all staff be trained in all five steps of EBP:
1. Posing well structured questions
2. Searching efficiently and effectively
3. Critically appraising related research
4. Integrating relevant information
5. Evaluating what happens?
– Or, will staff depend on others for steps 2 and 3?
– The process of EBP requires all staff to be skilled in carrying out steps 1,4 and 5
Gambrill (2005)

– Whether to implement needed organizational changes; e.g. see Evidence-based Health Care
(Gray, 2001):
• Hire knowledge managers (see Gray, 1998)
• Arrange access to relevant databases
• Set up a system for identifying errors so avoidable ones can be minimized and educate all staff that errors are typically systemic in cause
• Collect questions that arise in everyday practice from line staff and clients. High frequency questions should guide research efforts (e.g. systematic reviews, new studies)
Gambrill (2005)

• Arrange facilitating incentive systems (e.g. reward staff for blowing the whistle on harmful and ineffective practices)
• Administrators and supervisors model critical appraisal of claims; they raise questions regarding current practices and policies and welcome such questions from others; they avoid propaganda strategies such as ad hominem arguments
Gambrill (2005)

Questionable excuses for not taking a systemic view
• Critical appraisal and search skills are too hard for you and your staff to learn
Not true; learning key questions to raise regarding different kinds of research including research reviews is easy, especially with the help of user friendly books such as How to Read a Paper (Greenhalgh, 2001). Don’t let the researchers and academics fool you; learning such skills is not that difficult. Len Gibbs teaches them to undergraduate students.
Gambrill (2005)

Questionable excuses, cont.
Regarding research reviews
• Is the question addressed clear and relevant?
• Do the authors clearly describe how they searched, where they searched, and what criteria they used to appraise studies?
• Was a thorough search conducted using relevant databases?
• Did the search cover unpublished as well as published work?
• Were rigorous criteria used to review research?
For effectiveness questions:
• Was the sample size adequate?
• Was there a comparison group?
• Were subjects randomly distributed to different groups?
• Were ratings of outcomes blind?
• Was there a follow up period?
• Was there an intention to treat analysis?
Gambrill (2005)

– Keep a copy of Carl Sagan’s Baloney Detection
Kit available
– Be familiar with Hugh Rank’s cues to propaganda pitches: Hi, Trust me, You need, Hurry, Buy
– Read Thouless’ Straight and Crooked Thinking
(1974) describing informal fallacies such as appeal to popularity and emotional reasoning
Gambrill (2005)

Questionable excuses, cont.
• We researchers and academics don’t have time to prepare systematic reviews
Masters degree students in the evidence based social work program at Oxford complete Cochrane Reviews in one year. True, many will take longer and require considerable resources. But others will not.
• We don’t need a systemic approach
Services are unlikely to improve in quality in a fragmented approach to change.
Gambrill (2005)

• Increase in the percentage of systematic compared to narrative reviews prepared related to questions of direct concern to line staff and clients
• Increase in the percentage of clients involved as informed participants
• Increase in the percentage of line staff, clients and administrators who have acquired skills in posing well-structured questions related to decisions that must be made and in critically appraising research.
Gambrill (2005)

• Increase in the percentage of all involved parties who can spot human service propaganda such as inflated claims about “what we know”
• Increase in the percentage of all involved parties who ask probing questions regarding the effectiveness of services and who avoid propaganda methods in related discussions (such as distorting the views of others)
• Increase in the percentage of services purchased that have been critically tested and found to be effective
• Decrease in the use of harmful and ineffective services
Gambrill (2005)

Arrogance
• Preference for authority-based practices and policies
(examples):
• A reluctance to be transparent
• Inflated claims of effectiveness
• Use of the term “evidence-based” to refer to business-as-usual
Prevalence of pseudoscience and propaganda and underestimating vulnerability to their influence (e.g. see
Science and Pseudoscience in Clinical Psychology,
2003).
Gambrill (2005)

A justification approach to knowledge in which we search for data to confirm our views.(Use of word
‘proven’.)
The symbiotic relationship between clients’ wishes to be helped and professionals’ desire to help.
Gambrill (2005)

The philosophy and practice of EBP as described by its originators has implications for:
• Professional education
– Formats used and skills emphasized.
• Research
– Who decides on questions pursued
– How research is conducted and reported, for example rigorous (likely to answer questions posed), clear descriptions of limitations, accurate rather than inflated claims of “what we know”, involvement of consumers in critiquing research.
Gambrill (2005)

•
In Summary, cont.
–
Practice
Clients are involved as informed participants regarding the evidentiary status of services
– Staff are accurately informed regarding the evidentiary status of services
– Staff are skilled in carrying out the process of EBP
(e.g. posing well structured questions that help them to draw on relevant research findings)
– Staff evaluate the effects of their services
– Services purchased from agencies have been shown via critical tests to help clients
Gambrill(2005)

• Administration and management
– Needed tools are provided (such as access to relevant databases)
– Needed training is provided (e.g. in posing wellstructured questions)
– Needed feedback is provided (including supervisory) to maintain and enhance skills
– A system is in place to harvest errors that occur so avoidable ones can be minimized
– A user friendly complaint system is in effect and data gathered are used to improve services
– Incentive systems are in place that support EBP
Gambrill(2005)

• California has an opportunity to take a leadership role:
– Involve line staff and clients in identifying vital questions and outcomes and involving them as informed participants.
– Prepare systematic reviews related to key questions of direct concern to line staff and clients using
Cochrane and Campbell protocols.
– Avoid human service propaganda such as bogus claims of effectiveness.
– Create a workforce which is skilled in the process of EBP.
Gambrill (2005)

The Road Ahead, cont.
– Create organizational cultures that support
EBP (e.g. provide access to needed databases)
– Maximize opportunities to honor ethical obligations
• To help clients
• To avoid harm
• To involve clients as informed participants
• To maximize self-determination/autonomy
Gambrill (2005)

• Many challenges, especially a preference for authority-based practice (The “trust me” approach)
• Many exciting advances, especially in options for integrating research and practice at the line staff level and honoring ethical obligations to clients.
Gambrill (2005)