CHS 213
Research in Community
Health Education
Fall 2009
Instructor:
Donald E. Morisky, Sc.D., M.S.P.H (dmorisky@ucla.edu)
Office: CHS 46-071, phone: 310-825-8508
Class Time:
Thursday 8:00-9:50, Room 61-262 CHS (Lecture/Discussion)
Laboratory Session: Tuesday 8:00-10:50, Room A1-241 CHS (Computer Lab)
Office Hours:
Monday and Wednesday 3:00-4:00 and by appointment
Special Reader
Office Hours:
Akiko Sato <akisato@ucla.edu>
Tuesdays 11am-1pm, CHS 46-071
Reader: http://www.ph.ucla.edu/class/chs/chs213
Username: chs213
Password: chs213syllabus
Text:
Norusis, Marija J., SPSS 17.0 Guide to Data Analysis, Prentice-Hall, 2008
This course is designed to provide students an opportunity to review and discuss the conceptual
framework, research methodology, research design, implementation, and evaluation aspects of
previous and ongoing community/patient health education programs. The course will emphasize
the identification, application and understanding of various research methods used to design,
implement and evaluate community-based education programs.
Thursday: Lecture/Discussion Format
The lecture will address research methods to be considered in the development,
conceptualization, implementation and evaluation of community-based research programs. The
Course Reader contains selected large-scale health education/health promotion community-based
studies, such as the Stanford 3-Community Project, the Stanford 5-City Project, and other
community based intervention programs. These readings will form the basis of class discussion
each Thursday. The lecture/discussion session will also cover evaluation concepts, including
evaluation terminology, evaluating program effectiveness, selecting an evaluation design, sample
size determination, establishing a comparison group, issues in data collection, evaluation
techniques, reading frequencies, tables, descriptive statistics, analytical methods, and
reliability/validity assessment. These concepts will be integrated into the Tuesday laboratory
discussion sessions.
Tuesday: Laboratory/ Discussion Format
The laboratory session will meet each Tuesday from 8-10:50 am. The Special Reader will give a
60-90 minutes overview of the lab assignments for that day, followed by 30-60 minutes for
student article presentations and discussion. Afterwards, students will have 50-60 minutes to
work in groups for lab assignments. Students who have a 10:00 am class on Tuesdays can leave
early and continue on their own to finish their lab assignments later that week. The SPSS Guide
to Data Analysis and the Data Files used in the assignments will be used throughout the
laboratory sessions. You will set up a data file, run SPSS for Windows, define your data,
conduct simple descriptive univariate statistics, summarize your data, conduct bivariate analysis,
change the coding scheme, test hypotheses about two independent means/two dependent means,
comparing several means (ANOVA), measure associations, interpret correlation coefficients,
calculate reliability coefficients, and test regression hypotheses. Please bring a flash drive to
the computer lab on the first day and following sessions to download programs as well as to save
laboratory assignments throughout the quarter.
Each student will be given a research article that pertains to the current test(s) covered in the
homework (a sign-up sheet will be distributed during week 1). The student is responsible for
giving a short synopsis of the independent/dependent variables, research question and major
hypothesis being tested. Students will then discuss how specific significance tests were used to
answer the research questions. Finally, the student will comment on the appropriateness of the
test for the question and available data.
This will allow time to review questions from the homework, review the concepts for the next
homework, and prepare for the presentation of the relevant research article with discussion.
Those students who have a 10am class can leave and those who stay can start work on the
homework and get assistance.
Pre-Final Exercise and Final Exam
Pre-final Exercise
Laboratory exercises will be turned in each week, up to the seventh week of class. Then, a new
data set (HIV/AIDS Prevention in the Philippines) will be available with new exercises. The
data set is from an ongoing research program in the Philippines. You will be given a laboratory
exercises on the Philippines data set to complete prior to receiving the go signal for your final
laboratory exam. The questionnaires and code books for the HIV/AIDS Philippines
Longitudinal Research Project will also be available at this time on the web. The data sets will
be used by all students during weeks 8-10, as described below. Additional readings include
reliability and validity assessment, scaling, and sample size determination, and will be included
on the web.
Final Exam
Your final exam is already included in this syllabus and once you have completed all lab
assignments and the lab exercise on the Philippines data set, you can begin your final exam. For
the final exam, students will be requested to pose your own research questions, develop several
hypotheses and use the skills you have learned throughout the class to answer these research
questions. Performance will be based on how thoroughly you have used the available data and
how results, interpretation and discussion are presented.
Evaluation
Laboratory Exercises
Lab Presentation
Pre-Final Exercise
Final Exam (take home)
TOTAL
40%
10%
15%
35%
100%
LECTURE SCHEDULE
Week 1
9/24
Week 2
10/1
The class will meet on Thursday, September 24 to review the syllabus, scope
of the course, readings, assignments, evaluation criteria, etc. Introduction
and research related experiences will be discussed to give you an idea of how
this class will equip you with the skills and methodological competencies
necessary for solid scientific program design and evaluation. Please feel free
to make an appointment with the instructor during orientation week.
Overview of the laboratory component of the course, availability of the
Grad Pak at Ackerman for SPSS; assignments, grading, and student presentations.
Overview of the scope of the course, requirements, expectations and
evaluation criteria, discussion of evaluation concepts, and overview of the
Stanford 3-community Program (Shea and Basch articles, 01 and 02).
Week 3
10/8
Evaluation Research: Domains and Interest Groups, Stanford 5-City
Program (Farquhar JW, et al. (06); Fortmann, et al., (07)
Week 4
Evaluation Research: Domains and Interest Groups, Critiques by [03] Leventhal,
et al.]; [04] Kasl; and [05] Meyer, et al. Further Stanford 5-City Program,
Winkleby MA, [08 and 09]; North Kerelia, Finland Cardiovascular Risk
Reduction Program; [10]. Puska P et al; [11]. Salonen et al.,
Week 5
10/22
Understanding Reliability and Validity-exercises and
discussion—class handouts: Key Questions for Various ArticlesPart 1.pdf; Application of Principles from the Readings-Part II.pdf;
Reliability Recap.doc; Validity Recap.doc; Autonomy Factors.doc;
Autonomy Figure 1.doc; Autonomy Figure 2.doc; Autonomy Figure
3.doc; —[14]. Morisky et al., 2002; Continued discussion on
reliability assessment, factor analysis, and application of scaling
principles and analytical methods, and social desirability.
Week 6
10/29
Measurement Issues in Data Collection (Sample Size Determination; [15]
Statistical Methods for Rates and Proportions. Fleiss, 1981)
Week 7
11/5
Data Collection Methods and Simple Methods to Analyze Program Data;
Discussion of lessons learned and homework assignments on reliability and
Program Effectiveness - [12]. Morisky et al., 1983; 13]. Morisky et al., 1986;
Week 8
11/12
Discussion of lessons learned and homework assignments on reliability and
Program Effectiveness to prepare for final paper. Readings [100-105].
Week 9
11/19
Discussion of lessons learned and homework assignments on reliability and
Program Effectiveness to prepare for final paper. Readings [106-110].
Week 10
11/26
Happy Thanksgiving
Week 11
12/3
Evaluation concepts, including evaluation terminology, evaluating program
effectiveness, selecting an evaluation design. Establishing a comparison group,
issues in data collection, evaluation techniques, reading frequencies, tables,
descriptive statistics. Discussion of statistical methods used to explain outcome
assessments, such as regression modeling.
COURSE READER I: TABLE OF CONTENTS
01. Shea S, Basch CE. A review of five major community based cardiovascular
disease prevention programs. Part I: rationale, design, and theoretical framework.
American Journal of Health Promotion, 1990; 4:203-213.
02. Shea S, Basch CE. A review of five major community based cardiovascular
disease prevention programs. Part II: Intervention strategies, evaluation methods, and
results. American Journal of Health Promotion, 1990; 4:279-287.
03. Leventhal H, Safer MA, Cleary PD, Gutmann M. Cardiovascular Risk
Modification by Community-Based Programs for Life-Style Change: Comments on
the Stanford Study. J of Consulting and Clinical Psychology 1980; 2:150-158.
04. Kasl SV. Cardiovascular risk reduction in a community setting Some
comments. J of Consulting and Clinical Psychology 1980; 48:143-149.
05. Meyer AJ, Maccoby N, Farquhar JW. Reply to Kasl and Leventhal et al. J of
Consulting and Clinical Psychology 1980; 48:159-163.
06. Farquhar JW, Fortmann SP, Maccoby N et al. The Stanford Five-City Project:
Design and Methods. Am J of Epidemiology 1985; 122:323-334.
07. Fortmann SP, Flora JA, Winkleby MA, et al. Community intervention trials:
reflections on the Stanford Five-City. American Journal of Epidemiology 1995;
142:576-586.
08. Winkleby MA, Taylor B, Jatulis D, Fortmann SP. The Long-Term Effects of a
Cardiovascular Disease Prevention Trial: The Standard Five-City Project. American
Journal of Public Health 1996; 86(12): 1773-1779.
09. Winkleby MA, Feldman HA, Murray DM. Joint Analysis of Three U.S.
Community Intervention Trails for Reduction of Cardiovascular Disease Risk. J
Clin Epidemiol 1997; 50(6): 645-658.
10. Puska P, Tuomilehto J, SALONEN J, et al. Changes in Coronary Risk-Factors
During Comprehensive 5-year community Program to Control CardiovascularDisease (North Karelia Project). British Medical Journal 1979; 2 (6199): 1173-1178
11. Salonen JT, Puska P, Mustaniemi H. Changes in Morbidity and Mortality
During Comprehensive Community Program to Control Cardiovascular-Disease
during 1972-1977 in North Karelia. British Medical Journal 1979; 2 (6199): 11781183.
12. Morisky DE, Levine DM, Green LW, Shapiro S, Russell P. et al. Five-Year
Blood Pressure Control and Mortality Following Health Education for Hypertensive
Patients. American Journal of Public Health 1983; 73:153-162.
13. Morisky DE, Green LW, Levine DM. Concurrent and Predictive Validity of a Self-Reported
Measure of Medication Adherence and Long-Term Predictive Validity of Blood Pressure
Control. Medical Care 1986; 24:67-74.
14. Morisky DE, Ang A, Sneed C. Validating the Effects of Social Desirability on
Self-Reported Condom Use. AIDS Education and Prevention 2002; 14:351-360.
15. Fleiss JL. Statistical Methods for Rates and Proportions. Chapter 3-Determining Sample
Sizes Needed to Detect a Difference Between Two Proportions. John Wiley & Sons, p33-59,
1981.
Other Community-Based Intervention Programs
16. Barlow JH, Turner AP, Wright CC. A Randomized Controlled Study of the
Arthritis Self-Management Programme in the UK. Health Education Research 2000;
15(6); 665-680.
17. Tabet S, Sanchez J, Lama J, et al. HIV, Syphilis and Heterosexual Bridging
Among Peruvian Men Who Have Sex with Men. AIDS 2002; 16(9):
1271-1277.
18. Hewitt M, Denman S, Hayes L, et al. Evaluation of “Sun-safe”: A Health
Education Resource for Primary Schools. Health Education Research 2001; 16(5):
623-633.
19. Koenig LJ, Whitaker DJ, Royce RA, et al. Violence During Pregnancy Among
Women With or At Risk for HIV Infection. American Journal of Public Health 2002;
92(3): 367-370.
20. Fortenberry JD, McFarlane M, Bleakley A, et al. Relationships of Stigma and
Shame to Gonorrhea and HIV Screening. American Journal of Public Health 2002;
92(3): 378-381.
21. Baker SA, Beadnell B, Stoner S, et al. Skills Training versus Health Education to
Prevent STDs/HIV in Heterosexual Women: A Randomized Controlled Trial Utilizing
Biological Outcomes. AIDS Educ and Prev 2003; 15:1-14.
22. Vanable PA, Ostrow DG, McKirnan. Viral Load and HIV Treatment Attitudes as
Correlates of Sexual Risk Behavior Among HIV-Positive Gay Men. J of
Psychosomatic Res 2003; 54:261-269.
23. Jones R. Relationships of Sexual Imposition, Dyadic Trust, and Sensation Seeking with
Sexual Risk Behavior in Young Urban Women. Res in Nursing and Health, 2004; 27:185-197.
24. Carey MP, Carey KB, Maisto SA, et al. Reducing HIV-risk behavior among adults receiving
outpatient psychiatric treatment: Results from a randomized controlled trial J of Consulting and
Clinical Psychology 2004; 72: 252-268.
25. Richardson J, Milam J, McCutchan A, et al. Effect of Brief Safer-Sex Counseling by
Medical Providers to HIV-1 Seropositive Patients: A Multi-Clinic Assessment. AIDS 2004;
18:1179-1186.
26. Norr KF, Norr JL, McElmurry BJ, et al. Impact of Peer Group Education on HIV Prevention
among Women in Botswana. Health Care for Women International 2004; 25:210-226.
27. Hosain GMM, Chatterjee N. Beliefs, Sexual Behaviors and Preventive Practices with
Respect to HIV/AIFSS among Commercial Sex Workers in Daulatdia, Bangladesh. Public
Health 2005; 119:371-381.
Readings from the HIV/AIDS Prevention Community-Based Program in the
Philippines that provide more background information and impact and
outcome results of the longitudinal study.
100. Tiglao T, Morisky DE, Tempongko S, Baltazar J, Detels R. A Community Participation
Action Research Approach to HIV/AIDS Prevention among Sex Workers. Promotion &
Education 1996; 3:25-28.
101. Morisky DE, Tiglao TV, Sneed CD, Tempongko SB, Baltazar JC, Detels R, Stein J. The
Effects of Establishment Practices, Knowledge and Attitudes on Condom Use among Filipina
Sex Workers. AIDS Care 1998; 10 (2):213-220.
102. Sneed CD and Morisky DE. Applying the Theory of Reasoned Action to Condom use
Among Sex Workers. Social Behavior and Personality 1998; 26(4):317-328.
103. Morisky DE, Peña M, Tiglao TV, Liu K. The Impact of the Work Environment on Condom
Use among Female Bar Workers in the Philippines. Health Education and Behavior 2002;
29(4):461-472.
104. Morisky DE, Stein JA, Sneed CD, Tiglao TV, Tempongko SB, Baltazar JC, Detels R, Liu
K. Modeling Personal and Situational Influences on Condom Use Among Establishment-Based
Commercial Sex Workers in the Philippines. AIDS and Behavior 2002; 6:163-172.
105. Morisky DE, Chiao C, Stein JA, Malow R. Impact of Social and Structural Influence
Interventions on Condom Use and Sexually Transmitted Infections among Establishment-Based
Female Bar Workers in the Philippines. Journal of Psychology and Human Sexuality 2005; 17:
45-63.
106. Chiao C, Morisky DE, Ksobsech K, Rosenberg R, Malow R. The Relationship Between
HIV/Sexually Transmitted Infection Risk and Alcohol Use During Commercial Sex Episodes:
Results From the Study of Female Commercial Sex Workers in the Philippines. Substance Use &
Misuse 2006; 41(10-12):1509-1533.
107.
Morisky DE, Stein J, Chiao C, Ksobiech K, Malow R. Impact of a Social Influence
Intervention on Condom Use and Sexually Transmitted Infections among Establishment-Based
Female Sex Workers in the Philippines: A Multilevel Analysis. Health Psychology 2006;
25(5):595-603.
108.
Withers M, Dornig K, Morisky DE. Predictors of Workplace Sexual Health Policy at
Sex Work Establishments in the Philippines. AIDS Care 2007; 19(8): 1019-1024.
109.
Chiao C, Morisky DE. Clinic Appointment Attendance for Sexually Transmitted
Infection Screening among Filipina Sex Workers: A Multilevel Analysis. AIDS CAREPSYCHOLOGICAL AND SOCIO-MEDICAL ASPECTS OF AIDS/HIV 2007;
19(9):1166-1170.
110.
Chiao C, Morisky DE, Ksobsech K, Malow R. Promoting HIV Testing and Condom Use
among Filipina Commercial Sex Workers: Findings from A Quasi-Experimental Intervention
Study. AIDS and Behavior 2008 Jun 10. [Epub ahead of print]
LAB SCHEDULE
In addition to the weekly homework assignments, the following articles will be discussed during
the corresponding lab session. Please read them before the lab to maximize your understanding
of the material and ability to participate in discussion. Articles will be available on the course
web-site.
Article (Author/Title)
Date due
Week 1
No article due the week Introduction of the Special Reader, office hours,
Chapters 1-3
assignment expectations, Lab orientation, logging on to SPSS, navigating
9/29/2009
the system, use of the tutorial, and other things to help get you started.
Week 2
Chapters 4-5
01 Deichmann et al. (1999). Improvements in Diabetic Care as Measured
by HbA1c After a Physician Education Project.
10/6/2009
Bar Graphs
02. Islam et al. (2008). Cohort study of medication adherence in older
adults (CoSMO): Extended effects of hurricane Katrina on medication
adherence among older adults.
03. Bissell et al. (2004). Evidence of the effectiveness of health sector
preparedness in disaster response: the example of four earthquakes.
Week 3
Chapters 6-10
04. Braverman et al. (2001). World Health Report 2000: how it removes
equity from the agenda for public health monitoring and policy.
10/13/2009
Scatterplot
and Boxplot
05. Young and Hade (2006). Holidays, Birthdays, and Postponement of
Cancer Death
Binomial Test
06. Clayton et al. (1999). Public health in managed care: A randomized
controlled trial of effectiveness of the postcard reminders.
Week 4
07. Juniper et al., (1996). Measuring quality of life in children with asthma.
Chapters 11-14
10/20/2009
One-sample ttest
Paired t-test
Independent
sample t-test
and normal
distribution
08. Lauby et al. (2000). A Community-Level HIV Prevention Intervention
for Inner-City Women: Results of the Women and Infants Demonstration
Projects.
09. Nelson et al. (2003). A comparison of national estimates from the
National Health Interview Survey and the Behavioral Risk Factor
Surveillance System.
Week 5
Chapters 15-17
10. Bollella et al. (1999).Nutrient Intake of Head Start Children: Home vs.
School
10/29/2009
One-way
ANOVA and
Tukey
correction
11. Aikens et al. (2005). Adherence to maintenance-phase antidepressant
medication as a function of patient beliefs about medication.
Two-way
ANOVA
Week 6
Chapters 18-20
12. Bryson et al. (2008). Alcohol screening scores and medication
nonadherence.
11/3/2009
Chi-square
and
Somers’ d
13. Marx et al. (2006). Diarrheal illness detected through syndromic
surveillance after a massive power outage: New York City, August 2003.
Linear
Regression
Wilcoxon
paired test
and MannWhitney
Wilcoxon
rank sum and
KruskalWallis
Week 7
Chapters 21-23
14. Van Griensven et al. (1998). Evaluation of a targeted HIV prevention
programme among female commercial sex workers in the south of Thailand.
11/10/2009
Multiple
Regression
Week 8
11/17/2009
15. Wang et al. (2002). Noncompliance with antihypertensive medications:
The impact of depressive symptoms and psychosocial factors.
Turn in last HW assignment
Review additional SPSS procedures
Receive Pre-Final Exercise
Week 9
11/24/2009
Research questions/Hypotheses due
Review additional SPSS procedures
Work on Pre-Final Exercise
Week 10
12/1/2009
Pre-Final Exercise due ( on or before this date)
Review additional SPSS procedures
Receive Final Exam
Finals Week
12/14/2009
Final Exam due on or before 12pm on December 14, 2009
NOTE:A HARD COPY of the entire exam must be received on or
before Monday, December 14, 2009. Please e-mail your entire exam
(including syntax) to both the Special Reader and Dr. Morisky by
December 14, 2009.
Additional Readings will be added and /or students can identify their own readings for
class presentation/discussion
Laboratory Homework Assignments-SPSS 17.0 Guide to Data Analysis
Week
Week 1:
9/29/2009
Content
Chapters 1-3.
Week 2:
10/6/2009
Chapters 4-5.
Week 3:
10/13/2009
Chapters 6-10.
Week 4:
10/20/2009
Chapters 11-14.
Week 5:
10/27/2009
Chapters 15-17.
Week 6
11/3/2009
Chapters 18-20.
Week 7:
11/10/2009
Chapters 21-23.
Exercise
Statistical Concepts - #3.2, 3.3, 3.6
Due
10/6/2009
Statistical Concepts - #4.1, 4.3, #5.2, 5.9
Data Analysis: #4.1, 4.5, 4.12, #5.4, 5.6
10/13/2009
Statistical Concepts: #6.1, 8.1 8.2 #9.2, #10.2
Data Analysis: #6.6, #7.4, #8.7, #8.20; #9.10, #10.1
10/20/2009
Statistical Concepts: #11.7, #12.3, #13.2, #14.4
Data Analysis: #11.5, #12.7, #13.10, #14.16
10/27/2009
Statistical Concepts: #15.1 #16.3, #17.3
Data Analysis: #15.2, 15.10, #16.7, #17.16
11/3/2009
Statistical Concepts: #18.2, #20.3
Data Analysis: #18.4, #19.7, #20.3, 20.5
11/10/2009
Statistical Concepts: #21.3, #22.3, #23.4
Data Analysis: #21.3, 21.5, #22.5, #23.8
11/17/2009
Week 8-10. Once all assignments have been turned in, students will be given the Philippines HIV/AIDS
data set. Use this time to become familiar with this new data set. You will have to merge three data sets
in order to calculate change scores. You will be given two laboratory exercises to complete prior to
receiving your final laboratory exam on this data set.
CHS 213 Fall 2009
Criteria for Laboratory Presentations:
The purpose of the laboratory presentations is to reinforce content discussed in the laboratory
sessions. Additionally, the purpose is to allow for interaction among students and the teaching
assistant. Therefore, active participation (i.e. questions and constructive feedback) are expected
and encouraged. Remember, constructive feedback pertains to feedback regarding the methods
used in the papers, not necessarily your classmates’ presentation styles. Presentations do NOT
need to be formal (i.e. PowerPoint, etc.).
Please adhere to the following guidelines in the preparation of your presentation.
•
•
•
•
•
•
•
•
•
•
•
•
What is the problem being researched in the article?
What is the explicit research question?
What is the purpose of the research?
What is/are the research hypothesis/es
o This should be stated such that it is pertinent to the type of test used or covered
from the week prior.
o Keep in mind that the study hypothesis/es may be different from the hypothesis/es
for the tests being used.
o State both the null and alternative hypotheses.
What are the variables used (i.e., independent and dependent variables)
What types of variables are they?
What type of test was used? What is the null hypothesis of the test?
What are the assumptions of the test?
o Were the assumptions met?
Was the test appropriate?
o If not, then what would you do differently (i.e. different control variables, etc.)
What were the limitations of the study?
Results
Conclusions
CHS 213 Fall 2009
Final Exam Guidelines:
These guidelines are to assist you in preparing your final examination. Prior to this examination,
you will have completed a set of pre-final questions designed to help you familiarize yourself
with the HIV/AIDS Philippines data set, to be used for this final. In the final, you must develop a
research problem of interest, develop the theoretical rationale (based on literature), create an
analysis plan, and present the results.
The following assignments are due to Dr. Morisky and the Special Reader electronically:
• Research Questions/Hypotheses due by 11/24/09 by 5:00pm PST
• Pre-Final Exercise due by 12/1/09 by 5:00pm PST
• Final Exam paper due on or before 12:00 pm, PST noon on December 14 PST (both
hardcopy AND electronic)
In order for your final Exam paper to be complete, the following MUST be included:
Your Final Exam paper
• Should be 10-15 pages in length
• Output should be included within the text of the paper, not at the very end (appendix
style)
• All syntax for your analysis needs to be in the order it appears in the paper, added as an
appendix.
• You are to e-mail the paper with Syntax AND deliver a hardcopy (no Syntax) to the
Special Reader and Dr. Morisky on or before noon on December 14 PST.
The paper should include:
• A brief introduction/background
o The introduction should include a description of the purpose of your analysis
(research question). You must also include at least three references from the
literature, as well as an appropriate theoretical framework and pertinent concepts.
This should not be lengthy; however, it should be informative enough to allow a
reader to understand the purpose of your analysis.
o Dr. Morisky has written numerous papers using this data set, which can be found
on the class website. It is your responsibility to pull whatever is pertinent for the
questions you want to answer with your analysis.
• Detailed methods section (Main part!!!)
o Include characteristics of the data set (sample and variables to be used).
o Dr. Morisky has discussed sampling, sample sizes, etc. Use what you know.
o Include the rationale for the tests to be used
o Include the assumptions for the tests to be used
o Include the methods for testing the assumptions.
• Results: Bring it all together.
o The type of sequential analyses used should be detailed here.
o Any statistics used should be explained and justified.
o If you transform variables, that should be mentioned and explained, in detail.
•
•
Discussion:
o Interpret your results
o Discuss limitations of the data (including the type of sample, if relevant) or
limitations of the analysis
o Provide alternative interpretations
Include your bibliographic references