Title: The Genetic Counseling Outcome Scale: A New Patient Reported Outcome Measure
for Clinical Genetics Services.
Marion McAllister PhD (corresponding author), Medical Genetics Research Group, School of
Biomedicine, The University of Manchester, Manchester Academic Health Sciences Centre,
Nowgen, 29 Grafton Street, Manchester M13 9WU, United Kingdom. Telephone number:
+44 161 276 8979, fax number: +44 161 276 4058; e-mail address:
marion.mcallister@manchester.ac.uk. No conflicts of interest.
Alex M Wood PhD, School of Psychological Sciences, The University of Manchester. No
conflicts of interest.
Graham Dunn PhD, School of Community-Based Medicine, The University of Manchester.
No conflicts of interest.
Shoshana Shiloh PhD, Department of Psychology, University of Tel Aviv. No conflicts of
interest.
Chris Todd PhD, School of Nursing, Midwifery & Social Work, The University of
Manchester. No conflicts of interest.
ACKNOWLEDGEMENTS
We particularly thank Alastair Kent and Melissa Hillier from Genetic Alliance UK, along
with all the participating patients and patient group members who completed questionnaires.
We also thank all the clinicians in Genetic Medicine, St Mary’s Hospital, Manchester for
1
recruiting their patients for Study 2. This research was funded by the Medical Research
Council, and supported by the NIHR Manchester Biomedical Research Centre. The views
expressed in this paper are those of the authors, and not those of the funding body. Ethical
opinions were provided by the University of Manchester Committee on the Ethics of
Research with Human Subjects (Study 1; Ref. 08306) and (2) the North West 7 NHS
Research Ethics Committee (Study 2; Ref. 09/H1008/111).
2
ABSTRACT
Purpose: The aim in this study was to develop a Patient Reported Outcome Measure (PROM)
for clinical genetics services.
Methods: Previous research was used to develop a draft 84-item questionnaire, which was
completed by 527 members of patient support groups. Responses were subjected to
exploratory factor analysis (EFA). Parallel analysis was used to identify the number of factors
to extract using oblique rotation. 24 questions were selected to form the Genetic Counseling
Outcome Scale (GCOS-24). 241 patients completed a questionnaire pack before and after
attendance at a genetics clinic that included the GCOS-24, and validated measures of health
locus of control, perceived personal control, anxiety, depression, satisfaction with life and
authenticity. Concurrent validity of the GCOS-24 was assessed using bivariate correlation.
Sensitivity to change of the GCOS-24 was assessed using analysis of variance.
Results: EFA identified a single overarching construct consisting of 7 dimensions. Internal
consistency (α= 0.87) and test-retest reliability (0.86) are good. The GCOS-24 demonstrates
convergent and divergent validity, and sensitivity to change over time with a medium-to-large
effect size (Cohen’s d = 0.70).
Conclusion: The GCOS-24 has potential as a clinical genetics-specific PROM.
KEYWORDS: scale development, empowerment, evaluation, psychometrics, PROM,
outcome measures.
3
INTRODUCTION
Evaluating interventions in clinical genetics is difficult because there is little agreement about
suitable outcome measures (1-3). In other specialties, measures of health status combined
with measures of Health Related Quality of Life (HRQoL) are commonly used (4). However,
although pharmaceutical and surgical options may become important in some genetic
conditions (5), measures of health status are unlikely to be generally applicable because
interventions offered by clinical genetics rarely affect health status (3). There is also evidence
that health status is not valued as an outcome from clinical genetics, but that HRQoL is
valued by patients and clinicians (6). HRQoL measures have been recommended as useful in
clinical trials in medical genetics (7).
HRQoL is usually measured using Patient Reported Outcome Measures (PROMs). PROMs
are short self-completion questionnaires capturing aspects of patient health status or HRQoL
that come directly from the patient, without interpretation of responses by a clinician (8).
Generic measures of HRQoL are available, but there is no condition-specific PROM of
HRQoL for genetic conditions that captures all important patient benefits (9,10).
Our previous qualitative research suggests that patient benefits from clinical genetics could be
summarised using a new construct, empowerment (9,10), defined as the beliefs that one:
(1) Can make important life decisions in an informed way (decisional control)
(2) Has sufficient information about the condition, including risks to oneself and one’s
relatives, and any treatment, prevention and support available (cognitive control)
(3) Can make effective use of the health and social care systems for the benefit of the
whole family (behavioural control)
4
(4) Can manage one’s feelings about having a genetic condition in the family (emotional
regulation)
(5) Can look to the future having hope for a fulfilling family life, for oneself, one’s
family, and/or one’s future descendents (hope)
Empowerment can be considered an aspect of HRQoL (11). When compared with outcome
measures used in evaluations of clinical genetics, empowerment is conceptually most similar
to the construct captured by the Perceived Personal Control questionnaire (PPC) (12).
However, PPC does not capture all patient benefits summarised in empowerment e.g.
emotional regulation and benefits to other relatives (9,10).
The aim in this study was to develop a robust, theory-driven clinical genetics-specific PROM.
The theoretical framework driving PROM development in this study is the grounded theory
model of empowerment (9,10). Specific objectives were to (a) develop an instrument to
measure empowerment (b) perform preliminary psychometric evaluation of the instrument,
including reliability, some construct validity and sensitivity to change. This PROM could be
useful as a clinical genetics-specific measure of HRQoL for use in evaluating clinical genetics
services.
METHODS
Questionnaire Construction (item pool)
An item pool of 84 questions (Table 1) was generated using (i) qualitative data (9,10) (ii) the
PPC items (12) and (iii) the emotional representations sub-scale of the Revised Illness
Perceptions questionnaire (13) adapted for use in genetic conditions. To maximise content
validity, items were designed to comprehensively sample from the qualitative data, resulting
in a large item pool with approximately equal numbers of positively and negatively worded
5
items. Ease of understanding and face validity was checked in one focus group and 4
interviews with patient group representatives, resulting in rewording of some items.
Study 1: Factor analysis, reliability and development of short questionnaire
Factor analysis was conducted to inform item selection for the final questionnaire. Some
psychometricians recommend sample sizes of at least 5 times the number of items to be
analysed (14,15), in this case, 420 participants. To facilitate large-scale data collection, paper
and online versions of the 84-item questionnaire were created. A sample of people was sought
for whom having a genetic condition in the family was a current concern, but amongst whom
levels of empowerment were not likely to change significantly. This stability was required for
test re-test reliability testing. Members of patient support groups for genetic conditions were
targeted, rather than a patient sample as it was expected that levels of empowerment in a
patient sample would be more likely to change over time.
Patient support groups were recruited, using approved informed consent procedures by
Genetic Alliance UK (GAUK), a UK alliance of over 130 patient organisations supporting
families affected by genetic conditions. Organisations were asked by GAUK, by letter or
email, to recruit their members into the study. Whether paper questionnaires were sent to
members, or whether members were sent an e-mail link to the online questionnaire, or both,
was decided at the discretion of each organisation.
Study 2: Construct validity and sensitivity to change of the GCOS-24
To test sensitivity to change over time, and construct validity of the GCOS-24, all patients
(>18 years old) attending a CGS for a first appointment between February and July 2010 were
asked to participate in the study by the clinical team, using approved informed consent
6
procedures. If the person referred was a child, his/her parent was approached. Parents of
affected children can be thought of as “patients” in clinical genetics, since they may be at risk
for having an affected child in their next pregnancy. Participants were sent a questionnaire
pack prior to their first CGS appointment, which included the GCOS-24, the
Multidimensional Health Locus of Control Scales (MHLC) (16), the State-Trait Anxiety
Inventory short form (STAI-6) (17), the Perceived Personal Control scale (PPC) (12), the
Personal Health Questionnaire (PHQ-9) (a measure of depression) (18), the Satisfaction with
Life questionnaire (SWL) (19) and the Authenticity scale (20). Patients indicating on their
first questionnaire that they were happy to be sent a further questionnaire 2-4 weeks after their
appointment, were sent a copy of the same questionnaire pack. For this study, all participants
were sent a paper questionnaire pack in the post.
RESULTS
Study 1: Factor analysis, reliability and development of short questionnaire
Sample characteristics:
In total, 549 participants completed the 84-item questionnaire. 290 responses (52.8%) were on
paper, and 259 responses (47.2%) were online. Twenty-two questionnaires completed online
were excluded because of excessive (>5%) missing data. All completed paper questionnaires
were included. For questionnaires with minimal missing data (<5%), the mean of all
responses from other subjects on a given item was used to “impute” missing values. The final
sample size was 527. Because the questionnaire was available online, and because for ethical
reasons, the research team did not have access to patient support group mailing lists, we
cannot report accurate participation rates. We estimate participation rates to be 16.7% based
upon the number of paper questionnaires requested and returned. This is likely to be an
7
underestimate, because all those sent a paper questionnaire also had the option to complete
the questionnaire online, and may have done so in preference.
Respondents described themselves as predominantly female (79.5%) and White British
(89.6%). 35.2% were aged 17-40 years; 33.4% were aged 40–49 years; 31.4% were aged 50–
80 years. A wide range of genetic conditions was represented (see Table 2). 219 respondents
(41.5%) described themselves as affected by a genetic condition, 352 (66.8%) as the parent of
an affected child; 122 (23.1%) as concerned that they may be at risk for developing a genetic
condition themselves, and 318 (60.3%) as concerned that they may be at risk for having an
affected child. There were, of course, overlaps between these four groups.
Exploratory factor analysis:
Maximum likelihood exploratory factor analysis (EFA) was conducted, followed by oblique
(promax) rotation (21-23), using SPSS for Windows 16.0 (24). Oblique rotation was used
because (a) factors were theoretically expected to correlate based on the theoretical structure
of the empowerment construct (9,10), and (b) it was found to provide a solution closer to
simple structure than varimax rotation, providing empirical evidence that factors are likely to
be correlated in this sample. Bartlett’s test suggested the data were suitable for EFA, χ2 =
25021.344, p < 0.001. The Kaiser-Meyer-Olkin test (25) indicated that a factorial solution
could be found in the item pool (0.946). The sample size exceeded Gorsuch’s 5 participants
per item recommendation (14).
Methods for deciding how many factors to extract in EFA include the scree test, Kaiser’s
eigenvalues-greater-than-one test and Parallel Analysis (PA) (15, 26). Of these methods,
recent evidence suggests PA to be the most reliable. PA was conducted using SPSS syntax to
8
randomly generate 1000 random data matrices, each with 527 cases and 84 variables (26). In
the real dataset, the first 15 eigenvalues (and associated % of variances) were 23.47 (27.9%),
6.02 (7.2%), 3.23 (3.8%), 2.80 (3.3%%), 2.43 (2.9%), 2.11 (2.5%), 1.85 (2.2%), 1.56 (1.9%),
1.46 (1.7%), 1.36 (1.6%), 1.29 (1.5%), 1.19 (1.4%), 1.13 (1.3%), 1.11 (1.3%), 1.05 (1.24%).
These eigenvalues exceeded the seventh but not eighth eigenvalues that emerged from PA
(respectively, 1.67 and 1.64), suggesting an optimal 7-factor structure (as only seven factors
are larger than would be expected from pure chance dataset characteristics). On this basis,
seven factors were extracted and rotated (Table 1). Factor inter-correlations are shown in
Table 3.
Theoretically, each of the seven factors should exist under a super-ordinate empowerment
construct (9,10). To test this, we conducted a higher order factor analysis, by assigning each
participant a factor score on each of the seven extracted factors, and submitting these factor
scores to maximum likelihood EFA. The eigenvalues (and associated % of variances) were
3.17 (45.3%), 1.23 (17.5%), 0.95 (13.6%), 0.63 (9.1%), 0.49 (7.1%), 0.31 (4.4%), 0.22
(3.1%). PA indicated that only the first eigenvalue exceeded the eigenvalues that emerged
from 95% of the random datasets (respectively, the first and second eigenvalues were 1.23
and 1.14). This indicates that the seven factors exist under a single higher order construct,
“empowerment”.
Creating the shorter questionnaire:
The aim was to develop a short questionnaire that represents the entire outcome domain, so 35 items were selected from each factor. Items were not selected solely on the basis of the
highest factor loadings, as is commonly done, because for some of the factors, the items with
the highest factor loadings were judged to be too similar to each other, or to include too many
9
negatively worded items. In addition, although the factor structure was stable as sample size
grew over 450, the rank order of items within factors was subject to sampling variation. Items
were selected for sub-scales, informed by (a) the rank order of items within factors following
EFA, (b) those issues that our qualitative research suggested were most troubling for families
affected by genetic conditions e.g. feelings of guilt; concern about how to communicate to
other at risk family members about the condition (9,10,27-29) and (c) clinical judgement
about which items would provide useful information to clinicians about interventions to
improve patient outcomes. This resulted in a final questionnaire, the Genetic Counseling
Outcome Scale (GCOS-24), composed of 24 items (Table 1). Item distributions are shown in
Table 4.
Reliability analyses:
Internal consistency for the questionnaire is 0.87. A test retest reliability study was conducted
on a sample of 64 participants, asked to complete the 24-item questionnaire at two time-points
with an interval of 2-10 weeks. Test retest reliability was estimated by an intraclass
correlation obtained through the use of one-way analysis of variance and was found to be
0.86.
Study 2: GCOS-24: Construct validity and sensitivity to change
Sample characteristics:
In total, 395/1205 (32.8%) patients invited to participate in the study completed and returned
a questionnaire pack prior to their first CGS appointment (T0). 374 of these (94.7%) attended
for their appointment, and of these, 241 (64.4%) completed a second questionnaire pack 2-4
10
weeks after their CGS appointment (T1). For questionnaires with minimal missing data
(<5%), prorating was used to “impute” missing values.
Respondents at T0 ranged in age from 18–79 years, and were predominantly female (74.4%)
and White British (86.6%). A number of different types of genetic conditions were
represented, as follows: 238 (60.3%) cancer; 39 (9.9%) dysmorphology / learning disability;
18 (4.6%) neurological / neuromuscular conditions; 35 (8.9%) cardiac conditions; 12 (3%)
genetic eye conditions; 12 (3%) chromosome abnormalities and 41 (10.4%) other conditions.
155 respondents (39.2%) were referred because they were affected by a (potentially) genetic
condition; 62 (15.7%) because they were the parent of an affected child; 159 (40.3%) because
they might be at risk for developing a genetic condition themselves; 15 (3.8%) because they
might be at risk for having an affected child; 3 (0.8%) for a predictive genetic test (data was
missing for one person). There were no significant differences in gender, ethnicity, condition
type or reason for referral between the samples responding at (a) baseline only (T0) and at (b)
both baseline and Time 1 (T0+T1). However, the T0+T1 participants were significantly older
than T0 only participants (t = -4.69 (390 df); p < .001 (–8.73 < 95% CI < - 3.57)).”
Construct validity and sensitivity to change:
We explored the relationships between empowerment and gender, ethnicity, condition type,
reason for referral, and whether patients described themselves as active in a patient support
group at baseline (n=395). Patients who described themselves as active in a patient support
group were significantly more empowered than those who did not (F = .152; p = .014; 2.13 <
95% CI < 18.72). This provides some evidence of known-groups validity.
11
To test the relationships between empowerment and health locus of control (HLC); PPC,
anxiety; depression; SWL; and authenticity, we correlated GCOS-24 scores with measures of
these constructs at baseline (n=395) (See Table 5). As expected, empowerment was
significantly positively correlated with internal HLC, PPC, authenticity and SWL, with the
highest correlation being with PPC. PPC explains ~30% of the variance in empowerment;
SWL explains ~16.8%, internal HLC explains ~7.3%, and authenticity explains 4.2%. This
demonstrates good convergent validity. As expected, empowerment was significantly
negatively correlated with depression (explaining ~10.5% of the variance in empowerment),
and was not significantly correlated with either fate HLC or powerful others HLC,
demonstrating good divergent validity. Unexpectedly, empowerment was significantly
positively correlated with anxiety. However, the correlation was low, and less significant than
the other significant correlations, with anxiety explaining only ~1.8% of the variance in
empowerment.
To further explore construct validity, we tested empowerment scores before and after clinic
attendance using a paired samples t-test (n=241). Levels of empowerment were significantly
higher following clinic attendance. This also provides evidence of sensitivity to change of the
GCOS-24, with a medium-to-large effect size (Cohen’s d = 0.70).
DISCUSSION
This research has developed a new 24-item genetics-specific PROM, the GCOS-24, which
captures the construct of empowerment (9, 10), and demonstrates good internal consistency,
test-retest reliability, sensitivity to change, and some evidence of construct validity. The
empowerment construct captured by the GCOS-24 combines some aspects of other constructs
that have even operationalized in measurement scales, such as PPC (12) and decisional
12
conflict (30), as well bringing in other dimensions not traditionally included in PROMs such
as hope. An important strength of this research is that it is grounded in extensive qualitative
research with families affected by genetic conditions, and genetics clinicians. Items in the
GCOS-24 reflect outcomes from using clinical genetics that are valued by those stakeholders
(9,10,27-29, 31). The questionnaire differs from questionnaires developed to evaluate clinical
genetics constructed using a priori theorizing e.g. PPC.
PROMs capturing patient empowerment have been developed and validated, specific to some
conditions e.g. the Empowerment Scale (mental healthcare) (32), the Diabetes Empowerment
Scale (diabetes) (33) and the Patient Empowerment Scale (cancer) (34). The present research
has developed a PROM of patient empowerment specific to genetic conditions for use in
evaluating clinical genetics services. The GCOS-24 may also be useful for collecting PROMs
data in routine clinical genetics practice. There are moves in the UK towards using PROMs to
evaluate routine clinical practice for continuous quality improvement (35, 36).
An important strength of this research is the large sample sizes achieved. Although estimated
participation rates for the EFA were low in Study 1, this is unlikely to significantly affect the
findings, because such a large number of people for whom having a genetic condition in the
family is a current issue did complete the questionnaire and the factor structure was stable.
Furthermore, the GCOS-24 is designed to provide information to clinicians on specific patient
benefits, so items in the questionnaire may be useful to clinicians wishing to develop new
interventions to improve patient benefits.
One limitation of this research is that the factor analysis was conducted with a sample of
patient support group members (Study 1). Study 2 provided evidence suggesting that
13
members of patient support groups may be more empowered than patients who are not
support group members, and so the Study 1 sample could be a homogeneous sample in terms
of empowerment, which may affect the validity of the factorial analysis. Although the Study 1
sample, on the face of it, appears to represent a broad range of genetic conditions, there was
no way of confirming whether participants actually came from families affected by a genetic
condition. Also, since participation in patient support groups, as well as participation in both
Study 1 and Study 2 was voluntary, participants may not represent all patients.
Another limitation of this study is that the samples in both Studies 1 and 2 were overrepresented by people describing themselves as female and White British. Furthermore, the
sample in Study 2 was (a) over-represented by patients referred for familial cancer risks, with
very small numbers of patients in some of the other categories e.g. neurological /
neuromuscular conditions, chromosome abnormalities and genetic eye conditions and (b)
under-represented by patients referred for reproductive risks and predictive genetic testing. It
is not known whether or not this reflects the profile of new referrals to the centre, because for
ethical reasons, we could not collect any data from patient files for patients who declined to
participate. In addition, although 43% of participants were offered a genetic test in clinic,
none of these had a result from this by the time they completed their post-clinic questionnaire.
In conclusion, psychometric testing suggests that the GCOS-24 has potential to be a useful
PROM to evaluate clinical genetics services. However, as is commonly the case with new
psychometric instruments, further testing is needed before the GCOS-24 can be unreservedly
recommended for routine evaluation in clinical genetics. In particular, longer follow-up
studies, with non-cancer as well as cancer genetics samples, are required to explore the impact
of genetic counseling and genetic test results on patient empowerment in clinical genetics. It
14
will also be important to explore the impact of reproductive genetic counseling and testing on
patient empowerment, and to test discriminant validity of the GCOS-24 in intervention
studies (37, 38). Now that the GCOS-24 is in the public domain we plan to set up an
anonymised data repository for future analysis and look forward to hearing from interested
parties, who would like to contribute to this repository and/or collaborate in future research. A
collaboration of this kind is currently underway with colleagues in Scotland who will be using
the GCOS-24 for a service evaluation of clinical genetics practice during 2011, and will be
contributing the anonymised data collected to our repository.
15
REFERENCES
1. Wang C, Gonzalez R, Merajver SD. Assessmnet of genetic testing and related counselling
services: current research and future directions. Soc Sci Med 2004: 58: 1427-1442.
2. Payne K, Nicholls S, McAllister M, MacLeod R, Donnai D, Davies L. Outcome
measurement in clinical genetics services: a systematic review of validated measures.
Value in Health 2008: 11(3): 497-508.
3. Williams MS. Quality in clinical genetics. Am J Med Genet 2009: 151C: 175-178.
4. Bowling A. Research Methods in Health, 2nd edn. Maidenhead: Open University Press,
2004.
5. Brooke BS, Habashi JP, Judge DP, Patel N, Loeys B, Dietz HC. Angiotensin II blockade
and aortic root dilation in Marfan’s syndrome. NEJM 2008: 358(26): 2787-2795.
6. Payne K, Nicholls SG, McAllister M, MacLeod R, Ellis I, Donnai D, et al. Outcome
measures for clinical genetics services: a comparison of genetics healthcare professionals
and patients' views. Health Policy 2007: 84(1): 112-122.
7. Stevenson DA, Carey JC. Health-Related Quality of Life Measures in Genetic Disorders:
An Outcome Variable for Consideration in Clinical Trials. AM J Med Genet 2009 :151C:
255-260.
8. Valderas J, Alonso J. Patient Reported Outcome Measures: A Model-Based Classification
System for Research and Clinical Practice. Qual Life Res 2008: 17(9): 1125-1135.
9. McAllister M, Payne K, Nicholls S, MacLeod R, Donnai D, Davies L. Patient
Empowerment in Clinical Genetics Services. J Hlth Psychol 2008: 13(7): 895-905.
10. McAllister M, Dunn G, Todd C. Empowerment: A New Patient Reported Outcome for
Clinical Genetics Services? Eur J Hum Genet online publication 6 October 2010; doi:
10.1038/ejhg.2010.160.
16
11. Tengland P-A. Empowerment: A goal or a means for health promotion? Medicine, Hlth
Care & Philosophy 2007: 10: 197-207
12. Berkenstadt M, Shiloh S, Barkai G, Bat-Miriam-Katznelson M, Goldman B. Perceived
Personal Control (PPC): A new concept in measuring outcome of genetic counseling. Am
J Med Genet 1999: 82A: 53-59.
13. Moss-Morris R, Weinman J, Petrie KJ, Horne R, Cameron LD, Buick D. The Revised
Illness Perception Questionnaire (IPQ-R). Psychol & Hlth 2002: 17(1): 1-16.
14. Gorsuch RL. Factor Analysis (2nd edn.). Hillsdale, NJ: Erlbaum, 1983.
15. Cattell RB. The Scientific Use of Factor Analysis. New York: Plenum, 1978.
16. Wallston KA, Wallston BS, DeVellis R. Development of the multidimensional health
locus of control scales. Health Education Monographs 1978 6:160-170.
17. Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the
Spielberger State-Trait Anxiety Inventory (STAI). Br J Clin Psychol 1992: 31:301-6.
18. Gilbody S, Richards D, Barkham M. Diagnosing depression in primary care using selfcompletion instruments: UK validation of PHQ-9 and CORE-OM. Br J Gen Pract 2007:
57:650-2.
19. Pavot W, Diener E. Review of the Satisfaction with Life Scale. Psych Assessment 1993:
5(2):164-72.
20. Wood AM, Linley PA, Maltby J, Baliousis M, Joseph S. The Authentic Personality: A
Theoretical and Empirical Conceptualization and the Development of the Authenticity
Scale. J Counseling Psychol 2008: 55(3):385-399..
21. Fabrigar LR, Wegener, D.T., MacCallum R.C., Strahan, E.J. Evaluating the Use of
Exploratory Factor Analysis in Psychological Research. Psychol Methods 1999: 4(3):
272-299.
17
22. Floyd FJ, Widaman KF. Factor Analysis in the Development and Refinement of Clinical
Assessment Instruments. Psychol Assessment 1995: 7(3): 286-299.
23. Tinsley HEA, Tinsley DJ. Uses of Factor Analysis in Counseling Psychology Research. J
Counseling Psychol 1987: 34(4): 414-424.
24. SPSS [computer program]. Release 16.0. Chicago, Illinois: IBM inc.
25. Kaiser HF. A second generation little jiffy. Psychometrika 1970: 35: 401-415
26. O'Conner BP. SPSS and SAS programs for determining the number of components using
parallel analysis and Velicer's MAP test. Behav Res Methods 2000: 32(3): 396-402.
27. McAllister M, Payne K, Nicholls S, MacLeod R, Donnai D, Davies LM. Improving
service evaluation in clinical genetics: identifying effects of genetic diseases on
individuals and families. J Genetic Counsel 2007: 16(1): 71-83.
28. McAllister M, Davies L, Payne K, Nicholls S, Donnai D, MacLeod R. The emotional
effects of genetic diseases: implications for clinical genetics. Am J Med Genet 2007:
143A(22): 2651-2661.
29. McAllister M, Payne K, MacLeod R, Nicholls S, Donnai D, Davies L. What process
attributes of clinical genetics services could maximise patient benefits? Eur J Hum Genet
2008: 16: 1467-1476.
30. O’Conner AM. Validation of a decisional conflict scale. Medical Decision Making 1995:
15(1): 25-30
31. Payne K, Nicholls S, McAllister M, MacLeod R, Middleton-Price H, Ellis I, Donnai D,
Davies LM. Outcome measures for clinical genetics services: a comparison of genetics
healthcare professionals and patients’ views. Health Policy 2007: 84(1):112-122.
32. Rogers E, S., Chamberlain J, Ellison ML, Crean T. A consumer-constructed scale to
measure empowerment among users of mental health services. Psych Services 1997:
48(8): 1042-1047.
18
33. Anderson RM, Funnell MM, Fitzgerald JT, Marrero DG. The Diabetes Empowerment
Scale: A measure of psychosocial self efficacy. Diabetes Care 2000: 23(6): 739-743.
34. Bulsara C, Styles I, Ward AM, Bulsara M. The psychometrics of developing the patient
empowerment scale. J Psychosoc Oncol 2006: 24(2): 1-16.
35. Darzi A. High Quality Care for All: NHS Next Stage Review final report, 2008. Available
at:
http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGu
idance/DH_085825 Accessed 1 July 2009.
36. UK Department of Health, 2008. Guidance on the routine collection of Patient Reported
Outcome Measures (PROMs). Available at:
http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGu
idance/DH_092647 Accessed 1 July 2009.
37. McAllister M. Outcome Measures in Clinical Genetics Services. In: Kristoffersson U,
Schmidtke J, Cassiman J-J, editors. Quality Issues in Clinical Genetics Services. London:
Springer, 2010.
38. Streiner DL, Norman GR. Health measurement scales: A practical guide to their
development and use. Oxford: Oxford University Press, 1995.
19