Running Head: QUANTITATIVE CRITIQUE
Quantitative Critique
Amanda George & Quinn Bonner
N310
St. Francis Xavier University
JoAnne Macdonald
April 12, 2013
QUANTITATIVE CRITIQUE
2
Introduction
The following is a critique of a quantitative research study by Manojlovich (2005) titled
Linking the Practice Environment to Nurses’ Job Satisfaction Through Nurse-Physician
Communication. The purpose of this research study was to investigate direct and indirect
relationships among the practice environment, nurse-physician communication, and job
satisfaction. The following critique will evaluate the rigour of this research study by examining
the internal and external validity, reliability, and careful assessment of validity and reliability of
the instruments used. The research studies overall usefulness and contribution to nursing practice
is also assessed to determine potential inclusion in our nursing practice.
Validity
Internal validity is the degree to which the experiment treatment, not an uncontrolled
condition, resulted in the observed effects (Lobiondo-Wood & Haber, 2013). To establish
internal validity the researcher attempts to rule out other factors or explanations of the
relationships between variables. For example, selection bias is a potential threat to internal
validity that we identified in this research study. It is possible that only participants who were
highly motivated to improve nurses’ job satisfaction in the context of the hospital environment
were willing to participate. This threat compromised internal validity because we are unable to
determine if the researcher controlled for this extraneous variable and the potential effect on
research outcomes.
External validity concerns the generalizability of the researcher’s findings to additional
populations and to other environmental conditions (Lobiondo-Wood & Haber, 2013). To achieve
external validity, variation in the conditions and types of participants should lead to the same
results (Lobiondo-Wood & Haber, 2013, p.210). Selection effects were identified as a threat to
external validity in this research study. This identified threat compromises generalizability
QUANTITATIVE CRITIQUE
3
because the researcher did not provide adequate rationale for the proposed sample size of 500
participants. Based on this we are unable to determine if the ideal sample population was
obtained for this research study, especially because over 200 participants were not included
based on inclusion criteria, or did not provide usable data. This selection effect significantly
reduced our confidence in validity because it raises concerns of whether or not the remaining
sample is representative.
Instrumentation
Instrumentation validity refers to whether the instrument accurately measures what it is
intended to measure, and is promoted through content validity, criterion-related validity, and
construct validity (Lobiondo-Wood & Haber, 2013). Content and construct validity of CWEQ-II
and IWS was established through use in previous research studies. Construct validity was
established for PES-NWI and the five subscale structure was supported by confirmatory factory
analysis that tests whether measures of a construct are consistent with the researcher's
understanding of the nature of that construct. Construct and content validity are essential for
overall instrumentation validity because they ensure the content is representative of the content
domain under study, and demonstrates the degree of agreement between the theoretical concept
and its measurement (Lobiondo-Wood & Haber, 2013).
Validity was compromised through changes made to established instruments to remove
subscales that researcher suspected would produce repetitive data. For example, PES-NWI, IWS,
and ICU Nurse-Physician Questionnaire were modified for use in this study and were not
retested for construct validity prior to use. This decreases our confidence in validity because we
are unable to determine the extent to which the instruments listed measures the theoretical
construct or trait under study. There was limited justification for the use of the ICU Nurse-
QUANTITATIVE CRITIQUE
4
Physician Questionnaire on the basis that it was used in a recent study of medical-surgical nurses
and was recommended as appropriate for use in other settings by the author. However, for the
researcher to ensure validity of this instrument for the use with the current sample population
construct validity should have been evaluated.
Reliability of research instruments refers to the extent that the instrument yields the same
results on repeated measures and can be established through methods to ensure stability, internal
consistency, and equivalence (Lobiondo-Wood & Haber, 2013). Reliability was promoted
through the report of Cronbach’s alpha for the instruments used to measure organizational work
environment (CWEQ-II .90 and PES-NWI .93), nurse-physician communication (ICU RN-ND
Questionnaire .93), and nursing job satisfaction (IWS .92). The alpha levels reported for this
research study enhanced the reliability because it indicates a high level of internal consistency
and ensures the instrument measures the concepts the researcher intended to measure. Although
evidence within the research study demonstrates high levels of reliability, this was not congruent
with evidence of validity and prompted us to question the credibility of the research findings.
Reliability
The research design selected for this study was a nonexperimental survey design and was
appropriately selected on the basis that the purpose of this research study was to explore
relationships between variables. The use of a survey design allowed the researcher to collect
detailed descriptions of existing variables and the data was used to assess current conditions ie.
the practice environment of nurses. Appropriate selection of the research design increases
reliability and is a crucial element in the research process because it provides the plan for
answering the research question (Lobiondo-Wood & Haber, 2013). The use of a nonexperimental
design requires a clear and concise research problem based on a theoretical framework. The
QUANTITATIVE CRITIQUE
5
Nursing Role Effectiveness Model (NREM) was identified as the theoretical framework and was
used in examining the three explicitly stated research questions. Although the framework was
presented clearly in the literature review reliability was compromised on the basis that job
satisfaction is absent from the model until presented within the data analysis.
A random sample of 500 nurses was drawn from a list of acute care nurses. Only nurses
who had roles involving patient contact were included as an attempt to create a homogenous
sample, and therefore 284 staff nurses were included. Reliability was compromised because the
researcher provides no rationale for how the sample size was selected, and does not state the use
of power analysis in this process. Sample size is particularly important in quantitative research
because a small sample size can influence the interpretation of the results. When power analysis
is not used research studies may be based on sample sizes that are too small which may lead to a
lack of support for the researchers hypotheses and to a type 1 error (Lobiondo-Wood & Haber,
2013).
We should talk about data analysis here….level of significance (was set at 0.05), are the
inferential statistics used appropriate for the hypothesis, do tables and graphs clearly enhance the
text (I don’t find, if anything, they are confusing), what descriptive statistics were given (ie
mean, median, mode), what tests were done

Pearsons r was calculated as a screening tool to test for associations b/w all
variables of interest, except for hospital environment

kendalls tau

ANOVAS (inferential)

Look at the p and r values recorded for each variable in the findings section… is
there statistical significance ie <0.05
QUANTITATIVE CRITIQUE

6
Regression analysis
Conclusion
The relevance of this research study to nursing practice is clear because research shows
that job satisfaction is still an elusive concept for many nurses. By an improved understanding of
structure variables and process variables that are related to job satisfaction the research
demonstrates that improvements to practice environments may promote job satisfaction among
nurses. Although the study’s findings support the configuration of the NREM there is significant
issues with the sampling procedures and instrumentation that impacts both the reliability and
validity of this research study. We felt that crucial elements of the study were compromised
based on the lack of rationale provided for sampling procedures and difficulties in ensuring the
construct validity of instruments. Given these concerns we find it difficult to determine
generalizability and would not use this article in our own nursing practice.
QUANTITATIVE CRITIQUE
7
References
LoBiondo-Wood, G., Haber, J., Cameron C., & Singh, M. (2013). Nursing research in Canada:
Methods and critical appraisal for evidence-based practice (3rd ed.). Toronto, ON:
Mosby Elsevier.
Manojlovich, M. (2005). Linking the practice environment to nurses’ job satisfaction through
nurse-physician communication. Journal of Nursing Scholarships, 37(4), 3367-373.
Appendix A
Component
Introduction
Validity
Instrumentation
Reliability
Conclusion
Student Contribution
Amanda/Quinn
Amanda/Quinn
Amanda/Quinn
Amanda/Quinn
Amanda/Quinn