Appendix 1. Survey results
A. Demographics
Participants
Men
Value
242
92% (222)
Years in practice
0-5
6-10
11-20
21-30
34% (83)
22% (53)
30% (73)
14% (33)
Geographic location
North America
Europe
Other
52% (126)
37% (90)
11% (26)
Specialization
General orthopaedics
Orthopaedic traumatology
Shoulder and elbow
Hand and wrist
Other
4.6% (11)
37% (89)
16% (39)
36% (86)
7% (17)
What is your primary practice/institution setting?
Academic
Multispecialty group
Large private practice, 10 or more providers
Small private practice, less than 10 providers
Hospital-employed practice
51% (123)
13% (31)
12% (30)
5.4% (13)
19% (45)
Supervise trainees
88% (214)
How do you perceive the risk of litigation in your practice environment?
Very low
Low
Neutral
High
Very high
14% (35)
39% (95)
23% (56)
19% (47)
3.7% (9)
How many hours a month do you spend on keeping up with published research
(journal clubs, reading journal articles, visiting meetings, etc)?
13±10
Political orientation
Very liberal
Moderately liberal
Libertarian
Moderately conservative
Very conservative
27% (66)
33% (80)
8.3% (20)
17% (40)
15% (36)
Do you believe in God, any other deity/deities?
Yes, God is a major force in my life
Yes, but I don’t practice religion often (eg, go to church)
No opinion
No, but I consider myself an agnostic (I don’t know if these exist) rather than an atheist
27% (66)
33% (80)
8.3% (20)
17% (40)
No, I am an atheist
15% (36)
Would you consider yourself a religious person?
Yes, and I worship regularly
Yes, but I do not worship regularly
Moderately, or I don’t really think about it
No, not really, but I don’t rule it out
Not at all, I am an atheist (I believe there is no God)
20% (49)
25% (61)
13% (32)
26% (62)
16% (38)
B. Uncertainty
How much uncertainty is there in your daily medical decisions?
The majority of what I do is understood and established in the literature; there is little
uncertainty in daily medical decisions I encounter in my practice
I occasionally come across something that I recognize as being uncertain in my practice
I frequently regard medical decisions as being uncertain
I almost always see uncertainties in my daily medical decisions
In my practice, the medical decisions I make are always uncertain to various degrees,
and there is almost always no absolute truth to the matter at hand
What percentage of your patient contacts is associated with substantial
uncertainty in diagnosis or treatment options?
Less than 10%
10%-30%
30%-50%
50%-70%
70%-90%
More than 90%
How confident are you in the correctness of your daily routine diagnostic or
therapeutic decisions?
My decisions are based on the established literature and are therefore always correct
I am nearly always sure of the decisions I make
Approximately 25% of the time I feel an element of insecurity about the decision I am
making
I am unsure of about half of the decisions I make
I am unsure of more than half of the decisions I make
When I make a decision about an uncertain topic that has no published evidence
supporting it, I feel:
This does not happen, because just about everything is well worked out and the
decisions I make are supported by evidence
Confident that I am making the right choice, although I recognize that there is no "right
answer”; I don't really think that I am wrong, but rather that the optimal answer may not
exist
Neither confident nor unconfident; the decision is made based on the standard of care
established by my specialty
Not at all confident; I feel uneasy about making these types of decisions because I think
that I might be wrong
Very uneasy; decisions like this make me very uncomfortable and I try to avoid them; I
am probably wrong
C. Trust in the orthopaedic literature
Has the acquisition of more and more knowledge in your field made you more
confident in your decisions?
13% (31)
36% (88)
35% (84)
9.5% (23)
6.6% (16)
40% (96)
40% (95)
16% (38)
5.0% (12)
0.41% (1)
0% (0)
3.7% (9)
51% (124)
41% (100)
3.7% (9)
0% (0)
1.2% (3)
62% (149)
30% (72)
6.6% (16)
0.83% (2)
No, more and more experience and "knowledge" has only made me see that nothing is
really certain at all
No, I have gained more knowledge but with more knowledge has come more recognition
of what we don't know
I am not sure
Yes, I am more confident but my decisions are still at times uncertain; I recognize the
limitations of what we know
Yes, with more experience and knowledge I am much more confident now and I make
better decisions
It is permissible and justified to chose to do a surgery based on my personal
estimation that it will help the patient, even if the evidence does not support it?
Agree strongly
Agree, but not strongly
Sometimes agree, depending on the type of intervention
Disagree, but not strongly
Disagree strongly
Do you think that evidence-based medicine guidelines should limit physicians’
autonomy in making surgical decisions?
No, I do not think that my autonomy should ever be constrained; the evidence applies
only to populations, not to individuals, and we physicians should be free to decide on
treatment plans even if it goes against established evidence
No, but when we make a decision that goes against established evidence or guidelines,
we should be expected to provide reasons why before the treatment is accepted or
covered by insurance
No, but when we make decisions that go against established guidelines, we are in a very
gray area and we have an ethical obligation to tell our patients and insurance providers
that we are not sure
Not sure
Yes, but there should be times when we can petition the governing body/insurance
carrier to allow nonevidence-based treatments
Yes, we should not prescribe treatments that are not evidence-based and that do not
have published evidence supporting them
4.7% (11)
35% (82)
3.4% (8)
46% (108)
11% (25)
13% (30)
27% (64)
50% (116)
5.1% (12)
5.1% (12)
14% (33)
43% (101)
22% (51)
7.7% (18)
11% (25)
2.6% (6)
How often do orthopaedic research studies draw incorrect conclusions?
Most published research articles are false or misleading
Approximately half of published articles have incorrect or misleading conclusions
A minority of published articles have incorrect or misleading conclusions
It’s rare that published articles have misleading or incorrect conclusions
0.85% (2)
44% (102)
55% (129)
0.43% (1)
D. Confidence bias
Rate your diagnostic skills compared with other practitioners in your field.
Top 5%
Top 25%
Above average
Average
Below average
Bottom 25%
Bottom 5%
20% (47)
37% (87)
26% (60)
17% (40)
0% (0)
0% (0)
0% (0)
Rate your surgical skills compared with others in your field.
Top 5%
25% (58)
Top 25%
Above average
Average
Below average
Bottom 25%
Bottom 5%
If a paper from a leading institution says that 80% of their patients had good or
excellent outcomes, the chance that in your patients will have a similar outcome
is:
Far better; my patients almost always exceed published outcome
Slightly better than the published results
The same; no reason to think my results would be any different
Worse by a little; the expected outcome in the literature is idealized and likely biased and
it is likely that my patients will not exceed published outcome reports
Much worse; I am likely to be not as skilled as the authors, given their prominence and
reputation
29% (69)
30% (70)
15% (34)
0.85% (2)
0.43% (1)
0% (0)
2.1% (5)
9.4% (22)
57% (133)
30% (71)
1.3% (3)
E. Statistical understanding
A patient has a painful shoulder; imaging studies do not show a rotator cuff tear;
however, tendinosis is noted; the patient is not satisfied after physical therapy for
several months and has had no relief with three cortisone injections; an
arthroscopic decompression and bursectomy are done and the bursa was noted to
be hypertrophic at the time of surgery; the patient’s pain resolves completely and
he has no more symptoms and complete return to function; does this prove that
the hypertrophic bursa was the cause of the pain and that the surgery cured the
condition?
Yes, this is proof beyond doubt; the tendinosis was the cause of the pain and the surgery
cured it
Yes, to the degree that things like this offer proof, this does “prove” it; the pain could
have resolved on its own, but the surgery likely got rid of the painful bursa and therefore
“cured” the patient
I can't say
Not in a strict sense; proof of causation is not established here but it is very probable that
the cause of the pain was the hypertrophic bursa and the surgery got rid of the pain
generator, curing the symptoms
No, this is a logical fallacy; the pain relief and surgery are associated but there is a
causal relationship has not been demonstrated (correct)
1.7% (4)
16% (37)
12% (27)
47% (110)
24% (56)
In interpreting a paper on outcome of a surgical intervention that shows 80% good
and excellent outcomes, when applied to a particular patient, 80% good/excellent
means:
That there is an 80% chance that in this patient the result will be good or excellent
That of 100 patients, 80 would be expected to have good or excellent outcome (correct)
That this patient can expected to be 80% healed after treatment
12% (27)
87% (204)
1.3% (3)
An MRI you are going to order for a patient has a (hypothetical) sensitivity of 90%
and a specificity of 90%; if there is a 10% chance the patient has the condition,
then:
A positive result means that there is a 90% chance of the condition being present
The test increased the probability of a positive result to only 50% (correct)
A positive result increases the probability to less than 20%
58% (135)
11% (26)
6.0% (14)
We cannot calculate this because only a subjective probability is given and not a base
rate
A (hypothetical) newly developed MRI technique has a sensitivity and specificity of
95% to detect a specific cartilage lesion; approximately one in a thousand people
have this lesion; which of the following statements is true?
A positive result on this test means that there is a 95% chance that the result is true (a
true-positive)
A positive result on this test means that the chance the patient has the condition is
approximately 50%
A positive result on this test means that the chance the patient has the condition is
approximately 2% (correct)
A negative result on this test means that the disease is ruled out to 95% probability
There are 4 cards as shown, and each card has number on one side and a letter on
the other: which cards do you have to turn over to validate or invalidate the rule “if
there is a vowel on one side, the other side has an even number”?
Just the A card
The A and the 4 cards
The A and the 7 cards (correct)
The D and the 4 cards
p value < 0.05 means:
The chances are less than 1 in 20 that a difference would be found again if the study
were repeated
The same as a p value of < 0.01
The probability is less than 1 in 20 that a difference this large could occur by chance
alone (correct)
The chance is better than 95% that the study is correct
F. Other questions
You are confronted with an uncertain situation, perhaps a decision to operate or
treat nonoperatively, where both choices seem reasonable and the decision is
difficult; your response to a patient asking you a question about how the outcome
would be different between the two is (pick your most typical response):
I don’t know which treatment is best
I’m not entirely sure, and I am going to try to look this up
We, in the medical community, don't know the answer to that; some things are just not
known
I make a guess based on what is most probable, because patients do not respond well to
uncertainty, and my role is to not only treat to the best of my ability, but to provide
reassurance
This does not happen often enough to me in my field of specialty practice for me to have
a typical response; most of what I do is quite certain and well studied
When it comes to decisions that are uncertain and without an established
consensus, I make a decision based on (pick the best):
My past experience
What the least invasive choice is
What the patient wants
What I think the medical community would most accept
What I am feeling at the time
25% (59)
57% (127)
4.0% (9)
13% (28)
26% (59)
20% (44)
61% (137)
13% (30)
5.4% (12)
7.2% (16)
2.7% (6)
80% (178)
10% (23)
22% (49)
13% (30)
40% (90)
19% (43)
4.9% (11)
54% (121)
13% (30)
23% (51)
7.2% (16)
2.2% (5)
How has your confidence changed since you graduated training?
Decreased
Unchanged
Increased
7.6% (17)
7.2% (16)
85% (190)