What Does 95% Effective Mean? Teaching the Math of Vaccine Efficacy by Dashiell Young-Saver The New York Times Learning Network December 14, 2020 Lesson Overview Featured article: “2 Companies Say Their Vaccines Are 95% Effective. What Does That Mean?” by Carl Zimmer Last week, after a trial showed that Pfizer’s coronavirus vaccine had an efficacy rate of 95 percent, the Food and Drug Administration approved the vaccine. What is “efficacy,” and how is it calculated? In this lesson, you will analyze the data from Pfizer’s successful vaccine study to get a practical sense of how the vaccine performed. (Spoiler: It did well.) Then, you will analyze what factors could make or break the vaccine’s success as it becomes widely available. Warm Up Watch this video, from 0:56 to 3:00, of Dr. Anthony S. Fauci, the nation’s top infectious disease expert, speaking about the efficacy of new vaccines at a briefing of the White House’s coronavirus task force on Nov. 19. Respond to the following questions in writing or in class discussion: ● ● ● Dr. Fauci says that vaccines have “crushed” previous outbreaks of diseases like smallpox, polio and measles. You probably received one or more of those vaccines when you were young. Did you know what those vaccines were when you received them? Did you know those diseases had deadly outbreaks before vaccines were introduced? Dr. Fauci calls the 95 percent efficacy rate of Pfizer’s coronavirus vaccine “extraordinary.” Does that rate sound impressive to you? Why or why not? Toward the end of the video, Dr. Fauci implies that the ultimate effectiveness of the vaccine will depend on how it is rolled out and received by the general public. What factors might influence how effective a vaccine is when it’s widely released? Activity: Infection Risks and Efficacy Read the featured article. Then, proceed through the following sections to break down the Pfizer study and its data. Study Design: Answer the following questions: ● What is a placebo? Why do you think Pfizer and other drug companies use placebos in their studies? ● ● The Pfizer study enrolled 43,661 people. Why so many people? Would the company’s results be as convincing if it enrolled, for example, just 100 people? Half of the subjects (the study volunteers) were given a placebo. The other half were given the vaccine. This created two groups: the placebo group and the vaccine group. The study used a random process to choose which subjects were assigned to each group. Why would subjects be randomly assigned to the groups? Infection Risk: The 43,661 subjects were split evenly between the placebo and vaccine groups (about 21,830 subjects per group). In the placebo group — the group that got a “fake” vaccine — 162 became infected with the coronavirus and showed symptoms. In the vaccine group — the group that got the real vaccine — that number was only eight. Group Group Size Number Infected Placebo 21,830 162 Vaccine 21,830 8 Infection Risk If in U.S. Population Answer the following questions (and fill in the corresponding table cells): ● ● ● What percentage of the placebo subjects became infected? Hint: Divide the number infected by the total group size, then multiply by 100. What percentage of the vaccine subjects became infected? These numbers are each group’s “infection risk.” Put these numbers in the appropriate places of the “infection risk” column. At the bottom of this lesson we provide a link where you can check your work. Answer the following questions: ● ● Both infection risks are lower than 1 percent. Would you describe the placebo and the vaccine infection risks as low or high? Explain. Is the vaccine group’s infection risk that different from the placebo group’s? Explain. Scale Up: It can be hard to see a vaccine’s impact in terms of percentages. Let’s answer the same questions as above after calculating what the infection risks indicated by Pfizer’s study would look like if applied to the entire population of the United States. Answer the following questions (and fill in the corresponding table cells): ● About 328 million (328,000,000) people live in the United States. Assume all of these people had the same infection risk as those in the placebo group during the months of this study. How many people in the country would you expect to contract the ● ● coronavirus? Hint: Multiply the U.S. population by the infection risk (before multiplying, write the infection risk as a decimal: 0.74% = 0.0074). Assume the U.S. population had the same infection risk as the vaccine group. How many people would you expect to become infected? Put these numbers in the appropriate places of the table’s final column. At the bottom of this lesson we provide a link where you can check your work. Calculating Efficacy Now that we’ve analyzed infection risks, let’s calculate the efficacy. First, find the difference between the risks in both groups. Check your work: _________________________________ So, the vaccine reduced the infection risk by 0.7 percentage points (less than 1 percentage point). This seems small. However, as we know from our earlier analysis, small differences in infection risks can mean big differences in the number of people that become infected. How can we capture that difference in impact? Let’s scale the percentage point difference in risks by the original infection risk. To do this, divide the difference (0.7%) by the infection risk of the placebo group (0.74%). Check your work: _ _______________________________ You just did the same calculations as the scientists at Pfizer. The number you got, 95%, is known as the efficacy. This tells you that, under the same conditions as the study, the vaccine reduces the risk of infection by 95%. Going Further Part I: Threats to Effectiveness Read the following excerpt from the article: Vaccines don’t protect only the people who get them. Because they slow the spread of the virus, they can, over time, also drive down new infection rates and protect society as a whole. Scientists call this broad form of effectiveness a vaccine’s impact. The smallpox vaccine had the greatest impact of all, driving the virus into oblivion in the 1970s. But even a vaccine with extremely high efficacy in clinical trials will have a small impact if only a few people end up getting it. Answer the following questions: ● ● ● Because vaccines prevent new infections, they also prevent further spread. Why would this effect be diminished if few people agreed to get the vaccine? Even impressive vaccines, like Pfizer’s, show a nonzero infection rate among the vaccinated group. Why does this fact make the widespread adoption of vaccines so important? If effective vaccines are taken by enough people, a population can develop “herd immunity” to a disease. What do you think that term means? Do you think the United States will eventually develop herd immunity to the coronavirus? Why or why not? Part II: Your Own Decision-Making A vaccine against Covid-19 is now being distributed in the United States and elsewhere. Do you plan on being vaccinated? Do you want your loved ones and friends to be vaccinated, too? We invite you to share your opinions in the comments of our related Student Opinion question. To check your work, visit this lesson plan on The Learning Network. Dashiell Young-Saver is a high school statistics teacher and the founder of S kew The Script. He moderates for the NYT Learning Network’s weekly feature “ What's Going On in This Graph?”
