Approved COVID-19 vaccines have been tested in tens of thousands of people in Phase 3 clinical trials before being issued for emergency use in the United States. The clinical trials assess the vaccines’ efficacy and safety.

Last year, the U.S. Food and Drug Administration (FDA) mandated that vaccine manufacturers would need to demonstrate an efficacy of at least 50% in order for a vaccine to be approved for emergency use. So far, vaccines made by Pfizer-BioNTech and Moderna have been authorized for use in the U.S. after their clinical trials demonstrated efficacy substantially above the FDA’s “goalpost,” and which were shown to be safe for widespread use.

A third vaccine made by Janssen Pharmaceuticals (Johnson & Johnson) was approved by the FDA last month. On April 13, the Centers for Disease Control and Prevention (CDC) and the FDA began reviewing data involving U.S. cases of a rare, severe type of blood clot in individuals who received the J&J vaccine. In accordance with CDC guidance and out of an abundance of caution, Sutter Health and other health systems are  pausing use of the Janssen/J&J vaccine.

You may have heard or read reports about the efficacy of vaccines being administered across the U.S. Here we describe what the numbers mean to help you understand the data:

Lowering the Risk

Efficacy is a measurement of how much a vaccine lowers the risk of an outcome (in this case, getting COVID-19). Vaccine manufacturers tested how many people who received a vaccine in clinical trials still got COVID-19. Then they compared that to how many people became infected with the disease after receiving a placebo. The difference in risk can be calculated as a percentage, and stated as efficacy.

For example, if a (hypothetical) vaccine demonstrated 90% efficacy, that means it reduced the risk of COVID-19 by 90% compared with placebo. It does not mean that 90% of people who received the vaccine won’t get COVID-19 and 10% will, but that a person’s risk of getting the disease decreases by 90% after vaccination.

Study’s Parameters Matter

Vaccine efficacy as demonstrated in clinical trials depends on the parameters of the study, such as where it took place and what strains of the virus were circulating during the study, whether the study measured efficacy against all disease or only severe disease, as well as a vaccine’s dosing schedule and its mechanism of action (or, how it works). Janssen, for example, tested its vaccine in clinical trials across the U.S., Latin America (Argentina, Brazil, Chile, Columbia, Mexico and Peru) and South Africa. The vaccine’s overall efficacy was lower than the efficacy shown in the U.S. alone.

Janssen and other researchers believe this may be because the study arm in South Africa took place after a COVID-19 variant began spreading across that country. The efficacy against all COVID-19 disease in South Africa was lower (at 64%) but equivalent to efficacy demonstrated in the U.S. (85% protection against severe, life-threatening disease).

Furthermore, because the vaccines use different ways to engage the immune system, the differing technologies could lead to varying degrees of efficacy. Pfizer-BioNTech and Moderna developed their two-dose vaccines using mRNA technology whereas Janssen’s single-dose viral vector vaccine uses a deactivated cold virus that is modified to stimulate an immune response to the SARS virus that causes COVID-19.

Different Outcomes Measured

Vaccine efficacy is reported based on different outcomes measured in the clinical trials. The three vaccines approved for use in the U.S. were measured by their overall efficacy, efficacy against severe forms of COVID-19, as well as their effectiveness in lowering hospitalizations and death. A vaccine’s efficacy rate against severe disease, for example, measures its ability to prevent hospitalizations and death.

“It is difficult to precisely compare the vaccines that have been approved in the U.S. because they were tested in different groups of people at different stages of the pandemic. In addition, vaccine efficacy was measured in the clinical trials in different ways for one- versus two-dose vaccines,” says Jeffrey Silvers, M.D., Sutter’s medical director of infectious diseases. “But the value of these approved vaccines is in preventing severe-to-critical COVID-19, and it cannot be understated how important that is as we continue efforts to help end the pandemic.”

‘Effectiveness’ Being Studied Now

Efficacy means how well a vaccine works based on testing in clinical trials, while “effectiveness” refers to how well these vaccine will work in the real world, in much larger groups of people. Vaccines tested in Phase 3 clinical trials will continue to be studied for their effectiveness with further, widespread use. In the coming months, researchers will assess how much the approved vaccines reduce the risk of COVID-19 in millions of people versus tens of thousands who participated in the clinical trials, and to what extent they may remain effective in preventing COVID-19 as new variants arise.

Which Is Best? Here’s the Bottom Line 

It may be tempting to rank these vaccines, but the bottom line is that “all three vaccines approved for use in the U.S. to date greatly reduce the risk of COVID-19, and are critical tools in our ability to quell the pandemic and prevent hospitalizations and death,” says Dr. Silvers.

Find answers to commonly asked questions about approved COVID-19 vaccines.   

Learn more about vaccine emergency use authorization and other phases of testing and review by the FDA.