BioEd Online

The sponsors of the largest ever HIV vaccine trial yesterday hailed a "historic" moment as they formally announced the trial's results at an international AIDS vaccine meeting in Paris. The results received rapturous applause from an audience of more than 1,000 HIV researchers.

But some scientists are much more sceptical of the findings, arguing that the response of the HIV research community, long deprived of any good news from vaccine trials, is based more on hope than on rigorous science.

The US$119-million phase III trial, sponsored by the health ministry of Thailand and the US Army, started in Thailand in 2003. It enrolled 16,402 people, who were split evenly into a control group and a vaccine group. People in the vaccine group were given four shots of ALVAC-HIV, an attenuated canarypox virus carrying HIV genes, and two shots of AIDSVAX, a recombinant form of the gp120 HIV surface protein.

The trial's results were published on 20 October, to coincide with the meeting, in the New England Journal of Medicine¹. Researchers have been impatient to see the full data since the team announced on 24 September that the treatment had cut the the risk of HIV infection by nearly one-third (see Vaccine protects against HIV virus). The results are a "milestone in HIV vaccine research", says Supachai Rerks-Ngarm, the paper's first author, and a researcher at the Thai Ministry of Public Health.

"Because the history of preventive interventions against HIV has been so poor, the HIV research community has seized upon this," says Peter Smith, a tropical epidemiologist at the London School of Hygiene and Tropical Medicine, and an expert on statistics. "But at most it is a glimmer of hope. There is not much evidence from the data that it protects at all."

Data crunch

The trial was set up to measure the number of people in each group who became infected with HIV, and the amount of the virus that was circulating in the blood (the viral load) of those who became infected during the trial.

The teams analysed infection rates in three ways. One, denoted 'intention-to-treat' (ITT), included the full cohort of 16,402 people. This analysis found that 56 of those in the vaccine group became infected, which was 20 people — or 26% — fewer than in the control group. That difference was not statistically significant.

Another analysis, called 'per protocol', excluded 3,860 people who failed to strictly adhere to the trial protocols, such as the calendar of vaccinations. This also showed a difference of 26%, again not statistically significant.

A third analysis — the only one to be presented on 24 September before publication of the full data — used a 'modified ITT' data crunch, which excluded seven participants who contracted HIV between the time they enrolled in the trial and their first vaccination. This lowered the number of people who became infected in the vaccine group from 56 to 51, and the number in the control group from 76 to 74, yielding a 31% difference between the groups. This just scraped into statistical significance, with a p-value of 0.04.

The small numbers of infected individuals in the trial — 132 across both vaccine and control groups — also meant that none of the subgroup analyses was statistically significant.

Caution and scepticism

The ITT analysis is generally considered as the main yardstick of the outcome of drug clinical trials, although an mITT analysis is also acceptable if agreed by independent experts. But for vaccine trials, the per-protocol analysis is the most valid, says Adel Mahmoud, former president of Merck Vaccines and now a molecular biologist at Princeton University in New Jersey.

"The results of this trial should be treated with caution and some scepticism," says Tim Peto, a researcher in tropical diseases and clinical medicine at the University of Oxford, UK. "Taken together with the disappointing results of previous vaccine studies, it is likely that the results could have been due to chance alone," he says. "The authors do not seem to acknowledge this possibility."

"My view is that a more balanced interpretation of the data is that the results show some evidence that the vaccine might be effective and that, unlike previous vaccine studies, this study cannot clearly rule out that the vaccine is ineffective," says Peto.

Nelson Michael, a researcher at the Walter Reed Army Institute of Research and director of the US Military HIV Research Program, which co-organized the trial, argues that including people who didn't stick to their shots reflects a real-world vaccination scenario, and that excluding those who were HIV-positive before the trial began was justified. Although the team admit that any protection from the vaccine is "modest", Michael says that even the trends that are not significant are worth exploring, as they might open up new avenues of research.

Blood work

The trial also failed to detect any difference between the viral load in the two cohorts. Mahmoud describes this as "very, very disturbing", because an effective vaccine would be expected to at least reduce the viral load in infected subjects. Michael, however, suggests that this finding might still prompt new vaccine leads, and that scientists should see if they can uncover the immunological origin of any possible vaccine protection.

Dan Barouch, a HIV vaccine researcher at Harvard University in Cambridge, says that the results are ultimately positive. "We don't understand why we saw the protection that we did, and the results are only modest, but nevertheless the Thai trail provides the first evidence of vaccine protection in humans. This suggests that it is in fact possible to develop an HIV vaccine."

"Everyone is saying let's try to have hope, and this is a hope that the results mean something," says Mahmoud, "but raising expectations with no fundamental scientific base is dangerous."

"The field is desperate for some kind of positive result," agrees one HIV vaccine researcher, speaking on condition of anonymity. "There is a strong tendency in the community to want to believe - it's hope versus rational science."