New studies show the complexity and importance of HIV epidemiological modelling.

In Isaac Asimov’s Foundation science fiction series, which is set far in the future, mathematical modelling of human society has reached such a sophisticated level that the characters can predict the fall and rise of their civilisation thousands of years ahead. The series’ main mathematician is even able to determine a set of interventions that will shorten the period of barbarism between the collapse of his society and the rise of the next one.

Alas, in 2012, we are barely able to predict the trajectory of a single disease in one country even a few years into the future.

It may seem like fiction of another kind that mathematical models could ever generate enough controversy to involve the president of a country. Yet when the Actuarial Society of South Africa (ASSA) published their AIDS models in the early 2000s, this is exactly what happened. Public debate ignited because contrary to the AIDS denialist views of then President Mbeki, the ASSA models showed that millions of people in South Africa had HIV, hundreds of thousands were dying annually of AIDS, life-expectancy had plummeted and – in the absence of antiretroviral treatment (ART) – the worst was yet to come.

Mathematical models of epidemics estimate important information such as the prevalence, incidence and effect on life expectancy of an epidemic disease at time-points for which there have been no direct measurements. They usually tell us about the state of an epidemic now and in the future under different scenarios.

[box]

Incidence: The number of new cases of a disease that occur in a specific population at or over a given time.

Prevalence: The total number of cases of a disease in a certain population at or over a set time.

[/box]

Model myths

It is seldom that people outside the field have a good understanding of the details of mathematical models. Models can often seem mystical. Perhaps this is why there is a myth that epidemiological models are just based on abstract mathematics and not on real-world measurements.

But this is not true. Good models have a large number of parameters that must be set using the best available peer-reviewed data. Such parameters might be, for example, the risk of a sexual act resulting in HIV transmission, or the effect of ART on a person’s infectiousness. Once all the parameters are set models must be calibrated against reliable epidemiological data, so that their outputs match what is known about the epidemic. This is analogous – albeit in a much more complex way – to calibrating a scale before you weigh yourself. You make sure that the scale points to zero, so that when you stand on it, it will not understate or exaggerate your weight. Likewise, a good mathematical modeller will make sure that if the countrywide HIV epidemic was measured by a reliable survey to be, say, 9% in 2001, that when a model is run, it calculates close to 9% prevalence for that year. If it does not, then the modeller has to revisit the model’s parameters and calculations. It is hard work and good modelling is a highly skilled undertaking.

There is also a belief that models are entirely dependent on the assumptions and biases of the people who develop them, which is partially true, and therefore have no value, which is not true. A household budget is a mathematical model, and many readers of this article have no doubt made one. If done carefully, they are based on real-world measurements and usually predict future expenditure quite well. Most of us find them very valuable. At the risk of oversimplification, the difference between a household budget and the most sophisticated mathematical models of the HIV epidemic is mainly one of complexity.

A comparison of models

Today we see new controversies in HIV demographic modelling, albeit much more interesting and rational than the AIDS denialist arguments of the early 2000s. In 2009 Reuben Granich and colleagues published the results of their model in the journal The Lancet. They predicted that the HIV epidemic in South Africa could be eradicated by 2050 if universal voluntary testing and immediate treatment for all people with HIV were introduced. But using different assumptions about treatment uptake and drug resistance Bradley Wagner and Sally Blower of the University of California Los Angeles reached different conclusions. They published the results of their model in a paper titled, “Voluntary universal testing and treatment is unlikely to lead to HIV elimination: a modeling analysis”.

The open access journal PLoS Medicine has published a set of articles looking at the cost and effectiveness of using antiretroviral treatment to reduce HIV incidence. These papers debate the assumptions, methodologies and conclusions of mathematical models and consequently the affordability and benefits of treatment as prevention.

Dramatic benefits

One of these papers was co-written by the developers of 12 different epidemiological models, including the Granich model. Jeffrey Eaton, Timothy Hallett and colleagues explain, “Each of these models has predicted dramatic […] benefits of expanding access to ART[. B]ut models appear to diverge in their estimates of the possibility of eventually eliminating HIV using ART, the cost-effectiveness of increasing the CD4 threshold for treatment eligibility, and the benefits of immediate treatment compared to treatment based on the current World Health Organization eligibility guidelines. Directly comparing the models’ predictions is challenging because each model has been applied to a slightly different setting, has used different assumptions regarding other interventions, has been used to answer different questions, and has reported different outcome [figures].”

The aim of the research described in the paper was to systematically compare the 12 models by standardising a set of antiretroviral treatment scenarios and reporting a common set of outputs. The intervention scenarios were consistently implemented across the different models with the purpose of controlling “several aspects of the treatment programme and isolat[ing] the effects of [variations in the] model[s] …”

Three variables were controlled across the models and systematically varied: the CD4 threshold for starting treatment, the proportion of eligible people who access treatment and the retention of patients on treatment.

PLoS Medicine’s editor explained the methodology: “To exclude variation resulting from different model assumptions about the past and current ART program, it was assumed that ART is introduced into the population in the year 2012, with no treatment provision prior to this […I]nterventions were evaluated in comparison to an artificial […] scenario in which no treatment is provided.” To compare the models, the authors used a standard scenario based on the World Health Organization’s recommended threshold for ART initiation, although it does not reflect current provision in South Africa.

The methodology of these twelve models varied greatly. For example:

• They used two different modelling methods. Four used microsimulation. In such models, each individual in a population is simulated. Random events that affect their risk of HIV infection are applied to them. This is the most computationally intensive of the modelling methods. Microsimulations can take hours or even days to produce results. The remaining eight models divided the population into groups “according to each individual’s characteristics and HIV infection status and use[d…] equations to track the rate of movement of individuals between these groups.”

• Ten of the models explicitly provided for both sexes and heterosexual HIV transmission.

• Six of the models included age, but the degree to which age affects disease progression, the risk of HIV infection, and the risk of transmission varied amongst these models.

• One model simulated the HIV epidemic in Hlabisa, KwaZulu-Natal, while the remaining models dealt with the national South African epidemic.

The models were compared under three different CD4 cell count thresholds: 200 or lower, 350 or lower, and treatment for all regardless of CD4 count. The proportion of eligible individuals who eventually began treatment was also varied as follows: 50%, 60%, 70%, 80%, 90%, 95%, and 100%. So was the percentage of people still on treatment after three years, excluding those who died, as follows: 75%, 85%, 95%, and 100%.

Outcomes of the different models

The estimates of adult male HIV prevalence in South Africa in 2012, assuming there was no ART, ranged from 10% to 16% across the models. Female prevalence ranged from 17% to 23%. Male incidence ranged from 1.1 to 2.0 per 100 person-years and female incidence ranged from 1.7 to 2.6.

Under the scenario where no treatment is provided, the models varied in their predictions about the future path of the epidemic. Calculations ranged from almost no change in HIV incidence to a 45% reduction over the next 40 years. All the models predicted that treatment would reduce incidence by a large percentage compared to the no-treatment scenario. Their estimates varied, but only across a narrow range. For example, if 80% of HIV-positive individuals began treatment a year after their CD4 count dropped below 350 and 85% remained on treatment after three years, the estimated drop in incidence ranged from 35% to 54% lower eight years after the introduction of ART compared to not providing ART at all. The number of person-years of ART per infection averted over eight years ranged between 5.8 and 18.7. As expected, the further into the future the models went the more their results diverged. This scenario, incidentally, reflects current WHO treatment guidelines coupled with the UNAIDS definition of universal treatment access. i.e. reaching 80% of those in need.

Effect of ART on incidence in South Africa

The authors then did a separate analysis using seven of the models to determine the effect of the actual ART rollout in South Africa on HIV incidence by comparing it with a no-treatment scenario. Models either used their own existing calibrations of the number of people on ART in South Africa or were calibrated using estimates of the number of adults starting and already on ART in each year from 2001 to 2011.

All of the models predicted that ART has reduced HIV incidence. The estimates ranged from 17% to 32% lower in 2011 than in the absence of ART. Interestingly, the models gave widely different estimates of the effect of ART on prevalence. For example, one model estimated prevalence at 8% higher than it would have been without treatment, while two others calculated no net change in current prevalence. It is worth bearing in mind that an increase in prevalence does not mean a failed response to the HIV epidemic. On the contrary, the only way prevalence can decrease is if more people die than are infected. Since ART keeps people alive, it is unsurprising that several models predict an increase in prevalence. Incidence, not prevalence, is the measure of the success of prevention efforts.

Test and treat

The impact on incidence of a CD4 threshold of 200 versus 350 versus treatment with very high rates of HIV screening and the removal of CD4 eligibility – the latter known as the test-and-treat approach – were also compared across the 12 models. Here the results were not consistent. Some models showed that moving from 200 to 350 would not make a substantial difference, but that moving from 350 to treating everyone would. Others found that moving from 350 to treating everyone made little difference.

In an intervention treating all HIV-positive adults with 95% ART access and 95% retention, only three of nine models predicted that HIV incidence would fall below 0.1% per year by 2050, the virtual elimination threshold proposed by Granich and colleagues.

Explaining the differences between models

The authors put forward three hypotheses to try and explain the differences between their models. These were (1) differences in the fraction of transmission that occurs after people become eligible for ART in the no-treatment scenario, (2) differences in how effective ART is at reducing transmission and (3) different assumptions about what happens to patients who drop out of care. These hypotheses were tested and only accounted for some of the differences in model outcomes.

Although the models’ estimates diverge, collectively they can help policy makers and provide approximations of how successful antiretroviral treatment will be at reducing incidence. Also, we should remember that the models were not all designed to answer the identical questions.

The effectiveness of treatment as prevention is a question that will have to be answered more definitively with clinical trials as well as observational studies of actual practice. Over the next few years, controlled trials in South Africa, Zambia, Tanzania and Botswana will hopefully provide these answers.

The future of mathematical modelling

It is important to realise that disease modelling at the level of sophistication seen in these models is a relatively new field, made possible by the tremendous increase in computer power over the last few decades. Microsimulations in particular stretch the capabilities of even today’s computers and computer programmers. The widely different methodologies and assumptions used should be seen as pioneering efforts in a new science. Hopefully over time, and as the predictions of models are compared to what actually happens, modellers will be able to identify techniques that are robust and standardise the science. Just as the 95% confidence interval and the correspondence of a p-value less than 0.05 with significance are a standard part of medical statistics today, similar standard concepts might emerge in the modelling field. And just as R and STATA are standard software tools used by the vast majority of medical statisticians, so there will hopefully one day be standard tools for both deterministic and microsimulation mathematical models.

An effort to standardise modelling is already underway. Wim Delva and colleagues have published an article in the same PLoS Medicine collection summarising extensive discussions between mathematical modellers. They describe nine principles for modellers and those who depend on HIV models to make policy decisions. The (edited) principles are:

1. The model must have a clear rationale, scope and objectives.

2. The model structure and its key features must be explicitly described.

3. The model parameters must be well-defined and justified.

4. The way the model has been calibrated must be explained and justified.

5. The model’s results must be clearly presented including uncertainties.

6. The model’s limitations must be described.

7. The model must be contextualised. In other words previous studies must be referenced and the similarities and differences explained. Differences in the results of the model and previous models must be described.

8. The model must show epidemiological impacts that can be used for studies of health economics.

9. Models must be described in clear language.

These principles surely apply to all disease modelling, and not just to HIV.

Disease modelling is important. Models help us understand the relative contribution of different factors to the present state of the HIV epidemic. They also give us some understanding of what would happen in the future under different interventions. Models are valuable for making policies with short and medium-term impacts. Longer-term projections, such as to the year 2050, are less useful given that so many unpredictable technological and demographic changes are likely to occur over such a long period of time. Aside from the practical value of disease modelling, this is a fascinating theoretical field featuring elegant mathematics and computer algorithms.

Sources: Granich RM et al. ,‘Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model.’ Lancet 373: 48–57. doi:10.1016/S0140-6736(08)61697-9. (2009). http://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2808%2961697-9/abstract; Wagner BG and Blower S. 2009. ‘Voluntary universal testing and treatment is unlikely to lead to HIV elimination: a modeling analysis.’ doi:10.1038/npre.2009.3917.1 (2009) http://precedings.nature.com/documents/3917/version/1/html; Eaton JW et al.. ‘HIV Treatment as Prevention: Systematic Comparison of Mathematical Models of the Potential Impact of Antiretroviral Therapy on HIV Incidence in South Africa.’ PLoS Med 9(7): e1001245. doi:10.1371/journal.pmed.1001245 (2012) http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1001245 ; Cohen M et al.. ‘HIV Treatment as Prevention: Debate and Commentary—Will Early Infection Compromise Treatment-as-Prevention Strategies?’ PLoS Med 9(7): e1001232. doi:10.1371/journal.pmed.1001232 (2012). http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.1001232 ; Delva et al. 2012. ‘’HIV Treatment as Prevention: Principles of Good HIV Epidemiology Modelling for Public Health Decision-Making in All Modes of Prevention and Evaluation.’ PLoS Med 9(7): e1001239. doi:10.1371/journal.pmed.1001239. http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.1001239

 

By Nathan Geffen