Reevaluating health metrics: Unraveling the limitations of disability-adjusted life years as an indicator in disease burden assessment

Abstract

In 1993, the World Bank released a global report on the efficacy of health promotion, introducing the disability-adjusted life years (DALY) as a novel indicator. The DALY, a composite metric incorporating temporal and qualitative data, is grounded in preferences regarding disability status. This review delineates the algorithm used to calculate the value of the proposed DALY synthetic indicator and elucidates key methodological challenges associated with its application. In contrast to the quality-adjusted life years approach, derived from multi-attribute utility theory, the DALY stands as an independent synthetic indicator that adopts the assumptions of the Time Trade Off utility technique to define Disability Weights. Claiming to rely on no mathematical or economic theory, DALY users appear to have exempted themselves from verifying whether this indicator meets the classical properties required of all indicators, notably content validity, reliability, specificity, and sensitivity. The DALY concept emerged primarily to facilitate comparisons of the health impacts of various diseases globally within the framework of the Global Burden of Disease initiative, leading to numerous publications in international literature. Despite widespread adoption, the DALY synthetic indicator has prompted significant methodological concerns since its inception, manifesting in inconsistent and non-reproducible results. Given the substantial diffusion of the DALY indicator and its critical role in health impact assessments, a reassessment is warranted. This reconsideration is imperative for enhancing the robustness and reliability of public health decision-making processes.

Key Words: Disability adjusted life years; Cost-utility analyses; Outcome research; Public health; Burden of disease

Core Tip: Despite its adoption for conducting burden of disease studies, the disability-adjusted life years synthetic indicator has raised substantial methodological concerns since its introduction, leading to inconsistent and non-reproducible outcomes, and consequently, flawed results and public health rankings.

INTRODUCTION

In 1993, the World Bank released the "World Development Report 1993: Investing in Health"[1], a comprehensive international global report exploring the intricate interconnections among human health, health policy, and economic development. This report investigated the economic viability of health promotion initiatives to enhance debt repayment capacity. To evaluate the impact of significant diseases on health, the report introduced the disability-adjusted life years (DALY) as a novel health indicator.

The DALY, a synthetic metric incorporating temporal and qualitative data, shares similarities with the quality-adjusted life years (QALY) indicator. However, unlike the QALY, which relies on preferences regarding Quality-of-Life situations, the DALY utilizes preferences related to disability conditions, encompassing cognitive, developmental, intellectual, mental, physical, or sensorial impairments, as well as those resulting from multiple factors.

In the 1990s, Murray and Lopez[2] introduced the DALY concept at Harvard University for the inaugural global burden of disease study. They employed the DALY to quantify the health impacts of over 100 diseases across eight world regions, as outlined in the 1993 World Bank report[1]. However, the immediate and widespread criticism of the DALY arose[3]. Notably, the World Health Organization Advisory Committee in Health Research established the DALY review group[4,5], which asserted that the DALY approach was "an obscure theoretical exercise which remains unvalidated"[4].

Consequently, the DALY indicator saw limited use since the late 1990s due to the ensuing methodological controversy. During this period, the QALY indicator gained prominence as the preferred reference metric. The National Institute for Health and Care Excellence (NICE) in the United Kingdom[5] and other Health Technology Assessment agencies, including those in Canada and Australia[6], endorsed the QALY as the benchmark indicator. Despite sharing similar methodologies and facing comparable methodological challenges[7-9], NICE continues to recommend the QALY as the reference case for aiding reimbursement decisions for innovative medicines in the United Kingdom. The prominence of the QALY in NICE drug submissions has played a pivotal role in bolstering the acceptance of the DALY indicator in the scientific literature.

Notably, with backing from the Bill & Melinda Gates Foundation, the DALY regained prominence in 2007 with the establishment of the Institute for Health Metrics and Evaluation. This institute was founded by the originator of the DALY concept[9]. Subsequently, in 2017, the institute secured a substantial funding of 279 million USD for the execution of the Global Burden of Disease project over a decade, relying on the DALY indicator[10].

CALCULATING DALY

According to Murray[3], calculating the number of years of life lost by an individual or a population has two components:

The difference between an optimal disability-free lifespan and the years actually lived. These are the years of life lost by premature death.

A virtual loss of years of life due to a duration of disability caused by an illness.

Unfortunately, Murray is careful not to define what an "optimal" lifespan is, or what a "premature death" is. How is the premature death estimated? How does one determine the age for an optimal life duration without disability? How is the life expectancy estimated in the population considered at the age of death?

Similar to the QALY approach, the DALY methodology combines information about morbidity and mortality in numbers of healthy years lost. In the DALY approach, each state of health is assigned a disability weight (DW) on a scale from zero (perfect health) to one (death).

To calculate the burden of a certain disease, the DW is multiplied by the number of years lived in that health state, which is added to the number of years lost due to that disease. Future burdens are discounted at a rate of 3% per year, and the value of the lifetime is weighted so that years of life in old age are less valued.

DALY = years of life lost (YLL) + years lost due to disability (YLD), where DALY = YLL + YLD.

Component YLL (years of life lost)

If N is the number of deaths in the study population and L the optimal lifespan duration without disability deducted from the average lifespan consumed in the cohort or in the generation studied, then: YLL = N × L.

Hence, YLL is the number of years of life "lost" as a result of the so-called premature mortality. Most of the time, YLL is calculated using complex models. Of note, in this formula, the years of life lost are supposed to be lived without disability, which is a quite curious hypothesis because it assumes that all premature deaths would only concern healthy people without disabilities (such as sudden deaths only).

Component YLD (years lost due to disability)

If I is the number of incident cases, DW a coefficient of disability, and L the average duration in the population considered of the disease up to the cure or up to death, then: YLD = I × DW × L.

L is the average duration of the disability (time with disability). It should be noted that this “L” is different than the one in the YLL formula.

The equivalent of this duration, but lived without disability, is assumed to be proportional to the average duration lived with disability noted α L, with 0 ≤ α < 1, then the average lifespan lost without disability is equal to L- α L = (1- α) L.

Considering that 1- α = DW, then the number of years of life lost as a result of this disability equal to L × DW, which is proportional to the average number of years lived with a disability.

Component DW (Disability coefficient)

Such as the Q of the QALY, which requires measuring a utility of Quality-of-Life, the D of the DALY requires measuring a usefulness of disability by the disability coefficient DW.

The disability coefficient DW presented by Murray[11] as a utility measure between 0 (perfect health) and 1 (death), thus represents an equivalence between the year of life lived with and without disability in the frame of time trade-off (TTO) type arbitrations[12], such as:

"You are anticipated to experience a year with a disability due to an illness you have contracted. What proportion of this year's duration (DW) would you be willing to forfeit to regain the remaining period free from disability?".

Responding to TTO questions is inherently challenging as it involves estimating the extent of one's willingness to sacrifice lifetime to avoid living with a specific disability. To circumvent the complexity of conducting intricate TTO surveys, Murray initially introduced a DW scale consisting of six disability classes based on "expert opinions," with predefined DW coefficients[11]. In the original publication, the disability coefficients were not derived from patients' direct preferences but were determined by a group of healthy "independent experts" assumed to represent the nebulous concept of "society." This "society" decided on the values of disability coefficients through a voting process, bypassing direct surveys with actual patients. The specific expertise of these independent experts and their recruitment process remain undisclosed, leaving room for alternative groups of experts to generate different sets of values. A systematic review by Whitty et al[13] in 2014 confirmed the presence of inconsistencies when incorporating public preferences as weights for priority setting. The well-known weaknesses of the TTO and preference revelation techniques directly affect the reliability of the DALY indicator.

ARBITRARY ESTIMATION OF THE YLD

The construction of the YLD formula is theoretically grounded in the TTO technique, wherein an individual considers trading off a portion of their remaining healthy life years to avoid experiencing disability during a specified time period[12]. For instance, if a respondent indicates a willingness to trade 6 years of full health to evade a defined disability (D) for 10 years, it is inferred that the "utility" of D is 60% of the utility of perfect health, resulting in the DW coefficient being assigned a value of 0.6.

It is crucial to note that the principle of the TTO method diverges from that of the standard gamble (SG) technique[12].

The TTO aims to directly estimate the overall utility (u) of the situation "living X life years with a disability Y" by assuming that u(X,Y) = t w(Y), where t represents time, and w(Y) denotes the utility of Y. In contrast, the SG technique is rooted in multi-attribute utility theory, and the TTO formula lacks a mathematical demonstration or underlying economic theory, rendering it arbitrarily defined.

This elucidates why different individuals or expert panels may yield markedly distinct TTO results when estimating DW. For instance, a significant number of people might be inclined to trade more healthy years to avoid blindness compared to paraplegia, while others may hold the opposite preference.

Anand et al[14] published that the conceptual and technical basis for DALYs is flawed, and its assumptions and value judgements are open to serious question, especially because implications of age-weighting and discounting are found to be unacceptable.

Consequently, validating any social tariff for Disability Weights becomes challenging, as it essentially reflects the researcher's subjective choices. The proponents of the DALY approach tend to advocate for the utilization of a specific DW social tariff, such as the ones they originally published[11]. However, it is essential to recognize that any designated DW social tariff is inherently arbitrary and can significantly influence the ranking of DALY results, thereby impacting the scientific credibility of international DALY comparisons.

INDIVIDUAL AND COLLECTIVE DISABILITY WEIGHT

DW social tariff estimated either by expert opinions or by patient surveys would suggest to be able to “aggregate” the votes of the different experts or the preferences of different patients.

In order to do so, it would be necessary to aggregate the collected individual preferences from the subjects suffering of the same disease into a collective preference. This problem was raised by Nicolas de Condorcet (1743-1794)[15] and investigated in 1951 by the Nobel Prize in Economics Kenneth Arrow (1921-2017) who is the author of the impossibility theorem[16], a social-choice paradox illustrating that there is no procedure to define the preferences of a group based on the preferences of its members, if one wishes to satisfy four basic conditions (transitivity of preferences , respect for the principle of unanimity, non-manipulable aggregation procedure, rejection of dictatorial rule). Each attempt to obtain a collective preference therefore would require that the procedure has been manipulated or is not transitory. Whatever, the principle of transitivity seems essential if one wants to rank and compare diseases, which is also the primary objective of the global burden of disease project[11]. Arnesen et al[17] consider that published disability weights have been estimated using “forced” consistency between questions that address different issues and should be addressed as artefacts affecting the validity of the global burden of disease results. Hence, the conclusions of research initiatives using the DALY as the main health indicator should be interpreted with extreme circumspection considering that the DALY methodology can generate highly divergent results.

DALY AND SENSITIVITY TO CHANGE

The DALY formula is a simple additive and multiplication function, inspired from the QALY formula[7] but published without any description of its underlying assumptions. Because this formula is not supported by any scientific economic theory, there is no real assumptions to be able to validate using either mathematical or experimental demonstrations.

Whatever, should we consider the DALY as a simple synthetic indicator, it is at least possible to test its sensitivity to change, such as for any health indicator measure. That means that a change of value of one component of the formula should lead to different DALY results.

For this purpose, we could use the following example. Let’s assume that a subject lived 30 years in good health, then contracted a degenerative neurological disease that lasted 40 years, before dying at the age of 70 years old.

In this case, let’s consider that the DW disability coefficient for this disease is estimated at 0.4 (this means that a year of life with this disability is considered to be equivalent to 0.6 years of life without disability), while the lifespan without disability is estimated at 90 years.

In this example, the number of years of life lost as a result of premature death, i.e. a death before the "optimal" age of 90 years old equals to: YLL = 90 - 70 = 20 years.

The number of years of life lost due to the disability generated by the disease equals to: YLD = 40 × 0.4 = 16 years.

Then the number of DALY (i.e. the number of life years lost adjusted for disability) equals to the sum of these two components: 20 years + 16 years = 36 DALY.

However, a number of other health scenarios with the same disability coefficient (DW = 0.4) could also lead to the value 36 DALY, as presented in Table 1.

Table 1 Comparison of age profiles of death and disease durations with identical disability-adjusted life years values.

Age of death	Number of sick years	YLL	YLD	DALY
54	0	90-54 = 36	0 × 0.4 = 0	36 + 0 = 36
56	5	90–56 = 34	5 × 0.4 = 2	34 + 2 = 36
58	10	90–58 = 32	10 × 0.4 = 4	32 + 4 = 36
62	20	90–62 = 28	20 × 0.4 = 8	28 + 8 = 36
70	40	90–70 = 20	40 × 0.4 = 16	20 + 16 = 36
90	90	90–90 = 0	90 × 0.4 = 36	0 + 36 = 36

DALY: Disability-adjusted life years; YLL: Years of life lost; YLD: Years lost due to disability.

As demonstrated in this example, 36 DALY can represent a number of very different health situations which cannot be discriminated, confirming the poor sensitivity of the DALY indicator which can not differentiate situations as different as sick during 90 years, and never sick with death at 54 years old. This table illustrates how the DALY metric can result in the same value for vastly different situations, highlighting the potential inconsistencies and limitations of the indicator. By showcasing these varied scenarios side by side, the table demonstrates how different health conditions, age groups, or combinations of disability and mortality can all lead to identical DALY values, despite representing very different realities. These simple examples contribute to underscore the lack of sensitivity and specificity in the DALY indicator.

DALY AND LIFE EXPECTANCY VARIABILITY

The number of years of life "lost" (YLL) representing the premature mortality depends on the optimal life duration without disability (L), which is highly dependent on populations and countries.

Would it be the maximum life duration observed, such as 122 years in Europe[18]? Or would it be the mean life expectancy worldwide? But in this case how to take into consideration people who live longer that the mean?

This number of years of life without disability also depends on the diseases, which makes difficult cross disease comparisons. Consequently, international comparisons of results expressed in the number of years of life lost measured in DALY are far from reliable. For example, it is important to ensure that in each country the "optimal" lifespan used is the same. It is also essential to consider the variability of the disability weights either defined arbitrarily by expert opinions, or by non-reproducible individual preference surveys.

Should we assume that one social tariff of disability weights is robust and consistent in one country (which has never been the case…), this social tariff would not be relevant in any other countries. Also, life expectancies are not similar among countries. Given these major methodological deficiencies, it is effectively impossible to interpret any results expressed in DALY, such as the Global Burden of Disease initiative, for which any international comparisons or comparisons between chronic diseases remain highly hypothetical[19-22].

The ethical issues of the DALY indicator are deeply tied to these hidden methodological assumptions. One of the primary ethical concerns arises from the TTO technique's assumptions about age groups. The TTO often incorporates age weighting, which values individuals differently based on their age, typically assigning more value to the lives of middle-aged individuals compared to younger children or older adults. This approach implies that some lives are more valuable than others, leading to ethical concerns about discrimination against the very young and the elderly. Such weighting perpetuates the notion that the health and lives of older adults and young children are less important, thereby fostering ageist biases.

The impact of these assumptions on vulnerable populations is significant. For instance, elderly and disabled individuals could have their health needs deprioritized because the DALY framework tends to prioritize interventions for those perceived as having more "productive" life years remaining. Also, the choice of DALY as a health metric has far-reaching consequences on health equity, particularly in the context of resource allocation. By prioritizing interventions that reduce DALYs the most, health policies would favor diseases that affect younger and well-weighted disabilities, sidelining the needs of older adults or other disabilities, and leading to a reinforcement of existing social and economic inequities.

CONCLUSION

In 1997, the DALY review Group of the WHO Advisory Committee on Health Research had concluded that the DALY should be considered as mere intellectual exercises, as the DALY approach is too far from the reality to be used in priority settings[6]. This statement is consistent with the concerns about the QALY indicator, as any attempt to aggregate information about the length of life and Quality-of-Life in one single dimension raises many conceptual issues and impose to adopt a number of underlying assumptions[7]. This is because the DALY shares similar theoretical concerns of the QALY plus one specific issue, which is the necessary arbitrary valuation of the DW. This parameter is most of the time estimated by expert opinions who might be compelled to adopt discriminatory opinions. TTO surveys have also been proposed for estimating DW but it is not possible to construct any valid group preference from individual preferences unless one “dictator” imposes its choice[16].

From the ethical perspective, the concept of the DALY assumes that the life of disabled people would have less value than the life of people without disabilities[17]. Then diseases with higher DALY would lead to lower cost/DALY ratios, which raises questions about the profitability of health optimization investment strategies, as described in the 1993 World Bank Report “Investing in Health”[1].

The increasing economic pressures on healthcare systems around the world, particularly with the emergence of new respiratory virus pandemics, and the significant methodological issues and instability of the DALY indicator, underline the importance of using validated methodologies for conducting robust international epidemiological and health economic studies for assisting and prioritizing public health decisions. Establishing best practices for economic evaluation in public health research, either publicly or privately funded, would ensure that limited resources are optimally allocated to maximize health outcomes worldwide.

Not based on any multicriteria choice theory, the properties of the synthetic DALY indicator, such as validity, reliability, specificity, and sensitivity, have still not been validated to date, despite its recurring use in international comparisons whose results are therefore questionable and should be considered with utmost caution. Our analysis leads us to doubt the proper adherence of these properties for the DALY indicator.