JAIDS Journal of Acquired Immune Deficiency Syndromes

BACKGROUND

In resource-limited settings, the management of diseases is difficult due to limited diagnostic capabilities, inadequate treatment options, and overburdened staff. Checklists and treatment algorithms are an efficient way for staff with a wide range of skill sets and experience to quickly and accurately diagnose and treat patients, in a consistent manner. Although these techniques have significant advantages, their cost effectiveness is rarely examined. Maximizing the use of limited resources is essential for long-term sustainability. Creating cost-effective algorithms and checklists is one such method to achieve this objective.

The management of central nervous system (CNS) infections is difficult across all settings due to the high mortality and morbidity rates if proper treatment is not initiated promptly. Common CNS infections include bacterial meningitis, cryptococcal meningitis (CM), tuberculosis meningitis (TBM), viral meningitis, and various types of encephalitis and cerebral abscesses.¹ In resource-limited settings, case management of CNS infections is extremely difficult with in-hospital mortality rates of 17%–67% for bacterial meningitis, 40%–69% for TBM, and 19%–50% for CM in Sub-Saharan Africa.^2,3 Of clinical importance is the prevalence of HIV/AIDS, and how AIDS influences disease prevalence rates and outcomes. With an estimated 9.7–11.5 million children and adults in Sub-Saharan Africa requiring antiretroviral therapy (ART) in 2010,⁴ opportunistic infections are expected to continue for the foreseeable future requiring clinical tools to enhance diagnosis and treatment of CNS infections.

In an effort to reduce patient morbidity and mortality associated with nonoutbreak meningitis, a meningitis diagnostic algorithm was developed and implemented at the Mulago National Referral Hospital in Kampala, Uganda, and Mbarara Regional Referral Hospital in Mbarara, Uganda, where the most common causes of death are CNS infections, pneumonia, and tuberculosis (TB).⁵ With high patient loads, limited financial resources, and inadequate number of clinicians, CNS treatment algorithms and diagnostic checklists could be useful tools to assist in the diagnosis and treatment of meningitis; however, the overall costs and diagnostic yield of the algorithms should be assessed. Therefore, the goal of our analysis was to evaluate the costs, diagnostic yield, and cost effectiveness of the current CNS algorithm and develop and evaluate alternative algorithms that would enhance case management without draining resources.

METHODS

Decision Model

We used the methods of decision analysis to evaluate the costs and diagnostic yield associated with various meningitis diagnostic algorithms. We evaluated 3 diagnostic strategies for HIV-infected adults: (1) comprehensive testing, (2) stepwise testing, and (3) minimalist testing. We focused on these patients because the HIV prevalence in sub-Saharan Africa is high and differential diagnosis for meningitis symptoms among HIV-infected adults is broader, often including cryptococcosis and TB. In a decision tree framework, we assigned true disease status to a cohort of individuals with signs and symptoms for meningitis, based on the prevalence of 4 possible disease states: bacterial meningitis, CM, TBM, or “other” disease. On the basis of the sensitivities and specificities of the tests used, each strategy yielded 1 of 4 final diagnoses: bacterial meningitis, CM, TBM, or “other meningitis,” which triggered clinical management of the diagnosed disease. The “other meningitis” diagnosis includes suspicion of viral meningitis and individuals with negative results, with the assumption that supportive care treatment would be administered for all individuals when a definitive diagnosis was not achieved. Additionally, we assumed only 1 infectious etiology can occur at 1 time, and that all diagnostic tests are dichotomous (ie, positive for a particular etiology or negative for that etiology) and conditionally independent. Simultaneous CM and TBM coinfection can occur but are rare (0.2%).⁶ CD4 counts were not incorporated into the model as the time to receipt of CD4 results varies by health center.

The “comprehensive strategy” mimicked the diagnostic algorithm utilized in the Cryptococcal Optimal ART Timing trial (clinicaltrials.gov: NCT01075152), which was a randomized strategy trial to determine if early or deferred ART was optimal for 6-month survival. The comprehensive strategy is representative of a common “shotgun” approach many clinicians in middle and high-income countries use when ruling in and ruling out disease or infection. In this strategy, all (available) diagnostic tests would be ordered simultaneously by the clinician with the laboratory running all diagnostic tests simultaneously (see Figure S1, Supplemental Digital Content,https://links.lww.com/QAI/A405). The second, “stepwise strategy” limited the number and order of diagnostic tests performed, prioritizing tests with a high sensitivity for the most prevalent diseases. Diagnostic tests were run sequentially until a diagnosis was made or all available diagnostic tests were performed (see Figure S2, Supplemental Digital Content,https://links.lww.com/QAI/A405). The stepwise algorithm allowed for additional TB diagnostic testing (eg, Xpert MTB/RIF) on small subset of selected high-risk patients. The “minimalist strategy” used a similar sequential stepwise testing approach, except significantly limiting the number and order of diagnostic tests to high-yield tests only, eliminating tests with poor sensitivity/specificity (see Figure S3, Supplemental Digital Content,https://links.lww.com/QAI/A405).

Model Inputs

We used data obtained from 416 HIV-infected patients screened with suspected CNS infections from November 2010 to November 2012 at Mulago national Hospital in Kampala, Uganda, and Mbarara Regional Referral Hospital, Mbarara, Uganda. We estimated the underlying risk for 1 of 4 disease states: bacterial meningitis, CM, TBM, or “other” disease. Of the 416 patients, 4% had suspected bacterial meningitis, 8% had probable/definite TBM, 59% had laboratory confirmed CM, and 29% had suspected viral meningitis or other (nonmeningitis) diagnosis. In the sensitivity analysis, we used prevalence data from published studies conducted at GF Jooste Hospital in Cape Town, South Africa; at Queen Elizabeth Central Hospital in Blantyre, Malawi; and Harare Central and Parirenyatwa Hospitals in Harare, Zimbabwe (Table 1).^6–8 All studies had a robust sample size with a high prevalence (>75%) of HIV/AIDS.

The meningitis diagnostic algorithms utilized the most common diagnostic tests available. The diagnostic testing probabilities (sensitivity and specificity values) varied by strategy with the dichotomous cut-off values to determine positive and negative test status being obtained from the medical literature (Table 2).^9–29

The 2012 costs of diagnostic testing were obtained from the South African National Health Laboratory System. The total cost included reagents, quality controls, disposable laboratory supplies, labor, external quality assurance testing, laboratory overhead, and a margin.³⁰ We assumed an exchange rate of 1.00 USD to 8.34 South African Rand (January 1, 2013). Each diagnostic testing strategy included the total cost of all tests performed within that strategy. The model focused on the diagnostics cost only, without inclusion of treatment or hospitalization costs that would vary substantially by country and diagnosis.

Analysis

The model outcome values were clinical effectiveness and expected cost. Clinical effectiveness was defined as the percentage of cases for which the algorithm result matched the true disease state (ie, correct diagnosis). Average costs were based on the summation of all tests within a given strategy. The effectiveness of each strategy was determined by the weighted average cost of the summation of all of the pathways within the model.

To determine the relative benefit obtained for additional resources spent, we calculated the incremental cost-effectiveness ratios (ICERs) to determine the incremental gains in percent correct diagnosis compared with the additional costs incurred.³¹

The number needed to test (NNT) to prevent 1 missed diagnosis (and the associated adverse effects and cost) was also calculated for each strategy. NNT is helpful for understanding the relative effectiveness of the strategies and was used to measure the amount of effort required to obtain a positive change in outcome among the various diagnostic tests. NNT was calculated by dividing 1 over the incremental percent correct diagnosis. The decision model was analyzed in TreeAge Pro, 2013 edition (Williamstown, MA).

Sensitivity Analysis

A 1-way sensitivity analysis was conducted based on the prevalence of disease across all 3 countries to gain an increased understanding of the impact of varying disease prevalence rates have on the expected performance of the diagnostic algorithms. Due to the highly variable performance of TB diagnostics, a probabilistic sensitivity analysis (±10% SD) among TB diagnostics and disease prevalence [95% confidence interval (CI)] was performed.

RESULTS

In Uganda, the current comprehensive strategy of ordering a full diagnostic work-up to correctly identify a CNS infection resulted in 93.3% probability of a correct diagnosis at an average cost of $32.00 per patient. The stepwise strategy resulted in 93.8% probability of a correct diagnosis at an average cost of $9.72 per patient (95% CI: $5.56 to $34.85), and the minimalist strategy resulted in 91.1% correct diagnoses averaging $6.17 per patient (95% CI: $2.50 to $19.68). If the minimalist strategy was implemented, nearly 5 additional patients could be diagnosed for the same cost as 1 patient using the comprehensive strategy, although the percent of those diagnosed correctly would decrease by 2.2%. If the stepwise strategy was implemented, an additional 3.3 patients could be diagnosed for the same cost as 1 patient using the comprehensive strategy, and the expected effectiveness would increase by 0.5% (Table 3).

Within each strategy, the number of correct diagnoses varied by true disease state, with TB meningitis having the lowest percent diagnosed correctly across all strategies (22%–52%) in part due to the difficulty of diagnosis. In both the minimalist and stepwise strategies, the number of correct, confirmed diagnoses for bacterial meningitis were lower (70% and 90%, respectively); however, the comparison reflects an assumption of ideal testing characteristics with the lumbar puncture (LP) being performed before the administration of antibiotics. In reality, in many settings, antibiotics are given before performing the LP; thus, there is near zero additional diagnostic yield by bacterial cerebrospinal fluid (CSF) cultures after antibiotic administration.^32–34

The percent of correctly diagnosed “other meningitis” disease was high across all strategies, which can be partially attributed to the high specificity values of bacterial meningitis, TBM, and cryptococcal diagnostic tests (≥99%). The high percentage of false negative “other disease” results occurred primarily among individuals with true bacterial or TB meningitis. This can be attributed to the lower sensitivities of the diagnostic tests used to identify bacterial or TB meningitis.^14–16

The relative incremental costs of the 3 diagnostic testing strategies were compared to the increase in diagnostic yield. When implementing the stepwise strategy compared with the minimalist strategy, the incremental cost increase was $3.55 per person with a 2.7% improvement in diagnostic effectiveness. The increase in diagnostic yield was primarily for TBM. When the Xpert MTB/RIF Assay (Cepheid, Sunnyvale, CA) was included in the stepwise strategy, using a theoretical 60% estimated sensitivity, the number of correctly diagnosed TBM persons increased substantially from 22% to an estimated 52%. The Xpert MTB/RIF assay would be the most expensive of the diagnostic tests evaluated (∼$15 total assay cost using $9.98 cartridge cost + shipping + labor) and if included in the comprehensive strategy would increase cost to $47 per patient (with identical effectiveness as the stepwise strategy). By using a stepwise approach, an Xpert MTB/RIF (or similar higher tech TB diagnostics) could be included in a targeted approach for lower overall cost, yet delivering more care, when needed.

When implementing the stepwise strategy, the NNT to accurately identify the disease state of 1 person through improved diagnostic testing was 37.5 persons with a cost per additional diagnosis being $133. When implementing the stepwise strategy instead of the comprehensive strategy, the incremental cost savings was $22.28 per patient, with an additional 0.5% increase in effectiveness by focusing additional resources for selective TB testing. Thus, the stepwise strategy dominated the comprehensive strategy (Table 4).

Similar results were found when we used the prevalence rates from Malawi, South Africa, or Zimbabwe in that the minimalist approach had the lowest diagnostic yield and lowest expected cost; the stepwise had the highest diagnostic yield; and the comprehensive approach had highest expected cost per patient tested (Table 5).

Assuming a 6-month survival rate of approximately 40% for bacterial, TB, and CM,^2,35–38 the cost in diagnostics per life saved in Uganda for using algorithms compared with the minimalist strategy would be $333 plus treatment costs when using the stepwise and $2882 plus treatment costs for the comprehensive. Treatment costs vary by disease from $3 for meningococcal meningitis with 2 g of ceftriaxone intramuscularly once to $15 for 5 days of ceftriaxone for pneumococcal meningitis plus hospitalization cost.³⁹ Amphotericin-based treatment of CM costs $218–$402 per World Health Organization guidelines.²

Not all initial clinically suspected meningitis cases will ultimately have meningitis infection. Based on local institutional medical practice and patient acceptance rates of LPs, the prevalence of negative diagnostic LPs may range up to ≥50% (ie, persons receiving a diagnostic LP who do not have any meningitis).^6,7,40 To examine the impact of the prevalence of nonmeningitis individuals, we modeled the costs of LP diagnostics for these individual without meningitis within each diagnostic strategy. As the prevalence of persons without meningitis increased, the diagnostic testing costs increased such that when prevalence of no meningitis reached 50%, costs increased to $8.95 in the minimalist strategy, to $16.80 in the stepwise strategy, and to $48 in the comprehensive strategy per actual meningitis case (Fig. 1).

In the Ugandan setting, ceftriaxone was administered before LP in 34% of 281 persons diagnosed with either CM or TBM (28% received >1 day, mean 4.9 ± 3.3 days). This ineffective therapy ($2/d in 2012)⁴¹ accrued an estimated $936 in ceftriaxone expense, averaging $3.33 per person with CM or TB. Distributing this cost to the overall cohort resulted in an additional cost of $2.25 per person ($936/416).

DISCUSSION

Our findings demonstrate the importance of evaluating the cost effectiveness of diagnostic checklists and treatment algorithms, especially in resource-limited settings. The preferred strategy is ultimately dependent on societal or a health system's willingness (or ability) to pay for diagnostic certainty when identifying CNS infections. As the ability to pay varies across settings, understanding the health care system or the setting in which the algorithm or checklist will be implemented is essential to ensure long-term sustainability and maximal efficiency. Physicians have an ethical obligation to advocate for the highest yield testing, and formalizing a diagnostic algorithm can help achieve this as a health system's intervention.

When identifying the ideal implementation strategy, it was important to understand that although the percentage of correctly diagnosed individuals improved as additional tests were performed, costs also rose. The clinical benefits were impacted by the sensitivity and specificity values of the diagnostic tests. The number of correctly diagnosed individuals among the bacterial, TB, and CM states was based primarily on test sensitivity, whereas the “other” disease category was based primarily on the specificity of testing. Within each strategy, the diagnostic yield and costs did not substantially change with varying prevalence rates across the 4 sites; however, the overall cost effectiveness of the strategy was significantly impacted when prevalence rates varied.

Depending on the weight that a clinician or health system places on diagnostic accuracy, the reduced proportion of correctly diagnosed individuals accompanied by decreased costs associated with the diagnostic tests eliminated from the algorithm, could result in substantial cost savings. When evaluating the treatment algorithms, it is important to consider resource allocation. In the minimalist strategy, 91% of individuals were correctly diagnosed. Increasing the percentage of diagnostic certainty by 2.7% by using a stepwise algorithm increased the ratio of incremental costs to effectiveness by $133 per added diagnosis. In a resource-limited area, particularly when preemptive treatment is administered, the reduction in algorithm effectiveness may be acceptable to test and diagnose more of the population. Yet, simply reducing the number of diagnostic tests performed does not automatically equate to poor diagnostic efficacy if performed wisely. As can be seen through reducing the number of diagnostic tests in the comprehensive strategy to a stepwise strategy, this approach saved $22.28 per patient on average while also being able to add targeted TB testing, thereby increasing the overall effectiveness by 0.5%. By developing a stepwise algorithm based on a critical review of the literature and expert opinions, the number of missed diagnoses can be minimized and limited resources can be optimally allocated.

Understanding the impact disease prevalence and test characteristics have on treatment algorithms is crucial. When aggressively performing diagnostic LPs in populations in whom there is a high proportion of persons without meningitis (eg, 50% negative LPs), the costs per patient can increase significantly (by $5.56 in the minimalist strategy, $14.15 in the stepwise strategy, and $32 in the comprehensive strategy) per nonmeningitis patient. Furthermore, an algorithm with poor test sensitivity for diagnosis of a very common disease is not cost effective. The prime example is India ink testing, which has poor sensitivity (60%–84%) for cryptococcosis, the most common etiology of meningitis in East and Southern Africa.^26,42 India ink has a particularly poor sensitivity for persons presenting early in cryptococcal disease course with low CSF fungal burdens. As a result, diagnoses are missed, additional diagnostic work-up is pursued, and/or ineffective treatments are given, and lives are unnecessarily lost. This is true of any test with poor sensitivity when the pretest probability is high.

Additionally, it is important to understand the standard medical practice in the setting in which the algorithm is being implemented. For meningitis, where the majority of suspect cases receive empiric ceftriaxone treatment before LP,³ the large number of theoretically missed bacterial meningitis cases, 30% in minimalist and 10% in stepwise, reduced the effectiveness of the respective algorithms compared with the comprehensive strategy. Yet in actual use among this 416 person cohort, bacterial CSF culture was less sensitive than Gram stain and did not detect any additional cases. One problem with empiric antibiotic treatment for all meningitis cases is that a minority of persons presenting with suspected meningitis (5%–20%) in Sub-Saharan Africa outside of the meningitis belt actually have bacterial meningitis. In South Africa, the reported nationwide cases of Cryptococcus exceed all causes of bacterial meningitis combined.⁴³ The cost of ceftriaxone therapy rapidly exceeds the cost of either minimalist or stepwise diagnostic testing algorithms, and unnecessary use may propagate antibiotic resistance. The new point-of-care cryptococcal antigen lateral flow assay (Immy, Norman, OK),^28,29 and its low cost ($2.50 real world laboratory cost) should change “empiric” meningitis treatment to at least a minimalist diagnostic strategy of first testing for cryptococcal antigen before empiric antibiotics in high HIV prevalence settings.

Furthermore, the current practice of creating an algorithm or checklists based solely on effectiveness, without considering cost, is unsustainable over the long term, especially in resource-limited settings where extensive testing and treatment are often unavailable. For example, the cost of the current comprehensive algorithm of $32, which was designed to optimally diagnose 1 patient's condition (excluding treatment and hospitalization costs) is greater than the per capita annual health expenditure of $24 in Uganda.⁴⁴ The total operating budget for Mulago hospital was $17.20 per hospital bed per day in 2010. A further challenge is the necessary change in medical culture to shift individual physician ordering to a systems-based approach, trading autonomy for gains in efficacy. This culture shift to implement a laboratory-centric diagnostic testing approach may be a substantial challenge requiring well-written standard operating procedures and the ability to opt out when warranted by physician judgment.

Limitations of the study design included assuming conditional independence of diagnostic tests and assuming that only 1 infectious etiology could occur at 1 time. In reality, multiple etiologies are possible, albeit uncommon (0.2% in Cape Town).⁶ Categorizing all nondiseased and other disease (ie, aseptic viral meningitis) under 1 category overestimated the diagnostic certainty of the other disease state. Although the “other meningitis” state has many potential etiologies, such diagnostic testing is typically unavailable in most resource-limited laboratories, as is specific treatment. However, the cost implications associated with negative diagnostic LPs were accounted for in the analysis with costs increasing as the proportion of normal diagnostic LPs increased. Another limitation was that sensitivity and specificity of diagnostic tests were obtained from the literature. In practice, testing performance may have significant variability dependent on laboratory quality assurance, staffing levels, and techniques (eg, AFB stain). In addition, although the results of this analysis are most generalizable to populations with high HIV prevalence (eg, East and Southern Africa), this concept of using disease prevalence and diagnostic testing performance should be generalizable to other settings and other diseases. In other regions, local disease prevalence should guide the stepwise ordering of testing. Understanding regional disease prevalence is important for international guidelines. For example, World Health Organization guidelines for standardized treatment of nonepidemic bacterial meningitis in Africa (written from a West African perspective only)³⁹ are inappropriate in Eastern and Southern Africa by completely ignoring the diagnostic possibility of Mycobacterium tuberculosis and Cryptococcus.

To our knowledge, no previous studies have investigated the cost effectiveness of diagnostic algorithms in resource-limited settings. Our findings suggest that future research should be focused on performing cost-effectiveness analyses on diagnostic algorithms and checklists for additional disease states to inform policy on implementation of cost-effective treatment algorithms and outcomes. The use of diagnostic algorithms in high-income countries may also be warranted to increase cost effectiveness of diagnostic testing, but this is as yet a foreign concept. Increasing the awareness of creating cost-effective treatment algorithms, and checklists will not only improve the management of patient conditions but will also ensure their long-term sustainability in resource-constrained settings.

ACKNOWLEDGMENTS

The authors thank the input from Dr. Graeme Meintjes; Dr. Neal Wetherall for laboratory support; and Drs. Henry Nabeta, Kabanda Taseera, Radha Rajasingham, and Abdu Musubire for care of patients with meningitis in Uganda.

REFERENCES

Supplemental Digital Content