BODE3 Interactive League Table – how to use it

Last week we introduced the concept of league tables to compare interventions. This week we provide a brief ‘user guide’ for our just launched BODE³ Interactive League Table.  We walk through how to pull down tables and graphs of health gain (quality-adjusted life-years; QALYs), health system costs and cost effectiveness for 50+ interventions currently in the interactive league table.

In our blog last week we introduced the BODE³ Interactive League Table, and focused on how it is important to know what you are comparing – the intervention description, and what it is being compared to.  We argued that policy-making and research prioritisation could be improved with not only stand-alone simulation studies and cost effectiveness analyses of interventions, but also by comparing interventions side-by-side in a league table.

What is the BODE³ Interactive League Table, and how does it work?  First, what we have put up has data on 67 interventions evaluated by us (often variants around a theme like dietary salt reduction). This list will grow over time, and also hopefully grow to include Australian interventions in addition to the New Zealand ones.  Second, we are focusing on the ‘basics’ only in this version of the League Table: QALYs, net cost and (when not cost saving) incremental cost effectiveness ratios (ICERs) of the intervention.  Future versions of the League Table will have additional outputs that we will describe in coming weeks.  Third, you can select (ranked) tables and figures of QALYs, costs and ICERs.

Here is what the front page looks like:

By the red numbers shown on the picture:

1. The DOMAIN button. Select from domains of cancer, injury prevention, salt and tobacco (further domains, e.g. diet and obesity, will be added in the future).
2. The INTERVENTION button. Select from listed interventions (select all, or scroll down and individually click on/off interventions).  Interventions shown will depend on what domains (step 1) are selected.  Note that some of the interventions, for now, are what we call ‘heterogeneity’ evaluations, e.g. colorectal cancer screening by ethnicity.  In the future, they will be (de)activated by a separate button, for reasons we hint at the end of this blog, and will explain more in a future blog.
3. The TAB button. At the moment, you can select between Tables, Plots and Cost Effectiveness Plane options.  NB: When selecting the ’Plots’ button, it will take some seconds – up to 30 seconds – for the plot to be constructed, so we recommend you get your selections sorted using the Table Tab initially.
4. The INTERVENTION description and details. As per our last blog, the description is important – so you know what you are comparing with what.  The comparator column shows what the actual intervention is being compared with – which is usually what we call a “business-as-usual” or do nothing projection. Also note that by dragging the slider across at the bottom of the screen, you can also see a column on duration and frequency of the intervention, and lastly a link to the actual publication (which provides extra detail on the methods and results).  And, of course, there are the outputs of QALYs, costs and incremental cost effectiveness ratio (ICER; see more below).
5. The UNCERTAINTY button. Hit this, and extra columns are shown in the table with the 2.5^th and 97.5^th limits for the 95% uncertainty interval, for each of QALYs, costs and ICERs.  These uncertainty intervals are generated from the models run (usually) 2000 times, each time drawing different plausible values for each input parameter (e.g. % change in incidence rate, uptake, intervention cost, etc).
6. The PLOTTING VARIABLE button. Currently, the choices are QALYs, costs and ICERs – the list of options will expand in the future.

Once you have selected a range of interventions to examine, then hit the ‘Plots’ tab (i.e. 3 above).  Allow up to 30 seconds, and you will then have a histogram of interventions ranked by whatever ’plotting variable’ you selected.  For example, something like this for QALYs for selected salt interventions:

(Whilst not the focus of this ‘user guide’ blog, it is hard not to be struck by the large variation in intervention effectiveness – more on this in future blogs.)

To copy these plots into your own presentations and such like, right click on it, click ‘save image as’, and follow your mouse and own preferences.

Finally in this blog, some comments about the output at the BODE³ Interactive League Table so far:

• QALYs and net costs are for the entire eligible population over the remainder of their lifespans.
• The intervention cost is also shown. The net cost is the intervention cost minus (usually) cost-offsets due to reduced disease rates (costs the health system less) but also increased longevity (costs more – but way into the future).
• We have elected to show heterogeneity results (i.e. by age, ethnicity, and biological sub-type for cancers) in this first version of the League Table. But please note that these heterogeneity results are totals for the entire population. So, for example, the same QALYs gains for non-Māori and Māori actually means that per capita gains for Māori are much more. How much more?  This gets tricky, requiring expressing QALYs (and costs) per capita…. which requires defining eligible populations…. which is sometime easy, but sometimes not!  For example, should QALY gains from tobacco control be per capita across all Māori and non-Māori? Or just Māori and non-Māori smokers?  And should they be adjusted for age?  These are the issues we are working through now, to be part of the next release of the BODE³ Interactive League Table later this year.

We encourage you to explore this League Table.  We do not know of similar examples anywhere internationally.  There are static versions (e.g. the ACE-Prevention Report) and non-graphed versions (e.g. Tufts Cost Effectiveness Registry) – but nothing to our knowledge that currently has the functionality of the BODE³ Interactive League Table.

We want your feedback.  Is it useful, or not?  Are the plots useful?  Feedback through the comments to this blog is preferred, but if you prefer to not make your feedback public please email directly bode3@otago.ac.nz.

Contributions

Dr George Disney set up this prototype using the R Shiny App, and Dr Giorgi Kvizhinadze now runs and updates it.  Dr Nisha Nair put considerable time and effort into conceptualisation.  The rest of us supported and cajoled from the sidelines.