This blog responds to a recent online critique of a study that modelled how key components of the Smokefree Aotearoa Action Plan would affect smoking prevalence.
Given the interest in the study due to developments with the Action Plan and associated legislation before Parliament, we published our findings as a pre-print while it was undergoing peer-review at an academic journal.
The online critique made several criticisms of the methods of our study and the conclusion that mandated denicotinisation of smoked tobacco products would likely profoundly reduce smoking prevalence and health inequities. The key criticism – that we relied mainly on evidence from a single randomised controlled trial (RCT) of very low nicotine cigarettes (VLNCs) – is incorrect. Further, the critique and associated press coverage incorrectly imply that the case for mandated denicotinisation is weak and thus that the policy is not justified. The exact impact of mandated denicotinisation is uncertain because it has never been implemented outside of research studies which only partially simulate the policy. However, modelling studies, trials, other evidence and careful logical analysis of the policy strongly suggest it will be highly effective as the key policy to dramatically lower smoking prevalence and reduce health loss and inequalities when implemented in the Aotearoa/New Zealand (A/NZ) context.
In April 2021, the A/NZ Government published a discussion document for consultation that included proposals for measures to include in an action plan for achieving the Smokefree Aotearoa 2025 goal. The discussion document recognised the disproportionate impact of smoking on Māori and included a strong commitment to addressing smoking related inequities. It included a proposal to reduce nicotine in all smoked tobacco products to minimal (non-addictive) levels. A preliminary high level modelling study to estimate the likely impact of mandated denicotinisation led by NW was performed after publication of the discussion document. This work was published in a peer-reviewed journal in January 2022.1
The A/NZ Ministry of Health commissioned a more sophisticated modelling study from researchers at the University of Melbourne. This modelled the impact of the three key proposed policies (denicotinisation of smoked tobacco products, 90% to 95% reduction in tobacco retail outlets, and a smoke-free generation) on smoking prevalence and through to: health adjusted life years gained of the policies compared to business-as-usual (BAU); reductions in Māori:non-Māori mortality inequities; changes in future health expenditure; and changes in future income earnings of the A/NZ population.
The Regulatory Impact Statement prepared prior to release of the final Smokefree Aotearoa Action Plan in December 2021 referred to the findings of this study. The first output of the study, with co-authors from the University of Otago, was a report in January 2022 to the Ministry. The second output, with co-authors from University of Otago and Indigenous authors from A/NZ and Australia, included slightly updated modelling based on a more formal expert knowledge elicitation process (described below). It was published as a pre-print in July 2022 for transparency whilst it was undergoing peer-review at an academic journal. The findings (described below) were very similar to those in the report submitted to the Ministry. Further outputs that focus on economic and morbidity impacts are in preparation.
On 7 November 2022, a highly negative online critique of the pre-print paper was published together with a media article.
Response to the online critique
The critique alleges 10 problems with the modelling study. The key assertion, which underpins problems 1-5, is that the estimated impact of the denicotinisation policy on quitting among people who currently smoke is exaggerated because it derives from a misinterpretation of findings from a single NZ randomised trial of denicotinised cigarettes (Walker et al).2
This assertion is incorrect, as Table 2 of the pre-print makes clear; this table describes the model parameters and states the estimated impact of denicotinisation on quit rates was derived from an expert elicitation process.
This process occurred after considerable discussion among the authors of the available evidence (RCTs of VLNCs, responses from smokers to questioning on whether they would quit with mandatory denicotinisation, the physiology of nicotine and addiction, and other relevant research) and the A/NZ policy context. For example, we discussed that participants in trials of VLNCs still live in a world where they can buy and access regular-nicotine content cigarettes – a very different situation from mandated denicotinisation when only VLNC tobacco products are available in retail outlets.
The senior author (TB) led the process, positing his estimates first of the most likely, least and greatest reduction in smoking prevalence among people who currently smoke over BAU that would occur five years after a mandatory denicotinisation policy was implemented. TB then elicited the same estimates, independently, from five co-authors of the study with the necessary content expertise. TB then constructed a beta distribution (see excerpt from Appendix of the pre-print that is at the end of this blog) using the average of the authors’ estimates (most likely, lowest, highest) as the mean, 2.5th and 97.5th percentiles, respectively, of the beta distribution.
Appended at the end of the blog is the text from table 2 of the pre-print and additional text from an appendix that will be included in the published paper outlining how the elicitation process informed the modelling process.
One of the authors of the online critique was apprised about how we derived estimates of impact on cessation through the expert elicitation process at a meeting with two of the study’s senior authors shortly before publication of the online critique.
The results of this expert elicitation process were a posited median reduction of 85.9%, with a 95% uncertainty interval of 67.1% to 96.3%. This equates to a median 32% increase (with a 95% uncertainty interval of 20% to 48%) in the net cessation rate above the BAU cessation rate each year for five years.
The graph below shows what the predicted future smoking prevalence was for a sex by age cohort with 25% initial smoking prevalence and a 5% BAU net cessation rate in the analysis for the pre-print paper. Notably, this graph looks almost identical to the falling tobacco prevalence estimates included in the initial report to the NZ Ministry of Health for the first five years, other than smoking prevalence is a bit higher beyond five years (as the earlier report to the Ministry assumed a larger impact of denicotinisation on quitting beyond five years). Regardless, the reductions in smoking prevalence are similar and profound in both estimates.
Because the model assumptions did not rely only on findings from the Walker et al trial (although it was included among the evidence considered), the extensive critique assuming otherwise is irrelevant.
The online critique authors also did not acknowledge that we addressed in the pre-print text the inevitable uncertainties in input parameters and hence in the modelling estimates produced, nor that we used a range of impact assumptions to generate uncertainty intervals for our estimates, including a pessimistic scenario assuming a smaller impact on quitting. The is reflected in the dashed lines in the graph above that form the 2.5th and 97.5th percentiles of estimates that propagated through the modelling to generate uncertainty in our output estimates.
We consider that our estimate of a substantial increase in quitting is highly plausible, and aligns broadly with the findings of a similar expert elicitation process that was carried out for a previous modelling study undertaken by the US FDA and published in the New England Journal of Medicine. Indeed, the critique authors state: “It is unlikely, therefore, that many smokers would choose to spend their money on tobacco with minimal nicotine content,” which suggests that they fundamentally agree with a decline in smoking prevalence that very likely falls somewhere within the dotted uncertainty lines in the figure above.
Our assumption that denicotinisation will trigger greatly increased quitting also aligns with the science which recognises that nicotine is the main component of cigarettes and tobacco that maintains addiction and makes quitting difficult. Again, the authors of the critique seem to agree, acknowledging that “a vast literature characterises tobacco use as primarily a nicotine-seeking behaviour and concludes that nicotine is the reason people smoke”.
Our estimates are also consistent with extensive evidence from randomised trials and other studies investigating the impact of denicotinised cigarettes on smoking behaviour (summarised in previous blogs here and here) and considered by the experts contributing to the elicitation process.
The trials report that people who smoke and who are provided with denicotinised cigarettes and tobacco find these products unsatisfying; as a result, they typically (i) smoke less; (ii) are more likely to try to quit, and (iii) are more likely to succeed in their quit attempts compared to people given regular strength cigarettes, or (iv) “cheat” and purchase regular strength nicotine cigarettes outside the trial supply. This evidence base has been described in recent comprehensive reviews,3 4 including reviews of impacts among priority populations.5 For example, quit rates were 18% among people provided with denicotinised cigarettes vs 4% for people given regular strength cigarettes in this recent study with people with mood or anxiety disorders who smoked and who were not motivated to quit.6
However, such trials likely underestimate the impact of a mandated denicotinisation policy on quitting because participants motivated to quit are often excluded (as in the above example), study participants can easily buy regular nicotine tobacco products from retail stores (which would not be the case with a denicotinisation policy), and there is no cost-related motivation to quit, given denicotinised cigarettes are usually provided free in these trials.
Our assumptions about quitting are also consistent with anticipated quitting or switching to vaping in the event of a denicotinisation policy for cigarettes; as reported by people who smoke in A/NZ (26% in the NZ ITC study, 54% in Māori participants in the TAKe study) and by 50% of participants with experience of using denicotinised cigarettes in a US study.
The online critique made three other criticisms of note. First, it criticised the assumption that the increase in quitting is maintained at the same level over the five year period. We consider a sustained and consistent increase in quitting is plausible because cigarettes and tobacco will no longer be addictive or satisfying resulting in an ongoing stimulus to quit, enhanced quitting success and reduced risk of relapse. In addition, people who continue to smoke VLNCs will likely be smoking less intensively, and in the A/NZ context less expensive vaping products containing nicotine and other smoked tobacco substitutes are easily available as alternatives to switch to.
Second, the authors claimed that the modelling study ignored the ‘real-world dynamics’ of a market intervention and possible behavioural responses people who smoke might make, such as switching to vaping or buying tobacco products on the illicit market. This is also incorrect.
If the illicit market was to increase and that increase was sustained, and assuming that people using illicit cigarettes are less likely to quit than those using VLNCs, then the prevalence curve (see graph above) will tend to level off at the illicit market-driven prevalence as opposed to trending towards 0%.
However, whilst we acknowledge this issue in the discussion of the pre-print paper, we also note that we consider illicit trade and home grown tobacco are unlikely to be a major alternative source of smoked tobacco in the A/NZ context and hence our method did not include explicit modelling of changes in the illicit market.
We have explained our reasoning in more detail a previous blog. In brief, context matters: A/NZ has over 1000km of water in every direction and strong border controls due to biosecurity concerns. The government has committed to providing additional resources to monitoring and enforcement activities. A/NZ also has a low risk of corruption and other factors that facilitate tobacco smuggling. Our pack collection studies in A/NZ have not demonstrated a significant increase in the illicit market size over time. This is despite major policy changes including a decade of substantial annual tobacco excise increases and plain packaging – policies that tobacco companies frequently (though usually incorrectly) predict will greatly increase the illicit tobacco market.
Thus the large-scale smuggling of illicit tobacco to A/NZ, whilst not impossible, is much less likely than in other countries. The high capacity of A/NZ to combat illicit tobacco trade has been recognised in the literature with A/NZ ranked top (along with Singapore) on an Illicit Tobacco Trade Index that considered dimensions of (1) general governance, (2) tobacco control policies, and (3) trade and customs practices.
Furthermore, the model included pathways for switching partially or completely to vaping products. Intriguingly, the critique’s authors, who have argued for greater vaping product availability and encouragement to switch to vaping to achieve smokefree goals, appear to downplay the impact of vaping and the likelihood that denicotinisation will encourage people who smoke to switch to vaping. This oversight occurs despite the very substantial and well-recognised potential synergies from removing the nicotine from smoked tobacco products in ‘pushing’ people away from smoking.7 Such potential synergies are particularly relevant in a jurisdiction like A/NZ where alternative sources of nicotine are easily available, cheaper than tobacco products and widely used.
Put another way, A/NZ is the ideal country to pioneer a mandatory denicotinisation policy.
Finally, the critique argued that the timelines in the modelling study assumed implementation would occur unrealistically soon. This point is debatable but ultimately not material as it will not affect the size of estimated impacts, just their timing.
More general problems with the online critique included its failure to acknowledge that, even if our estimated impact of denicotinisation on quitting proved optimistic, the policy would still likely have a profound impact on reducing smoking prevalence, improving population health, and reducing health inequities and hence be highly justifiable. Furthermore, the critique focuses only on quitting and does not discuss benefits from greatly reducing smoking uptake among youth and young adults that is highly likely to follow denicotinisation.
Critical review of research is important and there were some helpful suggestions in the online critique mixed in with the inaccuracies. These helpful suggestions included that further studies could elicit estimates of impact with additional experts outside of the study team and could (assuming technical and practical feasibility) model the impact of an increase in use of illicit tobacco products and changes to regulatory approaches to vaping. We have offered to consider re-running models using appropriately justified estimates that the authors of the critique provide.
Foundations of the modelling study
As scientists we are committed to academic integrity through subjecting our work to critique and debate within our research teams, from our stakeholders and submitting it to objective peer-reviewed academic journals. Central to this process is transparency and engaging in fair and respectful critique with a focus on extending the evidence base for effective tobacco control.
We are also committed to ensuring our work is relevant to the needs and aspirations of those most affected by tobacco related harm. In many countries and jurisdictions with colonial histories the Indigenous populations invariably bear a disproportionate burden of harm from the impacts of commercial tobacco. This is attributable to their ongoing experiences of colonisation that continue to the present day. In research, this can be perpetuated by the ways in which Indigenous peoples are engaged in research, the way research questions are framed, how it is carried out and how findings are disseminated. In developing this paper we were cognisant of these issues, particularly in terms of our obligations in relation to the Treaty of Waitangi and Māori.
Accordingly, Māori helped shape the paper and key Māori stakeholders were briefed before the paper was made available as a pre-print. Leading Indigenous academics based in Australia were also co-authors for this paper and played a kaitiaki (guardian) role as the lead authors were also Australia-based. More broadly, the presence of Indigenous co-authors in the writing team aligned the study to Indigenous clauses within the Framework Convention on Tobacco Control (ensuring Indigenous engagement in tobacco control policy). This also allowed consideration of how the paper could contribute to Indigenous Data Sovereignty principles and recommendations for future research and data management practices. Given our paper was focused on Māori and smoking inequities, we were particularly disappointed that the online critique did not make any reference to Māori or Indigenous peoples, nor was there engagement with key Māori stakeholders prior to the critique being published.
Conclusions
To conclude, we acknowledge modelling studies can only provide predictions of future outcomes. All modelling involves uncertainties, particularly, as in this case, where the evidence to inform assumptions is constrained by the policy not yet having been implemented at a country-level. However, the estimates in our study are plausible, as they are based on a robust modelling process with assumptions and parameters grounded in a substantial evidence base, including from randomised trials.
Every policy intervention has to be implemented somewhere for the first time, and innovation and progress in public health and in ending the tobacco epidemic depends on governments taking substantive and decisive actions. We believe our findings, alongside extensive supporting evidence, support implementing a mandated denicotinisation policy for smoked tobacco products as the key backbone of a tobacco endgame strategy, and suggest this intervention will make a major positive contribution to achieving a Smokefree Aotearoa and eliminating inequities in tobacco harm. However, it is also imperative with any substantial policy intervention, and particularly when that intervention is novel, that it is thoroughly evaluated to test whether its anticipated benefits are fully realised and to inform decision-makers’ judgements about whether the policy should be maintained and emulated.
Author details: Authors are from the University of Otago Wellington (RE, NW, AW, JS); the University of Melbourne (TB, DAO, TW); Australian National University (RM, RL); and the University of Queensland (CG).
Appendicies
Text from Table 2 of Pre-print
“Using an expert knowledge elicitation (see Appendix D), smoking prevalence (i.e. X=prevalence in states CS and DU) with mean 15.2% (SD 7.84%, X=Beta (3.19, 17.78), median 14.1%, 95%UI: 3.7% to 32.9%) of BAU smoking prevalence five years after low nicotine policy implementation, due to quitting or switching to vaping (i.e. disregarding initiation impacts that will additionally impact prevalence among 20-24 year olds in first five years of the model). Implementation was as X^(t/5) scalar applied to BAU CS and DU prevalence, where t is the 1 to 5 years after intervention. For the sixth and subsequent years, the transition probabilities were twice those in BAU (due to an ongoing higher NCR, given non-addictive levels of nicotine in tobacco).”
[Key: CS = current smoker, DU = dual user (smoking and vaping), NCR = Net Cessation Rate]
Appendix D: Expert knowledge elicitation process (Excerpted from Appendix of paper under review)
There is no firm evidence on the amount by which smoking prevalence will reduce – as a result of quitting – with denicotinising tobacco. On the one hand, given nicotine is the addictive component there is reason to believe the vast majority of smokers will quit if only tobacco with sub-addictive levels of nicotine is available. This optimistic stance is further strengthened by possible positive reinforcement of denormalisation and suppliers exiting the market. On the other hand, some smokers maintain they will smoke regardless of nicotine, and for many smoking is a socially conditioned habit as well as a biological addiction.
To parameterise our model, many of the co-authors of this paper were asked to independently estimate the most likely percentage reduction in smokers after five to ten-years compared to ongoing BAU trends with not denicotinisation. And to estimate a pessimistic and optimistic percentage reduction in smoking (that we interpreted as each individual’s 95% range of likely values). Table S22 shows the separate estimates, resulting in an average of 84.8% reduction as the most likely estimate, and average pessimistic and optimistic scenarios of 65.8% and 96.6%, respectively (equating to an approximated SD of 7.84%). Specifying this a beta distribution (17.8, 3.2) gave a mean of 84.8%, median of 85.9%, 2.5th percentile of 67.0% and 97.5th percentile of 96.3%. Figure S2 below gives the probability density function.
Table S22. Co-authors estimates of the effect of denicotinisation on smoking prevalence
| Co-authors |
Pessimistic |
Most likely |
Optimistic |
| RE |
50% |
80% |
95% |
| RM |
50% |
78.5% |
95% |
| AW |
80% |
95% |
99% |
| DAO |
75% |
85% |
98% |
| TB |
80% |
90% |
97.5% |
| NW |
60% |
80% |
95% |
| |
|
|
|
| Average |
65.8% |
84.8% |
96.6% |
Figure S2. Probability density function applied to average effect of denicotinisation on smoking cessation
To operationalise this in the model, we randomly drew from this distribution (assume the mean of 84.8% was drawn), and applied the following additional net cessation rate in the first five annual cycles of 1 – (1-84.8%)^(1/5) = 31.3%. That is, after five cycles the ‘survivorship’ will be 1 – (1-31.3%)^5 = 15.3% (which is 1 – 84.8%). Given that this reduction was compared to future BAU, the percentage impact on the net cessation rate (NCR) was on that after the BAU NCR. For example, if BAU NCR in the next cycle was 5%, then under denicotinisation it would become 5% + (1-5%).31.3% = 34.8%.
This additional intervention NCR for denicotinisation was applied for the first 5 cycles, then the NCR reduced to 2x BAU NCR.
This post was originally published on the Public Health Expert Blog
