|Year : 2019 | Volume
| Issue : 3 | Page : 192-203
Proving non-conventional methods: A paradigmatic paradox
Lex A. L. B. Rutten
Independent Researcher, Breda, Netherlands
|Date of Submission||18-Aug-2019|
|Date of Acceptance||26-Aug-2019|
|Date of Web Publication||4-Oct-2019|
Dr. Lex A. L. B. Rutten
Aard 10, 4813 NN Breda
Source of Support: None, Conflict of Interest: None
Originally, it was thought that Evidence-Based Medicine (EBM) would supplant decision-making based on intuition or plausibility. Later, it appeared that the gold standard in EBM, the Randomised Controlled Trial (RCT), was not as 'hard' a reference point as had been supposed, and an assessment of credibility was needed. After some decades, 'credible' RCT-centred EBM has led to the dismissal of therapies deemed to be implausible, such as Homoeopathy. Nevertheless, such therapies remain widely appreciated by patients who use them alongside conventional medicine. Nearly two hundred RCTs of Homoeopathy showed no difference in efficacy between Homoeopathy and comparable conventional trials. There is no proof that conventional trials are of better quality and there is no proof of harm by Homoeopathy. However, selective analysis of evidence shows statistically insignificant results, then interpreted as unscientific 'confirmation' of the hypothesis that Homoeopathy is a placebo. As a result, the use of Homoeopathy instead of antibiotics in acute respiratory tract infections has been discouraged, despite the absence of evidence of the efficacy of antibiotics for this indication and an established risk of harm. Complex statistical interpretations of RCT evidence lead to impractical and even harmful advice. Credible proof is actually based on many subjective (often continuous) variables. As an endpoint for EBM, Bayesian probabilities, based on more than RCT evidence, would provide a more practical and personalised type of advice for patients, and would develop the diagnostic process into a prognostic framework, offering alternatives if one particular solution was to fail.
Keywords: Bayes' theorem, Credibility, Evidence-based medicine, Homoeopathy, Hypothesis testing, Paradigm
|How to cite this article:|
Rutten LA. Proving non-conventional methods: A paradigmatic paradox. Indian J Res Homoeopathy 2019;13:192-203
'No number of sightings of white swans can prove the theory that all swans are white. The sighting of just one black one may disprove it'.
- Karl Popper.
| Introduction|| |
In 1992, the Evidence-Based Medicine Working Group (EBMWG) stated: 'Medical practice is changing, and the change, which involves using the medical literature more effectively in guiding medical practice, is profound enough that it can appropriately be called a paradigm shift'. This paradigm shift refers to Kuhn. The EBMWG stated: 'Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research'.
Criticism of evidence-based medicine (EBM) has gradually increased and is, according to some, evolving into a crisis. Greenhalgh, among others, commented on the inappropriate influence of vested interests, statistical significance, management (and neglect) of patient-centred care and multimorbidity. Most clinicians will find it impossible to interpret the vast amount of existing information or to assess its credibility. Reviews are meant to guide clinicians, but at the beginning of this century, there were more than 100 systems available to rate the quality of research; there was extensive disagreement between these and none appeared to be particularly useful for clinical practice. Since 2004, the GRADE guidelines have been accepted as the method of choice to assess the credibility of evidence. Although the GRADE guidelines are meant to prevent different conclusions being drawn from the same information, the interrater reliability of GRADE is assessed variously as 'low' to 'considerable' by different authors.
For the last quarter of a century, the Journal of Evaluation in Clinical Practice has been active in encouraging the leaders of EBM to embrace authentic scholarship and to engage properly with their critics. Its main criticism of the current interpretation of EBM is its reductionistic approach which makes it inadequate for chronically ill patients and those with multimorbidity. EBM recognises the need to integrate patients' values and preferences, but it is not clear how to do this. Furthermore, a considerable number of patients make choices that appear contradictory to the fundamental beliefs of many practitioners.
A clear example is the use of complementary and alternative medicine (CAM). Patients who have derived inadequate benefit from conventional medicine, especially those with chronic complaints, seek alternatives such as CAM. Patients who use both conventional medicine and CAM appear to value both and tend to be less concerned about their medical doctor's disapproval than about his or her inability to understand or incorporate CAM therapy. Most patients do not disclose their use of CAM to their conventional doctors, who may particularly disapprove of methods such as acupuncture and Homoeopathy because these are implausible according to their paradigm or pathophysiologic rationale. Nevertheless, these two are the most frequently used therapies of more than 700 CAM modalities.
Basing an acceptance of CAM methods on evidence may be more complicated than was originally expected. In 1993, the Dutch Health Board recommended that CAM methods should be recognised after 'repeated' evidence. Gaining such evidence, however, turned out not to be straightforward; in 2005, Ioannidis stated that most published research findings in medicine were false. 'Evidence' was replaced by 'credible evidence,' and since then, most evidence has been discarded. The pathophysiologic rationale is, however, a large part of the existing paradigm and this strongly influences what is perceived as credible. Kuhn stated (page 78): '… scientists fail to reject paradigms when faced with anomalies or counterinstances. They could not do so and still remain scientists'.
The European Academies Science Advisory Council (EASAC) stated, after examining 176 randomised controlled trials (RCTs), that Homoeopathy 'may pose significant harm to the patient if incurring delay in seeking evidence-based medical care and that there is a more general risk of undermining public confidence in the nature and value of scientific evidence', because 'authoritative and impartial bodies, state that a placebo effect may appear in individual patients, but there is no robust, reproducible evidence that homoeopathy is effective beyond the placebo effect'.
In recent decades, the existing interpretation of EBM has not brought about a paradigm shift that brings CAM and conventional medicine closer together. On the contrary, the current EBM questions the judgement of large numbers of patients who use both Homoeopathy and conventional medicine, as well as that of the very many doctors who apply both methods. This raises certain questions: Is it possible to prove beyond doubt that a CAM method is not a placebo, and how relevant is it to try to do so? Why do patients keep on using CAM? This article is based on the existing literature and the personal experience of doctors trying to understand and reconcile a particular paradigm with patients' preferences for using both Homoeopathy and conventional medicines. This means understanding the patient's perspective, how a doctor's belief in Homoeopathy can change, and why CAM therapy can be effective after conventional treatment has failed.
| The Patient's Perspective|| |
The median 12-month prevalence of all use of Homoeopathy worldwide is 3.9% (range, 0.7%–9.8%). Many patients suffer long-lasting complaints before consulting a homoeopathic practitioner (Witt et al.'s study: 79%, average duration of 8.8 ± 8 years). Disease severity decreased significantly during and after homoeopathic treatment (P < 0.001) between baseline and 24 months (adults from 6.2 ± 1.7 to 3.0 ± 2.2; children from 6.1 ± 1.8 to 2.2 ± 1.9). People frequently seek homoeopathic treatment after other treatments have failed (Güthlin et al.'s study: 70%). Patients also value the longer consultations which are a necessary feature of Homoeopathy.
The fact that a considerable number of patients do not respond well to conventional medicine, even if there is good evidence for it, is a disquieting reality. RCT evidence using a placebo control shows that the verum works for more patients than the placebo does, not that it works for every patient. Patients who do not fit the inclusion criteria for a RCT are even less likely to benefit. RCT evidence is also absent or insufficient for many conventional treatments. Patients may prefer to avoid the use of antibiotics or may have experienced that antibiotics failed to work for their complaint. The Cochrane Collaboration states that there is no evidence of benefit from antibiotic use in the common cold or for persisting acute purulent rhinitis in children or adults, and there is evidence that antibiotics cause significant adverse effects. There is insufficient evidence for the use of antibiotics as a means to reduce the risk of otitis or pneumonia in children, but nevertheless, acute respiratory tract infections (ARTIs) remain responsible for 75% of the total amount of prescribed antibiotics in high-income countries. The number needed to treat (NNT) to prevent one serious complication in ARTI and otitis media using antibiotics is over 4000. Recurrent acute otitis media occurs more frequently in children previously treated with amoxicillin.
| Doctors Practising Homoeopathy|| |
There are homoeopathic doctors in more than 70 countries worldwide and more than 280,000 in India., Many doctors prescribe both conventional and homoeopathic medicines. They begin as conventional doctors, because it is rare to find any instruction in Homoeopathy in universities. These doctors are well aware of the plausibility problem with homoeopathic medicines: extreme dilutions cannot produce any drug–receptor interactions. However, the vigorous shaking between dilution steps is believed to create a new substance. History shows that even phenomena once thought impossible, such as quantum mechanics, do not completely overturn the existing physicochemical knowledge base. The EBMG stated: 'The study and understanding of basic mechanisms of disease are necessary but insufficient guides for clinical practice. The rationales for diagnosis and treatment, which follow from basic pathophysiologic principles, may in fact be incorrect, leading to inaccurate predictions about the performance of diagnostic tests and the efficacy of treatments'.
A scientific attitude should not prevent doctors from making their own observations. Scepticism gradually fades away after numerous observations of unexpected cures and hearing remarkable stories from their patients. They value the complementary possibilities which Homoeopathy offers, and they are able to use fewer antibiotics in ARTI (odds ratio [OR]: 0.43 and 95% confidence interval [CI]: 0.27–0.68) and in otitis media. Antimicrobial resistance (AMR) is expected to become the principal cause of death in a few decades and already costs over 700,000 lives each year. Doctors also value the safety of Homoeopathy: Posadzki et al. found only four deaths possibly related to the use of Homoeopathy in the literature in all languages over 34 years. Serious problems caused by delay in seeking conventional treatment were found in only 16 cases. However, medicines containing high doses of toxic natural ingredients can at times be falsely labelled as 'homoeopathic'. Homoeopathic medicines should be of guaranteed safety and manufactured to high standards.
It is unlikely that our knowledge of physics and chemistry is complete. One of the factors which counters the belief that medicines can only work via molecular interactions is that the effect of Homoeopathy in chronic or multiple complaints differs considerably from conventional medicines: it builds up gradually and the effect of one medicine extends over different complaints. Homoeopathy appears to be useful in chronic complaints and multimorbidity after conventional therapy has failed. Another experience that confirms the difference between conventional medicines and Homoeopathy is the fact that a homoeopathic medicine cannot be prescribed on the diagnosis of a condition alone. The choice of the homoeopathic medicine, say Belladonna, must be based on a match between the patients' symptoms and the consensus among prescribers regarding which symptoms and characteristics indicate this medicine, for example, in otitis media:
- Ear pain less in a half-sitting position
- Perspiration during fever
- Grinding the teeth during sleep.
On the other hand, Belladonna could also be used in other conditions, such as respiratory tract infections, ADHD or migraine. Doctors who practise both Homoeopathy and conventional medicine do not dismiss RCT evidence, but they value the reliable manner in which Homoeopathy can offer rational options after the failure of conventional therapy. When choosing alternative options, reproducibility provides a more scientific basis than simple intuition or personal experience. Patients prefer CAM to be practised by medically qualified doctors, citing safety reasons.
To prescribe homoeopathic medicines successfully requires an extensive knowledge of a large number of medicines, with hundreds of characteristics per medicine. For a single indication such as otitis media, homoeopathic practitioners must choose between more than 10 medicines. RCT evidence by Steinsbekk et al. may demonstrate that the untrained use (self-treatment) of Homoeopathy with only three different remedies shows no difference between Homoeopathy and placebo, but of course, this finding may well have other causes, such as the inefficacy of the homoeopathic method.
| Evidence for Homoeopathy|| |
In 1991, Kleijnen et al. concluded that although there was more RCT proof for Homoeopathy than expected, this might be caused by publication bias. A meta-analysis by Linde et al. in 1997 then proved that publication bias was an unlikely cause of the positive outcome for Homoeopathy. A meta-regression analysis by Linde et al. in 1999 showed a smaller effect in higher-quality studies, but there was no linear decline of effect with better quality: the mean OR of trials with the highest Jadad score was 2.00 (95% CI: 1.37–2.91). Vandenbroucke stated that the funnel plot of Homoeopathy trials was comparable to the best conventional results. Shang et al.'s hypothesis  was that quality in Homoeopathy, especially in smaller trials, would be lower than in conventional trials, but the outcome was the opposite: 21 out of 110 (19%) higher-quality trials in Homoeopathy, 9 out of 110 (8%) in conventional medicine; there were 13 smaller, higher-quality trials in Homoeopathy versus 3 in conventional medicine. The median sample sizes and effect sizes were similar.
The systematic review by Shang  is the only comparison of homoeopathic and conventional RCTs. The funnel plots showed no difference in effect [Figure 1]. Meta-regression of results was originally proposed by the authors to extrapolate effects, on the assumption that smaller trials were of lower quality. However, it is not valid to compare two methods by meta-regression if one has a better quality of smaller trials, because small trials will naturally show stronger effects because of better selection of patients.
|Figure 1: Funnel plots of 110 Homoeopathy trials (above) and 110 conventional trials (below). Source: Shang et al|
Click here to view
This comparative analysis is often cited as proof of the inefficacy of Homoeopathy, but the conclusion 'weak evidence for Homoeopathy, but strong evidence for conventional medicines' was based on undisclosed subsets of eight Homoeopathy and six conventional trials. After disclosure, the pooled Homoeopathy subset (OR: 0.88, 95% CI: 0.65–1.19) of 'larger, higher-quality trials' proved to be highly heterogeneous with respect to indications [Table 1], and only two out of the eight trials were compared with conventional trials. The confidence interval of larger and better Homoeopathy trials overlaps with that of larger and better conventional trials (OR: 0.58, 95% CI: 0.39–0.85): note that a difference in statistical significance is not a statistically significant difference.
|Table 1: Subset of eight ‘larger,’ higher-quality Homoeopathy trials in Shang's review. (n = sample size) Two (Jacobs, Papp) were compared with conventional trials|
Click here to view
In this case, the definition of 'larger trials' was unusual: 'with standard error in the lowest quartile'. The more usual definition of 'sample size above median' would almost have given a significant positive outcome for Homoeopathy (OR: 0.82, 95% CI: 0.71–1.02). One single indication (muscle soreness after marathon running) was the main cause of the loss of statistical significance. A meta-analysis of four trials based on this indication gave a nearly significant negative effect (OR: 1.30, 95% CI: 0.96–1.76).
Reviews which draw negative conclusions are all based on selections of fewer than 10% of all trials. Many different criteria are used for these selections, e.g., the National Health and Medical Research Council (NHMRC) of Australia uniquely chose a minimum sample size of n = 150. The way in which selection leads to different outcomes is illustrated by the evidence for Homoeopathy in ARTIs:
- Shang (2005): Pooling of eight trials showed 'a substantial beneficial effect (OR 0.36, 95% CI 0.26–0.50) and there was neither convincing evidence of funnel-plot asymmetry nor evidence that the effect differed between the trial classified as of higher reported quality and the remaining trials'
- NHMRC (OPTUM, 2013): 'no reliable evidence that homoeopathy is as effective as the other therapies for the treatment of upper respiratory tract infection'. This conclusion was based on '… especially one medium-sized, good-quality trial (251 participants) did not detect a difference between homoeopathy and placebo'
- Hawke et al. (Cochrane, 2018): 'Pooling of two prevention and two treatment studies did not show any benefit of homoeopathic medicinal products compared to placebo on recurrence of ARTI or cure rates in children'.
The first review pooled eight trials, showing a positive effect. The latter two reviews concluded that Homoeopathy should not replace conventional therapies, such as 'anti-inflammatory drugs, antibiotics or other therapies', in ARTI. The NHMRC did not pool trials, while Hawke restricted the analysis to children and pooled in pairs of two. Hawke did not pool the primary outcome 'disease severity' of different trials because of different outcome measures.
The evidence for Homoeopathy may be inconclusive, but it is no worse than in comparable conventional studies – the conclusions reached depend heavily on the selection of trials.
| Credibility|| |
Patients who find that a CAM therapy has worked for them after conventional medicines have failed, complain that conventional doctors question their credibility. They try to convince their doctor by citing remarkable results which would not usually be seen in conventional practice. Some doctors may be intrigued by this, and if they try the method themselves, they may then see it confirmed; they then start to doubt their former beliefs, especially if they also obtain remarkable results. The response to remarkable cases is paradoxical, however, it can work in both directions: while the observer may be led to doubt their prior beliefs, their own objectivity may then be doubted by others. Similarly, carrying out a RCT in Homoeopathy poses a danger for the credibility of the researchers: they must be biased, because it is assumed that Homoeopathy cannot work. Demanding credible RCT evidence from CAM researchers, thus, also becomes a contradiction.
The RCT as a method now enjoys lower credibility than was the case 25 years ago. Credibility depends on statistical issues (such as power, false positives or negatives), methodological choices, reporting, financial and other interests and prior beliefs. Methodological choices and vested interests can undermine the credibility of individual RCTs. No objective and consistent quality criteria have yet been established, and this is also true of GRADE. Criteria such as a large sample size, highly valued by NHMRC, are disputable. As an example, a study showed that adding Homoeopathy to usual treatment produced a higher survival rate 6 months after severe sepsis (75.8% vs. 50.0%, P= 0.043). This is highly relevant to clinical practice, despite the sample comprising only 70 patients. Insisting upon large sample sizes is therefore ethically questionable.
Experts in Homoeopathy may have different views of the quality of trials from those who lack such expertise. For example, the Steinsbekk trial, which investigated the effect of just three medicines chosen by parents, is low-quality Homoeopathy, but this trial is regarded as being of high or moderate quality by non-experts (NHMRC, Hawke). For the NHMRC, this trial was considered to be the only one acceptable for ARTI. For Hawke, it was one of the two acceptable trials for 'prevention' of ARTI. Unlike in former reviews, the ARTI trials were divided into 'prevention' studies and treatment studies. In practice, only patients who have previously had infections come for homoeopathic treatment, so this is not real prevention, and they are then treated both during and between episodes. It would also be interesting to have a Cochrane review which was not restricted to children, such as Keneally's Cochrane review for antibiotics in ARTI. Homoeopathy is of especial value to patients and practitioners when it is used as an addition to conventional medicine, e.g., in multimorbidity. Neither this added value nor the safety aspects are considered in reviews.
Shifting quality standards makes it difficult to design study protocols that continue to hold good into the future. The trial by Lange-de Klerk (1994) was conducted by one of the Holland's best research institutes in an Amsterdam university hospital; in 1997, it was regarded as one of the best Homoeopathy trials, but it is now considered to be of low quality. Credibility is inevitably a subjective matter – the recognition of Homoeopathy would create a considerable emotional upset for many – and this makes it difficult to combine with 'hard' criteria such as statistical significance or P < 0.05. Shang found a statistically significant effect in ARTI with no quality bias, but claimed that this was not credible because of biases 'shown in our study'. However, their research actually showed that there was less bias in the Homoeopathy trials.
| Falsification|| |
Despite 176 Homoeopathy RCTs, it still seems impossible for the research community to acknowledge Homoeopathy, even though the quality of these RCTs and the effect sizes shown are no different from comparable conventional RCTs. Even if there is a problem with the credibility of RCTs in general, there is no evidence that Homoeopathy RCTs are themselves less credible. The pathophysiologic rationale is always given as the reason for the low a priori belief in Homoeopathy. In 1993, a small number of RCTs seemed sufficient to cause a paradigm shift, but then the credibility of the RCT itself came into question. At present, insufficient credible proof is regarded as actual proof that Homoeopathy is a placebo effect. However, the absence of evidence of an effect is not evidence of the absence of an effect (the argumentum ad ignorantiam fallacy). The editor of the Lancet, based on an undisclosed subset of 8 out of 110 trials, proclaimed: 'Now doctors need to be bold and honest with their patients about homoeopathy's lack of benefit'. The result was, in fact equivocal, positive, but not statistically significant: OR, 0.88 and 95% CI, 0.65–1.19. These eight, highly heterogeneous trials were pooled with the aim of confirming the placebo hypothesis, while the same heterogeneity was considered a reason to avoid pooling the results in other reviews.
The NHMRC  states: 'there are no health conditions for which there is reliable evidence that homoeopathy is effective'. This is a hasty generalisation from the results obtained by selectively sampling the evidence base; the NHMRC selected 5 out of 176 Homoeopathy trials for 61 conditions. It is no surprise that a failure to pool results should lead to an inconclusive outcome, and the generalisation to all other conditions cannot be valid. The negative outcome in ARTI was based on a single trial involving self-medication, while Shang (2005) reported a strong effect with no quality bias based on eight trials. The NHMRC advises: 'Homoeopathy should not be used to treat health conditions that are chronic, serious, or could become serious'. This is a straw man fallacy: in reality, many patients with chronic, or even serious, conditions that do not respond to conventional therapy find relief from Homoeopathy. For them, Homoeopathy is a therapy with reproducible results, and this is a more scientific stance than relying on simple intuition or unsystematic clinical experience.
There are four possible outcomes of a RCT: placebo works better, statistically significant or not, and verum works better, statistically significant or not. An outcome which is not statistically significant is inconclusive. However, many reviews regard any outcome that is not 'statistically significant positive' as positive evidence that Homoeopathy is a placebo. Antonelli found that 25 out of 61 reviews concluded that Homoeopathy is a placebo. Many of those, like Shang's (OR: 0.88 and 95% CI: 0.65–1.19), are positive but not statistically significant, i.e., inconclusive. Moreover, inconclusive outcomes are seen only in subsets based on various arbitrary definitions of quality and homogeneity.
According to Popper, any attempt to confirm a hypothesis such as 'all swans are white' is unscientific, and similar hypotheses such as 'Homoeopathy is a placebo effect' can only be falsified. Recall that a funnel plot of all Homoeopathy trials was similar to a comparable conventional plot [Figure 1], but with higher quality in the Homoeopathy trials; this is not consistent with subgroup reviews which appear to confirm the placebo hypothesis. Shang's finding in ARTI is not consistent with those of the NHMRC and Hawke. Does restricting trials to those in children support the conclusion that Homoeopathy is a placebo in ARTI? Does the higher credibility of negative ARTI reviews confirm the inefficacy of bad Homoeopathy, the placebo hypothesis or uncertainty about the interrater reliability (validity) of the GRADE guidelines? It is indeed uncertain whether fewer than 5% of infinite repetitions of the one or two trials on ARTI selected as credible would show no effect above placebo; does this fact justify the conclusion that antibiotics should be used and Homoeopathy should be avoided in ARTI?
| Bayes' Theorem|| |
Bayes' theorem is used to explain why RCT evidence for Homoeopathy is not seen to justify an acknowledgement of Homoeopathy. However, Bayes' theorem is about updating beliefs after consecutive observations, not a statement about truth, especially not after one observation. It describes how our beliefs change after an observation:
Posterior odds = LR × prior odds.
LR = Likelihood ratio = (prevalence of an observation in a target population)/(prevalence of the observation in the remainder of the population).
Similar considerations can be applied to diagnostic tests in rare diseases: a positive test outcome increases the chance that the patient has the disease, but this chance could still be very low. It would be a mistake, however, to disregard the test result simply because of this low posterior chance – confirmation by other tests or symptoms could increase the probability step by step. The rate at which posterior chance increases after a few further tests or symptoms may appear surprisingly fast to people not used to handling odds. The essence of Bayes' theorem is probabilistic reasoning, with the adjustment of the probability after each new observation. It describes how we learn from the past experience.
Bayes' theorem also offers the possibility of comparing the likelihood of different hypotheses, using another variant of the formula:
Where Hi is a certain hypothesis (i) and E is the evidence. Expressing this equation in words: the probability of hypothesis i, given evidence E, is the probability of evidence E, given hypothesis i, times the probability of hypothesis i, divided by the sum (for all possible hypotheses i) of the probabilities of the evidence, given each hypothesis, multiplied by the probability of each hypothesis.
A famous example of (mis) using Bayes' theorem in this way can be found at the trial of OJ Simpson (OJS), who was suspected of murdering his wife. The finding of DNA material from OJS at his wife's home was explained by his lawyers as having been planted by the police. Using this formula, they compared the probabilities of different hypotheses: (1) OJS left his DNA at the murder site; (2) the DNA belonged to someone else; and (3) the DNA was planted. Putting this into Bayes' formula gives us [Table 2].
|Table 2: The defence's Bayesian comparison in the OJ Simpson trial of chances that his DNA was at the crime scene because OJ Simpson was there (H1), or the DNA was not his (H2), or the DNA was planted. The prior chance of H1 was supposed to be very low and the likelihood that the DNA was OJ Simpson' very near 100%. The joint probability of H1 and its likelihood is the product of both. The posterior chance of each hypothesis is the joint probability divided by the sum of joint hypotheses|
Click here to view
The outcome of this Bayesian comparison of hypotheses in OJS' case was that the existing evidence mostly supported the hypothesis that the evidence had been planted (P = 0.94). Adducing the hypothesis that the DNA was planted then reduces the probability of OJS's guilt to P= 0.06. Intuitively, such a conclusion makes us feel uneasy. The catch lies in the subjective nature of the prior chance estimate, the weakest point in Bayesian reasoning when it comes to hypothesis testing. Why should this prior chance be so low? In this case, because it was based on the assumption that OJS could have been anywhere in the US at that time. If that assumption is wrong, the prior chance of OJS being at the murder site is also wrong.
Something similar happens in scientific disputes: if the assumption that there is no possible mechanism of action other than molecular interactions is wrong (the assumption is a reductio ad absurdum), the extremely low prior chance attributed to Homoeopathy is wrong. The real issue here is the ability to estimate a prior chance objectively or a willingness to accept the observations of others. Are the observations of a doctor who has experience of both Homoeopathy and conventional medicine less credible? Did looking through a telescope make someone less credible in the debate about the Copernican worldview? If this credibility depends on the assumed prior chance we are caught in circular reasoning. Bayes' theorem, however, performs well in an expert system which advises users how to interpret their observations; the system then becomes data driven, rather than theory driven.
| Diagnosis/prognosis|| |
The greatest advantage of Bayesian reasoning is that it describes expectations based on previous experience as probabilities, rather than as binary, true-or-false options. Beliefs shift after a series of new observations, and this also reflects the natural process of diagnostic reasoning in medicine. Maximising the probability of a diagnosis remains the core skill of medicine. The hypothesis that a medicine works no better than placebo is a null hypothesis, a proxy for the hypothesis that the medicine works. This enables falsification, but it is not real falsification in the Popperian sense (the first black swan), because all it actually indicates is that the chance of a false-positive outcome is 5% or less. The expected chance that the therapy will improve the patient's condition would be a more informative parameter than the chance that this is not a placebo effect. There are good reasons to extend diagnostic reasoning to prognostic reasoning. Prognostic reasoning in Homoeopathy shows how similar the two can be, as shown in [Figure 2]. The step-by-step increments in probability of the diagnosis/prognosis can be explained by Bayes' theorem: the LRs of symptoms are greater than unity because the symptoms occur more frequently in patients with pneumonia or in patients who respond well to Belladonna. This hypothetical case shows how the homoeopathic medicine Belladonna could be chosen if symptoms indicating Belladonna are present. The medicine would be expected to work only if sufficient indicating symptoms were presented by this particular patient. The wider the knowledge which the practitioner has of different medicines, the more patients he or she will be able to help. The estimates of the accumulating chances in this case are based on expert opinion and consensus. Such estimates can be made more accurate by research into diagnostic/prognostic factors, not only for diagnostics tests, but also for symptoms such as 'rapid breathing' in pneumonia. Similarly, we can also assess the prognostic relationship between a medicine such as Belladonna and symptoms such as 'grinding the teeth during sleep'.
|Figure 2: Hypothetical comparison of a diagnostic procedure and a prognostic procedure in Homoeopathy. Chances are estimates by experts|
Click here to view
The existing EBM framework will not support a patient's choice of therapy if there is no 'credible' RCT evidence, or if there is multimorbidity, or if there is no effect despite evidence that there could be. Bayesian statistics can give an actual probability that a therapy will work, given the patient's particular characteristics. Bayesian reasoning complements frequentist reasoning; it is less suited to the falsification of hypotheses, but better suited to making individual prognoses. Prognostic factor research (PFR) can assess all the relationships between personal factors and future outcomes. In the case of AMR, or if existing treatments offer no personal benefit, a prognosis of, say, 60% that a specific medicine will work is more relevant than any degree of certainty that this is not a placebo effect. Advice based on PFR is also better than relying only on intuition or personal experience.
| Discussion|| |
Based on nearly two hundred RCTs, the current state of EBM has led medical authorities to conclude that Homoeopathy is a placebo and its use could be harmful. Antibiotics should be preferred over Homoeopathy in ARTI. EBM, however, simultaneously indicates that antibiotics may do more harm than good in ARTI: The NNT for antibiotics is over 4000 and there are over 700,000 yearly deaths caused by AMR. Despite all this, ARTI remains the prime indication for prescribing antibiotics. There is no proof that Homoeopathy does any harm, nor does RCT evidence demonstrate that Homoeopathy is any less effective than comparable conventional medicine. The conclusions which have been drawn from various meta-analyses of trials of Homoeopathy in ARTI vary from a strong effect to no effect over placebo, depending on the manner in which trials are pooled for analysis. Apparently, 'insufficient' evidence is regarded as a contraindication for non-conventional therapies, outweighing the established inefficacy and harm of conventional therapies.
The authorities policing the use of EBM and users of Homoeopathy appear to inhabit different worlds. EBM is concerned that patients suffering chronic complaints will neglect EBM using Homoeopathy, while users of Homoeopathy have already experienced the limitations of EBM before trying Homoeopathy. It seems that these two worlds do not communicate well and this contributes to a paternalistic attitude by EBM. Cooperation between experts in non-conventional methods and those who lack such expertise is conspicuously absent.
If there exists a similarly insufficient level of evidence for two therapies for the same disease (ARTI) and demonstrable harm from only one of them, a preference for the harmful therapy cannot be right. Logical fallacies may be involved and a chain of these could lead to the choice of antibiotics rather than Homoeopathy: (1) reductio ad absurdum, molecular interaction is the only possible mechanism of action; (2) argumentum ad ignorantiam, absence of evidence of an effect is evidence of absence of an effect and (3) straw man fallacy, use antibiotics rather than Homoeopathy in ARTI.
Fallacies may be obscured by complex, biased reasoning, e.g., by combining frequentist (hypothesis testing) and Bayesian (probabilistic) reasoning. We can use Bayesian prior belief to explain how we update our belief, but the correct Bayesian outcome is a rapid strengthening of this belief after repeated observations, not a frequentist lack of statistical significance after the first observation. How credible a proof appears depends on belief, which in turn is influenced by a large number of (often continuous) variables. This results in a variety of possible choices based on unvalidated and therefore arbitrary criteria, leading to a range of statistical significance values, each with an unknown probability. The outcome becomes nothing more than, e.g., 'if the selection of these two trials is credible enough, the chance of a false-positive outcome is more than 5%. If we select these eight trials, the chance of false-positive outcome is below 5%'. The first option leads to the rejection of an implausible method, but would the second choice lead to acceptance? Or do we just avoid the second choice? Bias in RCT evidence became apparent in conventional RCTs, but this was then considered to be an explanation for the unexpectedly positive outcomes of Homoeopathy RCTs. This constitutes the first step away from falsification – the original goal of RCT – towards confirmation, the unscientific use of evidence.
Plausibility becomes less relevant to patients when plausible therapy fails. For practitioners, plausibility is much more relevant because the pathophysiologic rationale is fundamental to their knowledge. However, a scientific attitude requires that this rationale is considered in the light of observations. For a few CAM methods, there is an extensive body of shared experience between patients and practitioners, allowing a consensus to be reached about prognostic factors for specific interventions.
In their daily practice, medical practitioners apply Bayesian reasoning to update their beliefs with respect to diagnosis and prognosis, based on their own experience. Is medicine a Popperian science anyway? Treatment starts with the Bayesian probability of a diagnosis, built up by consecutive symptoms and tests which are relevant to that particular patient. It then goes on to assume a 'certainty' about therapy, which is not a real certainty at all, but a frequentist P < 0.05 value relating to a proxy outcome (not better than placebo), applicable to an undefined subset of patients; credibility considerations then add a probability distribution to the P value. Neither diagnosis nor the results of treatment are dichotomous variables; a prognostic framework is needed in which probability-based decisions can be arrived at.
Introducing the notion of credibility into hypothesis testing presents a spectrum of probabilities of statistical significance. With so many nuances of credibility existing across 'proven' and 'unproven' therapies, hypothesis testing in medicine has become a broad spectrum of shades of grey in the Popperian white swan. The end result is a subjective choice which varies according to the position of the observer.
Bayesian reasoning can offer a rational personalised therapy, based on shared experience, which can further be improved by PFR. The weakness of research based only on consensus and observation is that it cannot prove causality and is more liable to bias; it can only be used to improve medicine, to get better results. Other incentives, such as academic recognition or financial profits deriving from patents, are liable to introduce bias. However, is not financial profit one of the main problems with the credibility of RCT evidence as well? PFR helps doctors to form their own opinions about CAM, and they can test how reproducible Homoeopathy is by prescribing a homoeopathic medicine if they encounter a patient with recognisable signs/symptoms.
| Conclusion|| |
If the credibility of evidence is to be defined by medical authorities, the current application of EBM risks becoming authoritarian; this will estrange patients, result in simplistic decisions and lead to prejudice towards CAM users and researchers. This prejudice is already evident in the unscientific 'confirmation' of the placebo hypothesis with respect to Homoeopathy. In cases of ARTI, this leads to the perpetuation of the harmful use of antibiotics in preference to Homoeopathy because of 'insufficient' evidence. The current application of EBM is becoming untrue to its principle of data-driven science because medical authorities decide subjectively which data are 'credible'. Patients and practitioners who provide data that do not fit within existing paradigms are denied influence. To demand that evidence is provided by researchers who are simultaneously refused, is an absurdity, particularly when the credibility of this evidence is based on existing beliefs.
Hypothesis testing in EBM is irrelevant to many patients, as it gives no real predictions about which treatments will be effective for the individual patient. EBM should acknowledge the reality that many patients fail to respond to conventional medicine, even when the placebo hypothesis is falsified. These patients also need advice about the expected benefits and possible harm of alternative treatments. Like diagnosis, prognosis is based on probabilities, not certainties. In clinical practice, prognostic reasoning is the natural extension of diagnostic reasoning, and prognostic research is more informative for more patients.
Grateful thanks to Christien Klein, Mieke Goossens, Martine Rutten, Erik Stolper, José Eizayaga and Galen Ives for their comments. Thanks also to Galen Ives for providing the final English version.
Financial support and sponsorship
Conflicts of interest
| References|| |
Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992;268:2420-5.
Kuhn TS. The Structure of Scientific Revolutions. 2nd
. Chicago: The University of Chicago Press; 1970.
Greenhalgh T, Howick J, Maskrey N. Evidence Based Medicine Renaissance Group. Evidence based medicine: A movement in crisis? BMJ 2014;348:g3725.
Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: A quarter century on. Lancet 2017;390:415-23.
Norris SL, Bero L. GRADE methods for guideline development: Time to evolve? Ann Intern Med 2016;165:810-1.
Miles A. Evidence-based medicine – 2018. Quo vadis? J Eval Clin Pract 2018;24:3-6.
Jong MC, van de Vijver L, Busch M, Fritsma J, Seldenrijk R. Integration of complementary and alternative medicine in primary care: What do patients want? Patient Educ Couns 2012;89:417-22.
Eisenberg DM, Kessler RC, Van Rompay MI, Kaptchuk TJ, Wilkey SA, Appel S. Perceptions about complementary therapies relative to conventional therapies among adults who use both: Results from a national survey. Ann Intern Med 2001;135:344-51.
Dutch Health Board. Alternative Modes of Treatment and Scientific Research. The Hague 1993.
Ioannidis JP. Why most published research findings are false. PLoS Med 2005;2:e124.
EASAC. Homeopathic Products and Practices: Assessing the Evidence and Ensuring Consistency in Regulating Medical Claims in the EU. EASAC; 2017.
Relton C, Cooper K, Viksveen P, Fibert P, Thomas K. Prevalence of homeopathy use by the general population worldwide: A systematic review. Homeopathy 2017;106:69-78.
Witt CM, Lüdtke R, Baur R, Willich SN. Homeopathic medical practice: Long-term results of a cohort study with 3981 patients. BMC Public Health 2005;5:115.
Güthlin C, Lange O, Walach H. Measuring the effects of acupuncture and homoeopathy in general practice: An uncontrolled prospective documentation approach. BMC Public Health 2004;4:6.
Adam D, Keller T, Mühlbacher A, Hinse M, Icke K, Teut M, et al.
The value of treatment processes in germany: A discrete choice experiment on patient preferences in complementary and conventional medicine. Patient 2019;12:349-60.
Schork NJ. Personalized medicine: Time for one-person trials. Nature 2015;520:609-11.
Kenealy T, Arroll B. Antibiotics for the common cold and acute purulent rhinitis. Cochrane Database Syst Rev 2013;6:CD000247.
Alves Galvão MG, Rocha Crispino Santos MA, Alves da Cunha AJ. Antibiotics for preventing suppurative complications from undifferentiated acute respiratory infections in children under five years of age. Cochrane Database Syst Rev 2016;2:CD007880.
Petersen I, Johnson AM, Islam A, Duckworth G, Livermore DM, Hayward AC. Protective effect of antibiotics against serious complications of common respiratory tract infections: Retrospective cohort study with the UK general practice research database. BMJ 2007;335:982.
Bezáková N, Damoiseaux RA, Hoes AW, Schilder AG, Rovers MM. Recurrence up to 3.5 years after antibiotic treatment of acute otitis media in very young Dutch children: Survey of trial participants. BMJ 2009;338:b2525.
Manchanda R. Research implementation in Education and clinical practice. Indian J Res Homoeopath 2019;13:1-3.
Rutten L, Mathie RT, Fisher P, Goossens M, van Wassenhoven M. Plausibility and evidence: The case of homeopathy. Med Health Care Philos 2013;16:525-32.
Grimaldi-Bensouda L, Bégaud B, Rossignol M, Avouac B, Lert F, Rouillon F, et al.
Management of upper respiratory tract infections by different medical practices, including homeopathy, and consumption of antibiotics in primary care: The EPI3 cohort study in France 2007-2008. PLoS One 2014;9:e89990.
Sinha MN, Siddiqui VA, Nayak C, Singh V, Dixit R, Dewan D, et al.
Randomized controlled pilot study to compare homeopathy and conventional therapy in acute otitis media. Homeopathy 2012;101:5-12.
O'Neill J. Tackling Drug-Resistant Infections Globally: An Overview of Our Work the Review on Antimicrobial Resistance; 2016.
Posadzki P, Alotaibi A, Ernst E. Adverse effects of homeopathy: A systematic review of published case reports and case series. Int J Clin Pract 2012;66:1178-88.
Steinsbekk A, Bentzen N, Fønnebø V, Lewith G. Self treatment with one of three self selected, ultramolecular homeopathic medicines for the prevention of upper respiratory tract infections in children. A double-blind randomized placebo controlled trial. Br J Clin Pharmacol 2005;59:447-55.
Kleijnen J, Knipschild P, ter Riet G. Clinical trials of homoeopathy. BMJ 1991;302:316-23.
Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, et al.
Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet 1997;350:834-43.
Linde K, Scholz M, Ramirez G, Clausius N, Melchart D, Jonas WB. Impact of study quality on outcome in placebo-controlled trials of homeopathy. J Clin Epidemiol 1999;52:631-6.
Vandenbroucke JP. 175th
anniversary lecture. Medical journals and the shaping of medical knowledge. Lancet 1998;352:2001-6.
Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JA, et al.
Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 2005;366:726-32.
Sterne JA, Egger M, Smith GD. Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ 2001;323:101-5.
Lüdtke R, Rutten AL. The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. J Clin Epidemiol 2008;61:1197-204.
Hahn RG. Homeopathy: Meta-analyses of pooled clinical data. Forsch Komplementmed 2013;20:376-81.
Optum. Effectiveness of Homeopathy for Clinical Conditions: Evaluation of the Evidence Australian Govt: National Health and Medical Research Council; 2013.
Hawke K, van Driel ML, Buffington BJ, McGuire TM, King D. Homeopathic medicinal products for preventing and treating acute respiratory tract infections in children. Cochrane Database Syst Rev 2018;4:CD005974.
Lytsy P. Creating falseness-how to establish statistical evidence of the untrue. J Eval Clin Pract 2017;23:923-7.
Ioannidis JP. Why most clinical research is not useful. PLoS Med 2016;13:e1002049.
Frass M, Linkesch M, Banyai S, Resch G, Dielacher C, Löbl T, et al.
Adjunctive homeopathic treatment in patients with severe sepsis: A randomized, double-blind, placebo-controlled trial in an intensive care unit 2005. Homeopathy 2011;100:95-100.
Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ 1995;311:485.
The end of homoeopathy. Lancet 2005;366:690.
National Health and Medical Research Council. NHMRC Information Paper Evidence on the Effectiveness of Homeopathy for Treating Health Conditions. Canberra: National Health and Medical Research Council; 2015.
Antonelli M, Donelli D. Reinterpreting homoeopathy in the light of placebo effects to manage patients who seek homoeopathic care: A systematic review. Health Soc Care Community 2019;27:824-47.
Popper K. The Logic of Scientific Discovery. New York: Basic Books; 1959.
Rosendaal FR, Bouter LM. Errors in methodology (conclusion). XXXIX. The ultimate truth. Ned Tijdschr Geneeskd 2002;146:304-9.
Rutten LA. Flawed statistics and science confirming existing paradigms. J Eval Clin Pract 2018;24:1273-6.
Woodworth GG. Biostatistics: A Bayesian Introduction. New Jersey: Wiley & Sons; 2004.
Gill CJ, Sabin L, Schmid CH. Why clinicians are natural bayesians. BMJ 2005;330:1080-3.
Shahar E. A Popperian perspective of the term 'evidence-based medicine'. J Eval Clin Pract 1997;3:109-16.
Croft P, Altman DG, Deeks JJ, Dunn KM, Hay AD, Hemingway H, et al.
The science of clinical practice: Disease diagnosis or patient prognosis? Evidence about “what is likely to happen” should shape clinical practice. BMC Med 2015;13:20.
Wagner JS. Pneumonia. In: Bajaj L, Hambidge S, Nyquist AC, Kerby G, editors. Berman's Pediatric Decision Making. 5th
ed. Philadelphia: Elsevier/Mosby, 2011. Chapter: p. 754-59.
Rutten AL, Stolper CF. Diagnostic test evaluation by patient-outcome study in homeopathy: Balancing of feasibility and validity. J Eval Clin Pract 2009;15:1230-5.
Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, et al.
Prognosis research strategy (PROGRESS) 2: Prognostic factor research. PLoS Med 2013;10:e1001380.
[Figure 1], [Figure 2]
[Table 1], [Table 2]