Gerry Dendrinos is interviewed by Alan V. Schmukler. Gerry works as a Classical Homeopath and Western Herbalist and serves as Vice President of the Australian Homoeopathic Association (AHA). He spent eighteen months investigating the NHMRC negative report on homeopathy, that was used to undermined homeopathy. In this interview, he exposes the misconduct and ethical breaches that the NHMRC engaged in to contrive its specious report.
A note from Gerry: The Australian homeopathic community is calling on the support of the homeopathic community internationally. We need your help to bring attention to the misconduct involved in the National Health & Medical Research Council Homeopathy Review published in 2015. This is negatively impacting homeopathy everywhere.
We are asking two things of our friends and colleagues internationally: First, go to www.yourhealthyourchoice.com.au and sign up – signing up takes only 30 seconds and is easy. If you are from outside Australia, just click the box that states: “I live outside of Australia”. Second, please donate even a small amount to the campaign, to help the Australian Homoeopathic Association (AHA) bring justice to this issue. https://www.yourhealthyourchoice.com.au/campaigns/donate/ Running such campaigns is expensive for a small association such as the AHA, but together we can make a big difference for homeopathy everywhere.
Gerry Dendrinos –
AS: How did the NHMRC decide to do a review of the evidence for homeopathy?
GD: The story of the National Health & Medical Research Council (NHMRC) Homeopathy Review starts with the 2009 UK House of Commons Science & Technology Report (Evidence Check 2: Homeopathy). The UK report was a political, not scientific report and the UK Parliament rejected its recommendations in July 2010 in favour of user-choice.
In October 2010, the then NHMRC CEO, Prof Warwick Anderson, instructed NHMRC Council to develop a formal position statement on homeopathy on the sole basis of the UK report – without critically evaluating it, consulting any homeopathy subject or research experts, or independently assessing any research evidence. The same month, Prof Anderson submitted an article to the Medical Journal of Australia in which he declared his personal stance that homeopathy was a “retreat from reason” and an “alleged therapy”. Open anti-homeopathy bias and lack of procedural and scientific rigour was evident from the start of the process.
AS: As I recall, the NHMRC’s draft position statement was that homeopathy was ‘unethical’, ‘inefficacious’, implausible’. They betrayed their clear bias from the start. Can you elaborate on this?
GD: In December 2010, NHMRC Council approved the content of a draft position statement on homeopathy, which declared homeopathy to be “unethical”, “inefficacious”, “implausible” and “placebo”; it even declared the prescribing of homeopathy to be “deceptive”. The statement was strongly adversarial and no evidence was assessed and no homeopathic experts were consulted in coming to this ‘position’.
NHMRC then contracted a consumer group containing prominent Skeptics to conduct a “readability check” – to check the wording (not the content) so the public would understand the statement when released.
On the morning of 20 April 2011 the Australian Homoeopathic Association (AHA) accidentally learned of the existence of the draft statement: that afternoon it was leaked to the media. It appears anti-homeopathy advocates within the NHMRC’s network were anxious that it be released, in case AHA’s inadvertent discovery of the process created unwanted controversy. Indeed, controversy over bias and lack of procedural and scientific rigour engulfed NHMRC, causing it to abandon the draft statement process. It was at this point that NHMRC decided to instigate a formal evidence review on homeopathy, to substantiate development of a formal position statement. So one could say that things did not get off to a promising start, with clear bias and opposition to homeopathy apparent throughout the process, from the top down.
AS: What happened next? Did biases and conflicts continue to influence NHMRC’s approach?
GD: To oversee the Review, in mid 2011 the NHMRC CEO personally selected and appointed four members to a ‘reference group’, which later became the Homeopathy Working Committee (HWC) when the Review formally commenced in April 2012. The NHMRC CEO refused to appoint any homeopathy subject or research experts to the HWC, in breach of NHMRC’s own mandatory guidelines, while also ignoring strong protest. This was despite a key criticism of NHMRC’s earlier draft statement process being the refusal to involve any homeopathy experts. History was repeating itself.
During this time, in July 2011, the NHMRC Chairman declared in an interview with the Australian Skeptics: “Let me assure you I am no supporter of homeopathy. As Chairman of NHMRC I can also assure you that NHMRC does not support homeopathy”. This was well before any evidence was assessed and it confirmed the bias.
Then in January 2012 a HWC member, Prof Peter Brooks, formally joined the anti-homeopathy Skeptics group ‘Friends of Science in Medicine’ (FSM), but he did not declare his conflict of interest.
In February 2012, the NHMRC CEO was personally warned of FSM’s attempts to influence NHMRC reviewers by an external party. Despite this warning, the CEO proceeded to appoint Prof Brooks as the initial Chair of the HWC – a fact NHMRC never declared for the duration of the Homeopathy Review.
In fact, NHMRC took no action at all on this conflict until mid-way through a first review of homeopathy it commissioned in 2012. Despite the conflict, NHMRC allowed Prof Brooks to remain on the HWC for the duration of the Review, without formally managing the conflict. Prof Paul Glasziou was appointed as the second Chair of the HWC.
AS: You mention that NHMRC did a ‘first’ review. Can you tell us more about this?
GD: Our investigation has revealed that between April and August 2012, NHMRC commissioned a first review of the evidence on homeopathy under a highly respected and experienced reviewer who had worked with NHMRC conducting evidence reviews for many years. This reviewer was even a principal author of NHMRC’s own guidelines on how to review health evidence. They had conducted and published multiple reviews using the seminal NHMRC method they co-developed.
However, NHMRC concealed the existence of this review, its findings and tax payer expenditure – as if it never existed! This raises serious questions concerning administrative and scientific misconduct. What does the NHMRC have to hide? We learned through Freedom of Information (FOI) documents that the reviewer’s contract was prematurely terminated within days of submitting their final Draft Report in early August 2012. These have also revealed that the first review was of good methodological quality. For example, in mid July 2012 a HWC member provided NHMRC with the following expert review of the reviewer’s work:
”Overall, the consistent use and reporting of the same criteria for each of the evaluations is a strength and reveals the careful systematic approach that has been brought to these evaluations. […] Overall, a lot of excellent work has gone into this review and the results are presented in a systematic, unbiased and convincing manner.”
Yet NHMRC terminated the reviewer’s contract just two weeks later.
AS: Why do you think the NHMRC would have terminated and concealed this report, especially given the reviewer completed the work and it received positive expert feedback regarding its quality?
GD: The NHMRC has refused to provide any details about the first review, but our investigation has revealed it almost certainly reported positive, good quality research for homeopathy in a number of medical conditions. Most disturbing, it appears this was a primary factor influencing NHMRC’s decision to terminate the reviewer’s contract. Based on explicit bias demonstrated in the process, it appears NHMRC was ideologically opposed to allowing the reporting of any positive research for homeopathy. Allowing this would in effect be an admission by Australia’s peak medical research institute that homeopathy is not only ‘plausible’, but it can be ‘efficacious’ – a fact well known to the international homeopathy community and the hundreds of millions of people that regularly use it.
NHMRC’s demonstrated bias and strong connections with the medico-skeptics community, members of which it directly involved in the Review, represented a fundamental conflict that impacted the objectivity of the process.
AS: So the final report that NHMRC published in March 2015 concluding there was “no reliable evidence” homeopathy works was conducted by another contractor?
GD: That’s right. After sacking the first reviewer – under suspicion of improper purpose (because they reported positive research evidence for homeopathy) – in October 2013 the NHMRC started again under a second contractor, Optum Insight. This was the Review published in 2015.
What happened next was perhaps even more controversial than what had occurred to date. Our investigation has uncovered the lengths to which NHMRC went to essentially engineer the final conclusion of “no reliable evidence” and in the process, breaching accepted standards of ethical administrative and scientific conduct. The seriousness of this issue is accentuated by the fact that the NHMRC is not Big Pharma – it is a taxpayer funded, government institution. Therefore, it not only has to demonstrate adherence to ethical scientific standards, but also is legally obliged to uphold standards of ethical administrative conduct.
AS: Could you explain the concept of “reliable evidence”?
GD: The NHMRC Review concluded, ‘there is no reliable evidence that homeopathy is effective in any health condition’, on the basis that there was ‘no good-quality, well-designed studies with enough participants for a meaningful result’. This conclusion largely rests on two criteria, that for a trial to be “reliable” it had to: 1) have more than 150 trial participants and 2) be rated 5 out of 5 on the ‘Jadad’ quality rating scale or equivalent on other scales (that is, ‘100% quality’). NHMRC also incorporated other criteria, such as a unique “modified GRADE” tool to assess the ‘level of confidence’ in the results. These are concepts most people would not understand or know how to question. At face value, the concept of “reliable evidence” appears reasonable, until one realises that the concept and its component criteria are 100% arbitrary! This is why the framework has never been used before or since by any other research group in the world, including NHMRC. It was developed specifically for the Homeopathy Review. But it is the story of how and when the framework was developed that tells a disturbing story, one that indicts the NHMRC for misconduct.
AS: How and when was the “reliable evidence” framework developed?
GD: In hindsight, the answer to that question is simple, ‘hidden in plain sight’. But it took us over a year and a half of hard work to put all the pieces of the puzzle together. This occurred through a combination of documents released under FOI (such as HWC meeting minutes) and the invaluable work of the Homeopathy Research Institute (HRI). The HRI supported the investigation by conducting a thorough scientific analysis of NHMRC’s methods and how they influenced the Review’s findings.
This highlights a bizarre fact, that despite the NHMRC publishing over 944 pages of report documentation, the most important details about its methods and procedures were not reported. Following is a summary of what our investigation uncovered – the ‘missing pieces’ of the report that the NHMRC did not disclose to the public, the government or other scientists:
FOI documents reveal that the research protocol for the Optum review was agreed and finalised between all parties (the NHMRC, HWC and Optum) in late December 2012.
Optum then completed the evidence assessment in March 2013, with a meeting convened on 18 March to discuss the results. The minutes of the HWC meeting show that when developing the draft evidence statements, the agreed original protocol was modified in order to alter the results to reach more definitively negative conclusions. However, this was just the start of a systematic process of retrospective manipulations to the research protocol that occurred over the coming months.
Between April and June 2013, the NHMRC set up a special ‘HWC Sub-Group’, co-led by senior NHMRC staff and the HWC Chair, Prof Paul Glasziou. The purpose of the Sub-Group was to develop and apply the criteria for the evidence statements for each of the 61 medical conditions assessed. This was done entirely retrospectively and not reported.
Our investigation has revealed that all the criteria that formed the basis of NHMRC’s published conclusion that ‘there is no reliable evidence that homeopathy is effective’ were developed during this process, months after the research protocol had been finalised and well after Optum had already completed the assessment:
- The ‘modified GRADE’ tool to assess level of confidence in the results was created in May 2013
- The ‘150 sample size’ threshold to determine whether a trial was ‘reliable’ was created and retrospectively applied in mid July 2013
- The ‘100% quality’ threshold to determine whether a trial was ‘reliable’ was created and retrospectively applied later again (in late July/ early August 2013).
AS: Are you saying, then, that the NHMRC essentially invented the criteria midstream and thereby fabricated the Review’s findings?
GD: Yes – that is what our investigation has revealed, all fully documented. A core requirement of ethical scientific investigation is that any and all changes to the research protocol must be fully justified and disclosed, to safeguard against bias (where researchers may consciously or unconsciously tamper with the methodology according to pre-held expectations of the outcome). In the Homeopathy Review, NHMRC did not disclose or justify any of the retrospective changes made to the protocol outlined. You will not find this anywhere in their published report. The Optum Overview report even included a ‘Changes to the protocol’ section, which did not reveal any of these changes.
Not only did the NHMRC not disclose these significant post-hoc changes to the methodology, it also did not disclose their impact on the Review’s published findings. The HRI analysis went back to the original data, study by study, showing that of the 176 studies NHMRC included in scope of the Review, the results of 171 of the studies (97%) were dismissed from any consideration in the Review’s findings:
- The ‘N=150 sample size’ threshold retrospectively dismissed the results of 146 (83%) of the studies from contributing to the Review’s findings;
- The ‘100% quality rating’ threshold retrospectively dismissed the results of 25 out of the remaining 30 studies from contributing to the findings.
This effectively reduced the entire Review to only 5 “reliable” studies. It is hardly surprising, then, that the NHMRC concluded there is “no reliable evidence” that homeopathy works. It was a manufactured outcome using arbitrary criteria developed post-hoc.
Yet in March 2015, NHMRC misled the public that its findings were, “the result of a rigorous assessment of over 1800 papers”, which was what the media then reported.
AS: That is a fundamental breach of ethical scientific conduct. Did NHMRC seek expert review of its method and findings?
GD: Yes – but NHMRC ignored and did not disclose crucial expert feedback. For example, in 2013, NHMRC sought external review of its methods by the Australasian Cochrane Centre (ACC), which informed NHMRC:
“If the intent is to provide general statements about the effectiveness of homeopathy, then ‘no reliable evidence’ may not adequately reflect the research. For example, when a substantial proportion of small (but good quality) studies show significant differences, […] ‘no reliable evidence’ does not seem an accurate reflection of the body of evidence.”
It also ignored and withheld from public disclosure another expert reviewer’s feedback in 2014, that:
“The dismissal of positive SRs compounded with the lack of an independent systematic review of high quality RCTs leaves me uncertain of the definitive nature of the Report’s conclusions. […] if I were to dispassionately consider the evidence of efficacy, I am still left with niggling doubts that there are unanswered questions around the evidence. Contributing to those doubts is the absence of non-clinical in vivo and in vitro studies that examine the effects of homeopathy, where placebo effects are not relevant.”
AS: Was there anything else your investigation uncovered?
There is so much more. For example, on 9 April 2014 the NHMRC released its Draft Report for public consultation. This was the day before World Homeopathy Awareness Week – a coincidence? On 8 April – the day before the official release – the Skeptics group FSM published a letter congratulating the NHMRC on its findings! Clearly, FSM had received advance notification. Key stakeholders such as the AHA and other natural medicine groups received no notification whatsoever; they only found out through media reports.
As a requirement of the public consultation process, NHMRC called for any extra evidence ‘it may have missed’ to be submitted. It accepted an additional 40 studies for review, 37 of which reported positive results for homeopathy. Oddly, a different contractor assessed these studies, which were also assessed separately to the 176 studies considered by Optum.
NHMRC failed to disclose that in May 2014 it contracted a FSM supporter to review these additional 40 studies – another major unmanaged conflict of interest. All these studies were dismissed from consideration in the Review’s findings, on the basis that external stakeholders submitted them (‘self-selection bias’) – despite NHMRC having formally invited external parties to submit them. Thus, the entire public consultation process was a sham from start to finish.
Further, the FSM contractor belonged to a research group that was directly associated with FSM’s Vice President, Prof Alastair MacLennan who had declared homeopathy to be “snake oil” in FSM’s letter to the NHMRC CEO of 8 April 2014. A year later upon release of the final report, the NHMRC CEO reiterated this message when he publically declared homeopaths to be “snake oil merchants wanting to take your money” and “charlatans”. There is little more to say.
AS: This was clearly a deliberate and highly contrived plot to discredit homeopathy. Since the NHMRC is a taxpayer funded government institution, would intentional violation of research protocol also be a violation of the law? If so, how can they be held accountable?
GD: Yes, that is precisely what the AHA has asked Australia’s Commonwealth Ombudsman to investigate (a government agency that has powers to investigate the conduct of other government agencies). Under its legislation, the Ombudsman has the power to investigate both administrative and scientific misconduct. The NHMRC Homeopathy Review involved both.
With the help of the Homeopathy Research Institute, the AHA conducted an investigation into NHMRC procedures and methods and last year submitted a formal Submission of Complaint to the Ombudsman. A copy of the Executive Summary of the Complaint can be accessed at: www.nhmrchomeopathy.com and https://www.hri-research.org/resources/homeopathy-the-debate/the-australian-report-on-homeopathy/.
AS: If the Ombudsman agrees that the NHMRC has engaged in misconduct, what happens next?
GD: The Ombudsman can make a recommendation to the Australian government, but its recommendation is not binding. This is why the AHA has instigated the ‘Your Health Your Choice’ campaign (https://www.yourhealthyourchoice.com.au/) – to bring this issue to national and international attention and pressure the Australian government to act on (what we hope) is a favourable decision, and ensure the report is set aside. The campaign calls for a government inquiry into the NHMRC’s conduct, to support the Ombudsman complaint process and generate more pressure.