Secret trials

Transparency is at the heart of medical science. Every day decisions are made about when to stop and start treatment and how best to invest large sums of money in ways to protect the public from disease. All these rely on knowing as much as possible about the benefits compared to the risks of action or inaction.

No medical treatment is perfect or suitable for everyone – that’s why balancing risks and benefits is crucial. But healthcare is big business; it’s where science meets big money and not all research evidence makes it into the public domain – specifically into medical journals where doctors and academics glean their information.

Medical history is replete with examples of the benefits of a treatment being overhyped and potentially serious side-effects being buried, leading to poor decisions. This wastes public money and can cost lives.

Take the case of the drug lorcainide, used to regulate the heartbeat during a heart attack. In the early 80s, researchers in Nottingham carried out a study of the drug in 95 people using a method known as a randomized control trial. They noticed that nine out of the 48 people taking the drug died, compared to only one out of 47 who got a sugar pill, or placebo, instead.

At the time, the researchers thought that the high number of deaths in those given lorcainide might have been due to chance rather than the drug itself. For commercial reasons, the drug was not developed any further and the results of the trial were never published. However other, similar, heart drugs did make it onto the market and were widely used. But they too had serious safety problems and many were withdrawn.

According to Sir Iain Chalmers, a long-standing champion of transparency in medical research, the lorcainide trial might have been an early warning of trouble ahead for these other heart drugs. At the peak of their use in the late 80s, these medicines are estimated to have caused between 20 000 and 70 000 premature deaths every year in the US alone.

This is a particularly stark example of what might happen when critical evidence remains unavailable to doctors and researchers. Even when individual drugs do make it onto the market and have overcome the regulatory hurdles, information about their risks and benefits might well be hard to come by.

What the public doesn’t know

In western countries, legislation dictates that companies have to provide regulators with a thorough scientific dossier on all trials conducted on a drug so the data can be scrutinized before the drug is allowed onto the market. They are then required to do follow-up studies looking at any adverse reactions that might not have been picked up in the pre-market research. They must inform the authorities about what they find.

Many companies, however, have been reprimanded – mainly in the US courts – for hiding troubling side-effects of drugs, including: anti-depressants, such as Seroxat (known as Paxil in the US; generic name paroxetine) and painkillers, such as Vioxx (rofecoxib).

But it’s not always the companies which are unforthcoming about safety concerns; the regulators have dragged their feet too. Last year, the diabetes drug Avandia (rosiglitazone) was suspended from the market in Europe and severely restricted in the US because of an increased risk of heart problems. But this was long after both the manufacturer, GlaxoSmithKline (GSK), and the US regulator had reason to suspect an increase in serious side-effects.

Rather than the regulators – whose remit is to protect the public – it was the actions of the then New York attorney general, Eliot Spitzer, in a 2004 court case of GSK’s Seroxat, that led to the side-effects of Avandia coming to public attention. As part of a settlement with the state over its hiding of data on heightened suicide risk in teenagers who took the drug, GSK agreed to post results from its recent clinical studies on a website. And this included studies of the drug Avandia, many of which had been unpublished until then.

Three years later, Dr Steven Nissen, chairman of cardiovascular medicine at the high-profile Cleveland Clinic in the US, decided to analyse all the studies of Avandia on the website. Using a research method called meta-analysis, he pooled all the results together to see what they said overall. He found that the risk of having heart problems in people with diabetes who took the drug rose by 43 per cent compared to those who had diabetes and did not take it.

The following years entailed investigations into GSK’s conduct by the US Senate; intense deliberations by national drug regulators; questions about how we regulate medicine; and now pending class actions. But what really broke the case open was enforced transparency. ‘It’s important to realize what an important role publicly available trial results data played in the rosiglitazone story’, said Jerry Avorn, professor of medicine at Harvard Medical School.

During an investigation in collaboration with BBC’s Panorama in September 2010, the British Medical Journal looked into the different drug regulators’ attitudes towards transparency. In the US, the Food and Drug Administration’s (FDA) advisory committee discussions are held in public in front of the national press. Most of the relevant scientific documents are made available on a website in advance. Before the deliberations start, each panellist is required to declare any conflicts of interest in line with US legislation to increase transparency.
The UK: a need for transparency
But gaining an overall perspective of discussions within the European and UK regulators was far trickier. The BMJ attempted to speak to people who had sat on panels for them both, but they were bound by confidentiality clauses. Nor would Europe’s regulator release the names of the members of the scientific advisory group discussing the drug under the Freedom of Information Act (FOIA).

Doctors and the public in the UK had not been told that the national regulator had voted unanimously to take Avandia off the market several months before the European agency came to the same decision. If the European vote had gone the other way, who knows if the views of the UK’s panel would ever have been revealed.

Some say that open discussions and more transparency do not necessarily lead to better decisions. But documents obtained from the European regulators under the FOIA showed that advisers had concerns about Avandia’s side-effects from the outset. And knowing about these could have lent support to other academics who were ‘intimidated’ by the company, according to a 2007 report by the US Senate Finance Committee.

Manipulation of data

In 1999, when the drug was first licensed, Dr John Buse, a professor of medicine at the University of North Carolina who specializes in diabetes, told attendees of academic meetings that he was concerned that while Avandia lowered blood sugar, it also caused an increased risk of heart problems.

Concerned about the effects that his comments would have on their drug that had been touted for blockbuster status, executives at GSK (then SmithKline Beecham) devised “what appears to be an orchestrated plan to stifle his opinion”, the Senate Finance Committee report stated – in the light of internal company documents it had seen.

The report goes on to state that GSK executives labelled Buse a “renegade” and silenced his concerns about Avandia by complaining to his superiors and threatening a lawsuit. GSK prepared and required Buse to sign a letter claiming that he was no longer worried about cardiovascular risks associated with Avandia. Then, after he signed the letter, GSK officials began referring to it as Buse’s “retraction letter” to curry favour with a financial consulting company that was evaluating GSK’s products for investors. GSK has denied all allegations in the report, describing them as “absolutely false”.

Years later, Buse wrote a private email to a colleague detailing the incident with GSK: “I was certainly intimidated by them. … It makes me embarrassed to have caved in several years ago.”

Meanwhile, over on the other side of the Atlantic, EU drug agencies were drawing similar conclusions that the drug increased the risk of heart problems during their premarket discussions. In March 2000, Buse sent a letter to the FDA, saying Avandia might raise patients’ risk of heart attacks, and he criticized the company’s marketing, saying it employed “blatant selective manipulation of data” to overstate the drug’s benefits and understate its risks. Doctors may not have prescribed the drug if they had known from the outset there were issues around its safety.

Tamiflu and hidden data

But data transparency doesn’t just mean exposing harm done, it can also help to establish how well something works ‹- and that reported benefits aren’t just hype. Major international decisions are made on how best to tackle impending health crises based on how well a medical invention works as reported in journals, for example the UK government’s decision to stockpile the influenza drug Tamiflu.

Back in 2009, during the swine flu pandemic, the internationally respected Cochrane Collaboration, a network of independent academics, was commissioned by the NHS to look at the evidence about the benefits and risks of using Tamiflu – a drug the UK had spent around £500m on to treat all those infected in the outbreak.

The academics, led by Christopher Del Mar at Bond University in Australia, scoured the medical literature to find all the different relevant studies of the drug to pool together all the results to see what they said. They were also aware that there had been reports of suicides in Japan – the biggest consumers of Tamiflu – and they wanted to find out more.

But when they went about surveying the medical literature, not all of the trials they knew existed about the effects of the drug in healthy people appeared in the medical press. To fairly reflect the evidence, they needed to know exactly what all trials said. But they couldn’t access all the data they needed – the majority of trials were unpublished. This included the biggest, and therefore arguably the most important, trial conducted.

The UK government at the time had based its decision to stockpile Tamiflu in such large quantities on one particular piece of research published in 2003. This paper showed the dramatic benefits of giving Tamiflu to healthy people who got the flu and not just those who were at particular risk of getting sick. It claimed that the drug reduced the number of people taken to hospital with the flu by a half and reduced serious complications by around the same amount. Little wonder that health officials, concerned about the strain on the NHS, stockpiled the red and yellow pills in such vast quantities.

But this piece of research was funded by the drug’s manufacturer, Roche. It relied upon eight unpublished studies, each given code names, and used the company’s own statisticians to draw conclusions about the data. The two independent researchers named on the paper – who are supposed to be accountable for the content of the research – could not produce the unpublished studies when the Cochrane Collaboration asked them.
Medical research relies heavily on the ability to replicate the findings of another piece of research. This helps to show that a finding wasn’t fraudulent or simply due to chance.

But the Cochrane Collaboration couldn’t replicate the 2003 findings. Its calculations based on the publicly available papers were at odds with the claims made and it needed to see the unpublished studies, so it turned to the company.

Despite asking Roche repeatedly for the full complement of research documents showing that Tamiflu would stop so many healthy people from going into hospital, the whole set were never forthcoming. What it did provide was limited in detail and not what the Cochrane Collaboration needed. Roche did nothing illegal – it is its commercial information. But its commercial information has huge repercussions for public health spend – both in terms of direct costs of the drug and its distribution, but also on what economists call the opportunity costs. Half a billion spent on Tamiflu is half a billion not spent on some other wonder drug.

Del Mar and his team were left to wonder if these bold claims really did stack up – and if the unpublished trials really were the best of the lot, why were they unpublished?

What should have been a straightforward exercise to confirm the evidence base for current policy and practice became instead a complex investigation involving the Cochrane Collaboration, the BMJ and Channel 4 News. Not only did this unmask the extent of unpublished data, it found that the person who actually wrote some of the journal papers was never credited – known in the trade as ghostwriting.

This is not the benign undertaking it is in celebrity autobiographies. Commercial medical writing firms team up with drug companies to draft a series of academic papers aimed at medical journals to promote a carefully crafted message. In the case of Tamiflu, it was that the drug helps to reduce serious complications.

The lead investigator author who was named on the biggest trial ­ which was unpublished – said that he couldn’t remember ever having participated in the trial when the BMJ/Channel 4 News asked him. And the investigation revealed that documents submitted to Nice (the National Institute for Health and Clinical Excellence) show different investigator names appended to the key Tamiflu trials at different points – nowhere is it totally clear who took overall responsibility for all of the studies.

Behind closed doors

In a later twist, an investigation the BMJ conducted with the Bureau of Investigative Journalism revealed that experts who had been paid to promote Tamiflu were also authors of influential World Health Organisation (WHO) guidance on the treatment and prevention of pandemic flu. Nowhere were their conflicts of interest made public, despite the WHO having a specific policy to exclude those with such major competing interests from crafting guidelines. And when the scientific evidence pointed to a serious global outbreak of swine flu in early 2009, the WHO pulled together an international expert panel called the Emergency Committee. Keeping up the trend of opacity that had been a recurrent feature of pandemic planning, the committee executed its decisions – which the former health secretary, Alan Johnson, said would lead to “costly and risky” repercussions – behind closed doors in Geneva. An internal WHO investigation conducted by Harvey Fineberg, president of the US Institute of Medicine, criticized the lack of transparency and timely disclosure of conflicts of interest in May last year.

After an inauspicious start – with experts from within the US regulatory agency saying the benefits of healthy people taking the drug were marginal at the outset – Tamiflu sales sky-rocketed. This, coupled with a mild strain of flu and an abject lack of transparency, allowed conspiracy theories to ferment that alleged the WHO was in league with big pharma and had fostered fears of a pandemic in order to boost sales of drugs. And with blogosphere rumours abounding, not only has the WHO’s reputation taken a hit, scepticism might well accompany future warnings of serious flu outbreaks.

“Open access should be the default setting”

Yet again the role of the regulators comes into the spotlight. Roche said that it had supplied all the required data to US and EU regulatory authorities. Only after five months of chasing drug regulators with FOI requests, asking for the full study reports of trials that Roche submitted for its market approval, did the Cochrane Collaboration get some of what it asked for.

“Open access should be the default setting for drug trials once the drug is registered. The public pay for the drug, the public should have access to the facts, not sanitized versions of them”, one of the Cochrane collaborators, Dr Tom Jefferson, said. He believes that drug regulators should make data accessible once a drug comes onto the market. Others suggest that the regulators should also publish the data of drugs that have failed to make it onto the market. That way the situation that happened with locainide would be avoided.

This, too, might be helpful for those charged with making decisions about which drugs health services should use, such as Nice. Writing in the BMJ last year, researchers from the official German drug assessment body charged with synthesizing evidence on the antidepressant Edronax (generic: reboxetine) reported they had encountered serious obstacles when they tried to get unpublished clinical trial information from the drug company that held the data.

Once they were able to integrate the astounding 74 per cent of patient data that had previously been unpublished, their conclusion was damning: Edronax (reboxetine) is “overall an ineffective and potentially harmful antidepressant”. This conclusion starkly contradicted the findings of other recent studies that pooled the data published by reputable journals.

But the amounts of data submitted to regulators can be voluminous – another reason why overstretched and underfunded drug authorities could benefit from the safeguard of publicly available data that academics could analyse. The Cochrane Collaboration is now in possession of over 24 000 pages to peruse and distil. But this kind of volume doesn’t deter researchers; they are actively asking for it.

In June this year, Medtronic, a medical technology company, drew widespread criticism in the US for its alleged failure in published research papers to mention the side-effects of a spinal treatment it manufactures. Capitalizing on the company’s dip in public opinion, Harlan Krumholz, professor of medicine and public health at Yale University, approached Medtronic to take part in a transparency programme for industry that he had set up. He wanted access to all data it had on file – published and unpublished – to commission two independent reviews of it to see what it really said about safety.

“Industry’s reputation has really dropped substantially. People are concerned. They’ve lost confidence and trust in these companies,'”Krumholz said, adding: “Marketing has sometimes gotten the best of the companies and there have been some episodes that have tarnished their reputation. So they are in great need to show to the public that they are really interested in the societal good and want to contribute in ways that are meaningful.”

The company obliged and described its move as “unprecedented in the medical industry”. Needless to say, not all companies are keen on having their data analysed by independent researchers. When Krumholz first approached manufacturers asking them to allow the scientific community to vet their data when safety concerns had emerged, he was rebuffed at every turn. Nevertheless, he hopes this will change and transparency will become expected rather than simply celebrated. He hopes his scheme will make it impossible for other companies – particularly when questions are being raised about the safety of their products – to simply say that they are not going to share all the information they have that may be relevant.

Betraying trust

But there is a broader ethical aspect to selective publication. People often participate in clinical trials because they want to help grow scientific knowledge. And the very nature of many trials means there is a level of uncertainty of what a drug or device may do. This includes any potential benefits and it also involves risks.

According to Chalmers, those who don’t publish all the studies are betraying the trust of those who have volunteered themselves to medical science. “If a patient takes part in a clinical trial – which is essentially an experiment – they are doing their service to humanity and putting themselves at the disposal of science. Unless patients are explicitly told that the results won’t be published if the trial does not show what the researchers or the company want before they start the trial, there is a dereliction of duty on behalf of the researchers.”

Chalmers is uncompromising on what the fate of doctors who are complicit in the burying of bad results should be – they should face discipline that might include the loss of their right to practise medicine or conduct research. His mood reflects a growing concern about the moral duty of medical scientists to publish their results. Journal editors have railed against what they consider a distortion of the medical literature.

But for many years there has been comparative silence from organisations representing people conducting medical research. In the UK, the charge for transparency has been led by the Faculty of Pharmaceutical Medicine in London. Over a decade ago, it said: “Pharmaceutical physicians have a particular ethical responsibility to ensure that the evidence on which doctors should make their prescribing decisions is freely available.”

In June this year, the Royal Statistical Society followed suit and released a statement saying it is “committed to transparency in scientific and social research”. It said it is “crucially important that the results of scientific research should be made publicly available and disseminated as widely as is practical in a timely fashion after completion of the scientific investigation provided that there is no conflict with any legislation on confidentiality of data”.

Chalmers is critical of organisations who represent people conducting medical research – such as the Academy of Medical Sciences and the Royal College of Physicians – which refuse to sign up to a bill of transparency.

Attempts have been made to limit a researcher’s ability to hide trials that they may not want to come to light. Registers of trials sprang up. In 2005, the International Committee of Medical Journal Editors said its journals would only publish trials that were fully registered before they started – which should make trials that went missing much easier to spot. Then, in 2007, the US implemented legislation to ensure that all trials protocols are listed on a public searchable website called clinicaltrials.gov. Companies are supposed to update the information with changes or highlight when and where their research has been published. But the BMJ has found instances where the information on the website is out of date. And, unless someone goes through the database systematically to identify what studies have surfaced publicly, it’s hard to pin down exactly what impact the register has had on publication bias.

But once again, Europe trails behind in terms of transparency. The names of the trials being conducted in the EU appear on the EudraCT database. But crucial details of the study design and where it’s taking place are not on the website.

Europe: an example of how bad it can get

If data transparency is an issue for drugs, the opacity surrounding medical device governance is in a different league. Medical devices cover a wide range of products from adhesive bandages and syringes to heavy duty implantables, such as hip prostheses, pacemakers and stents.

Representatives of the drug industry marvel at how devices get away with a comparative lack of government and public oversight both in the US and the EU. Debates about the perceived flaws in the US system have been hammered out in public – the media weighing in on what they considered to be a failure of their regulators to protect the public adequately. Front page coverage of hip replacements failing and heart devices misfiring has forced discussions about inadequacies in their system into the US Congress.

But this has not happened to the same extent in Europe. One senior US official asked me why the European media has not scrutinized device regulation in the way that the American press had. In the States, Europe has been held up as an example of how bad things can actually get – with patients on this side of the Atlantic having been described as “guinea pigs”.

A joint BMJ/Channel 4 Dispatches in May this year didn’t do much to quell concerns. The EU system of approval by agreement between manufacturer and a commercial regulatory body operates under conditions of almost total commercial secrecy and is overseen in a hands-off manner by national regulatory authorities. Manufacturers submit data to a private body, which then assesses it to see if it is fit for market, and it is then allowed to display a CE mark. It is the same process that non-medical products such as mobile phones and toys go through.

As Nick Freemantle, professor of epidemiology at UCL, said: “The current European regulatory framework – CE marking – might provide sufficient safeguards for electric toasters and kettles, but it is not adequate for treatments that can affect symptoms, health related quality of life, serious morbidity and mortality.”

Representatives of device manufacturers say that the European light touch regulation approach is fine – that there is no evidence it is any worse than America’s. But, as the medical adage goes, absence of evidence is not evidence of absence.

There is no way of knowing what percentage of serious medical devices are faulty, poorly designed or have had to be recalled, because the European authorities have no centrally maintained register listing the devices on the market. In short, they do not know exactly what patients have had put into them in the first place.

Nor do they know on what evidence market entry was based. No European governmental regulator has it – scientific data sits with the manufacturers and the private companies that “approve” the device. As the head of device regulation in the US, Dr Jeffrey Shuren, said: “For the public in the EU, there is no transparency. The approval [requirements] are just what deal is cut between the device company and the private [organisation].”

Even data about devices that have been pulled from the market is virtually impossible to come by. When the BMJ – together with two doctors from Oxford University – contacted 192 manufacturers of withdrawn medical devices requesting evidence of the clinical data used to approve their devices, they denied us access, claiming that “clinical data is proprietary information”, that it was “company confidential information” and that they could discuss only “publicly available information” – of which there is very little.

Likewise, when we asked the relevant commercial regulatory bodies for the scientific rationale for approval of various devices that had been recalled, the results were stark. This information was classed as confidential because they were working as a client on behalf of the manufacturers – not the people who have them implanted in their bodies.

Even the Freedom of Information Act is of little help in obtaining information on any adverse events. The BMJ/Channel 4 Dispatches attempts to get access to adverse incident reports for specific implantables from the UK national regulator through the act were thwarted because it is overridden by medical device legislation. Article 15 of the EU Medical Devices Directive states: “Member States shall ensure that all the parties involved in the application of this Directive are bound to observe confidentiality with regard to all information obtained in carrying out their tasks.”

Even the Association of British Healthcare Industries, a trade organisation of device manufacturers, agrees that the lack of transparency leads to misunderstanding and mistrust. “Today it is very hard for anyone, even manufacturers and authorities, let alone citizens, to find out what products are approved to be on the market. We would like to see enhanced transparency and information to patients, citizens and all EU government authorities.”

Signs of change

So what does this mean? It means that doctors and patients are left to trust the companies to provide them with information about the benefits and harms of using their products. But with little scrutiny, oversight and transparency, there are no guarantees of this being a fair reflection of what their data – where they have it – actually says.

But there is a movement for change. As Krumholz says: “I think one day people will look back and say now wait a minute. Half of the data were beyond public view and yet people were making decisions every day about these products? How did you let that happen? And I’m not sure how we let it happen.

“But I hope we’ll enter an era where that will be over, and in fact there will be a great sharing of data, that we’ll be able to have a public dialogue that’s truly informed by the totality of evidence, and that we’ll be able to make choices that are based on all of that evidence, knowing that there are no perfect drugs. That’s always going to be a trade off. But we ought to be informed by all the evidence when we’re making these decisions.”