Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« The Hard News on Cancer | Main | Play by Play in the Lab »

December 9, 2005

A Vioxx Bomb Drops. Or Does It?

Email This Entry

Posted by Derek

You don't see many articles in the medical literature titled "Expression of Concern". But that's how yesterday's release from the New England Journal of Medicine starts out (PDF).

The editors are expressing their concern over the published VIGOR trial data from 2000. It turns out that some time ago, the journal's editors had found a disk which had been part of the submission of the paper, and it contained earlier versions of the manscript. These had a blank table for cardiovascular events, and in what looks like another example of the wonders of the "track editing changes" function, it appeared that the table had been deleted by Merck two days before the final version. The journal wasn't sure what to make of this, and kept quiet (for four years) until their editor, Dr. Gregory Curfman, went through a deposition during this latest trial. He then saw a complete version of the earlier manuscript, which seemed to show more adverse cardiovascular data than appears in the final published version. (Forbes is all over this story; check that link for the full details).

This comes after the jury in the first Federal Vioxx trial had started deliberations. (For updates on how that's been going up until now, take a look at some of the Merck entries on Point of Law). What's it going to mean? Well, I would assume that the jury can't hear about this, for one thing, since none of this was introduced in testimony during the trial. It would seem to be fodder for a mistrial motion, though, if the verdict doesn't go the plaintiff's way. Should it be?

I've gone both ways on that question in the last few hours, but now I don't think so. While this story makes Merck look bad, idiotically bad, on closer inspection there isn't as much here as you'd think. The data in question are three heart attacks in the final weeks of the VIGOR trial. But the adverse cardiovascular event data in the paper, as published, didn't reach statistical significance, and they don't seem to reach it with these added in, either. On top of that, these data were submitted to the FDA during the drug's approval process, and (according to Point of Law) are on the Vioxx package insert itself.

So Merck might be guilty of making their data look better for the New England Journal of Medicine, but they're not guilty of hiding it from the world. And I'm not sure about that first charge, either. The lead author on the VIGOR study, Claire Bombadier of Toronto, told Forbes that the paper accurately disclosed the data and the she and the other authors are working on a response to the journal. But the headlines today are going to be variations on the theme of "Merck Hides Data". But as far as I can tell, they should be "Merck Looks Like Gang of Idiots, Blasts Away At Own Foot For Fourteenth Time". But that's not a crime. Yet.

Comments (21) + TrackBacks (1) | Category:


COMMENTS

1. tom bartlett on December 9, 2005 10:32 AM writes...

I think it is irresponsible for NEJM to come out with this. I would regard pre-press communications between journals and authors to be confidential, unless subpeoned. This SERIOUSLY undermines my faith in NEJM's ethics.

Permalink to Comment

2. Brian on December 9, 2005 10:46 AM writes...

I also find NEJM's actions here very unsettling...that they would delve into the revisions of a submission by using the "track changes" function may be legal in the strictest sense, but it sure as hell is not ethical.

There are any number of changes that a manuscript could go through prior to submission for any number of reasons. I've been known to put "fake" data into a manuscript in progress just as a placeholder to work out formatting, for example. Or to write in throw-away notes for sections I don't want to work on right away, such as "discuss BS paper by Smith et al. and why it's wrong". All of which could be greatly misconstrued by someone--especially someone with an agenda.

Permalink to Comment

3. Giraffe on December 9, 2005 12:25 PM writes...

I find Tom's comment totally perplexing. If the NEJM is to be faulted, it's for sitting on this for the past five years. The Journal's responsibility is to its readers, not to the multi-national companies that are jockeying for coverage within its pages.

Permalink to Comment

4. tom bartlett on December 9, 2005 1:11 PM writes...

"The Journal's responsibility is to its readers, not to the multi-national companies that are jockeying for coverage within its page"

If I edit my Will and Testament, I hardly think the prior edits are for public consumption in the absence of a court order. Perhaps the "left out" data seemed irrelevant to one or more of the interested parties at the time. I think the mag's obligation is to cooperate in any legal "Discovery " process, but dragging out old edits just seems an invasion of Merck's privacy to me.

Permalink to Comment

5. TFox on December 9, 2005 1:52 PM writes...

I think it's perfectly legit to go delving into the documents you've been sent, particularly if questions of malfeasance are on the table. The Jan-Hendrik Schon scandal was definitively proven through such means. I don't think the NEJM can be faulted for this. But...

What doesn't make sense is Curfman et al's claim that "certain calculations and conclusions" are made incorrect. Sure, the total MI percentage goes from 0.4% (as reported in the article) to 0.5%. It's a matter of rounding down vs. rounding up, and the 3 new events take us across the line. Whether this is important is a separate question; the more insignificant figures you display, the larger the change appears to be. In junior high I learned that you aren't supposed to display insignificant digits, this seems to be a lesson that Curfman et al didn't learn: check out their display of 3 sig figs on a number (the Relative Risk) which isn't known even to the order of magnitude (see the CI). These are basic numeracy skills, which even newspaper editors ought to know, much less medical journal editors.

The real question is whether conclusions are changed. Considering all patients, there was a significant difference in MI even without the new data, as correctly reported in the paper. For the nonaspirin group, there was no difference reported in the paper, and when you throw in the new events, there's still no difference (as Derek pointed out). Again, the paper remains correct. Finally, the most important question, where the risk comes from. The paper ascribes the added risk to the high risk aspirin group. The risk differential here is so large that even 3 new nonaspirin events don't change it; high risk patients on rofecoxib are keeling over at a rate 20x that of their low-risk cohorts (8+% vs 0.5%). Curfman calls this conclusion "misleading", but the new data gives you the same conclusion as the old!

So are calculations made "incorrect"? Only in a way that an innumerate reader would care about. Are the conclusions "incorrect" and/or "misleading"? No and no. This is basic stuff -- I doubt this "Editorial" would have made it through peer review without changes. I wonder what process NEJM used to assess it before going ahead and publishing? And will they be reconsidering that process going forward?

Permalink to Comment

6. Insider on December 9, 2005 1:59 PM writes...

Lets try and imagine the situation at the time. Who did what and why?

Merck have to explain, as have the investigators.

The NEJM are just doing what the medical editors code suggests in such circumstances.

The bigger question is: how many other diskettes are out there? Is there a whistleblower ready to come forward and help?

Permalink to Comment

7. burt on December 9, 2005 2:19 PM writes...

"how many other diskettes are out there?"

How quaint-- I haven't seen a floppy since-- well, since about 2000. If they weren't a publishing house, they probably wouldn't have equipmenty to read them.

Permalink to Comment

8. Daniel Newby on December 9, 2005 4:27 PM writes...

The ethical problem is not in publishing pre-press tidbits of unknown relevance. It is in only publishing a single such tidbit, carefully chosen for maximum mass media impact.

Permalink to Comment

9. Kram on December 9, 2005 6:06 PM writes...

What software was used ? How can anyone be sure that hovering over the cell displaying "Merck" is not forged ? Were digital signatures used ? This does not seem like CFR21 part 11 compliance at all. Would this evidence hold before a court ?

Permalink to Comment

10. Martin on December 10, 2005 1:18 AM writes...

It amazes me that the main concerns being expressed here are (a) the ethics of the NEJM in releasing this information and (b) whether of not the additional data made a statistical difference or not.

Surely the most important question is why this information was not reported by the authors of the study? And if the data was subsequently reported to the FDA why wasn't the journal told about it as well? It is not about Merck making itself looking silly: it's about whether we can trust any data submitted by industry-funded researchers.

Permalink to Comment

11. Kay on December 10, 2005 6:27 AM writes...

To Martin: One of the reasons for the reactions above is that Merck has long been seen as one of the white-hat members of our industry. I agree with your concerns for one simple reason: one of the Authors of the NEJM report was THE Vice President of Clinical Research for Merck.

This indicates that black hats got inside, and they brought their copies of "Skullduggery Basics" (PhRMA Press, Wash. DC, Old as the Hills) to their new job.

Derek: Have they lost most/all of their credibility?

Permalink to Comment

12. TFox on December 10, 2005 10:46 AM writes...

Martin,
Merck claims that the data as published are correct for the prespecified cutoff for analysis. Are they lying? It'd be an odd thing to lie about, considering that you can't exactly fake a prespecified cutoff, and it's not like nobody's going to bother to check them on this. Curfman seems to believe they are lying, if the Forbes quote ("We're not buying into that") is accurate. I guess we'll have to wait to see whose reputation stands in the end, as this story is certainly not over.

Permalink to Comment

13. Insider on December 11, 2005 3:58 PM writes...

At the risk of confusing the issue. Derek had some thoughts about Merck after the Pargluva debacle(What were they thinking? was the phrase, I believe).

I'm afraid I remain dubious about Mercks intentions. The document that listed Vioxx sceptic MDs who needed to be "neutralized" using various tactics including "show me the money" was, for me, very telling.

Permalink to Comment

14. tgibbs on December 11, 2005 6:17 PM writes...

If the data were beyond the cutoff date for the study, then strictly speaking Merck did the correct thing by not including them in the analysis. Questions of propriety would arise only if Merck chose to include other data beyond the cutoff, such as some GI events. If the cutoff had altered the statistical conclusions, Merck would have been ethically obligated to disclose this, but that appears not to have been the case.

On the other hand, it does seem that the NEJM has violated the bounds of acceptable practice for an academic journal. Vioxx is off the market, so there is not a public health issue to justify a rush to publication. Given the journal's concerns, the ethical action would have been to notify Merck of the Journal's intent to address their concerns about the data in an editorial, and give Merck adequate time to write a response for publication in the same issue.

Permalink to Comment

15. Insider on December 11, 2005 6:33 PM writes...

"I think this is terribly damaging to Merck,'' said Howard Erichson, a professor at Seton Hall Law School. "The fact that a prominent medical journal is accusing Merck with hiding information about Vioxx goes straight to the heart of their defense that they did nothing wrong.''

Permalink to Comment

16. JSinger on December 12, 2005 11:41 AM writes...

Given the journal's concerns, the ethical action would have been to notify Merck of the Journal's intent to address their concerns about the data in an editorial, and give Merck adequate time to write a response for publication in the same issue.

I completely concur. At a minimum, they should have made an attempt to resolve this with the authors instead of blindsiding them.

Permalink to Comment

17. Still Scared of Dinosaurs on December 12, 2005 1:08 PM writes...

If the analysis was prespecified then judgment would have to be exercised as to whether to include information collected outside of the time frame. One issue noted above is whether it changes the conclusions, in which case the inclusion should occur as appended data with a "however..." attached.
Another concern is whether the new events of note are part of information representing a sufficient sample to revisit prior conclusions. The timing of three events just after the cutoff period could just be bad luck. Also, if the 3 MIs were reported through expedited serious adverse event reports that would argue in favor of not reporting them until all the data for the period in which they occurred is collected. Both of the suppositions here would likely ultimately suggest that the risk was higher with Vioxx, but if there is bias or bad luck in the data collection that could make the increase appear worse than it really is. A more neutral possibility would be when more patients in a comparator arm drop out, in which case the risk is best expressed as events/patient years.
I do think that in the end the decision to include the data ultimately rests with the clinicians not employed by the sponsor. And that they should be given the opportunity to respond when challenged in the manner chosen by NeJM.

Permalink to Comment

18. Ragout on December 12, 2005 7:01 PM writes...

My guess is that the most damning charges are yet to come. The NEJM editors write: "In addition, the memorandum of July 5, 2000, contained other data on cardiovascular adverse events that we believe would have been relevant to the article."

As I read the NEJM note, they felt ethically obliged to report the facts they are sure about (the 3 missing MIs) as soon as possible. After all, Vioxx is still on the market in some countries (for example, Canada). I assume that they are waiting for the authors to reply before discussing other data that should have been included in the article.

In my view, the original NEJM article does everything possible to downplay the cardiovascular risk of Vioxx, short of lying. There's no table of cardiovascular risk data, only discussion in the text. Results are reported as reductions in MI due to Naproxen, not increases due to Vioxx. There's speculation that the results are due to the cardioprotective effects of Naproxen (unsupported at the time, and since debunked -- Naproxen may be somewhat cardioprotective, but not nearly enough to explain a 400% difference in MIs). Finally, there's the implication that the problem isn't Vioxx, it's that patients weren't taking aspirin. But aspirin isn't remotely protective enough to offset a 400% increase in risk. So, I'm not surprised that the NEJM editors feel like they've been misled.

Permalink to Comment

19. bala on December 12, 2005 9:16 PM writes...

Perhaps a closer look at the timeline of events would be illuminating here. Granted, these dates are from Merck's own website, but it seems to me that they are easily confirmed elsewhere and unlikely to be inaccurate. According to this timeline, the preliminary data from Vioxx, an ~9 month study involving about 8 000 patients, were first available to Merck in March 2000 and were submitted as preliminary data to the FDA that same month. In May of that year, these same preliminary data were submitted in manuscript form to NEJM (herein lies the controversy - this submission apparently omitted the three events occuring after the pre-determined study cutoff).

Now here's where things gets interesting. The finalized data from VIGOR were submitted to the FDA in the form of a supplemental NDA in June of 2000. These data, as I'm sure everyone agrees, included all events in the study, both those before and after the cutoff (and including the infamous three MIs). The NEJM paper was published in November - over four months AFTER the FDA had all that data in its hands.

Now, if you were going to commit scientific fraud and omit data from your manuscript, would you do that? It would be monumentally stupid to do so. Perhaps you can accuse Merck of splitting hairs here, but that's the worst you can do. The complete data set was where it needed to be - at the FDA and being incorporated into the label.

One should also take into account what else Merck knew at the time. I find it strange that no one seems to remember that the other large trial that ran concurrently with VIGOR, the Alzheimers treatment and prevention trial, showed no difference between Vioxx and placebo with respect to cardiovascular events. Not to mention that ultimately the study that led to withdrawl, APPROVe, itself found no difference in events between Vioxx and placebo until 18 months in. This is well beyond the time period of the VIGOR trial. Simply put, the VIGOR results did not repeat. Why? Your guess is as good as mine - maybe naproxen really is that protective, maybe the VIGOR trial was a statistical glitch. Who knows? How could anyone make sense of fragmented safety data like that?

In my opinion Merck has been villified in this forum, as well as in the general media, all too willingly. I have yet to see any evidence that this company knowingly did anything wrong. Did they make a mistake in developing a Cox-2 inhibitor? Yes, no doubt. Did they foist it on an unsuspecting public, knowing full well this drug class has mechanism-based toxicity? No, they did not. On the contrary, they studied the hell out of this drug - it was Merck's own trial (not mandated by the FDA, by the way) that ultimately revealed the danger of Cox-2 inhibition.

This is not a gun manufacturer or big tobacco, people. This is a company that recently developed a vaccine to prevent cervical cancer. Let me say that again - a vaccine that prevents cancer! That is just one of the many accomplishments that this company has had over the years. But maybe those days are over. We seek the downfall of this company at our peril.

Permalink to Comment

20. steve on December 13, 2005 12:09 AM writes...

I saw some numbers in the wall street journal about the vioxx lawsuit. The WSJ doesn't talk about the statistical significance, but 4 heart attacks out of 4,000 verses 17 out of 4,000 is not statistically significant. You can accurately say "4 times as many heart attacks" but when it's such a low percentage and the total number affected is less than 30, it's a nebulous area that even statisticians try to avoid at all costs. There are all kinds of adjustment factors to be applied and you can always argue about which factors to apply. It gets into too many method of application (social and philosophical) issues. The precision of how the study is consistently implemented would have to be within about 4 out of 4,000. For example, the instructions read to the patient and how adverse effect reports are handled. I don't think the medical community can do anything with that kind of precision except when simply measuring something on an analytical balance. It was probably a multi-center trail, so the consistency between medical centers would have to be that precise.

Permalink to Comment

21. Ragout on December 13, 2005 9:26 AM writes...

4 heart attacks out of 4,000 verses 17 out of 4,000 is not statistically significant.

Yes it is statistically significant: see the NEJM editorial Derek linked to.

Permalink to Comment

TRACKBACKS

Listed below are links to weblogs that reference A Vioxx Bomb Drops. Or Does It?:

NEJM v. Merck from blogs for industry
Derek Lowe has a post about NEJM concerns about a manuscript from Merck on Vioxx safety. NEJM published Expression of Concern: Bombardier et al., “Comparison of Upper Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with Rheumatoid Arthr... [Read More]

Tracked on December 13, 2005 2:21 AM


EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
Life Is Too Short For Some Journal Feeds
A New Look at Phenotypic Screening
Small Molecules - Really, Really Small
InterMune Bought
Citable Garbage
The Palbociclib Saga: Or Why We Need a Lot of Drug Companies
Why Not Bromine?
Fragonomics, Eh?