Pharmaceutical Research

Scientific Validation of Herbs Part 5

Read Part 1.  Read Part 2. Read Part 3. Read Part 4.

Research Bias:  Pharmaceutical Research
We’ve been talking about research that is what I might call “pure” research. Pharmaceutical drugs studies actually fall outside of traditional scientific research channels, so it is best to consider them separately.

Pharmaceutical companies are interested in getting their product okayed by the FDA as a drug. They develop the drug, pay for the studies, and run the studies themselves in conjunction with medical test sites. Their research is inherently biased. The people running the studies are paid by the pharmaceutical company. The data they collect is prepared in house and submitted to the FDA for approval. There is no peer review process where the data is evaluated before publication. Publication of the research is not the objective, the objective is to demonstrate the safety of the drug and that it can be used to treat a certain disease. They are focused on “selling” their drug to the FDA so they can get it on the market. There is an assumption that they are not falsifying data.

Another bias is present in the FDA review process itself. Members of FDA advisory panels are allowed to have financial ties to drug companies and still participate in the approval process. For example, in recent years the contraceptive drugs Yaz and Yasim were implicated in 100 deaths, demonstrated a three-fold increase in blood clot risk, and 12,000 unhappy patients were suing for damages. The FDA finally decided to investigate the drugs. The advisory panel voted 15 to 11 to keep the drugs on the market. Four or five of the panel members had financial interests in the company that produced the drugs. They were all allowed to vote and voted to keep the drug on the market. Another doctor, that had written an article calling for the drugs to be removed from the market, was kicked off the panel for an “intellectual conflict of interest”.

In addition to bias within the advisory panel, it is not uncommon for the FDA staff to have worked in the industry they will be regulating as part of the FDA. Further, key policy makers may also have financial ties to the industry they regulate.

In addition, sometimes drug studies come up with a significant effect that is clinically irrelevant. A hypothetical example might be a drug that significantly reduces cholesterol. That sounds really good until you look at the clinical reality that the drug has a bunch of side effects and the reduction of cholesterol was only 1%. This means if someone has a high cholesterol value of 245 the drug would bring it down to 243. The reduction is actually meaningless and the patient still has high cholesterol.

You might wonder, would a doctor actually prescribe a drug that was so worthless. Maybe. Doctors receive their drug education along with gifts, free samples, and perks from drug companies. They would be told the hypothetical drug is FDA approved for the reduction of cholesterol. Why would they question it further?

The hypothetical situation demonstrates how this might work, but this is exactly what is going on with anti-depressants. It has long been know that anti-depressants work just as well as placebo in patients with mild, moderate and even severe depression. Indeed, they only show true clinical efficacy in patients that are very severely depressed. Yet, doctors still prescribe them. They are expensive placebos with unpleasant side effects.

Colleen O. Davis, Red Tape Tightrope:  Regulating Financial Conflicts of Interest in FDA Advisory Committees, 91 WASH. U.L. Rev. 1591 (2014)

John Kelley, Antidepressants: Do they “work” or don’t they?, Scientific American, Mar2, 2010. 

Research Bias

Scientific Validation of Herbs Part 4

Read Part 1.  Read Part 2. Read Part 3.

4) Research Bias
The results of research can be distorted based on conflicts of interest. The most common conflict of interest is when researchers have a financial investment in the research showing a positive result. The most obvious form of this is when the researcher is testing a product or service that they sell personally or they own stock or interest in a company that does. This conflict of interest would include a relative or friend selling the product they are testing. In that case they would have an interest in the success of their relative or friend.

Although, researchers are supposed to indicate any conflict of interest, they may want to present their interest as unbiased and will neglect to mention any conflict of interest that is not a direct financial gain. Indeed, sometimes the funding agency is a foundation that upon closer inspection appears to have been set up so that a researcher can launder the financial contribution to the study by having the funding come from the foundation rather than directly from the manufacturer of the test preparation.

Direct financial gain isn’t the only way researchers may make money on studies. Many researchers need to publish interesting research in order to maintain their careers and ensure future research funds. Researchers make a living by getting someone to fund their research. An interesting research proposal will gain funding, but a researcher needs to report satisfactory progress each year to maintain that funding. A researcher may feel pressure to make a study work. I once heard a statistician say, “If you think there is an effect, we can find a way to make the statistics show it.”

In addition, researcher may be motivated by fame or just wanting to be right. I talked to a former student once that worked for a researcher that had her run chemical analysis on samples over and over until, by a fluke, the data matched the researcher’s hypothesis. The researcher would review the data and circle the samples to rerun. The student reported that the repeated samples would give the same result, but researcher would have her rerun them over and over. Eventually, a run would give the value that fit with what the researcher wanted. The student could see the farce in this, but as long as the researcher had an actual result, they did not feel like they were falsifying data.

Pharmaceutical Research
to be continued tomorrow..

Incomplete and Inaccurate Research Reports

Scientific Validation of Herbs Part 3

Read part 1Read part 2.

3) Incomplete and Inaccurate Research Reports
Let’s consider the study design we were talking about earlier regarding Cimicifuga and hot flashes. Perhaps someone decides to run that study and finds out that a standardized extract of Cimicifuga (30 mg three times a day) does not reduce the number of hot flashes in Caucasian women that have just undergone a hysterectomy. By the time the results hit the news all you learn is that the study demonstrated that Cimicifuga doesn’t work. However, this doesn’t mean that a woman undergoing natural menopause wouldn’t see a reduction in hot flash intensity with using half that dose. The results of the study are interesting, but may not apply to you unless your situation is identical.

In addition, many people rely of scientific summaries or research abstracts for information. Unfortunately, these are often designed to be sensational or intriguing. Even if they are accurate, they are written with a world limit and a lot of information has to be left out. I have found that upon reading the full article the results are not really as clinically relevant as the abstract would lead one to believe.

Why don’t people read the full article? It is an access issue. Researchers and students have access to the full scientific reports as part of their institutional perk. For a private individuals one article can sometimes cost more than $30. There has been a shift in recent years to require research done with public funds to be accessible to the public, but it doesn’t really solve the problem of restricted access to other research. Hard to make an educated decision when you are unable to get access to the full report.

4) Research bias
Read More…