Research Flaws

The Scientific Validation of Herbs Part 2

(Read part one)

2) Research Flaws
Flawed research is really an extension of what I was just writing about. It is important to consider research flaws, because if the research used as the basis of an evidence-based practice is flawed, then people are not getting the highest quality of care. Scientifically, there is not a study out there that is flawless. Indeed, researchers get together weekly, review other people’s work, and discuss the limitations (which is an nice way of saying they tear the research apart).

Actually one of the most flawed scientific designs from a practical perspective is what is considered the gold standard of clinical research: the double-blind, randomized, placebo-controlled study. The assumption behind this research design is that substances have inherent power and efficacy and that people’s opinions, attitudes, beliefs, expectations, and intentions also have power. This research is designed to tease apart the inherent power of the drug from the inherent power of the mind. This is a worthy goal, but this research tends to devalue the mind’s power and suggest a substance’s power is more real or valid. Indeed, the placebo effect is often eluded to as an effect that is not really real or worthy of notice.

Personally, I do not understand why, once people recognized the placebo effect, researchers didn’t focus more on optimizing the inherent healing ability of the mind. If the mind is so powerful, why are we even bothering with drugs? I suppose this may have something to do with the economics of selling drugs. In addition, this type of research is just a left-over from the limiting worldview based on Newtonian physics. In Newtonian physics a drug has an action and will always produce a predictable effect. This is a comfortable idea for many people. Quantum physics supports the idea that a wide range of possibilities are possible and that we, as humans, can influence the outcome of a drug’s effect.   Explaining the world as cause and effect will eventually fall by the wayside, along with double-blind, placebo-controlled studies, as our society adopts a worldview more in alignment with quantum physics.

What is the greatest limitation of research that is based on the double-blind, randomized, placebo-controlled study? Well, it is possible that some substances are more active when combined with the mind. This important interaction is neglected in a study designed to take all human connection and intention out of the picture. Further, human variation is amazing. It is possible that people will gravitate to the right treatment for their health problem and the idea of randomly assigning them to a test group limits the broad applicability of the results. A treatment that “doesn’t work” with the population tested, may actually be the perfect cure for certain individuals that might naturally seek out that treatment.

Other common flaws in herbal research are:

Dosing does not reflect clinical practice. I’ve seen this in research on herbs versus the common cold. I, personally, can stop any cold using Berberis. The key things are starting at first onset of cold symptoms and using high and frequent doses as needed to stop symptom progression. If I start when I just feel tired, I just need a squirt or two of tincture every once and awhile. If I start after my throat feels sore, I need a squirt of tincture every half hour until my throat doesn’t feel sore, then I can cut back.

Research was done on a single herb that is usually used in combination. Or vice versa, the research tests a proprietary blend. I was taught that herbs are feeble, but that the target effect of a combination of herbs can have a bigger impact. Still, many herbs are tested using the drug model that ignores the synergistic effects of herbal combinations.

Research was done on a form of the plant not usually used clinically. Often times companies want to validate their product and they fund research on their preparation. It is common to assume that all preparations of the plant will work similarly, but that is just an assumption. It is also assumed that each batch of the plant will work similarly, but because of natural product variability, you cannot even count on that.

Natural products and extraction methods are variable. This is not really a flaw in the research, but does represent a limitation of using research reports about herbs. I used to get menstrual migraines. I was using an alcohol extract of feverfew to reduce the frequency and intensity of those headaches. It seemed to be working. Then some research was released that demonstrated that the powered herb was effective, but not an alcohol extract. I believed it and stopped using the tincture. My headaches got worse. When I resumed taking the extract I once again experience fewer headaches with milder intensity.

Botanical product not characterized. Much of the older herb research doesn’t even report if they confirmed they had the right plant or the correct part of the plant (root vs leaf vs seed, etc). Sorry, scientists make the assumption if they buy something it actually is what they think it is, but herbs are not as straightforward as chemicals. They may also neglect to explain how the plant was extracted. In addition, the chemical analysis of the plant, which is critical to determining if one batch is equivalent to another, is rarely done or reported. Again, there was an assumption that natural products have no variability.

A traditionally used herbs was tested out of context. When herbs are used as part of traditional healing systems other components, such as ritual and/or song, may be necessary to activate the herb. For instance, in Tibetan Buddhism there are mixtures of herbs that allow meditative practitioners to go for long periods of time without food. These herbs have to be activated by special words (mantras) and are used in conjunction with specific practices.


Wild Nettles

Test subjects do not match you. The results of using an herb may differ based on the person’s lifestyle, diet, ethnicity, etc. For instance, nettles is a common nutritive herb and food. Indeed, in Britain it is sold as cream of nettles soup. In United States and United Kingdom it is considered to be completely safe to use in any quantity. It really is a food and many women use it during pregnancy because it is rich in minerals. The fresh plant is also a potent anti-histamine and perfect for allergies. I once recommended it to a pregnant friend with hayfever. She was early in her pregnancy and started having uterine cramping/contractions. Thanks to the internet, I was able to find out that nettles does have this effect on Indian women.

Delivery method is not traditional. Some of the research coming out of Europe, where use of herbs is a standard practice in medicine, uses injectable herbal extracts. The results of an injectable form may not reflect what the herb does if taken orally. One example is the treatment of Amarita mushroom poisoning using a milk thistle extract intravenously. In the United States, mushroom poisoning is treated with liver transplants. We don’t have the IV extract they have in Europe and the oral preparations will not protect a liver from Amarita mushroom poisoning. (If you are going to eat poisonous mushrooms, better do it in Europe. Think about the differences in cost and human suffering!)

Placebo used was not really inactive. It is hard to find an inert substance to match a strong tasting herb for a placebo controlled study. I’ve seen research on the common cold that used alfalfa as the placebo. Apparently the researchers didn’t know that alfalfa is an herb that is used in the treatment of the common cold. It’s not a primary defense, but it does support the body’s ability to carry waste out of the body.

Subjects can figure out if they have the placebo or herb. If the researchers pick something that is really just filler, many people can figure out they don’t have the active substance and the study is not longer “blind”. When I was at UC Davis a women enrolled in a long term study involving gingko and the prevention of Alzheimer’s called me. She was worried about developing the disease and did not want to spend years on the placebo. I told her to just open the capsule and taste the contents. It did have flavor, but she wanted to be sure it was gingko so she came over. I tasted it and assured her she was getting gingko. Opps! Study no longer placebo controlled.

Confounding factors matter. Research has shown that the color of the tablet makes a difference in psych medications. The timing of treatment also makes a difference.   Rats undergoing radiation therapy for cancer will either live or die depending on the time of day the treatment was given due to daily fluctuations in cortisol levels. Season and stage of the lunar cycle may also be important in treatment outcome. And, of course, intention and subconscious beliefs can influence treatment results.

3) Incomplete and Inaccurate Research Reports
Read more…

CREDITS:  Urtica dioica by Franz Xaver – Own work. Licensed under CC BY-SA 3.0 via Commons –


I’ve been so busy with work and school, it is so good to finally have some time to write.

This week we studied co-occurring disorders.  I was especially interested in the perspective presented on depression.  Mainstream medicine appears to be in denial around the use of anti-depressants.  Research consistently shows that antidepressants do not work any better than placebo.  So why do we still use them?  They get approved as drugs because they do have a statistically significant effect on depression.  However, that effect is not clinically significant.

The difference between clinical significance and statistical significance can be described like this:  Le’ts say that people start the study with depression reported as a 7.0 on a ten point scale with the most severe depression being 10.  And at the end of the study they have improved to a 4.0.  That’s great!  They are feeling better.  However, the placebo controls have also improved and they come in at a 4.1.  For our example the 4.0 is statistically different than the 4.1.  However the difference between placebo and anti-depressant is not clinically different.  People just don’t say, “my depression is down to 4.0 from 4.1 and I feel the difference.”

Different ailments appear to have respond differently to placebo.  Depression is highly responsive to placebo.  Research from 1999 found the effects of placebo on depression to be 75%.  However, these placebo effects on depression have been continuing to grow over the past couple decades, so that now we are finding certain antidepressants no better than placebo.  Menopausal hot flashes also have a nice placebo response.  And at least one third of the results of pain relief can be attributed to placebo effect alone.

Placebo, or context effect as I like to call it, has been extensively studied.  We know some of the things that will increase the likelihood of a treatment being effective.  These range from the color of the pill to patients perception’s of the health care practitioner.   Placebo effect is really not much different than marketing.

The mental health practitioner doesn’t realize that when they listen attentively to their client, label them with a disorder name (e.g. “You have bipolar depression.”) and tell the client that there is a pill that will “stabilize their brain chemistry” they are actually marketing the treatment.  They may also tell the client that the drug’s effects could “kick in” immediately, or they might occur slowly over a period of a couple months.  This gives the client options on how to respond to their “placebo/antidepressant”.  If the client then runs into someone that is on “their” antidepressant and they say it “saved their life”, they have additional context to boost the effectiveness of the treatment.

The word “placebo” has gotten a bad wrap, in the sense that only a fool would respond to placebo effect.  That is one of the reasons I prefer “context effects” over “placebo”.  Context matters and optimizing context to support healing seems like a good strategy, not a unethical strategy.  Certain settings, particular colors, and personal biases make a difference in treatment effectiveness.  Does optimizing context fall under the adage of “first do no harm” or does it harm someone to play into their natural tendencies to put faith in certain people, places and things?

Kirsch, I., (2008). Challenging Received Wisdom: Antidepressants and the Placebo Effect.  McGill Journal of Medicine, 11(2): 219-222.

Di Blasi, Z. & Kleijnen, J., (2003).  CONTEXT EFFECTS:  Powerful Therapies or Methodological Bias?  Evaluation & The Health Professions, 26(2), 166-179 DOI: 10.1177/0163278703252254.