Is The Study Reliable and Truthful?

Are we heading into an era of junk science? From all indications, this seems to be the case. Patients will suffer and doctors will be blamed. Yet, it is Big Pharma that sets many of the studies that are not reliable and the leaders of the many physician organization, associations, and academies that promote the guidelines based on unreliable studies and data.

If you think otherwise, read this article in the New York Times and this blog by Dr. Malcolm Kendrick about why studies are not done to refute or confirm the previous studies. Another blog that is interesting is this one by Tom Naughton.

All confirm how poor and poorly constructed the research studies of today are and give some idea of where medicine is headed.

Normally I do not reference articles appearing in the New York Times because they are written for sensationalism, but this article actually outlines many of the problems of today's research studies. The biggest truth is some of the steps you can find out about a study.

“First, if the study examined the effects of a therapy only on animals or in a test tube, we have very limited insight into how it will actually work in humans. You should take any claims about effects on people with more than a grain of salt. Next, for studies involving humans, ask yourself: What method did the researchers use? How similar am I to the people it examined?” If you are between 21 and 50, chances are in your favor you could be similar, but if you are above the age of 50 or younger that 21, a healthy dose of skepticism is in order.

“This random assignment is powerful. If done with enough people, it causes the two groups to be statistically identical to each other except for the experience of the treatment (or not). Whatever changes are observed can usually be attributed to that treatment with a good degree of confidence.”

“Though a randomized trial makes two groups statistically identical to each other — apart from treatment received — it still doesn’t mean either group is identical to you. If the individuals selected to participate in the trial happen to be very similar to you — similar ages, income, living environment and so forth — that increases the chances that the results would apply to you. But if you’re, say, a 65-year-old, middle-class New Yorker, a study whose subjects were poor 30-somethings in rural China may not translate to your experience.”

“This is one of the chief limitations of randomized trials. They’re typically focused on narrow populations that meet strict criteria — those most likely to benefit from treatment. Many drug trials exclude older patients or children because of ethical or safety concerns. Many, particularly much earlier trials didn’t include women. We know a lot less about how drugs affect groups who weren’t studied than we might like. Harm could even come if it was assumed that findings from those who were studied applied to people who weren’t.”

Dr. Kendrick says, “It has become clear that much of medical research is flawed, and so inherently biased that much of it/most of it simply cannot be relied upon. One of the strongest critics of this current situation is a brilliant statistician, Professor John P Ionnadis. His seminal paper on the subject of medical research, which is nearly ten years old now, was entitled ‘Why Most Published Research Findings Are False ‘.”

I think if you can read all three of the links in the second paragraph above, you will understand why many studies are so poor.