Recent studies have shown…

I bet you’ve heard it on TV or read it in articles: how dangerous eggs are because they cause cancer, or how healthy broccoli is because it lowers your risk of heart disease and makes you live longer.

These scare stories, designed to grab attention (and boost the author’s ad revenue), usually open with the same magic incantation: “Recent studies have shown.”

There’s probably no emptier claim than that, and yet people fall back on it whenever a debate suddenly leaves them out of arguments — they use recent studies to protect their opinions (or to hide that they don’t understand the subject at all).

What is a study?

First we need to be clear about what a study actually is. Basically, it’s nothing more than a kind of test in which a particular hypothesis is put to an experiment, and the test either confirms or refutes it.

Who pays for it?

But there’s one fundamental problem. Well-designed, well-documented clinical trials take an enormous amount of time and money, and somebody has to pay for them. Scientists have traditionally been too poor to pay for studies themselves, but somebody simply has to cover the costs.

It makes sense, really. These days people aren’t satisfied with the opinion of some self-proclaimed guru, so these rainmakers hide behind the magic incantation “Recent studies have shown.”

If you run into one of these charlatans, try asking them two questions first:

Most of the time all you’ll get is a shrug and an open mouth. People who use this incantation usually know nothing about the studies, or have never actually read them, or haven’t looked into any possible conflict of interest.

They just want to sound educated and important. Maybe they heard or read about some study somewhere and are simply parroting it, even though they’ve never seen the study with their own eyes. Why? Because studying studies is seriously hard work!

Dr. Attia has a whole series of articles on his blog about how demanding it is to study studies, and he also points out the common problems that studies tend to run into.

Not all studies are equal

Another important factor is what type of study we’re actually talking about, because the quality of a study is measured by the strength of its evidence.

Let me briefly explain this pyramid, which shows three basic types of studies:

RCTs

Leaving aside meta-analyses at the very top, the most time-consuming and expensive studies are randomized controlled clinical trials (RCTs), which meta-analyses and systematic reviews are then built from.

Only RCTs can prove causation — cause and effect. Since nothing of the kind has been done on human diet, it makes no sense to talk about cause and effect, and anyone who does is quite simply lying to you.

Epidemiology

In second place are observational studies. These studies are sometimes called epidemiological, and they most often take the form of a filled-out questionnaire.

People simply fill out a questionnaire like this one, stating how often they eat, what their diet consists of and how often they go out to restaurants. Based on such questionnaires, researchers then track similar eating patterns alongside the incidence of various diseases.

The important thing, though, is that epidemiological studies can’t prove causation, only correlation.

In this case it’s obvious to everyone that the firefighters came to put the fire out and had nothing to do with starting it. But swap firefighters for cholesterol, and suddenly everything is different.

We’ve all heard how cholesterol clogs your arteries, so you need to eat low-fat products and take drugs for high cholesterol, such as the popular statins.

The problem is that cholesterol is just an innocent firefighter that the body sends to the site of damage in a blood vessel to patch the holes caused by inflammation. Cholesterol isn’t the cause of the problem, because it didn’t cause the damage to the vessel.

Trying to lower the cholesterol in your body makes about as much sense as trying to cut the number of firefighters so that there aren’t so many of them at fires.

Probably the best-known example of a skewed scientific approach is the infamous Seven Countries Study by Ancel Keys (whose caricature is in the cover picture of this article).

Ancel Keys deliberately picked for his conclusions only the countries where fat consumption correlated with the incidence of heart disease, threw out the rest, and then even claimed that there was a causal link between eating fat and heart disease.

It’s like claiming that because every premiere of a new Nicolas Cage movie coincided with more people drowning in backyard swimming pools, the actor caused their deaths.

Case studies

Last in the ranking of relevance are so-called case studies, which should serve only to formulate hypotheses. They are mostly studies done on animals, typically on rodents.

The problem with taking conclusions from these studies is obvious: a human isn’t a rodent, doesn’t eat like a rodent, doesn’t live as long (or rather as briefly) as a rodent, and is on the whole (surprisingly) quite different from a rodent.