Posted: May 13, 2012 in Association and Causality, Crisis in Nutrition, Red Meat
Tags: carbohydrate, Meat Intake and Mortality, red meat,,
Tags: carbohydrate, Meat Intake and Mortality, red meat,,
TIME: You’re partnering with, among others, Harvard University on this. In an alternate Lady Gaga universe, would you have liked to have gone to Harvard?
Lady Gaga: I don’t know. I am going to Harvard today. So that’ll do.
– Belinda Luscombe, Time Magazine, March 12, 2012
What’s odd is that this is all described from a distance as if the study by Pan, et al (and likely the content of the blog) hadn’t come from Harvard itself but was rather a natural phenomenon, similar to the way every seminar on obesity begins with a state-by-state picture of obesity as if it were some kind of meteorologic phenomenon.
When the article refers to “headline writers,” we are probably supposed to imagine sleazy tabloid publishers like the ones who are always pushing first amendment rights in the old Law & Order episodes. The Newsletter article, however, is not any less exaggerated. (My friends in English Departments tell me that self-reference is some kind of hallmark of real art). It is not true that the Harvard study was urging moderation. In fact, it is admitted that the original paper “sounded ominous. Every extra daily serving of unprocessed red meat (steak, hamburger, pork, etc.) increased the risk of dying prematurely by 13%. Processed red meat (hot dogs, sausage, bacon, and the like) upped the risk by 20%.” That is what the paper urged. Not moderation. Prohibition. Who wants to buck odds like that? Who wants to die prematurely?
It wasn’t just the media. Critics in the blogosphere were also working over-time deconstructing the study. Among the faults that were cited, a fault common to much of the medical literature and the popular press, was the reporting of relative risk.
The limitations of reporting relative risk or odds ratio are widely discussed in popular and technical statistical books and I ran through the analysis in the earlier post. Relative risk destroys information. It obscures what the risks were to begin with. I usually point out that you can double your odds of winning the lottery if you buy two tickets instead of one. So why do people keep doing it? One reason, of course, is that it makes your work look more significant. But, if you don’t report the absolute change in risk, you may be scaring people about risks that aren’t real. The nutritional establishment is not good at facing their critics but on this one, they admit that they don’t wish to contest the issue.
Nolo Contendere.
“To err is human, said the duck as it got off the chicken’s back”
— Curt Jürgens in The Devil’s General
Deaths per 1,000 people per year | ||
| 1 serving unprocessed meat a week | 2 servings unprocessed meat a day | |
| Women | 7.0 | 8.5 |
| 3 servings unprocessed meat a week | 2 servings unprocessed meat a day | |
| Men | 12.3 | 13.0 |
Definitely less scary. For men, the absolute difference in risk going from 3 servings a week to 14 servings a week is less than 0.1 % per year (the units in the two columns of the table are different, presumably, in order to maintain the deception coefficient). Put another way, you would have to drastically change the eating habits (from 14 to 3 servings) of 1, 429 men to save one life. Well, it’s something. After all for millions of people, it could add up. Or would it? We have to step back and ask what this study is about.
It is commonly said that observational studies only generate hypotheses and that association does not imply causation. Whatever the philosophical idea behind these statements, it is not exactly what is done in science. There are an infinite number of observations you can make. When you compare two phenomena, you usually have an idea in mind (however much it is unstated). As Einstein put it “your theory determines the measurement you make.” Pan, et al. were testing the hypothesis that red met increases mortality. If they had done the right analysis, they would have admitted that the test had failed and the hypothesis was not true. The association was very weak and the underlying mechanism was, in fact, not borne out. In some sense, in science, there is only association. God does not whisper in our ear that the electron is charged. We make an association between an electron source and the response of a detector. Association does not necessarily imply causality, however; the association has to be strong and the underlying mechanism that made us make the association in the first place, must make sense.
What is the mechanism that would make you think that red meat increased mortality. One of the most remarkable statements in the paper:
“Regarding CVD mortality, we previously reported that red meat intake was associated with an increased risk of coronary heart disease2, 14 and saturated fat and cholesterol from red meat may partially explain this association. The association between red meat and CVD mortality was moderately attenuated after further adjustment for saturated fat and cholesterol, suggesting a mediating role for these nutrients.” (my italics)
This bizarre statement that saturated fat played a role in increased risk because it reduced risk was morphed in the Harvard News Letters plea bargain to “The authors of the Archives paper suggest that the increased risk from red meat may come from the saturated fat, cholesterol, and iron it delivers” although the blogger forgot to add “…although the data show the opposite.” Reference (2) cited above had the conclusion that “Consumption of processed meats, but not red meats, is associated with higher incidence of CHD and diabetes mellitus.” In essence, the hypothesis is not falsifiable — any association at all will be accepted as proof. The conclusion may be accepted if you do not look at the data.
The Data
In fact, the data are not available. The individual points are grouped together in quintiles (the red meat intake is broken up into five groups) so that it is not clear what the individual variation is and therefore what your real expectation of actually living longer with less meat is. Quintiles are some kind of anachronism presumably from a period when computers were expensive and it was hard to print out all the data (or, sometimes, a representative sample). If the data were really shown, it would be possible to recognize that it had a shotgun quality, that the results were all over the place and that whatever the statistical correlation, it is unlikely to be meaningful in any real world sense. But you can’t even see the quintiles, at least not the raw data. The outcome is corrected for all kinds of things, smoking, age, etc. This might actually be a conservative approach — the raw data might show more risk — but only the computer knows for sure.
Confounders
“…mathematically, though, there is no distinction between confounding and explanatory variables.”
— Walter Willett, Nutritional Epidemiology, 2o edition.
This is the essence of it. The underlying philosophy of this type of analysis is “you are what you eat.” The major challenge to this idea is that carbohydrates, in particular, control the response to other nutrients but, in the face of the plea of nolo contendere, it is all moot.
Who paid for this and what should be done.
We paid for it. Pan, et al was funded in part by 6 NIH grants. (No wonder there is no money for studies of carbohydrate restriction). It is hard to believe with all the flaws pointed out here and admitted by the Harvard Health Blog and others, that this was subject to any meaningful peer review. The plea of no contest does not imply negligence or intent to do harm but something is wrong. The clear attempt to influence the dietary habits of the population is not justified by an absolute risk reduction of less than one-tenth of one per cent, especially given that others have made the case that some part of the population, particularly the elderly may not get adequate protein. The need for an oversight committee of impartial scientists is the most important conclusion of Pan, et al. I will suggest it to the NIH.
