It is important to distinguish between “bad” and “wrong.” In the 2 X 2 table of good/bad versus right/wrong, all four cells are filled, and failure to distinguish among them inappropriately condemns the field. “Good and right” is devoutly to be wished. “Bad and wrong” is anathema. The discordant pairs are a problem. “Bad and right” can happen; what? 5% of the time? But the real problem is “good and wrong.” In fact, the HRT controversy is an example of how, in the progress of epidemiologic science, many may have gotten it wrong.
Critique of Epidemiology: Changing the Terms of the Debate
ReplyDeleteby Ruth Ness
History of Cardiovascular Disease Research Archive
University of Minnesota School of Public Health
http://www.epi.umn.edu/cvdepi/essay.asp?id=128
Reference
Ness, R.B., Rothenberg, R. 2007. Editorial. Critique of Epidemiology: Changing the Terms of the Debate. Ann. Epidemiol. 17:1011-1012.
Future person 1: You mean there was no deep fat? No steak or cream pies or hot fudge?
ReplyDeleteFuture person 2: Those were thought to be unhealthy, precisely the opposite of what we now know to be true. (From Sleeper (1973) by Woody Allen; a segment of conversation after he wakes up in 2173)
Gary Taubes recently mounted an assault on observational epidemiology in the cover story of The New York Times Magazine. If you only glanced at the byline, you would have read, “Much of what we’re told about diet, lifestyle and disease is based on epidemiologic studies. What if it is just bad science?” He focused on the controversy surrounding epidemiologists’ changing assessment of the effect of hormone replacement therapy (HRT). Several features of the article bear highlighting.
First, his opening paragraphs imply a cataclysmic about-face in this assessment of risk. That summary is in marked contrast to his more leisurely (and accurate) unfolding of the story a few paragraphs later. Second, he assumes the mantle of muckraker, uncovering fundamental flaws in epidemiologic thinking, and suggesting that epidemiologists hoodwinked ordinary citizens. As you read the articles, you realize that he is in fact reporting on the concerns and ruminations of luminaries in the field. Laudable self-appraisal, the source of his article, is transmogrified into self-serving incompetence.
Mr. Taubes is no stranger to attacks on epidemiology studies. He has cornered the market on “dissing” epidemiology since the publication of a nihilistic editorial in Science in 1995. Attainment of cover status in The New York Times Magazine has widely disseminated his skepticism.
We epidemiologists are our own severest critics. To have honest, thoughtful self-doubt thrust back in our faces surely grates. But our concern about public attack goes beyond the venting of petty annoyance. Public doubt in science, abetted by the current political environment (think: evolution and global warming) has been mounting for years. Epidemiology, by the nature of what it is and does, is especially vulnerable.
The distrust of epidemiology comes from several sources. First, over the past 60 years, epidemiology has developed a sophisticated analytic armamentarium that is less-than-accessible to the general public. But methods of data collection and the underlying quality of information have not kept pace. Such tension is a constant companion to the practice of epidemiology. As a result, epidemiology does not provide a quick fix. With its focus on risk factors that have long latency periods and interventions that necessitate lifestyle modifications, epidemiology quickly loses the sound-byte fight.
ReplyDeleteSecond, the public, quite rightly, reacts poorly to being told one day that butter should be assiduously avoided and the next that butter in moderation is not at all bad. Here, we stand justly accused. Those of us who use our research as public health pronouncements may be right on occasion, but fail to recognize the social ramifications of a rush to intervention.
So how do we regain the high ground? First, we must stop “dissing” Mr. Taubes (however gratifying), and start learning how to communicate the uncertainty inherent in our science. To begin, we might explore Mr. Taubes’s use of the terms “bad science.” It is important to distinguish between “bad” and “wrong.” In the 2 X 2 table of good/bad versus right/wrong, all four cells are filled, and failure to distinguish among them inappropriately condemns the field. “Good and right” is devoutly to be wished. “Bad and wrong” is anathema. The discordant pairs are a problem. “Bad and right” can happen; what? 5% of the time? But the real problem is “good and wrong.” In fact, the HRT controversy is an example of how, in the progress of epidemiologic science, many may have gotten it wrong. And even this, as Mr. Taubes himself remarks, may not be the final word. Only a body of evidence, fully accumulated over time, will inform causal thinking and prevention strategies. While this is happening, we must work to change the way in which the media report scientific findings and the public absorbs them.
ReplyDeleteA second component of gaining the high ground is to realize that traditional notions of causality, probably dating back to germ theory, are insufficient to capture the complexity of systems wherein multiple factors and multiple levels are operative. Think of the steps in the traditional sequence of research: a factor is found to be associated with a disease, usually in a case series or case-control study; a follow-up (small) cohort study is confirmatory; someone finally gets funded to do a large cohort study; and perhaps an even more costly randomized controlled trial (RCT) is ultimately organized. Mr. Taubes focuses on the dissonance between the penultimate and last step, but in fact there is uncertainty in every step, even after the RCT results are in. One way to resolve the uncertainty is to implement a public health program and see if the intervention reduces the target disease, but there are obvious downsides to that approach (fools rush in; wise persons dither).
Recently, there has been some interest in rethinking causal approaches by placing epidemiologic findings in the larger context of complex systems. Such systems modeling underscores the need to consider complexity, which RCTs alone do not, since they are designed to focus on a narrow question in a selected population. But this approach is to be developed, and will be no complete panacea.
A third pathway to the high ground is to place our successes in perspective. Life expectancy has skyrocketed in the past century, with marked improvements in healthy older life. The epidemiology underlying preventive practice can take credit for some of this, and much of the remainder: medical care, changing social circumstances, has been sorted out through epidemiologic analysis. In this arena, the critical distinction for policymakers and the public is the difference between individual risk and population impact (another distinction that Mr. Taubes clearly recognizes in his article). The individual stories do have power: a person with a disastrous family history of heart disease embraces a healthy diet and exercise, ups his or her statin when the coronary calcification score rises, and lives disease free into older age. But such stories have their contretemps: the young, fit runner who collapses and dies during a marathon (anyone remember Jim Fixx?). The clinical approach has limits and pitfalls, and we must make clear, and garner support for, the importance of population-based science.
Perhaps most important, we must lobby for energetic, unencumbered collection of epidemiologic data, from surveillance to association studies to clinical trials, on the basis that such data will bring future benefits. Would the public want to give up avian influenza surveillance? Are our communities happy to continue to add chemicals to the environment and never conduct human studies to assess their potential toxicity? Would people vote for public policies and accept medical interventions based solely on results in rats? The need to evaluate and reduce health risks in people is, in its own right, remarkably compelling.
ReplyDeleteOver 20 years ago, Lewis Kuller argued for an office in Washington to advocate for our profession, a good idea whose time has now come. For the first time in our professional history, a dozen societies representing epidemiologists have joined forces to form a Joint Policy Committee (JPC), under the banner of the Societies of Epidemiology. In concert with the long-established American College of Epidemiology Policy Committee, the JPC, now 9 months old, has conducted a survey of the impact of the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule on research; helped to secure an Institute of Medicine Committee on HIPAA and research; weighed in on the ongoing reconsideration of National Institute Health peer review; and began planning for a lobbying trip to promote epidemiology to Washington funders.
This is a start, but not enough. The profession of epidemiology should promote its interests as other organizations do, with lobbyists in the halls of Congress and organized letter-writing campaigns. We should align ourselves with patient and interest groups. We should find multiple avenues to educate the public. Such efforts are time consuming, expensive, and require great common will. In the best of all worlds, the science would speak for itself, but this is not the best world, it is the real world. We need to speak for our science.
Preventing Heart Attack and Stroke
ReplyDeleteA history of cardiovascular disease epidemiology
http://www.epi.umn.edu/cvdepi/
This site presents a history of the research of investigators among populations as they worked in tandem with clinical and bench scientists to seek the causes of common heart and blood vessel diseases and the potential for their prevention.