There is a fascinating article by Jeneen Interlandi ("An Unwelcome Discovery") on the case of Eric Poehlman, only the second researcher in United States history to be prosecuted for falsifying research data. While the case is disturbing -- and suggests that there are other cases of fraud in science that remain undiscovered -- it would be more damaging to overstate its significance.
First, as Interlandi herself notes, relationships based on trust are central within the scientific enterprise:
The principal investigator is not just a boss; he is also a teacher with knowledge and experience. “Trust is an essential component in any relationship, but especially between a student and mentor, especially in a research environment,” Tchernof [a former Poehlman postdoc who left to head a lab in Quebec City] told [Interlandi] in a telephone conversation last spring, before Poehlman’s sentencing hearing. “If you didn’t trust the person you were working with, you’d have to check every single raw data point. It simply would not work. But then it takes a substantial amount of doubt to overcome that established trust.”
The last point of that quote from Tchernof, that there is a default assumption of trust among scientific researchers -- a default assumption that is very difficult to overcome, is also important. If Tchernof is correct that trust is essential to the working of science, it would be disastrous if trusting relationships could too easily be destroyed; collaborators must be able to assume that they need not adopt a paranoid stance in their dealings with each other.
How does this fact jibe with Interlandi's attempt to suggest that the Poehlman case raises questions about the ability of the scientific process to be self-correcting? Here's Interlandi:
The scientific process is meant to be self-correcting. Peer review of scientific journals and the ability of scientists to replicate one another’s results are supposed to weed out erroneous conclusions and preserve the integrity of the scientific record over time. But the Poehlman case shows how a committed cheater can elude detection for years by playing on the trust — and the self-interest — of his or her junior colleagues.
My sense is that the Poehlman case is less alarming than Interlandi suggests, and that she perhaps failed to appreciate this because she failed adequately to consider one salient feature of the Poehlman case. The strategy that Poehlman employed in perpetrating his fraud is one that confirms a hypothesis that appeared in an earlier post on this site, "When Is Absence of Evidence Evidence of Absence? The Case of Fraud in Science ..." There I suggested that the most successful cases of fraud would also be the least revolutionary, because they would involve claims that would little repay other researchers' attempts at disconfirmation. This, however, is exactly what Interlandi's article suggests.
Thus, Poehlman's fatal faux pas that caused his lab technician, Walter DeNino, first to suspect that Poehlman was falsifying data involved the attempt by Poehlman to alter entries so that the data confirmed a widely-held hypothesis:
DeNino’s task was to compare the levels of lipids, or fats, in two sets of blood samples taken several years apart from a large group of patients. As the patients aged, Poehlman expected, the data would show an increase in low-density lipoprotein (LDL), which deposits cholesterol in arteries, and a decrease in high-density lipoprotein (HDL), which carries it to the liver, where it can be broken down. Poehlman’s hypothesis was not controversial; the idea that lipid levels worsen with age was supported by decades of circumstantial evidence. ... But when DeNino ran his first analysis, the data did not support the premise.
When Poehlman saw the unexpected results, he took the electronic file home with him. The following week, Poehlman returned the database to DeNino, explained that he had corrected some mistaken entries and asked DeNino to re-run the statistical analysis. Now the trend was clear: HDL appeared to decrease markedly over time, while LDL increased, exactly as they had hypothesized.
Indeed, the research that made Poehlman's reputation was, to use Kuhn's term, normal science, as opposed to revolutionary science:
Because most studies that examine the physiology of aging look at only one point in time, researchers can’t tell whether the differences measured are because of age, menopause or individual variation. Poehlman’s longitudinal study on menopause collected the same measurements from each person twice over a six-year period. This enabled him to show, for the first time, that some metabolic changes were from menopause, not aging. Published in 1995 in the Annals of Internal Medicine, the study confirmed a long-held assumption and helped establish Poehlman’s reputation. [Emphasis mine.]
This assessment is borne out by those Interlandi interviewed:
Some scientists believe that his ability to beat the system for so long had as much to do with the research topics he chose as with his aggressive tactics. His work was prominent, but none of his studies broke new scientific ground. (This may also be why no other scientists working in the field have retracted papers as a result of Poehlman’s fraud.) By testing undisputed assumptions on popular topics, Poehlman attracted enough attention to maintain his status but not enough to invite suspicion.
Now of course Poehlman's fraud was serious; the hundreds of thousands of dollars that Poehlman received could have funded genuine research. Notice, however, the amazing fact buried within the parentheses: no other scientists needed to retract papers as a result of Poehlman's fraud.
If anything, it is this fact that is perhaps most disturbing, particularly for those of us who work in the academy -- namely, how little of the research that is done, research receiving hundreds of thousands of dollars of support, in fact finds an audience outside of an often very tightly inscribed circle of one's closest associates. Although the lack of wider implications of the ten-year-long fraud unearthed by the investigation of the Poehlman case for the edifice of science should be reassuring to those who worry about the integrity of the scientific process, it should nevertheless be sobering for those who worry about the impact of much of the business of the academy on the world at large.
Recent Comments