Momentary Language

Flashbulb memories and 9/11

Over at the Frontal Cortex blog, Jonah Lehrer has a post today that is basically one big shout-out to research done by my colleague, Jennifer Talarico, on the reliability -- and the perceived reliability -- of flashbulb memories.  (See also, more recently, here.)

More on Fraud in Science: the Poehlman Case

There is a fascinating article by Jeneen Interlandi ("An Unwelcome Discovery") on the case of Eric Poehlman, only the second researcher in United States history to be prosecuted for falsifying research data. While the case is disturbing -- and suggests that there are other cases of fraud in science that remain undiscovered -- it would be more damaging to overstate its significance.

First, as Interlandi herself notes, relationships based on trust are central within the scientific enterprise:

The principal investigator is not just a boss; he is also a teacher with knowledge and experience. “Trust is an essential component in any relationship, but especially between a student and mentor, especially in a research environment,” Tchernof [a former Poehlman postdoc who left to head a lab in Quebec City] told [Interlandi] in a telephone conversation last spring, before Poehlman’s sentencing hearing. “If you didn’t trust the person you were working with, you’d have to check every single raw data point. It simply would not work. But then it takes a substantial amount of doubt to overcome that established trust.”

The last point of that quote from Tchernof, that there is a default assumption of trust among scientific researchers -- a default assumption that is very difficult to overcome, is also important. If Tchernof is correct that trust is essential to the working of science, it would be disastrous if trusting relationships could too easily be destroyed; collaborators must be able to assume that they need not adopt a paranoid stance in their dealings with each other.

How does this fact jibe with Interlandi's attempt to suggest that the Poehlman case raises questions about the ability of the scientific process to be self-correcting? Here's Interlandi:

The scientific process is meant to be self-correcting. Peer review of scientific journals and the ability of scientists to replicate one another’s results are supposed to weed out erroneous conclusions and preserve the integrity of the scientific record over time. But the Poehlman case shows how a committed cheater can elude detection for years by playing on the trust — and the self-interest — of his or her junior colleagues.

My sense is that the Poehlman case is less alarming than Interlandi suggests, and that she perhaps failed to appreciate this because she failed adequately to consider one salient feature of the Poehlman case. The strategy that Poehlman employed in perpetrating his fraud is one that confirms a hypothesis that appeared in an earlier post on this site, "When Is Absence of Evidence Evidence of Absence? The Case of Fraud in Science ..." There I suggested that the most successful cases of fraud would also be the least revolutionary, because they would involve claims that would little repay other researchers' attempts at disconfirmation. This, however, is exactly what Interlandi's article suggests.

Thus, Poehlman's fatal faux pas that caused his lab technician, Walter DeNino, first to suspect that Poehlman was falsifying data involved the attempt by Poehlman to alter entries so that the data confirmed a widely-held hypothesis:

DeNino’s task was to compare the levels of lipids, or fats, in two sets of blood samples taken several years apart from a large group of patients. As the patients aged, Poehlman expected, the data would show an increase in low-density lipoprotein (LDL), which deposits cholesterol in arteries, and a decrease in high-density lipoprotein (HDL), which carries it to the liver, where it can be broken down. Poehlman’s hypothesis was not controversial; the idea that lipid levels worsen with age was supported by decades of circumstantial evidence. ... But when DeNino ran his first analysis, the data did not support the premise.

When Poehlman saw the unexpected results, he took the electronic file home with him. The following week, Poehlman returned the database to DeNino, explained that he had corrected some mistaken entries and asked DeNino to re-run the statistical analysis. Now the trend was clear: HDL appeared to decrease markedly over time, while LDL increased, exactly as they had hypothesized.

Indeed, the research that made Poehlman's reputation was, to use Kuhn's term, normal science, as opposed to revolutionary science:

Because most studies that examine the physiology of aging look at only one point in time, researchers can’t tell whether the differences measured are because of age, menopause or individual variation. Poehlman’s longitudinal study on menopause collected the same measurements from each person twice over a six-year period. This enabled him to show, for the first time, that some metabolic changes were from menopause, not aging. Published in 1995 in the Annals of Internal Medicine, the study confirmed a long-held assumption and helped establish Poehlman’s reputation. [Emphasis mine.]

This assessment is borne out by those Interlandi interviewed:

Some scientists believe that his ability to beat the system for so long had as much to do with the research topics he chose as with his aggressive tactics. His work was prominent, but none of his studies broke new scientific ground. (This may also be why no other scientists working in the field have retracted papers as a result of Poehlman’s fraud.) By testing undisputed assumptions on popular topics, Poehlman attracted enough attention to maintain his status but not enough to invite suspicion.

Now of course Poehlman's fraud was serious; the hundreds of thousands of dollars that Poehlman received could have funded genuine research. Notice, however, the amazing fact buried within the parentheses: no other scientists needed to retract papers as a result of Poehlman's fraud.

If anything, it is this fact that is perhaps most disturbing, particularly for those of us who work in the academy -- namely, how little of the research that is done, research receiving hundreds of thousands of dollars of support, in fact finds an audience outside of an often very tightly inscribed circle of one's closest associates. Although the lack of wider implications of the ten-year-long fraud unearthed by the investigation of the Poehlman case for the edifice of science should be reassuring to those who worry about the integrity of the scientific process, it should nevertheless be sobering for those who worry about the impact of much of the business of the academy on the world at large.

The epistemology of acting however the poets require

It is useless to attempt to reason a man out of a thing he was never reasoned into. -Attributed to Jonathan Swift

This is my 100th post, and since I am, by training and inclination, an epistemologist, I thought it only fitting that I do a little philosophy. Here goes.

Knowledge is widely accepted to involve belief as a component. Belief is widely accepted to involve some sort of assignment of high probability as a component. Consider, however, this utterance:

[U] "I know that there are other women out there with whom I could be happy, nevertheless I still believe that she is the only one with whom I could be happy."

We can, I think, easily imagine someone reasonably making such a claim as [U]. This, however, causes problems for the widely accepted views of those concepts of knowledge and belief canvassed above.

If knowledge involves belief, then the claim above would involve its speaker in flat contradiction -- the speaker would be stating, among other things, that he believes that there are others with whom he could be happy and that there are no others with whom he could be happy.

The common strategy for dealing with such utterances is to claim that they involve a non-standard notion of knowledge -- not the philosophical notion of warranted true belief, but perhaps simply a notion involving a high degree of certainty. This however, also seems problematic.

As we noted above, belief generally is seen to involve an assignment of high probability. But on the high-degree-of-certainty reading of knowledge in the utterance above, this would involve interpreting the speaker as claiming that he has assigns a high probability both to the claim that there are others with whom he could be happy and that there are no others with whom he could be happy -- indeed, presumably, that he assigns a higher probability to the former than to the latter. However, it would seem to violate rationality norms on belief were one reasonably to be able to believe a proposition despite assigning a higher probability to another proposition that one knew to be incompatible with one's belief.

The solution I think I favor is this. The notion of belief involved in the utterance above is simply that of the proposition on which one is disposed to act. That is, what the speaker is saying in uttering [U] is that, despite the fact that he knows that there are others with whom he could be happy, nevertheless he is disposed to act as if the opposite were the case -- waiting on the object of his affection, being absolutely incapable of even considering other options, acting, in other words, however it is that the poets require.

Even this explanation of the reasonableness of uttering [U], however, would require further refinement. For presumably, betting is a form of action. But we could imagine that our speaker would, if offered a wager, bet against there being no other person with whom he could be happy, and bet on there being others with whom he could be happy, despite the fact that his other, non-betting, behavior would not reflect those probability assignments -- despite the fact, even, that he could be painfully aware that his other behavior would not reflect those betting assignments.

The upshot of all of this, the big news-flash for my 100th post, would seem to be this: reason is the servant of the passions. Whether that is something that the poets require, it certainly can't hurt when it comes to winning them readers.

How radical was Locke on testimony?

Those familiar with the literature on testimony will recognize the following quote, from Locke's Essay Concerning Human Understanding, Bk. I, Chap. IV, Sec. 24:

we may as rationally hope to see with other men's eyes as to know by other men's understanding ... The floating of other men's opinions in our brains makes us not one jot the more knowing, though they happen to be true. What in them was science is in us but opiniatrety. (Quoted in C.A.J. Coady, Testimony, 14)

I was reminded of this passage from Locke while reading Steven Shapin's "The House of Experiment in Seventeenth-Century England." Shapin quotes Sprat, who, in his History of the Royal Society, noted that there was a division of labor among the members of the Society, with those particularly skilled in experimentation responsible for carrying out experiments and reporting back on the results to the other members. Sprat writes:

Those, to whom the conduct of the Experiment is committed ... do (as it were) carry the eyes, and the imaginations of the whole company into the Laboratory with them." (Quoted in Shapin.)

What was initially striking for me about this passage from Sprat should be obvious: Locke's own contemporaries in fact did not find it irrational to hope to "see with other men's eyes"!
The underlying position that Locke took with respect to testimony, however, is in some sense borne out by Sprat. Locke's main target in the passage quoted above is opinion -- i.e., the conclusions drawn as the result of experimental evidence. And here the traditions of the Royal Society seem to be more in line with Lockean prejudices. Again, quoting Sprat:

though the Experiment was but the private task of one or two, or some such small number; yet the conjecturing, and debating on its consequences, was still the employment of their full, and solemn Assemblies. (Quoted in Shapin.)

Of course, this partial defense of Locke relies on the distinction between observation statements and conjectures, etc., made on the basis of those statements (a distinction that Coady, in his discussions of historical figures in Testimony, criticizes a number of philosophers for making), but this version of Locke seems less immediately vulnerable to criticism, at the very least.

When is Absence of Evidence Evidence of Absence? The Case of Fraud in Science ...

According to Horace Freeland Judson, author of The Great Betrayal,

grandees of the scientific establishment regularly proclaim that scientific fraud is vanishingly rare and that perpetrators are isolated individuals who act out of a twisted psychopathology. As a corrolary, they insist that science is self-correcting. ... They could not be so dogmatic if they had considered what evidence there is that might back up general conclusions, positive or negative, about the nature and incidence of scientific fraud. Their claims about fraud are unscientific." (26-7)

The implication here is that Judson himself is in a position to provide evidence that backs up negative general conclusions about the nature and incidence of scientific fraud. Let's see.

First modify the claim to the following:

[Modified Self-Correction] Science is structured in such a way that fraudulent research either has relatively trivial results (both in terms of its detrimental effects for the edifice of science and of its positive effects for the perpetrator of the fraud) or is doomed to be revealed as fraudulent.

That is, either the fraud concerns a result so trivial that it will not impinge upon central result in any area of science, nor bring great credit upon the perpetrator of the fraud, or the fraud concerns a result so central that it is bound to be discovered in relatively short order. I'll call this the MSC thesis. My suggestion is that none of Judson's evidence provides better support for another, more negative claim, than it does for MSC.

Judson himself recognizes that "the brute fact is that many published papers, tens of thousands every year, are ignored. They are not woven into the fabric [of science]; they are cited rarely or not at all except perhaps by their authors. Such vanishing papers are not part of live science." (37) If this is the case, however, then certainly fraudulent research leading to such "vanishing papers" would hardly pose a significant challenge to the edifice of science. Let's call these the insignificant results; they would seem to be covered by the first disjunct of MSC. So any support for a more negative finding by Judson would seem to rest on the significant laboratory results.

Judson suggests that "significant laboratory results don't normally get verified ... for three reasons ... . Two of them are institutional: repeating other scientists' work is not an enterprise that attracts funding, and journals rarely publish negative results. The third reason lies in the practical problems of laboratory research: some experiments can't be rerun." (39)

Let's grant that significant results are never verified -- in the sense of being performed again -- and that this is the case for the reasons Judson cites. Nevertheless, if the results are significant, as we are assuming, then the results will be indirectly verified. This will occur in at least two ways.
First,

when you publish an interesting new result, your closest competitor on reading your paper slaps his forehead and exclaims, "Why didn't I think of that!" But then he realizes, "If that's true, why then the next step is X, and I'd better get back to the lab pronto." Confirmation is indirect: if the new finding works, it can be built into the growing edifice. (36)

Indirect confirmation, however, is still -- at least provisionally -- confirmation. Indeed, there is no reason why such indirect confirmation shouldn't provide at least as much evidential confirmation as a successful repetition of an initial experiment.

Second,

In much science, confirmation comes about by the use of independent but neighboring and convergent approaches and results -- hence, triangulation. ... [T]he mid-nineteenth century philosopher and historian of science William Whewell called this by the satisfying phrase "the consilience of inductions." ... Triangulation can yield new findings. It allows the scientist building on another's claimed result to receive credit. It provides cross-checks that will undermine a claim or make it more robust. (36)

The example that Judson cites of triangulation (or consilience), the determination of Avogadro's number "from a variety of unrelated methods" at the beginning of the 20th century, is a stark example of the powerful evidence that such triangulation can provide.

Where does this leave the prospects for a negative finding concerning the significance of fraud in science? It seems that the only potentially damaging case would be a significant finding that wasn't potentially testable in terms of the first sort of indirect confirmation or of triangulation. Such a finding, however, could hardly be termed significant. Thus, it would seem impossible to give a case supporting a more negative thesis than MSC.

Initial Thoughts on Testimony and Defeat

In a recent post on "Testimony and Defeat" at Certain Doubts, Jon Kvanvig has suggested difficulties with a thesis that he attributes to Gary Gutting, namely,

(GT) One's judgment that one disagrees with an intellectual peer is a defeater for one's justification.

As an alternative, Kvanvig proposes what he terms the "irrelevance of competence" position. In the post, he suggests that this position amounts to the claim that "information about the intellectual competence of a disagree-er is never an underminer of present opinion." The example he gives to support this involves a case where I think someone is smarter than I am and has thought more about the matter in question, and thereby come to doubt my own belief. According to Kvanvig, "what is doing the defeating here is my view that you know more about this issue."

I'm confused -- I would've thought that knowing more about the issue just is competence. I take it that part of what is bothering Kvanvig is the distinction between general and specific competence. If I merely believe you to be an intellectual peer -- I have had conversations with you in which you picked up on all of my allusions to books and journals that I've read, and in which you've responded with interesting and thought-provoking allusions of your own, you've presented interesting arguments and picked up on potential weaknesses in arguments of mine, etc. -- without having any reason to suppose that you possess competence regarding a particular subject matter of interest, then that's general competence. If you have evidenced great interest and sustained thought about a particular subject matter, then that's specific competence. Perhaps we can then reformulate what Kvanvig means using something he writes in a comment to the initial post:

(IC) Knowledge that someone equally competent disagrees is not by itself a defeater.

My difficulty now in weighing (GT) against (IC) is that, as Kvanvig notes, (GT), the view that disagreement with an intellectual peer provides one with a defeater, is only plausible if the defeater in question is easily overridden. Let us so understand (GT). Given this understanding, however, Kvanvig's own examples in support of his position are not terribly helpful, as they all involve cases in which we are assuming that one has greater experience in the specific are than one's supposed intellectual peer -- and thus would presumably have grounds to override the defeater.

Couple this fact with the fact that specificity and generality come in a wide variety of flavors -- I could consider someone who also has a Ph.D., regardless of discipline, to be my intellectual peer, or only philosophy Ph.D.'s, or only philosophy Ph.D.'s from certain schools, or in certain subdisciplines, etc. This would also mean that whether one ought to embrace (GT) or (IC) will depend on the level of specificity of intellectual peerage involved in the case under consideration. My sense is that the more specific the intellectual peerage, the more plausible (GT) appears.

Philosophy's a tough field to figure out how disagreement with one's peers leads to defeaters. For the sake of the advancement of knowledge, I'm hoping it's something of an outlier in that respect. Let's use a different example. Suppose two scholars in 17th century German history have a dispute about whether Otto von Guericke performed his famous experiment in which a team of horses attempt to pull apart two metal half-spheres held together by a vacuum. Let's assume neither one is an expert on Saxony-Anhalt, Magdeburg, von Guericke, or the history of science in that period. One believes the experiment was performed with 6 horses, the other with 16. Let's suppose that, prior to their disagreement, each of the scholars would've confidently held their respective beliefs on the basis of what they would've picked up in the course of their extensive readings and conference experiences regarding that period in Germany. What should each do regarding their own belief?

I take it that (IC) would dictate that they may each still hold onto their respective beliefs, while (GT) would require them to suspend judgment until they receive further corroboration for their positions. Ignoring additional difficulties posed by the fact that we aren't taking into consideration what's riding on the accuracy of their beliefs, I have some preference for the outcome required by (GT).

Quick and Dirty Meets Truetemp

Recently, Geoff at Tiger! Tiger! (here) has presented a quick and dirty argument against the KK thesis. I wanted briefly to note that the argument will also work against the assertion that Truetemp-style cases provide evidence against plain-vanilla reliabilism.

Consider Chrysippus's dog. He tracks his prey to a fork in the path at which three branches diverge. He goes a bit down the first and second branches of the path until he realizes that he has lost the scent. Then, without hesitation, he runs down the third branch after his prey.

The quick and dirty argument is as follows:

1. Chrysippus's dog knows that the prey he is chasing has run down the third branch, the only one of the possible paths that the dog hasn't ruled out.
2. Chrysippus's dog doesn't know that he knows that the prey he is chasing has run down the third branch.
3. Thus, knowledge doesn't require knowing that one knows.

Now consider Truetemp. We're supposed to assume that Truetemp has, unbeknownst to him, had a device implanted in his brain that causes him spontaneously to form beliefs about the ambient temperature in his vicinity, and, further, that this device is a highly reliable source of information about the ambient temperature in Truetemp's vicinity. We must also assume that Truetemp has no reason of which he is aware either to accept or reject these spontaneously formed beliefs. (Let's suppose further that Truetemp otherwise wouldn't spontaneously form beliefs about the ambient temperature in his vicinity.) Then, the argument goes, plain-vanilla reliabilism must consider Truetemp's belief that it is 23 C in his vicinity to be an instance of knowledge, while our pretheoretic intuitions would stamp Truetemp's belief an instance of mere true belief.

Here's the catch: if Chrysippus's dog knows that his prey went down the third branch in the path, why doesn't Truetemp know that it is 23 C in his vicinity? My suggestion, obviously, is that there's no good reason to say that Truetemp doesn't know.

So how to explain the widespread intuition that Truetemp doesn't know? (I should note that these intuitions are extremely malleable. I've taught epistemology courses now in different ways, and, depending on how I begin the course -- say, with a discussion of sources, on the one hand, or with a discussion of decision theory and rational belief formation, on the other -- I tend to get different results as to my students' intuitions concerning Truetemp cases.) I still like the suggestion in Goldman's "Epistemic Folkways" article: around here, spontaneously forming beliefs about the precise ambient temperature in one's vicinity is highly unreliable, and this fact creates cognitive "noise" that prevents us from recognizing that the Truetemp example is in fact an instance of knowledge.

Poincare's Conjecture Follow-Up: Polling Intuitions

Suppose that the sort of analogical reasoning upon which one might base the belief that the 3-sphere is unique in its simplicity among three-dimensional manifolds is in fact highly reliable. (Is it the Nyaya school in classical Indian philosophy that considers analogical reasoning to be a basic source of knowledge?) Could one know that the 3-sphere is unique in that way on the basis of such reasoning? If not, how did this jibe with the fact that one can know a mathematical fact on the basis of testimony? (Could a disparity between our intuitions regarding the two cases, assuming there is such a disparity, be used to motivate Burge's idea that testimony preserves a priori warrant?)

Comments most welcome.

Poincare's Conjecture

In what became one of the so-called "Millenium Problems", Henri Poincare conjectured that the 3-sphere is unique in being the simplest 3-dimensional manifold. (More info here.) It is likely that the conjecture has now been proved by the St. Petersburg mathematician Grigori Perelman, who stands to earn $1 million for having solved one of the millenium problems.

Poincare himself proved that the 2-sphere is the simplest manifold in 2 dimensions, respectively. Given that, by 1982, Poincare's Conjecture was proved for n-spheres, where n is greater than or equal to 4, was it at all likely that it would fail to hold for n=3? That is, did we have had some prior justification in believing Poincare's Conjecture to be true, given that we knew that it held for all n other than n=3?

Still More on Knowledge and the Pragmatic

In a recent post (here), Jon Kvanvig seems to lament the recent emphasis of pragmatic features of knowledge in that emphasizing those features leads to (4), which Kvanvig considers an abominable conclusion, via the following argument:

(1) The epistemic goal is to get at the truth and avoid error now.

(2) Knowledge involves pragmatic features.

(3) If the epistemic goal is to get at the truth and avoid error now, and if knowledge involves pragmatic features, then knowledge is not purely epistemic.

Thus, (4) Knowledge is not purely epistemic.

I guess I just don't see the force of this argument.

My problem is with premise (3). Presumably, if the epistemic goal is to get at the truth and avoid error now, then some attribute will be purely epistemic to the extent that it is judged solely with respect to its achievement of that goal. However, attempting to answer the question of what will count as achievement of the goal -- e.g., how much error one should be required to avoid in the pursuit of as many truths as possible -- certainly can't be taken as straying outside of the realm of the epistemic. It'd be helpful to consider an example.

Thus, suppose that we are very bare-bones reliabilists, according to whom knowledge simply is true belief caused by a reliable belief-forming process. Presumably, whatever disagreements he has with this view as a correct account of knowledge, Kvanvig wouldn't want to deny that such a view is compatible with the claim that knowledge is purely epistemic. One holding such a view, however, must answer the question, How reliable must a faculty be in order to be knowledge-yielding?

Once one asks this question, however, one opens the door for pragmatic considerations -- nevertheless, in a way that doesn't run afoul of the intuition that knowledge is purely epistemic, in Kvanvig's sense. For once a project merely involves the application of pragmatic considerations in order to aid in the determination of, e.g., the level of reliability the attainment of which is sufficient to make a faculty a knowledge-yielding faculty, such a project may be seen to be merely in the service of setting the standards for the evaluation of states with respect to their attainment of the goal of achieving truth and avoiding error.

I haven't yet read all of the recent literature advocating a renewed attention to the pragmatic in the analysis of knowledge -- Hawthorne's book is on my night-table! -- but suggestions that pragmatic considerations may be relevant in establishing the standards of evidence sufficient for the attainment of knowledge seem to me to provide the strongest support in favor of recognizing a pragmatic component to knowledge.

Now, referring to David Owens's 2000 book, Reason Without Freedom, Fantl and McGrath write:

In a recent book, David Owens offers the following argument against evidentialism. "[My case against evidentialism] can be made simply by asking: how are you going to tell us, in purely evidential terms, what level of evidence is needed to justify belief? Unless this question can be answered, evidentialism (internalist and externalist) must be abandoned. ... Something in addition to evidence must 'complete' the justification. 'How,' he asks, 'could this something be anything other than the (perceived) needs and interests of the believer?'" (Owens 2000, 26)

Given what I've written up to now, I agree with Owens -- and Fantl and McGrath -- that the question as to how to determine standards for the level of evidence needed to justify belief. Where I part ways with Owens, Fantl, and McGrath is in taking this fact to be antithetical to the thesis in premise (1) above. I just don't see why the fact that pragmatic considerations affect standards of justification is itself antithetical to the idea that knowledge simply involves success in getting at the truth and avoiding error now.

To be fair to Kvanvig, I'm not clear that he would disagree. Thus, in response to a comment by Wayne Riggs, Kvanvig writes:

If salience of the chance of error is semantically tied to knowledge by playing a role in the theory of epistemic justification, and if salience could explain all of the cases where practical significance comes into play (a claim I have not argued for, but would like to be true!), then pragmatic considerations of this sort would only be indirectly connected with knowledge, and what I’m terming the Foley approach to epistemology could be retained.

If I'm right in thinking that my solution to (1) - (4) above is in line with Kvanvig's suggestions, then I'm not sure that I've yet appreciated the extent to which the pragmatic infect knowledge, according to the Owens/ Fantl/ McGrath line.

More on this later.