What lessons can we learn from the current flood of methodological confessions from scientists on Twitter? Most commentaries so far—and there have been a few from historians as well as scientists—assume that the #overlyhonestmethods meme shows that the methods of scientists are not what they are cracked up to be. I agree that the tweets are revealing, but I also think we need to be careful when deciding what they reveal. Here's why.
Reconstruction is not rhetoric
Some people have worried that the tweets will harm public trust in science, since they show that scientific articles misrepresent the research that goes into them. The assumption is that if an article does not describe accurately every step of the author's research, in the order in which they occurred, then there is something fishy going on.
It should be obvious that this assumption is wide of the mark. True, some omissions can be seriously misleading, like the one exemplified in this comic. But many omissions make little difference to what a reasonable and well-informed reader will take from an article. Who cares if “experiment was failing, because my partner was using a bulletbelt?” (@farmperfect) There's nothing fraudulent about omitting an experiment that failed for known reasons—especially when everyone has too much to read already and publishers charge exorbitant rates for journal space.
Moreover, the rhetorical losses of leaving things out are just as great as the rhetorical gains. The more details are published about an experiment, the easier it is to replicate. The more a scientist writes about hypotheses they entertained then soundly refuted, the more convincing she will be. A detailed narrative, vividly portraying the long hours spent at the workbench, might persuade readers that the author is at one with his apparatus. And as Robert Boyle knew, circumstantial details at least generate the appearance of honesty, if not the reality.
In sum, it is well known that scientific articles are carefully crafted reconstructions rather than full descriptions of research carried out. But this may have more to do with the practical pressures than with rhetorical sleight-of-hand.
Update: Jim Grozier makes similar points about the function of scientific papers in the second half of this post.
Error and contingency are not opposed to method
It is said that the tweets challenge our views about scientific method. One reason this is said is that some of the tweets describe failed experiments, fruitless hypotheses, and accidental discoveries. “There was no plan,” wrote @russelgarwood, “we just tried stuff we thought would be interesting until something interesting happened.” And @dr_leigh “did experiment2 because we don't know what the fuck to make of experiment 1.”
There is one view of scientific method that is challenged by confessions like these. It is the view that scientists do research in the same way they bake cakes: by mechanically following a small set of well-defined steps that lead reliably to a desired result.
Francis Bacon's account was a bit like this, but post-Bacon I don't know of any philosophers who have seriously endorsed such a paternilistic view of science. The most ambitious accounts of scientific method give a kind of rough outline of how all research should proceed, leaving the scientist to fill in the gaps as best she can.
For example, most readers will know that Karl Popper advised scientists to dream up a bold conjecture, derive a prediction from the conjecture, then try a new conjecture if the prediction was wrong. This leaves plenty of room for error: one of Popper's mottos was that scientists are people who learn from their mistakes. It also leaves plenty of room for contingency: Popper compared conjectures to the biological mutations that appear more or less at random in evolving organisms.
Consider also a book that has been seen as a kind of prophesy of #overlyhonestmethods, Peter Galison's How Experiments End. It is true that this book portrays the messy, local, and conventional aspects of scientific practice. But this does not stop the author from offering his own account of scientific method. In a way that resembles Bacon, Galison argues that experimenters proceed by “a process of exclusion,” trying to rule out all except one of their explanations of their data. And in a way that recalls another old-fashioned methodologist, William Whewell, he argues that the key to the experimenter's confidence in a result is that it is robust under a variety of conditions [1].
The upshot is that scientific method is compatible with “stuff-ups and serendipity,” as one person described the subjet-matter of the tweets on #overlyhonestmethods.
We can be surprised by good methods as well as bad
I have to admit that many of the tweets look like cases of genuinely bad scientific practice. For example: “For the sake of parsimony we excluded everything that contradicts our hypotheses and only present what supports us.” If all scientists did this all the time then we would have good reason to shut down their departments.
Suppose for a moment that tweets like this one are unembellished accounts of actual events (rather than irony or hyperbole). And suppose also that their authors are a representative sample of all practising scientists. Does this mean that we have finally lifted the lid on actual scientific practice?
No, because for every one of these examples of surprisingly inept method there is another example of surprisingly adept method. Sometimes experiments are left “for the precise time that it took for us to get a cup of tea.” But many other times they are left for a period of time that is carefully measured by some instrument that—though not perfect—is vastly superior to what was used for the same sort of experiment 100 years ago.
Similarly, sometimes experimenters use bluetack to hold their apparatus together; but on other occasions the apparatus is a device of bewildering complexity made up of a zoo of exotic materials. Sometimes experimental results are not replicated; other times the same result holds true in so many circumstances that we've lost count.
This point is so obvious that it is worth asking why we tend to forget it. One reason might be that there is a straw-mannish view that scientists are infinitely competent. This view is vulnerable to examples of less-than-competent practice because those examples can never be counter-balanced. They could only be balanced by examples of more-than-infinitely-competent practice—and good luck finding those.
I say “straw-mannish” because the view that scientists are infinitely competent is often (mostly?) put forward by people who want to show that science is not what it's cracked up to be. Admirers of science are usually not so rash as to say that scientists are infallible or that they have super-human powers. Usually they make the more modest claim that scientists are unusually competent—and sometimes remarkably competent—in their domains of expertise.
Scientists are not the only ones
Hank from AmericanScience has quoted the sociologist of science Michael Lynch on the “tension between rhetoric and reality” that results from air-brushed scientific reports:
Science is perhaps unique as an occupation for the way in which its shop work is made extensively accountable in written reports, and examined for its rational, logical and systematically achieved 'methods' as formulated in those accounts.I agree that scientists are unusual, perhaps unique, in the extent to which their written reports are meant to be rational and logical. However it does not follow that the mismatch between rhetoric and reality is greater in science than in other fields, such as history or literary criticism. True, the rhetoric is less rationalistic in those fields than in science; but perhaps there is a corresponding lack of rigour in the actual practice of those non-scientific fields. This point is frequently ignored in science studies. X will publish a study showing that there is a surprising amount of prejudice/convention/office politics in natural science. Y will infer from the study that the gap between scientific knowledge and everyday knowledge is smaller than we thought. The inference is faulty because we might find, if we looked, that we underestimated the amount of prejudice/convention/office politics in everyday knowledge as well as in scientific knowledge. As a historian I can vouch for the fact that historical articles are air-brushed versions of actual historical practice. Most of what we read in books and archives does not show up in our published articles. The information that does make it is usually rearranged for the sake of intelligibility. And many of our decisions are arbitrary, from the decision to use a particular archive to the decision to publish one quote from a text rather than three. It is no surprise that there is now a hashtag for #overlyhonesthistorians. It might be objected that at least historians are more open about these foibles than scientists are. This is questionable. It is rare to read a published historical article in which the author describes all the twists and turns in his or her research. The confessions on #overlyhonesthistorians are as absent from published historical articles as the confessions on #overlyhonestmethods are from published scientiic articles. Nor are people in science studies any less likely than scientists to wax on about the value of their chosen methods. Steve Fuller makes the point nicely:
Thus, while most STS scholars officially oppose the very idea of a universal theory of science, they are nevertheless inclined to promote a universal method for studying science, be it Bloor's four tenets of the Strong Programme, Harry Collin's eleven propositions on experiment, and Latour's seven rules of method [2].I am not arguing for the suicidal view that the humanities is a methodological slag-heap, or that science studies has said nothing new or interesting about how scientific research is done. My point is just that when it comes to tensions between rhetoric and reality, the humanities are no better—or worse—than the sciences. Most of us in the humanities would add that, despite this tension, our methods and training make us unusually competent in our domain of expertise, whether we are classicists or philosophers or anyone else. We should be equally charitable towards scientists when we read the tweets on #overlyhonestmethods. [1] How Experiments End, esp. pp. 246 and 277. The Whewellian idea I have in mind is that of a “consilience of inductions,” his idea being that we should believe a theory when it can be derived from multiple independent chains of inference. [2] Fuller, Thomas Kuhn: A Philosophical History For Our Times (Chicago: University of Chicago Press), p. 343. Expand post.
Malheureusement je ne traduis plus tous les posts que j'écrit sur ce blog. Si vous voudriez lire les posts qui ont été traduits, veuillez cliquer ici.
Agrandir ce message.
No comments:
Post a Comment