Scientific Publication Essentials
In examining options for peer review, GG has come to see that clarifying what he thinks is a scientific publication is worth a small digression. Here are the ingredients:
Science: Should be self-evident that a publication has at its core some possible scientific advance supported with observations and/or analysis of existing observations.
Peer review: Let’s break down the elements here.
Peer meaning some other scientists (more than one, please) familiar with the techniques, datasets, reduction approaches, and/or literature relevant to the paper at hand. Not whoever finds a webpage and opens an account so he or she can celebrate or lambast the paper’s conclusions.
Review: Not comments, not ratings, not flame wars, but methodical examination of the paper. Before publication. In private. Because nobody likes to be exposed in public, authors are far more likely to correct mistakes and adopt changes when all understand the manuscript is still a work in progress.
Publication. Not a posting, which is scientific propaganda; a publication. Such have editors who try to make the peer review be fair and appropriate and completed in a reasonable time. Such have organizations that assure that the publication doesn’t vanish when a web server dies or a faculty member retires. Ideally (but too rarely these days) there are also copy and graphics editors to make sure that the paper is clear.
Citable. Meaning a paper reaches a final form and is then left in that form. To build on science done before, you have to know what it is. If we shift to papers that change every time a new comment appears or as a new data point is added, we lose the roadmap for scientific papers. Even retracted papers need some marker in the literature so we can see what banana peel was stepped on. Hey, that work was based on v 2.1 of that paper, but did the v. 3.0 version make it incorrect? Who knows? Imagine reading a paper on the cosmological constant that predated Einstein erasing it from his papers after having deciding that the cosmological constant was a big mistake. You’d have no idea what was going on. Yes, this mean mistakes survive in the literature–but mistakes can have value too. But so do correct ideas sometimes thought to be mistakes. And sometimes bad ideas in one application are good ideas for other applications.
This is not to say there is no value in alternative forms of scientific communication; it is merely to say that such forms should supplant and not replace the core memory of science. Indeed, it could be that alternative forms of communication could lower the burden on publications, making the current problem of getting reviews less challenging. But pulling out one of the core elements listed above will cripple future scientific work.
This isn’t to say the modern system is perfect (it isn’t); it is to say what elements are making a positive contribution. Probably the biggest disagreements would be with publication and maybe peer review. The problem with an absence of publication is that peer review then is either absent or a wide-open mishmash more apt to produce flame wars than real insight. (Note, do not confuse “more apt to” with “must always”). Also, if carried to an extreme (e.g., publishing science as a blog), the science will vanish when the source does. As for peer review, we’ve been there before and so GG will just point at this and this and this…. suffice it to say that the problem is not the ideal but the implementation of peer review.