Going Peerless

Seems like every week or two, somebody is complaining about peer review–it is a barrier to scientific communication, it empowers gatekeepers, it destroys careers, etc. Now GG doesn’t buy into that (baby and bathwater territory, in his view), but perhaps as an exercise, what would happen if we outlawed peer review?

So you write up some science and want it shared with other scientists.  Let’s consider your options: email, blog post, paper server, journal.

So to start with, you email all the colleagues in your field with your paper–how does that go? Probably your email looks like spam to a bunch, but maybe you have cultivated connections well and several colleagues read the paper and want to incorporate its results in their work. How shall they cite it? Should they include your article with theirs, a kind of blockchain sort of thing? Hmmm, this doesn’t sound promising…

So you post your paper on a blog, kind of like this one.  Hey, now you have a URL and can list an access date, and even if your blog gets canned, folks can use the Wayback machine, right? Provided you are careful that your blog host doesn’t cripple web crawlers…So your promotion emails are a lot shorter, just maybe giving an abstract  and a URL. So your colleagues have something to cite now. But how about folks who weren’t on the email list–how do they find it? Yes, there is Google (or DuckDuckGo for those not wanting to feed the maw of Big Data Run Amok), but do you really want to trust them to point the researchers in your field in your direction? Just how good are those tools for turning up professional materials at ad hoc locations?

So maybe you engage in some mutual back-scratching.  I’ll have a list of interesting papers on my site that will include your papers if you include mine.  That way even if there isn’t a direct citation, you’ve embedded yourself in a network of like-minded researchers. And maybe you go a bit farther and join an actual group of mutual referrers…until you find that your site is now pointing at some really awful non-science that somebody in your group thought was cool. So maybe you have to indulge in pruning your pointers, or organizing new groups or…what were we here to do again? Because this doesn’t sound like fun.

So, you say, that maybe kind of worked, but can’t we just put papers somewhere where our colleagues will know to look for them?  A subject-specific online archive? Why of course you can! Now your paper proudly sits with the work of Nobel laureates, future laureates, esteemed colleagues, postdocs, graduate students, honors undergraduate students…home-schooled high schoolers…cranks and wannabes…predatory meetings and workshops.  Well, that’s OK, at least the search engine knows about science and the server is supported by a standing government grant…until, you know, somebody decides a different concern should run the server (maybe Elsevier!) and your server is mothballed–materials still available but the speed and accuracy of the search engine decline. Or maybe you find it hard to locate the science you want–the search terms you try miss some new papers that didn’t use quite the same terms and you post a paper that missed some other work that made your interpretations–or even your data–incorrect or irrelevant. And people notice.  Not good.

So you’d like a trusted source that can point you to the important and relevant work you need to see.  Maybe a colleague who reads a lot of stuff and can point you in a good direction. In fact, she does this so well that after awhile she has her own page of “recommended readings.” This is popular enough that people start to send her pointers to their papers in the hopes of making her list, and so she adds the ones she thinks are worthy.  But increasingly she finds herself looking at papers that seem interesting…but a technique being used, or a dataset, is unfamiliar.  Is this something good, or is this an incorrect application of some statistical test or lab procedure or piece of equipment? So she asks a colleague who knows more about that stuff to give it a look-see….

…and then she gets the cease-and-desist order from the Peerless Police, for she just broke the law against peer review. Just like the one received by all the folks who wanted to send their pre-posting paper to her to look over and spot any problems before a public posting.

What really is a journal but an editor giving advice and picking gems from trash? Yes, better journals have people who will copyedit a manuscript and make sure the graphics will render on most computers. But the soul of the thing is that somebody has made judgements that carry some value. Wheat has been separated from chaff.

Peer review and its companion, professional journals, are far from perfect. But are the alternatives that much better?