Archive | science RSS for this section

Don’t be a rube

An interesting article in The Guardian on the rise of the profit-oriented part of scientific publishing. One part of the article describes how companies like Elsevier and Pergamon make so much money: “It is as if the New Yorker or the Economist demanded that journalists write and edit each other’s work for free, and asked the government to foot the bill.” How much money? Try revenue of $24 billion. Elsevier’s profit margin: 36%.

Now some scientists have argued that journals are outdated and provide no added value; GG has argued this isn’t true. But with the existence of non-profit publishers, does it make sense to feed these very profitable monsters?

Well, no. Worse, many scientists don’t seem to understand that their science is no longer theirs once it is in one of those journals.

Some of us have sworn off of Elsevier journals, not reviewing for them or publishing in them (though we sometimes get dragged in by colleagues). That is walking away from a lot of poor journals and a few really good ones. In the days of paper journals, this was a clear choice. Even now, Elsevier’s tactics even for open-access have driven some away. But close examination of what societies are doing suggests that avoiding vendors many view as unscrupulous is getting harder and harder.

Read More…

When you have a hammer…

…all the world is a nail.  And the currently popular hammers are things like Twitter and Instagram and Tinder.  While some have long advocated the first two as important tools for scientists, the last has been used as a model for scanning through preprints.  Lots and lots of preprints. The Science story on this says “A web application inspired by the dating app Tinder lets you make snap judgments about preprints—papers published online before peer review—simply by swiping left, right, up, or down.”

Nothing says “science” like “snap judgment”.

While GG lambasted an effort to capture social media-ish solutions as a means of post-publication peer review, how about tools to let you find what cutting edge science is appearing? That Science report on social media linked above says that is what social media is good for.  Um, really?

GG studies the Sierra Nevada.  Try going to Twitter and searching on #SierraNevada.  Bet you didn’t think there were that many people so fascinated with taking pictures of beer bottles. Add, say, #science. Chaff winnowed some, but very little wheat. Add #tectonics. Crickets.

The idea of this new app (Papr) is that if only you were able to see lots and lots of stuff quickly, you’d find some gems to explore. Really?  Students complain bitterly about a firehose approach in the classroom, and the solution here is, um, a firehose? (To be fair, it appears the app developers are not necessarily expecting great things here).

Forget that.  What we want and need are tools to reduce chaff, not accelerate it.

What we need is something akin to Amazon’s suggestions tool.  Imagine visiting the preprint store to get a couple of papers you know you want.  One maybe is on a topic you care about–say, the Sierra Nevada.  Another maybe deals with a technique, say full waveform tomography.  A third uses some unusual statistical tests. You download these and the preprint store suggests a few other preprints based on the full text content of the papers you got. Why that instead of keywords? Keywords have a way of being too picky. You might call work “tectonics” and GG might call it “geodynamics” and thus the keywords searches might pass by each other. But if the text is still talking about changes in elevation, changes in lithospheric structure–those are less likely to get overlooked.  If this tool is smart enough to recognize quasi-synonyms and phrases, all the better.

Such a tool grows more powerful the more you work with it. While on that first try, you will also get recommendations on papers overlapping in non-interesting ways (say, applications of the techniques in paper 1, the geographic area under study in paper 2, and the measurement types in paper 3), the more you interact with this, the better it gets.

Here’s the sad thing: the tools to make something like this have been around for decades.  The best spam filters (like SpamSieve) use a form of Bayesian filtering based on message content in addition to black- and whitelists. Earth science got much of its literature into a single “preprint store” long ago in GeoScienceWorld. And yet here we are, swiping left again and again and again….

Citation Statistics Smackdown

Sorry, it isn’t that dramatic.  But in updating various web tools, GG noticed dramatic differences between his supposed citations between Google Scholar and Web of Science. In the past he has assumed the difference was because Google was capturing junk citations, but today decided to actually look at what is going on in detail.  Which may or may not interest you, dear reader….

The raw starting points for Web of Science is here, and for Google is here. At the very top, GG’s h index is 21 with Web of Science, 27 with Google (a significant difference for those who love those things, just a numerical quirk for others). The most highly cited paper has  252 citations from WoS but a staggering 338 in Google. Although this is tedious to work through, there is clearly a lot of fodder for comparison, so let’s dive in.

An oddity of Google’s citation listing comes into focus quickly: sorting on date only yields the last 15 papers.

Google overestimates citations in at least one situation: it repeated the citation to papers in the Chinese Journal of Geophysics, linking to both the English language version and the original Chinese html version of the papers. Another goofy thing is the Google will mess up from time to time and assign a citation from a previous paper in Nature with the article that starts on the same page as the citation. For instance, Google has an immunology paper citing the Zandt et al. tectonics paper. Google does end up with some number of duplicated citations: several preprints are counted along with the actual publication. Also some Chinese and possibly Russian papers are counted twice, once as Chinese versions, once in English versions.

Mostly, however, the difference is in theses and books, items Web of Science explicitly does not track. Since some theses contain papers published elsewhere, some of these are duplicates. More embarrassingly, there are some term papers on the web that are taken as citable materials.

What is the balance, though?

Of the 331 references identified overall, only 5 in Web of Science were not in Google.  Two were chapters in the Treatise on Geochemistry, two others were in GSA Special Paper 456, and the last was a G^3 article. So of the remaining 326, 247 were in WoS and so 79 more are in Google. Since 338-326=12, there are 12 outright duplicate entries in Google; what of the 79 other additional entries?

Five did not cites the Zandt et al. paper at all; these were outright mistakes.  Combined with the 12 duplicate entries, 17 of the 338, or about 5%, of the Google citations are simply wrong. The duplicates are sometimes multiple language versions of the same paper, or a preprint showing up as a separate item.

  • Theses: 28
  • Books: 16 (including 8 from GSA Memoir 212, which WoS should have had)
  • Foreign language (Chinese and Russian): 12 (Some of which might be duplicates or not even cite the paper at all)
  • “News” Journals (GSA Today, Eos): 6
  • Real journals missed by WoS: 6 (which, if you add the 8 from GSA Memoir 212, are 14 references that WoS should have had).
  • Miscellaneous: 6. A term paper was in there, a meeting abstract, an in press paper.

Which do you take to be more accurate? The 252 in WoS should clearly be at least 258 and probably over 260 with the GSA volumes that are supposed to be counted these days.  The 6 GSA Today+EOS science articles probably deserve inclusion, though the EOS articles are shakier. On the other side, the 338 reported by Google should be no higher than 320 (338 – 17 – 6 + 5). Theses are something interesting in this count, as they represent some kind of original research, but these days most thesis work worth anything is published.  If you take that view we are down to 292, 26 above the 266 WoS probably should have had.

This leaves as seriously gray at least 8 books, 12 foreign language papers, and the 6 news journals. So arguably the uncertainty on a citation count is in the 10-20% range.  If we say the correct number is 279 +/-13, the 252 of WoS is 27 low and Google is 59 high.

What does this mean, aside from apparently we can’t even count integers? Perhaps a first-cut approach would be to take as a closer approximation to a “true” measure of citations by going a third of the way from WoS to Google numbers (true = WoS + (Google-WoS)/3, or true = 2/3(WoS) + 1/3(Google)).

Scientific Publication Essentials

In examining options for peer review, GG has come to see that clarifying what he thinks is a scientific publication is worth a small digression.  Here are the ingredients:

Science: Should be self-evident that a publication has at its core some possible scientific advance supported with observations and/or analysis of existing observations.

Peer review: Let’s break down the elements here.

Peer meaning some other scientists (more than one, please) familiar with the techniques, datasets, reduction approaches, and/or literature relevant to the paper at hand. Not whoever finds a webpage and opens an account so he or she can celebrate or lambast the paper’s conclusions.

Review: Not comments, not ratings, not flame wars, but methodical examination of the paper. Before publication.  In private. Because nobody likes to be exposed in public, authors are far more likely to correct mistakes and adopt changes when all understand the manuscript is still a work in progress.

Publication. Not a posting, which is scientific propaganda; a publication.  Such have editors who try to make the peer review be fair and appropriate and completed in a reasonable time.  Such have organizations that assure that the publication doesn’t vanish when a web server dies or a faculty member retires. Ideally (but too rarely these days) there are also copy and graphics editors to make sure that the paper is clear.

Citable. Meaning a paper reaches a final form and is then left in that form. To build on science done before, you have to know what it is.  If we shift to papers that change every time a new comment appears or as a new data point is added, we lose the roadmap for scientific papers.  Even retracted papers need some marker in the literature so we can see what banana peel was stepped on. Hey, that work was based on v 2.1 of that paper, but did the v. 3.0 version make it incorrect?  Who knows? Imagine reading a paper on the cosmological constant that predated Einstein erasing it from his papers after having deciding that the cosmological constant was a big mistake. You’d have no idea what was going on. Yes, this mean mistakes survive in the literature–but mistakes can have value too. But so do correct ideas sometimes thought to be mistakes. And sometimes bad ideas in one application are good ideas for other applications.

This is not to say there is no value in alternative forms of scientific communication; it is merely to say that such forms should supplant and not replace the core memory of science. Indeed, it could be that alternative forms of communication could lower the burden on publications, making the current problem of getting reviews less challenging. But pulling out one of the core elements listed above will cripple future scientific work.

This isn’t to say the modern system is perfect (it isn’t); it is to say what elements are making a positive contribution. Probably the biggest disagreements would be with publication and maybe peer review. The problem with an absence of publication is that peer review then is either absent or a wide-open mishmash more apt to produce  flame wars than real insight. (Note, do not confuse “more apt to” with “must always”). Also, if carried to an extreme (e.g., publishing science as a blog), the science will vanish when the source does. As for peer review, we’ve been there before and so  GG will just point at this and this and this…. suffice it to say that the problem is not the ideal but the implementation of peer review.

Peers Review

GG has been skeptical of many suggestions about peer review over the years, things like post-publication review, public peer review, publishing via blogs, or outright elimination of peer review. But the latest wrinkle might bear some thought.

Editors for the journal Synlett decided to try something a bit different, something they describe as intelligent crowd review.  In essence, this is creating a forum populated by some range of experts (they recruited 100) and then tossing a submission into the forum and letting the experts do what they will.  Overall they got results faster and with greater insight than traditional peer review.

Why is this?  As the Ars Technica article on this suggests, in this environment, reviewers can just focus on what they know backwards and forwards.  Yeah, that is a fair introduction, or, no, that equation is inappropriate. So you dive in (maybe as lunchtime entertainment), shoot all the fish in your barrel and leave the rest of the manuscript for others.

The biggest advantage of this proposal is in reviewing complex multi-disciplinary papers where a reviewer either has to say “I can’t review this part of the paper because it is too far out of my expertise” or has to bone up on material he or she is unfamiliar with. Either of these tends to slow the review process down. Given the increased emphasis and visibility of such research, embracing such an approach might be a boon to editors and authors alike.

Of course there are problems that would have to be solved. Avoiding conflicts of interest could get challenging; this might get harder too if an author specifically requests certain individuals not be granted access to a submission. The system apparently requires the editor to assemble the resulting crowd review, which could in some cases require the editor to fill in gaps. Whether such a system would breed a new kind of burnout remains to be seen.

But this might be one of the better hopes for getting out of the peer-review rut we are presently in.  It is certainly worth careful consideration.

The Double Edged Sword of Science

Well, it is that time of year when we send off freshly minted graduates off into the real world.  They have sat through speeches imploring them to go out and make the world a better place from their elders and others reminiscing on their times in college before marching to a podium, getting a piece of paper, and discovering that the alumni association is really interested in them.

While the speeches heard quickly fade from memory, GG would like to take a stab at some advice for science graduates….without the need to sit under a hot sun wearing a giant trash bag and the most ill-fitting and unflattering hat on earth.

Congratulations New Scientists! You have completed a degree program viewed as Important by Important People like politicians (very few of whom have completed such a degree) and placement officers (ditto) and your professors (who generally do have such degrees). So you must have done something significant.

Why might this be so significant?  It is because you are now armed with a powerful weapon, a sword of science, if you will.  With this, you can cut through bias to find truth, you can drop superstition in its tracks, drive rumor into retreat and determine how the world really works.  You have encountered and hopefully mastered a mode of thinking that helps you to penetrate thickets of ignorance.

Others will defer to your better judgement because you wield this weapon. Some of you will even command comfortable salaries. Having passed through the travails of an academic program in science, you may find the way forward easier for having suffered to this point.

But don’t pat yourself on the back just yet–remember you are holding that sword of science. It might hurt.

Read More…

A different view of science

Today is Earth Day, and many folks will be out marching for science. There are lots of reasons one might do this–provides the grist for technological advance, allows for finding ways of living longer and better, etc–but GG would like to explore a different tack.

GG was recently in Ireland, and at geologic sites like Giant’s Causeway and the Cliffs of Moher he looked for books on geology in the visitor center stores.  There was one rather generic book on deep time at Giant’s Causeway (with a cover shot of the Grand Canyon) and nothing at the Cliffs of Moher.  There were, on the other hand, fistfuls of books on Irish history, most of which dealt with events nowhere near the places where the books were being sold.  This is hardly unique to these places, but it is sobering.  Here are landscapes–geological landscapes–that people have travelled some distance to see, and there isn’t enough interest for there to be even one book specific to the place?

The point? People seem to gravitate to history books but avoid science books.  Science books, it would seem, are too heavy for casual reading. Many of the marchers today may feel science is important, but you wonder if they would pick up a science book while on vacation. Now perhaps this reflects a certain inability of those writing science to author accessible books (but there are some really excellent science writers out there), but it might also mean that folks feel that science is best left to scientists.  Just give me my smartphone, thanks, and make sure the plane gives me a smooth ride home and all is good.

Science can be more though than just the mechanism by which we find our way to new and better gadgets. It can also provide the same kind of insights into the human experience that we seek in art and literature. But by limiting science to an academic pursuit, the public misses out on this facet of the scientific enterprise.

This is why GG wrote The Mountains that Remade America. The book fuses human history with geological history, so in some ways it is a bit deceptive–here is some nice tasty history, and oh by the ways, there is some geology here too–try it, just a smidgen, and you’ll get a bit more yummy history. Who we are, why we do things some ways, how we live in others is rooted in the geology of a region, but this aspect of history is buried in most of those history books cluttering visitor center shelves.  Hundreds of books address the California Gold Rush.  Few if any consider how the gold got to be there–or that the presence of different kinds of gold deposits dictated how the rush would affect later history. Or consider how John Muir’s idolization of empty landscapes might have been different had he been tramping in the Rockies with Indian settlements all around instead of the High Sierra where occupation was purely seasonal.

So today, as many trumpet how important science is as a practical matter, don’t forget that science is a human endeavor, and there is real gratification in learning the origin stories that make us what we are.