Archive | academia RSS for this section

What’d we do wrong?

One of the typical surveys run by Pew Research Center is one asking about the impact of different institutions on America. Not surprisingly, there are differences between Republicans and Democrats in views on things like the media, churches and labor unions. But the latest iteration of this survey had a bit of a surprise: the partisan divide suddenly yawned open and swallowed higher education. Republicans have suddenly turned against higher education in the past two years, making the partisan divide on education (at 36%) greater than any other institution, including the much-maligned media.

1_3

In other words, seven years ago higher ed was thought to have a positive impact by 58% of Republicans and 65% of Democrats, and while that slowly diverged in the following 5 years, the big change was over the 2016 election cycle.

The news stories out of this suggest that this is a backlash against higher ed because of high tuition and debt or views that they are liberal strongholds. But really? All of that has been going on a long time.

There are a couple of possibilities here.  One is that the members of the 2015 GOP who liked colleges were so turned off by the Trump campaign that they aren’t identifying that way any more, while the GOP attracted less well educated members of the Democratic party. After all, one of the great divides in the presidential vote in 2016 was on education. But then you might expect a sharp rise in the favorability of college among Democrats, and that number barely moved.

Another notion in the media is that colleges got dinged for making headlines about intolerance directed at right-wing speakers. But most of that postdated the election and the sudden decline was far earlier. Although accusations that college students were “snowflakes” have certainly increased, college free speech has long been viewed as questionable in conservative eyes.

No, it seems it was something a lot more specific, and GG would like to suggest it was, ironically, the arguments within the Democratic party about making college free.

Before you argue that such a program might be beneficial for a lot of Trump supporters, keep in mind that many of them oppose other government programs that might help them.  Their objection, as often as not, is that you shouldn’t be a “taker.” Getting a free ride through college probably made college less of something you do for self-improvement and more of an entitlement.

Whatever the cause, this is not good for public institutions.  If bashing colleges is in vogue, tuitions will rise where GOP candidates are successful–and ironically, they will rise most at schools that right now are most affordable. Is it really in the national best interest to make college even more of an elitist institution?

Dear Elsevier Editors…

…this is what you will get from the Grumpy Geophysicist if asking for a review for your journals:

Sorry, I am part of the anti-Elsevier resistance. Elsevier’s practices in both their extortive pricing and their inflexible and aggressive assertion of their ownership of anything carrying their copyright (as all <journal name here> articles do) prevents me from participating in the extreme monetization of scientific work as practiced by Elsevier.

While GG does not object to scientific societies holding copyright (as they are generally protecting the content from exploitation rather than for monetary gain, and their policies are subject to the wishes of their membership), he does object to copyright ownership being in the hands of a for-profit (and very profitable) company. We discussed a lot of this recently here.

GG feels bad in saying stuff like this, because the person asking is somebody essentially volunteering their time to help publish the journal. Making their job harder isn’t really pleasant. But, well, maybe they hadn’t appreciated how editing an Elsevier journal might be different than a society-run one. And maybe this might help them question their participation.

Or, it just angers a colleague with GG.  *Sigh*.

Don’t be a rube

An interesting article in The Guardian on the rise of the profit-oriented part of scientific publishing. One part of the article describes how companies like Elsevier and Pergamon make so much money: “It is as if the New Yorker or the Economist demanded that journalists write and edit each other’s work for free, and asked the government to foot the bill.” How much money? Try revenue of $24 billion. Elsevier’s profit margin: 36%.

Now some scientists have argued that journals are outdated and provide no added value; GG has argued this isn’t true. But with the existence of non-profit publishers, does it make sense to feed these very profitable monsters?

Well, no. Worse, many scientists don’t seem to understand that their science is no longer theirs once it is in one of those journals.

Some of us have sworn off of Elsevier journals, not reviewing for them or publishing in them (though we sometimes get dragged in by colleagues). That is walking away from a lot of poor journals and a few really good ones. In the days of paper journals, this was a clear choice. Even now, Elsevier’s tactics even for open-access have driven some away. But close examination of what societies are doing suggests that avoiding vendors many view as unscrupulous is getting harder and harder.

Read More…

When you have a hammer…

…all the world is a nail.  And the currently popular hammers are things like Twitter and Instagram and Tinder.  While some have long advocated the first two as important tools for scientists, the last has been used as a model for scanning through preprints.  Lots and lots of preprints. The Science story on this says “A web application inspired by the dating app Tinder lets you make snap judgments about preprints—papers published online before peer review—simply by swiping left, right, up, or down.”

Nothing says “science” like “snap judgment”.

While GG lambasted an effort to capture social media-ish solutions as a means of post-publication peer review, how about tools to let you find what cutting edge science is appearing? That Science report on social media linked above says that is what social media is good for.  Um, really?

GG studies the Sierra Nevada.  Try going to Twitter and searching on #SierraNevada.  Bet you didn’t think there were that many people so fascinated with taking pictures of beer bottles. Add, say, #science. Chaff winnowed some, but very little wheat. Add #tectonics. Crickets.

The idea of this new app (Papr) is that if only you were able to see lots and lots of stuff quickly, you’d find some gems to explore. Really?  Students complain bitterly about a firehose approach in the classroom, and the solution here is, um, a firehose? (To be fair, it appears the app developers are not necessarily expecting great things here).

Forget that.  What we want and need are tools to reduce chaff, not accelerate it.

What we need is something akin to Amazon’s suggestions tool.  Imagine visiting the preprint store to get a couple of papers you know you want.  One maybe is on a topic you care about–say, the Sierra Nevada.  Another maybe deals with a technique, say full waveform tomography.  A third uses some unusual statistical tests. You download these and the preprint store suggests a few other preprints based on the full text content of the papers you got. Why that instead of keywords? Keywords have a way of being too picky. You might call work “tectonics” and GG might call it “geodynamics” and thus the keywords searches might pass by each other. But if the text is still talking about changes in elevation, changes in lithospheric structure–those are less likely to get overlooked.  If this tool is smart enough to recognize quasi-synonyms and phrases, all the better.

Such a tool grows more powerful the more you work with it. While on that first try, you will also get recommendations on papers overlapping in non-interesting ways (say, applications of the techniques in paper 1, the geographic area under study in paper 2, and the measurement types in paper 3), the more you interact with this, the better it gets.

Here’s the sad thing: the tools to make something like this have been around for decades.  The best spam filters (like SpamSieve) use a form of Bayesian filtering based on message content in addition to black- and whitelists. Earth science got much of its literature into a single “preprint store” long ago in GeoScienceWorld. And yet here we are, swiping left again and again and again….

Citation Statistics Smackdown

Sorry, it isn’t that dramatic.  But in updating various web tools, GG noticed dramatic differences between his supposed citations between Google Scholar and Web of Science. In the past he has assumed the difference was because Google was capturing junk citations, but today decided to actually look at what is going on in detail.  Which may or may not interest you, dear reader….

The raw starting points for Web of Science is here, and for Google is here. At the very top, GG’s h index is 21 with Web of Science, 27 with Google (a significant difference for those who love those things, just a numerical quirk for others). The most highly cited paper has  252 citations from WoS but a staggering 338 in Google. Although this is tedious to work through, there is clearly a lot of fodder for comparison, so let’s dive in.

An oddity of Google’s citation listing comes into focus quickly: sorting on date only yields the last 15 papers.

Google overestimates citations in at least one situation: it repeated the citation to papers in the Chinese Journal of Geophysics, linking to both the English language version and the original Chinese html version of the papers. Another goofy thing is the Google will mess up from time to time and assign a citation from a previous paper in Nature with the article that starts on the same page as the citation. For instance, Google has an immunology paper citing the Zandt et al. tectonics paper. Google does end up with some number of duplicated citations: several preprints are counted along with the actual publication. Also some Chinese and possibly Russian papers are counted twice, once as Chinese versions, once in English versions.

Mostly, however, the difference is in theses and books, items Web of Science explicitly does not track. Since some theses contain papers published elsewhere, some of these are duplicates. More embarrassingly, there are some term papers on the web that are taken as citable materials.

What is the balance, though?

Of the 331 references identified overall, only 5 in Web of Science were not in Google.  Two were chapters in the Treatise on Geochemistry, two others were in GSA Special Paper 456, and the last was a G^3 article. So of the remaining 326, 247 were in WoS and so 79 more are in Google. Since 338-326=12, there are 12 outright duplicate entries in Google; what of the 79 other additional entries?

Five did not cites the Zandt et al. paper at all; these were outright mistakes.  Combined with the 12 duplicate entries, 17 of the 338, or about 5%, of the Google citations are simply wrong. The duplicates are sometimes multiple language versions of the same paper, or a preprint showing up as a separate item.

  • Theses: 28
  • Books: 16 (including 8 from GSA Memoir 212, which WoS should have had)
  • Foreign language (Chinese and Russian): 12 (Some of which might be duplicates or not even cite the paper at all)
  • “News” Journals (GSA Today, Eos): 6
  • Real journals missed by WoS: 6 (which, if you add the 8 from GSA Memoir 212, are 14 references that WoS should have had).
  • Miscellaneous: 6. A term paper was in there, a meeting abstract, an in press paper.

Which do you take to be more accurate? The 252 in WoS should clearly be at least 258 and probably over 260 with the GSA volumes that are supposed to be counted these days.  The 6 GSA Today+EOS science articles probably deserve inclusion, though the EOS articles are shakier. On the other side, the 338 reported by Google should be no higher than 320 (338 – 17 – 6 + 5). Theses are something interesting in this count, as they represent some kind of original research, but these days most thesis work worth anything is published.  If you take that view we are down to 292, 26 above the 266 WoS probably should have had.

This leaves as seriously gray at least 8 books, 12 foreign language papers, and the 6 news journals. So arguably the uncertainty on a citation count is in the 10-20% range.  If we say the correct number is 279 +/-13, the 252 of WoS is 27 low and Google is 59 high.

What does this mean, aside from apparently we can’t even count integers? Perhaps a first-cut approach would be to take as a closer approximation to a “true” measure of citations by going a third of the way from WoS to Google numbers (true = WoS + (Google-WoS)/3, or true = 2/3(WoS) + 1/3(Google)).

Scientific Publication Essentials

In examining options for peer review, GG has come to see that clarifying what he thinks is a scientific publication is worth a small digression.  Here are the ingredients:

Science: Should be self-evident that a publication has at its core some possible scientific advance supported with observations and/or analysis of existing observations.

Peer review: Let’s break down the elements here.

Peer meaning some other scientists (more than one, please) familiar with the techniques, datasets, reduction approaches, and/or literature relevant to the paper at hand. Not whoever finds a webpage and opens an account so he or she can celebrate or lambast the paper’s conclusions.

Review: Not comments, not ratings, not flame wars, but methodical examination of the paper. Before publication.  In private. Because nobody likes to be exposed in public, authors are far more likely to correct mistakes and adopt changes when all understand the manuscript is still a work in progress.

Publication. Not a posting, which is scientific propaganda; a publication.  Such have editors who try to make the peer review be fair and appropriate and completed in a reasonable time.  Such have organizations that assure that the publication doesn’t vanish when a web server dies or a faculty member retires. Ideally (but too rarely these days) there are also copy and graphics editors to make sure that the paper is clear.

Citable. Meaning a paper reaches a final form and is then left in that form. To build on science done before, you have to know what it is.  If we shift to papers that change every time a new comment appears or as a new data point is added, we lose the roadmap for scientific papers.  Even retracted papers need some marker in the literature so we can see what banana peel was stepped on. Hey, that work was based on v 2.1 of that paper, but did the v. 3.0 version make it incorrect?  Who knows? Imagine reading a paper on the cosmological constant that predated Einstein erasing it from his papers after having deciding that the cosmological constant was a big mistake. You’d have no idea what was going on. Yes, this mean mistakes survive in the literature–but mistakes can have value too. But so do correct ideas sometimes thought to be mistakes. And sometimes bad ideas in one application are good ideas for other applications.

This is not to say there is no value in alternative forms of scientific communication; it is merely to say that such forms should supplant and not replace the core memory of science. Indeed, it could be that alternative forms of communication could lower the burden on publications, making the current problem of getting reviews less challenging. But pulling out one of the core elements listed above will cripple future scientific work.

This isn’t to say the modern system is perfect (it isn’t); it is to say what elements are making a positive contribution. Probably the biggest disagreements would be with publication and maybe peer review. The problem with an absence of publication is that peer review then is either absent or a wide-open mishmash more apt to produce  flame wars than real insight. (Note, do not confuse “more apt to” with “must always”). Also, if carried to an extreme (e.g., publishing science as a blog), the science will vanish when the source does. As for peer review, we’ve been there before and so  GG will just point at this and this and this…. suffice it to say that the problem is not the ideal but the implementation of peer review.

Peers Review

GG has been skeptical of many suggestions about peer review over the years, things like post-publication review, public peer review, publishing via blogs, or outright elimination of peer review. But the latest wrinkle might bear some thought.

Editors for the journal Synlett decided to try something a bit different, something they describe as intelligent crowd review.  In essence, this is creating a forum populated by some range of experts (they recruited 100) and then tossing a submission into the forum and letting the experts do what they will.  Overall they got results faster and with greater insight than traditional peer review.

Why is this?  As the Ars Technica article on this suggests, in this environment, reviewers can just focus on what they know backwards and forwards.  Yeah, that is a fair introduction, or, no, that equation is inappropriate. So you dive in (maybe as lunchtime entertainment), shoot all the fish in your barrel and leave the rest of the manuscript for others.

The biggest advantage of this proposal is in reviewing complex multi-disciplinary papers where a reviewer either has to say “I can’t review this part of the paper because it is too far out of my expertise” or has to bone up on material he or she is unfamiliar with. Either of these tends to slow the review process down. Given the increased emphasis and visibility of such research, embracing such an approach might be a boon to editors and authors alike.

Of course there are problems that would have to be solved. Avoiding conflicts of interest could get challenging; this might get harder too if an author specifically requests certain individuals not be granted access to a submission. The system apparently requires the editor to assemble the resulting crowd review, which could in some cases require the editor to fill in gaps. Whether such a system would breed a new kind of burnout remains to be seen.

But this might be one of the better hopes for getting out of the peer-review rut we are presently in.  It is certainly worth careful consideration.