Why publish in a journal if you can disseminate online?

A new paper by Glenn Ellison documents something that I have suspected for some time. Economists in top departments are publishing less in top field journals. In addition, economists at Harvard's department of economics--which Ellison and many other consider to be the top department in the country at the moment--are publishing fewer papers in the top general interest journals. (The Quarterly Journal of Economics, which is sort of a house journal at Harvard, seems to be the exception.) The explanation?

Several pieces of evidence bolster the view that one factor contributing to these trends is that the role of journals in disseminating research has been reduced. One is that the citation benefit to publishing in a top general-interest journal now appears to be fairly small for top-department authors. Another is that Harvard authors appear to be quite successful in garnering citations to papers that are not published in top journals. The fact that the publication declines appear to be a top-department phenomenon (as opposed to a prolific-author phenomenon) suggests that a top-department affiliation may be an important determinant of an author’s ability to sidestep the traditional journal system.

This concerns me as an editor of REStat, but it is fairly consistent with my own experience.

Comments

[talboito]

Path dependency seems the likliest culprit.

I imagine journals will be in a greatly altered state within the next 5 years.

Perhaps they will be something like PLOS or the whole system will decentralize. Academics could start paying into some refereeing pool that would bless the paper for "journal" status before it is posted to the individual's website.

[Eric de Souza]

NBER requires a subscription, but the paper can be downloaded from Ellison's own website:
http://econ-www.mit.edu/faculty/index.htm?prof_id=gellison&type=paper

[Howard]

As editor of REStat, it concerns you. As an academic and an economist, does it seem better or worse that information is more freely accessible?

Certainly, there is a lot of value to the peer review process but there is also a lot of value to being able to get ideas out without the publishing latency and without the politics of peer review.

I suspect one of the bigger downsides is that it helps to be someone known to be found online and achieve credibility. Peer review and publication puts a stamp of approval of peers on a document and can help elevate the author.

[robertdfeinman]

Let's be honest, a great deal of scholarly publishing is done to secure tenure and/or promotion.

Once one has gotten tenure the need to publish in peer-reviewed journals declines. In fact we all know of many tenured professors (in all disciplines) who haven't published anything in decades.

So for the study to be meaningful one would need to separate out those with tenure from those without and see if the trend is the same for both groups.

A second fact is the attitude of tenure/promotion review committees. If they will treat informal publishing as they do peer-reviewed then there is less motivation to go through the hassle of journal publishing.

I imagine this attitude change also depends upon the school. The less prestigious the school the more they want to see their faculty appearing in prominent journals.

In economics there seems to be a special added factor. The large number of think-tanks which do their own publishing has blurred the lines between peer-reviewed and in-house publishing. This has made the acceptability of this avenue more likely than in other disciplines.

[jonfernquest]

"Why publish in a journal if you can disseminate online?"

Why expend energy on something that's going to gather dust on library shelves and only be available to an elite few, when you can disseminate to everyone who might benefit from your wisdom at near zero cost?

[Barkley Rosser]

Well, the issue is not do one or the other. Increasingly, most people put their papers up somewhere online. I do. They start getting cited or they do not. To the extent citations count more than pubs, who needs pubs?

I am writing as a journal editor (JEBO). I have seen papers I rejected because of comments by referees, go on to be cited in papers that I receive for submission quite copiously, making me feel either silly or maybe these people do not know what they are citing.

Also, as more senior type person, there has been an increase in the demands for endless revising. I can see many people, especially those who are getting cited for the paper anyway, say "why bother wasting time with this revision, especially if I think what they are asking is stupid." I happen to have two papers right now in a revise and resub mode, in which I am looking at exactly this question, with one of these a paper that is already getting cited. Why waste time when one would like to move on to the next project, new idea?

Clearly, of course, for junior faculty, especially those who are not superstar geniuses at Harvard with unpubbed papers getting cited like mad, they must pub, and preferably pub in more prestigious journals, in order to get tenure and promotion. And even after one makes full prof, or even gets that nice chair to sit in, further merit pay increases depend on something, although by then, one can often make the case more fully with citations than with actual pubs.

Which may lead us back to why pubbing in journals may not fully disappear, at least not for some time. The measure of citations that counts is things like ISI or SCOPUS, and these only count cites in pubbed journal articles, and only in journals that they count. So, one may be getting cited in all kinds of working papers, but one will have trouble convincing that Chair of the Harvard dept. that one is getting cited a lot if one is not actually showing up as getting cited in pubs that show up in these official sources.

Maybe someday someone will be officially counting all cites in all papers. But we are some ways away from that, for better or worse. So, Dani, I don't think you (or I) are quite ready yet to shut down the shop(s).

[jonfernquest]

"Well, the issue is not do one or the other."

There may also be differential benefits across disciplines and specialisations within disciplines. Peer reviewed online international journals have strategic potential to revive more marginal (but important) areas of research.

My area of research "pre-modern Burmese history" has hardly existed in the last 50 years and the few people who do research in this area are spread out over the world and don't even have consistent department affiliations, only a few are actually officially history professors, most have been sloughed off into the more virtual than real "Asian Studies" department. For the most part, they have published by finding little nooks and crannies in the journals of other disciplines but many paper journals are disappearing and obscure European published journal articles or books (Brill) about Southeast Asia can (ironically) not even be found in Southeast Asia. The Singapore government has stepped in and tried to monopolize the marginal Southeast Asian studies field and push everyone out. In fact if you look at the Burma editors at the Journal of Asian Studies in the US, three people have served as gatekeepers there for the last 20 years, one of whom has nepotistic family ties with the emergent Singapore hegemons, is on all critical publication committees, and brooks absolutely no criticism private or public of his work. The only real alternative is a flank attack with an international online peer reviewed journal and that is exactly what SOAS in London has offered for the last few years and I have availed myself of. The role of food supply in warfare has arisen recently in the Darfur conflicts and this is something pre-modern history with its frequent subsistence crises can provide insight into, so even though it is marginal, it can provide insight into current problems.

[Barkley Rosser]

Sorry to jonfernquest about the situation in your field. In econ, the problem is not too few journals, but arguably just the opposite, which may be partly feeding this disdain by big cheeses at top schools to bother with revising endlessly to actually publish.

There is another curious element in this: that putting a paper out online may be almost necessary to get published in a respectable journal, again at least in econ. One of the issues I have had to deal with has been that old chestnut of double versus single blind refereeing. Before I took over at JEBO, they did single blind refereeing, so that referees knew who the authors were, but not vice versa. I changed it to the more normal double blind procedure.

Now I am under pressure from members of my board to reverse that decision. The argument is that referees can find the paper online anyway with a little googling. When I replied that not all papers are put up online, I got back the reply "well, if a paper is not up online, it is probably not any good."

I think the person who started this new trend was Michael Woodford. Back in the late 1980s he was fought over by Princeton and Chicago and one or two other top schools, after finishing at Chicago. Now, he did pub a few major papers, most notably his "Learning to Believe in Sunspots" in Econometrica in 1990. But the majority of his major papers were not published anywhere, although some later were. They were very long working papers, too long to be pubbed as regular journal articles, but they circulated widely and got lots of attention and citations. He became a hot commodity, and basically broke the rules while getting away with it. But not everybody can pull a Michael Woodford.

[been though it]

There's another issue that may not have occurred to the academics in the crowd. Central banks, departments of finance and other government agencies with research outfits that put out working/discussion paper series already put their researchers through a review process before putting things out as a "working paper". This process is in many cases even more broken than the journal peer review process, because the "reviewers" are often senior managers who are not experts in a particular field and sometimes are not active researchers (some even have quite scant publication records themselves).

So there are cases of multiple rounds of editing and substantial major work required to get papers out -- and these compulsory revisions can even detract value from the paper. The put-upon authors are then too sick of the process to go through the revision mill again (often reversing the changes their managers insisted on) to submit them to journals, and the paper will get cited as a working paper anyway.

This shift online might lead to its own problems. In particular, you don't have to be a member of a particular club to get a good paper into a journal (despite rumours to the contrary). But if you want your working paper to be noticed, it helps to be an NBER member or associated with the CEPR.

[david]

Top departments, particularly Harvard / MIT, disseminate many of their results early through NBER. I suspect that if the NBER Working Papers were considered a "top journal" for the purposes of this study, this effect might decline.