OMT Web
Organization and Management Theory
Division of the Academy of Management
OMT Web
Organization and Management Theory
Division of the Academy of Management
Note: The article below is a guest commentary on the topic of management journal rankings sparked by a recent OMT Listserv debate on this topic. If you are an OMT scholar interested in making a contribution, please email This e-mail address is being protected from spambots. You need JavaScript enabled to view it . Enjoy!
By Teppo Felin, Brigham Young University
Crowdsourcing is an intriguing concept. It suggests that opinions and subjective guesses, lots of them, aggregated, can beat out expert and other forms of judgment. The central premise behind the crowdsourcing concept is essentially a version of the law of large numbers. Galton famously discovered that aggregating lots of opinions (even "dumb" ones) can, on average, lead to discovering the "truth" (see his 1907 article in Nature). Sure, not in all cases. Yes, absolutely, there are lots of the usual qualifications and contingencies. But crowdsourcing is an interesting concept nonetheless.
So, inspired by Kieran Healy's and Steve Vaisey's effort to crowdsource sociology department rankings (see here), I posted a survey to crowdsource management journal rankings. The survey was advertised via various Academy of Management listservs (including OMT) and the orgtheory.net blog, and eventually the results began to pour in. In pairwise fashion, scholars were asked to select the bettermanagement journal --- Technovation versus the Strategic Management Journal, Administrative Science Quarterly versus Academy of Management Review etc.
Scientists are still pouring over the data, but here are some (very) preliminary results (rank based on 'winning' percentage).
| Management Journals | % | |
| 1 | Administrative Science Quarterly | 90.43% |
| 2 | Academy of Management Journal | 90.41% |
| 3 | Academy of Management Review | 88.94% |
| 4 | Organization Science | 88.31% |
| 5 | Strategic Management Journal | 84.42% |
| 6 | Journal of Applied Psychology | 84.00% |
| 7 | Management Science | 82.56% |
| 8 | Journal of Management | 82.46% |
| 9 | OBHDP | 78.93% |
| 10 | Organizational Research Methods | 74.06% |
| 11 | Journal of Organizational Behavior | 72.79% |
| 12 | Personnel Psychology | 71.93% |
| 13 | Journal of Management Studies | 71.10% |
| 14 | Research in OB | 70.37% |
| 15 | Organization Studies | 69.68% |
|
|
The above represents some 69,000 pair-wise votes, by some 1000+ unique users sessions. We're still playing around with the data (additional variables etc), to see what they might tell us, if anything. (We ran a similar survey of law schools, with 300,000+ votes and 6000+ unique user sessions, the results are here.)
So, what can a crowdsourced ranking tell us? Nothing definitive is my guess, though I'm sure other rankings don't necessarily give us a definitive signal either. Though I do think that aggregated perceptions perhaps give us another data point when evaluating and comparing journals (along with impact and influence factors and other, more "objective" measures). These rankings can of course mirror extant rankings (raising causal questions). But they might also capture more up-to-date anticipations of future performance. For example, the UC Irvine Law School (established in 2008) has not graduated a single student, though the school is already well within the top 50 in the crowdsourced ranking.
Lots of other questions can be raised, specifically related to a management journal ranking like this. For example, should micro and macro journals be lumped together like this? And certainly disciplinary journals play a large role in management - should they be included (sociology, psychology, economics)?
Strategic "gaming" of the results of course can also occur. For example, I ended up having to delete some 25,000+ automatically generated votes (it looked like a computer script was created to throw the ranking off), votes that were explicitly cast to sabotage the effort (the African Journal of Management beat all the top journals according to this mega, robo-voter). Though, it is interesting to see how the "crowd" essentially plays a role in averaging bias and in putting a check on strategic voting.
Ironically, I'm actually not one to necessarily really care about journal rankings like this. I wonder whether article-effects trump journal-effects? (I believe Joel Baum has a provocative paper on this.) Of course I read and submit to "top" journals, but there are many "lesser" (ahem) journals that are just as much on my radar screen, for example Industrial and Corporate Change, Managerial and Decision Economics andStrategic Organization. Obsessions with journal standing can detract from ideas.
Pragmatically, yes, journal rankings matter: promotions indirectly depend on it, as do resource flows etc. So, perhaps a "democratic," crowdsourced ranking like this can provide additional information for decision-makers and scholars in the field.
