google scholar and high-impact publication

Academics feel narcissistic or anti-intellectual when we check citations to our work, but it isn't just an ego thing. Citations tell us who is using our research and who we should be reading -- a big help in making intellectual connections. If we really want people to read the work we spend so much time writing, then we need to figure out why some articles rise and others (ahem) drop from cite. Analysis can also reveal correctable mistakes. We may have written the right paper for the wrong audience or used a title or abstract that all but guaranteed our work would never be read or referenced.

I ran the numbers, but never looked much at citation indexes until seeing Google Scholar, which tends to be more inclusive and useful than other indexes. Editing thesocietypages.org, though, I'm starting to think we need new ways of measuring both scholarly and public impact. For example, I'm convinced that Lisa Wade and Gwen Sharp are having an enormous impact at sociological images, but it isn't (yet) counted in ways that make sense to the Social Science Citation Index or Google Scholar. I'm not just talking about hit counts -- increasingly, students and other scholars are adopting the site's sensibility and and its application to the visual social world.

For now, though, Google Scholar represents a huge advance over the sort of citation trackers we had just a few years ago. Seeing Philip Cohen's google scholar profile this morning, I made my own. A few observations:

1. Scale. Before constructing such a profile, you should know that some people and papers get cited a lot, but it takes most of us a few years to develop an audience. Nobody cited my stuff at all as an assistant professor, but folks began excavating the nuggets once a few pieces got a little attention. In Google, as elsewhere, try not to compare yourself against the standard set by the top senior scholars in your field (a.k.a. "Sampson Envy").

2. Inclusiveness. Google scholar is indeed more inclusive than other sources. For me, at least, it includes three times the citations and twice the number of writings than SSCI (2,578 citations in Google to 84 "things" (articles, chapters, grant reports, committee documents) and 767 citations in SSCI to 35 journal articles)). Some may find it overinclusive, but Google seems far more effective in bringing to light intriguing intellectual connections. For instance, I learned that a Swedish economist found use for one of my papers in a presentation on the "entrepreneurial life course of men and women" -- which jazzed up my own thinking about a project on entrepreneurship and prisoner reentry.

3. Bias? For me, at least, the Social Sciences Citation Index seems to give a pretty misleading picture of scholarly impact. Since SSCI doesn't count books or book chapters, it misses a couple more-cited pieces -- a book with Jeff Manza and a popular chapter in an edited volume. [Junior scholars are often told to avoid writing book chapters, but some of them seem to find a pretty good audience.] Also, when I rank the articles by citation count, Google seems to have better face validity -- it does a better job picking up the contributions that people ask me about than SSCI. As chair in a department that values both books and articles, the omission of books in any index is really problematic. I haven't done a careful analysis, but my sense is that Google Scholar is also better than SSCI at tracking my criminological and interdisciplinary work.

4. Flagships. But still .... articles in the so-called sociology flagships get cited way more often than articles in other journals or book chapters. By either index, my 3 most-cited pieces (and 6 of the top 16) appeared in American Sociological Review or American Journal of Sociology.

5. Future. I expect that people will always want to assess the scholarly and public impact of academic work, and that these tools will evolve rapidly. Google Scholar offers a great set of tools already, but I suspect we'll soon be able to run much more sophisticated searches that allow us to track impact across a broader spectrum of outlets. People are sure to debate "what counts" as a citation, but the really big honkin' question concerns "what counts" as scholarly publication. My sense is that journal impact will remain important, but we'll soon have the tools to identify and assess a more robust and varied set of impacts.