Validation of Google Scholar as an Impact Measure for Political Science
One interesting side finding of the citations data from my paper with Michael Marsh was the cross-validation, throuch comparison with other measures, of the results from Google Scholar compared to other bibliographic indexes. Facing the criticism that we ignored books by looking only at ISI and Scopus (article-based) indexes, Michael and I collected data from Google Scholar – which includes books and book chapters – and also collected separate data on book citations from the “Cited Reference Search” feature of the SSCI from ISI/Web of Science. Comparing this to Google Scholar, it seems that Google correlates the best with the total Cited Reference Search data. It also correlated better with all other indexes in the overall pair-wise correlation contest.
Inter-measure Correlations of Citation Counts
This figure shows the relationships between five different sources, taking in each case total citations: Google scholar, ISI, Scopus, ISI-CRS (books only) and ISI-CRS (all material). ISI has been the most widely used source to date despite concerns about its coverage. We can see that its results correlate quite well with those form Scopus (0.73), and also with Google Scholar (0.69) but less well with the full ISI-CRS where the correlation is only 0.52. Most strikingly, it correlates at only 0.25 with the books only citations from ISI-CRS. Scopus provides pretty similar results, correlating modestly with ISI-CRS (total cites) and more strongly with Google Scholar, but also very poorly with the book only measure from ISI-CRS. It is very clear that measures based just on books will give different results to those based just on articles, although article based measures do correlate more highly with total measures than do book only measures, suggesting that a majority of citations are to articles. The two most inclusive measures, Google Scholar and the full ISI-CRS, provide the highest correlation in the whole matrix, 0.84. In general the analysis here seems to “confirm"the worth of Google scholar. Not only does it correlate more highly with ISI and Scopus, but it also matches well with the total CRS citations (books and articles), suggesting that Google provides a valid aggregate measure that includes both books and articles.
