Robustness Of Journal Classifications In Ssh: An Empirical Analysis From Italy

In Humanities and Social Sciences, a two-tier journal classification implemented by selected scientific experts has been recently used in Italy in the context of the National Habilitation programme; for the same period, peer review is available for a large number of journal articles, developed in the framework of two national evaluation exercises (VQR 2004-2010 and VQR 2011-14). We take advantage of this rich dataset in order to check if journals classified as top class by scientific experts show higher impact and if articles published in those journals receive higher marks in peer-reviewed evaluation exercises. Results are rather mixed: the impact indicator does not seem to be significantly higher for top-class journals with respect to those classified in the second tier; on the other hand, median marks received by individual articles in the two evaluation exercises are significantly higher for top journals. We can draw two main conclusions from the analysis: the first is that journal classification performed by experts does not necessarily rely on impact measures, depending more instead on different criteria for journal selection. However, journal classification based on experts' judgement seem to proxy quite well on average the quality of individual articles as measured by peer review.