Severità di giudizio: dinamiche valutative nell'area della sociologia nella VQR 2004-2010

The results of the Italian Quality Research Assessment VQR 2004–10 in the area of Sociology were characterized by the scarcity of excellent ratings and a lack of consensus among referees. These two aspects, which affected the assessment of all social sciences and humanities, descend directly from the use of the peer review as a method of evaluation. As a matter of fact, peer review, although irreplaceable, poses different issues on which critical thinking is needed. In this paper, I aim to contribute to this discussion. Firstly, I focus on the system framework, i.e. an assessment exercise where in total almost 200,000 scientific publications from universities and research institutes have been analyzed. Then, the rules of the game have been reviewed and also the specific characters of the macro-area “14” where Sociology is located. The judgments formulated by the referees and the level of inter-rater agreement were descriptively considered, considering the main features of the referees, the publications and the authors. Finally, some possible interpretations are outlined together with recommendations for future improvements.