Ideas can... is the website of the Canadian Federation for the Humanities and Social Sciences, a non-profit charitable organization with a mandate to (from the website): promote the value of research and learning in the humanities and social sciences; support the dissemination of knowledge to the public and the public policy community; provide a range of services to individual members and member institutions.
The Federation has recently launched an ambitious project that aims at developing metrics for measuring the impacts of Social Sciences and Humanities within and beyond academe. Recently, a preliminary working paper, titled, “The Impacts of Humanities and Social Sciences Research” has been released.
The document first defines, then proposes some indicators for measurement of, impact around five dimensions.
Impacts on Scholarship – measured through bibliometric indicators, downloads from OA repositories, citations, prizes and awards, reputation as measured by surveys, …;
Impacts on Capacities through teaching and mentoring – measured through surveys for students and colleagues, employer surveys, …;
Impacts on Economy – measured through advisory roles, revenue opportunities resulting, income derived from patents, consulting contracts;
Impacts on Society and Culture – measured through number and quality of partnership with community groups, media coverage of research, engagement with the public, …;
Impacts on Practice and Policy – measured through citations in government documents, consulting for government, commissioned reports, ….
Two personal thoughts. On the one hand, this document is an important benchmark for future studies: going beyond impact as defined as a simple number of citations in the closed world of academia should be considered a priority for research policy, indeed. And an attempt at mixing academic dimensions with wider impacts on society, economy, and policy-making should be regarded as an important one. Future debates should thus focus on how the various dimensions should be balanced and how objective and subjective (surveys) indicators can dialogue.
On the other hand, the question should be still addressed on whether quality and impact of research can be 'measured', that is, objectively quantified rather than evaluated through a judgement. My stance, here, is that the aspiration at a comprehensive measurement of quality and impact is a fallacious one: we may build more complex and refined indicators and understand through them some dimensions of research. But there are dimensions in knowledge building, especially in Social Sciences and Humanities, that will always slip away from measurement. Put in other words, the moment of judgement will always be inescapable and there are traditions and cultures, for which no objective assessment can be set out: how could, for instance, the lesson of Foucault be measured?
For such reasons, all debates on 'alternative' metrics are important and welcome, to the extent that their aim is complementing, not replacing, the capacity to 'judge', which cannot but be grounded on experience and praxis.
(Simone Tulumello)