• Editorial announcement
  • ICT assessment: Moving beyond journal outputs
  • Katherine McCain: Recipient of the 2007 Derek de Solla Price Award of the journal Scientometrics
  • Katherine W. McCain wins the 2007 Derek John de Solla Price Medal
  • Knowledge network hubs and measures of research impact, science structure, and publication output in nanostructured solar cell research
  • Patent coupling analysis of primary organizations in genetic engineering research
  • Policy impact of bibliometric rankings of research performance of departments and individuals in economics
  • Preface
  • Q-measures for binary divided networks: Bridges between German and English institutes in publications of the Journal of Fluid Mechanics
  • Science and technology in standardization: A statistical analysis of merging knowledge structures
  • The declining scientific impact of theses: Implications for electronic thesis and dissertation repositories and graduate studies
  • The differentiation of the strategic profile of higher education institutions. New positioning indicators based on microdata
  • UK Research Assessment Exercises: Informed judgments on research quality or quantity?
  • ‘Triad’ or ‘tetrad’? On global changes in a dynamic world


Linda ButlerContact Information

(1)  Research Evaluation and Policy Project, Australian National University, ACT 0200 Canberra, Australia

Received: 12 February 2007  Published online: 14 November 2007

Abstract  There are increasing moves to deploy quantitative indicators in the assessment of research, particularly in the university sector. In Australia, discussions surrounding their use have long acknowledged the unsuitability of many standard quantitative measures for most humanities, arts, social science, and applied science disciplines. To fill this void, several projects are running concurrently. This paper details the methodology and initial results for one of the projects that aims to rank conferences into prestige tiers, and which is fast gaining a reputation for best practice in such exercises. The study involves a five-stage process: identifying conferences; constructing a preliminary ranking of these; engaging in extensive consultation; testing performance measures based on the rankings on ‘live’ data; and assessing the measures.

In the past, many similar attempts to develop a ranking classification for publication outlets have faltered due to the inability of researchers to agree on a hierarchy. However the Australian experience suggests that when researchers are faced with the imposition of alternative metrics that are far less palatable, consensus is more readily achieved.

Vincent Larivière1, 4 Contact Information, Alesia Zuccala2 and Éric Archambault1, 3

(1)  Observatoire des sciences et des technologies (OST), Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Université du Québec à Montréal, CP 8888, Succ. Centre-ville, Montréal, Québec, H3C 3P8, Canada
(2)  Rathenau Instituut, Den Haag, The Nederlands
(3)  Science-Metrix, Montréal, Québec, Canada
(4)  Graduate School of Library and Information Studies, McGill University, Montréal, Québec, Canada

Received: 12 February 2007  Published online: 14 November 2007

Abstract  Although the writing of a thesis is a very important step for scientists undertaking a career in research, little information exists on the impact of theses as a source of scientific information. Knowing the impact of theses is relevant not only for students undertaking graduate studies, but also for the building of repositories of electronic theses and dissertations (ETD) and the substantial investment this involves. This paper shows that the impact of theses as information sources has been generally declining over the last century, apart from during the period of the ‘golden years’ of research, 1945 to 1975. There is no evidence of ETDs having a positive impact; on the contrary, since their introduction the impact of theses has actually declined more rapidly. This raises questions about the justification for ETDs and the appropriateness of writing monograph style theses as opposed to publication of a series of peer-reviewed papers as the requirement for fulfilment of graduate studies.