Bibliometrics: Measuring
the impact of research
Sam Aston
http://www.flickr.com/photos/mad_house_p
hotography/4311409835/
Sam Aston
Objectives
Sam Aston
What are Bibliometrics?
Sam Aston
Definition
• ‘A term used in quantitative studies of
peer-reviewed scientific publications,
dealing with measurement of the spread
and impact of information published in
scientific media, based on the statistical
analysis of information in sources such as
the Index Medicus and Science Citation
Index’.
• "bibliometrics" A Dictionary of Public Health. Ed. John M. Last, Oxford University
Press, 2007. Oxford Reference Online. Oxford University Press. University of
Manchester. 26 October 2010 <http://www.oxfordreference.com>
Bibliometrics in a nutshell
What is a bibliometric?
What are bibliometrics used for?
Sam Aston
What’s good about bibliometrics?
What’s not so good?
Impact Measures
Sam Aston
(Journal) Impact Factor
• Journal Impact Factors (JIF) were devised
by Eugene Garfield
• Eg. If a journal has an impact factor of 4 in
2009 then in 2007 & 2008 then its papers
received an average 4 citations each.
Sam Aston
Sam Aston
200920082007
Source paper – published in 2009
Cited reference – published in 2007 or 2008
Citations
All
Previous
Years
2006 2010
2009 Impact Factor
More Measures
Sam Aston
Individual Measures
• ‘H’ Index developed by Jorge E. Hirsch
Sam Aston
Schematic curve of number of citations versus paper number, with
papers numbered in order of decreasing citations.
Hirsch J E PNAS 2005;102:16569-16572
Sam Aston
Services to measure
Web of Knowledge
&
Journal Citation Reports
Sam Aston
Using the Scopus database
for bibliometrics
H Index
SCOPUS author evaluator
• Andrey Geim
SCOPUS Journal analyser
• Scopus analytics
Sam Aston
Bibliography
• Chapman, Karen, 2010. Methods of
demonstrating article and journal impact,
Journal of business and Finance
Librarianship. 15:1, 2-13.
• White Paper Using Bibliometrics in
Evaluation Research
http://science.thomsonreuters.com/news/2
008-07/8465001/

Bibliometrics pres

Editor's Notes

  • #2 Sam
  • #3 Sam
  • #4 Martin
  • #5 Martin
  • #6 Martin
  • #7 Martin Followed by Sam Why are they so important to us? Flipchart exercise
  • #8 Sam
  • #9 Sam The recent THE world rankings and how citation reports make up the weighting. Handout why do we need to know about bibliometrics
  • #10 Martin
  • #11 Martin
  • #12 Sam It is important to remember that the measure that you choose must be the right one for what you want to find out.
  • #13 Sam
  • #15 Sam Eigenfactor is available from eigenfactor.org It is freely available and uses a similar Pagerank methodology to Google. SJR and SNIP are available through the Scopus database part of the Sciverse hub. SCImago Journal Rank (SJR) is a prestige metric based on the idea that ‘all citations are not created equal’. With SJR, the subject field, quality and reputation of the journal has a direct effect on the value of a citation. SJR- Is weighted by the prestige of the journal, thereby ‘leveling the playing field’ among journals- Eliminates manipulation: raise the SJR ranking by being published in more reputable journals- ‘Shares’ a journal’s prestige equally over the total number of citations in that journal- Normalizes for differences in citation behavior between subject fields SNIP Created by Professor Henk Moed at CTWS, University of Leiden, Source Normalized Impact per Paper (SNIP) measures contextual citation impact by weighting citations based on the total number of citations in a subject field. The impact of a single citation is given higher value in subject areas where citations are less likely, and vice versa. SNIP- Measures contextual citation impact by ‘normalizing’ citation values- Takes a research field’s citation frequency into account- Considers immediacy - how quickly a paper is likely to have an impact in a given field - Accounts for how well the field is covered by the underlying database- Calculates without use of a journal’s subject classification to avoid delimitation- Counters any potential for editorial manipulation
  • #16 Sometimes referred to as the hirsch index/number Measures both the impact and the quantity of research. The index is based on the set of the scientist&amp;apos;s most cited papers and the number of citations that they have received in other people&amp;apos;s publications. The index can also be applied to the productivity and impact of a group of scientists, such as a department or university or country. What the h index does not take into consideration is the time period, so early career researchers suffer. Also indexes are only comparable in the same discipline.
  • #17 Sam There are differences in the coverage of the products that are available ISI Web of Science has been the traditional measure of bibliometrics for 50 years. It has a broad subject base and a large number of journals not limited to Science, it covers a great deal of humanities materials. Times Higher Education Supplement use Web of Science to collate their University list Journal Citation Reports measure the impact of individual journal titles Journal Citation Reports® provides a systematic, objective means to evaluate the world’s leading research journals. It offers a unique perspective for journal evaluation and comparison by accumulating and tabulating citation and article counts from virtually all specialties in the sciences, social sciences, and technology fields. To help you compare journals and discover which are the most significant to you, Journal Citation Reports can show: ■ Most frequently cited journals in a field ■ Hottest journals in a field ■ Highest impact journals in a field ■ Most published articles in a field ■ Subject category data for benchmarking The growth of technology has meant that this has increased in the past few years. Scopus is the product from ElsevierGives a rough evaluation of an author’s research output Only papers published since 1996 count Can be represented graphically Overview of author’s publication history Overview of citation activity over time Publish or Perish has been created by Anne-Wil Harzing from the University of Melbourne and it collects its data from Google Scholar. Unfortunately it is not possible to try this today as we cannot download the small program that is required for this on University cluster however you may try this at home free of charge. A recent exercise here at JRUL undertaken by the Liaison Librarians. They checked 30 articles in each of the 5 fields, Medical and Life sciences, Engineering and physical sciences, business, social sciences and arts using each of the 3 services. Google Scholar was as competitive in most of the disciplines, often findng unique material, in all but Medical and life sciences. Therefore finding that it is useful to check your metrics in a couple of different places
  • #18 Sam Demonstration and exercise from work book page 12-13
  • #19 Martin
  • #20 Martin
  • #21 Martin
  • #22 Martin
  • #23 Martin
  • #24 Sam review Think about how you would use this and write it on a post-it. Is anybody willing to share? Martin for evaluation forms