Name of Institution
Assessing and Reporting
Research Impact – A Role for the
Library
Name of Institution
overview
• Impact metrics - the ususal suspects and more
• A view from the library
• Identifying needs
• Roles played by libraries
• Developing services
• Moving this space forward on the local and larger
level
Name of Institution
impact metrics-why do we care?
• Quantify and document research
impact
• Justify future requests for
funding
• Quantify return on research
investment
• Discover how research findings
are being used
• Identify similar research projects
• Identify possible collaborators
• Determine if research findings
are duplicated, confirmed,
corrected, improved or
repudiated
• Determine if research findings
were extended (different
human populations, different
animal models/species, etc.)
• Confirm that research findings
were properly
attributed/credited
• Demonstrate that research
findings are resulting in
meaningful health outcomes
• Discover community benefit as
a result of research findings
• Progress reports
• Tenure
• Promotion dossiers
https://becker.wustl.edu/impact-assessment/model
Name of Institution
Alternative Metrics
• Twitter
• Facebook
• Blogs
• Research Highlights
• Google+
• Main Stream Media
• Reddit
• Forums
• Q&A – StackExchange
• Pintrest
• LinkedIn
• FigShare
• F1000 Reviews
• GitHub
h
Articles
Datasets
Slides
Software
Webpages
Name of Institution
Understanding the impact of research is a massive task.
How do we understand the impact of a published work?
Aperson?
A researchcenter?
Auniversity?
What ISimpact?
Name of Institution
I
• We need to move assessment toward
“desired outcomes: research that is
productive, high-quality, reproducible,
shareable, and translatable (PQRST)”
• Productivity metrics should reward high-
influence science rather than least
publishable units and decrease publication
bias against negative results.
Research impact and assessment
Name of Institution
Go Beyond Counts!
https://becker.wustl.edu/impact-assessment
Investigate and adapt
frameworks to help put
things into context
More and more organizations and efforts are
considering the impact of their work through
frameworks.
The Becker Model
Frameworks from the IOM, CDC, and NIEHS
Snowball Metrics
Discipline-specific apporaches and more…
Name of Institution
Go Beyond Counts!
Investigate and adapt
frameworks to help put
things into context
Name of Institution
The Becker Model
• Provides a supplement to publication analysis to provide a more
robust and comprehensive perspective of biomedical research
impact.
– reporting templates, glossary of resources and terms, examples of
relevant indicators of impact across the research process, and readings
• Straightforward framework for tracking diffusion of research outputs
and activities to locate indicators that demonstrate evidence of
biomedical research impact
– individual, core, and institutional-level; modify for different disciplines
• Guidance for quantifying and documenting research impact as well
as resources for locating evidence of impact.
• Strategies for enhancing the impact of research
Name of Institution
Strategies for Enhancing the Impact of Research
h
Repetition, consistency, and an
awareness of the intended
audience form the basis of most
the strategies.
Optimizing discoverability and
access of your research is the
surest way to enhance its visibility
and impact.
The strategies focus upon
Preparing for Publication,
Dissemination, and Keeping
Track of Your Research.
Suggestions for researchers and
recommendations to reach out to their
library for assistance.
Name of Institution
Understandingtheimpactofresearchisa massivetask.
Howdowe understandtheimpactofa publishedwork?
Aperson?
Aresearch center?
Auniversity?
What ISimpact?
How do we scale up and operationalize this process?
How do we supportit locally from the library?
Name of Institution
Leverage ongoing efforts on the levelandbeyondlocal
• ORCID
• CASRAI
• CERIF
• VIVO-ISF ontology
• NISO Altmetric work
• CV/Biosketch
• Large-scale semantic search
• Institutional
repositories
• Research
networking systems
• Institutional
research
information and
management
systems
• Analytical and
benchmarking
efforts
• Tracking and
evaluation
Name of Institution
Vivo lends itself to research assessment & impact
• Facilitates a more robust research
ecosystem
– Diffusion of research products
– Discovery of shared products
– Reproducibility
– Translation
– Enables credit and visibility by
showcasing individual
achievements and expertise
• Supports team-based science and
collaboration
• Allows better large-scale understanding
of the research enterprise
– Temporal relationships (career
development, time from publication to
research synthesis)
– Peer comparisons
– Strategic planning and visualization
– Identify emerging trends
• Enables more efficient means of
collecting & representing meaningful
outputs en masse
• New experimental protocols
• Datasets
• Instrumentation
• Software code
• New diagnostic criteria
• New standard of care
• Curriculum guidelines
• Measurement instruments
• Continuing education
materials
• Clinical/practice guidelines
Name of Institution
• Plum Analytics gathers metrics across
five categories—usage, mentions,
captures, social media and citations.
• Metrics are gathered around artifacts
• Collected information is displayed in
visualizations, dashboards, and widgets.
• Customized for the institution or
organization.
ARTIFACTS
articles blog
posts
book chapters
books cases
clinical trials
conference
papers
datasets
figures
grants
interviews
letters media
patents
posters
presentations
source code
theses /
dissertations
videos
webpages
IR integration
Name of Institution
IR integration
Benefits?
1.Create a feedback loop for researchers that
gives them timely insight into the impact of
their research right from the start
2.Easy access to advanced metrics from
PlumX can help build buy-in from researchers,
increasing repository support by your
contributors
3.Aggregated metrics and reports from the
PlumX dashboard puts the repository at the
center of communicating about the impacts of
your organization’s research
Name of Institution
a spectrum ofpossibilities…
http://commons.wikimedia.org/wiki/File:Linear_visible_spectrum.svg
Integrate
altmetrics
in your IR
Lead or
support RNS
project
(VIVO, etc)
ORCID
services and
support
Run citation reports
Encourage prudent use
of online tools
Welcome deposit of
alternative outputs
in your IR
Keep
current
Collaborate with
TT office to
track outputs
Develop
standard
reports/visualiz
ations/analyses Hire an impact
and evaluation
librarian
Workshop on
strategies for
enhancing
research impact
Run
publication
data reports
Establish a
consultation
service
Help
researchers
obtain DOIs
and understand
data options
Understand
motivations
of funding
agencies
Name of Institution
local success depends on stakeholder engagement
• Stay up to date on scholarly issues and the scholarly workflow
• Stay up to date on funding/reporting requirements
• Brainstorm to identify and understand motivations of
stakeholders
• Anticipate need and present solutions
• Call on advocates
• Think beyond ‘business as usual’
• Be persistent!
Name of Institution
Acknowledgements
Support:
• Northwestern University
Clinical and Translational
Sciences Institute, NIH
award 8UL1TR000150-05
• VIVO - DuraSpace
• Thanks:
• Cathy Sarli, MLS, AHIP
• Karen Gutzman – NLM Fellow
• Andrea Michalek at Plum
@amichalek
• VIVO Community @VIVOcollab
• Galter Health Sciences Library

Important.ppt

  • 1.
    Name of Institution Assessingand Reporting Research Impact – A Role for the Library
  • 2.
    Name of Institution overview •Impact metrics - the ususal suspects and more • A view from the library • Identifying needs • Roles played by libraries • Developing services • Moving this space forward on the local and larger level
  • 3.
    Name of Institution impactmetrics-why do we care? • Quantify and document research impact • Justify future requests for funding • Quantify return on research investment • Discover how research findings are being used • Identify similar research projects • Identify possible collaborators • Determine if research findings are duplicated, confirmed, corrected, improved or repudiated • Determine if research findings were extended (different human populations, different animal models/species, etc.) • Confirm that research findings were properly attributed/credited • Demonstrate that research findings are resulting in meaningful health outcomes • Discover community benefit as a result of research findings • Progress reports • Tenure • Promotion dossiers https://becker.wustl.edu/impact-assessment/model
  • 4.
    Name of Institution AlternativeMetrics • Twitter • Facebook • Blogs • Research Highlights • Google+ • Main Stream Media • Reddit • Forums • Q&A – StackExchange • Pintrest • LinkedIn • FigShare • F1000 Reviews • GitHub h Articles Datasets Slides Software Webpages
  • 5.
    Name of Institution Understandingthe impact of research is a massive task. How do we understand the impact of a published work? Aperson? A researchcenter? Auniversity? What ISimpact?
  • 6.
    Name of Institution I •We need to move assessment toward “desired outcomes: research that is productive, high-quality, reproducible, shareable, and translatable (PQRST)” • Productivity metrics should reward high- influence science rather than least publishable units and decrease publication bias against negative results. Research impact and assessment
  • 7.
    Name of Institution GoBeyond Counts! https://becker.wustl.edu/impact-assessment Investigate and adapt frameworks to help put things into context More and more organizations and efforts are considering the impact of their work through frameworks. The Becker Model Frameworks from the IOM, CDC, and NIEHS Snowball Metrics Discipline-specific apporaches and more…
  • 8.
    Name of Institution GoBeyond Counts! Investigate and adapt frameworks to help put things into context
  • 9.
    Name of Institution TheBecker Model • Provides a supplement to publication analysis to provide a more robust and comprehensive perspective of biomedical research impact. – reporting templates, glossary of resources and terms, examples of relevant indicators of impact across the research process, and readings • Straightforward framework for tracking diffusion of research outputs and activities to locate indicators that demonstrate evidence of biomedical research impact – individual, core, and institutional-level; modify for different disciplines • Guidance for quantifying and documenting research impact as well as resources for locating evidence of impact. • Strategies for enhancing the impact of research
  • 10.
    Name of Institution Strategiesfor Enhancing the Impact of Research h Repetition, consistency, and an awareness of the intended audience form the basis of most the strategies. Optimizing discoverability and access of your research is the surest way to enhance its visibility and impact. The strategies focus upon Preparing for Publication, Dissemination, and Keeping Track of Your Research. Suggestions for researchers and recommendations to reach out to their library for assistance.
  • 11.
    Name of Institution Understandingtheimpactofresearchisamassivetask. Howdowe understandtheimpactofa publishedwork? Aperson? Aresearch center? Auniversity? What ISimpact? How do we scale up and operationalize this process? How do we supportit locally from the library?
  • 12.
    Name of Institution Leverageongoing efforts on the levelandbeyondlocal • ORCID • CASRAI • CERIF • VIVO-ISF ontology • NISO Altmetric work • CV/Biosketch • Large-scale semantic search • Institutional repositories • Research networking systems • Institutional research information and management systems • Analytical and benchmarking efforts • Tracking and evaluation
  • 13.
    Name of Institution Vivolends itself to research assessment & impact • Facilitates a more robust research ecosystem – Diffusion of research products – Discovery of shared products – Reproducibility – Translation – Enables credit and visibility by showcasing individual achievements and expertise • Supports team-based science and collaboration • Allows better large-scale understanding of the research enterprise – Temporal relationships (career development, time from publication to research synthesis) – Peer comparisons – Strategic planning and visualization – Identify emerging trends • Enables more efficient means of collecting & representing meaningful outputs en masse • New experimental protocols • Datasets • Instrumentation • Software code • New diagnostic criteria • New standard of care • Curriculum guidelines • Measurement instruments • Continuing education materials • Clinical/practice guidelines
  • 14.
    Name of Institution •Plum Analytics gathers metrics across five categories—usage, mentions, captures, social media and citations. • Metrics are gathered around artifacts • Collected information is displayed in visualizations, dashboards, and widgets. • Customized for the institution or organization. ARTIFACTS articles blog posts book chapters books cases clinical trials conference papers datasets figures grants interviews letters media patents posters presentations source code theses / dissertations videos webpages IR integration
  • 15.
    Name of Institution IRintegration Benefits? 1.Create a feedback loop for researchers that gives them timely insight into the impact of their research right from the start 2.Easy access to advanced metrics from PlumX can help build buy-in from researchers, increasing repository support by your contributors 3.Aggregated metrics and reports from the PlumX dashboard puts the repository at the center of communicating about the impacts of your organization’s research
  • 16.
    Name of Institution aspectrum ofpossibilities… http://commons.wikimedia.org/wiki/File:Linear_visible_spectrum.svg Integrate altmetrics in your IR Lead or support RNS project (VIVO, etc) ORCID services and support Run citation reports Encourage prudent use of online tools Welcome deposit of alternative outputs in your IR Keep current Collaborate with TT office to track outputs Develop standard reports/visualiz ations/analyses Hire an impact and evaluation librarian Workshop on strategies for enhancing research impact Run publication data reports Establish a consultation service Help researchers obtain DOIs and understand data options Understand motivations of funding agencies
  • 17.
    Name of Institution localsuccess depends on stakeholder engagement • Stay up to date on scholarly issues and the scholarly workflow • Stay up to date on funding/reporting requirements • Brainstorm to identify and understand motivations of stakeholders • Anticipate need and present solutions • Call on advocates • Think beyond ‘business as usual’ • Be persistent!
  • 18.
    Name of Institution Acknowledgements Support: •Northwestern University Clinical and Translational Sciences Institute, NIH award 8UL1TR000150-05 • VIVO - DuraSpace • Thanks: • Cathy Sarli, MLS, AHIP • Karen Gutzman – NLM Fellow • Andrea Michalek at Plum @amichalek • VIVO Community @VIVOcollab • Galter Health Sciences Library