1
Clinical Citations:
Telling the Story
of Clinical Impact
May 26, 2016
Andrea Michalek
President and Founder, Plum Analytics
andrea@plumanalytics.com
@amichalek | @plumanalytics
2
Agenda
• Our approach to analytics
• Clinical Citations
• Q&A
TELL THE STORY.
4
Over 250 PlumX Customers
Worldwide
5
6
7
What impact
has our
research had
in the past 12
months?
8
How do we
help our early
career
researchers
secure grants.
9
How do we
increase the
visibility of
our
research?
10
How do we
assess and
evaluate our
research?
11
What clinical
impact does
our research
have?
12
Our Approach
• Categorize
– Define the unique properties to measure
research output by
– Group research outputs your way
• Visualize
– See impact at a glance
• Analyze
– Discover your research stories
13
The universe
of metrics is
expanding
14
15
Journal citations are the traditional form of
Scholarly Measure
15
Journal
Impact Factor
3.161
16
17
Researchers have
Moved Online
18
Examples: Scholarly Data Exhaust
TECHNOLOGY HAS FINALLY
CAUGHT UP
Measuring data exhaust to tell the story…
Make
visible
what was
invisible
Make
measurable
what was
immeasurable
22
Defining altmetrics
• Altmetrics = Alternative ways of measuring
the use and impact of scholarship
• Altmetrics combine traditional impact
measures (citation counts) with non-
traditional measures
• Altmetrics = ALL METRICS
“Altmetrics are measures of
scholarly impact mined from
activity in online tools and
environments” – Jason Priem,
Co-Founder, ImpactStory
Using Altmetrics to Illustrate the Impact of Open Access
on Graduate Student Research (Barnett, Collister, Chan 2014)
23
Benefits of Altmetrics
• Broadness
– Measure the broader impact of research—beyond
science
• Diversity
– Measure the impact of scholarly products beyond just
the journal article.
• Speed
– Outputs can be measured immediately.
• Openness
– Obtaining and viewing the data is easier.
Woulters & Costas, 2012
24
Metrics
Categories
USAGE
(clicks, downloads, views,
library holdings, video plays)
CAPTURES
(bookmarks, code forks, favorites,
readers, watchers)
MENTIONS
(blog posts, comments, reviews,
Wikipedia links)
SOCIAL MEDIA
(+1s, likes, shares, tweets)
CITATIONS
(citation indexes, patent
citations, clinical citations)
25
USAGE
• Is anyone reading our work?
• Did anyone watch our videos?
• Usage is the #1 stat researchers want to know after
Citation counts
• PlumX is the only product that includes Usage
clicks, downloads,
views, library holdings,
video plays
26
• Captures indicate that someone wants to come back
to the work
• Early indicator of future citations
bookmarks, favorites,
readers, watchers
CAPTURES
27
• This category measures people truly engaging with
your research
• Automatically uncover the conversations about your
research
• Discover feedback, opinions, etc.
blog posts,
comments, reviews,
Wikipedia linksMENTIONS
28
• Social media measures how well a researcher is
promoting their work
• This is especially important for early career
researchers to measure and understand
• Track the buzz and attention around your research
output
+1s, likes, shares,
tweets
SOCIAL MEDIA
29
• Citations still a standard of long-term impact
• Including citations enables side-by-side analysis with
other metrics categories
• PlumX is expanding the definition of citations
– Patent citations, Clinical Citations, etc.
Citation Indexes,
Patent Citations,
Clinical Citations
CITATIONS
30
5 Categories in Action
Article level
metrics are the
building blocks
for PlumX
31
The Plum Print
• Includes 5 categories
of metrics
• Designed to
communicate
engagement without
a score
32
33
LIST OF
GRANTS
for the
Institution Example
of Analytics
Social Media
by Grant
34
Clinical
Citations
35
Clinical Citations
• Showcase your researchers and institution
• Help your researchers
– Get credit for their work
– Tell their stories for funding
• Understand the value of clinical and
translational research
36
…clinical medicine draws heavily on basic
science, but not vice versa. The result is that
basic medicine is cited three to five times more
than clinical medicine, and this is reflected in
journal impact factors.”
Clinical Research & Citations
“
37
Researchers
have difficulty
getting
funding.
38
Early-career
researchers
opt out of
translational
medicine.
39
Hospitals
cannot
showcase
their talents.
40
41
Clinical Citations
Clinical
Alerting
Services
Clinical
Guidelines
Systematic
Reviews
Clinical
Trials
42
DynaMed Plus Topics
• Clinical reference tool
• Designed for point-of-care
• Content: Extensive network of clinical
experts
– Review 500 journals and 120 guideline
organizations
– select the best and most appropriate evidence
– confirm the clinical applicability of content
– peer-review topics
43
Examples of guideline
organizations include:
• American Academy of Neurology (AAN)
• American Academy of Pediatrics (AAP)
• American Academy of Clinical Endocrinologists
(AACE)
• American Association for the Study of Liver
Diseases (AASLD)
• American College of Cardiology (ACC)
• American College of Chest Physicians (ACCP)
• American College of Obstetricians and
Gynecologists (ACOG)
• American College of Physicians (ACP)
• American Diabetes Association (ADA)
• British Thoracic Society (BTS)
• Canadian Cardiovascular Society (CCS)
• Canadian Medical Association (CMA)
• Canadian Task Force of Preventive Health
(CTFPHC)
• Centers for Disease Control and Prevention
(CDC)
• European Respiratory Society (ERS)
• European Society for Medical Oncology (ESMO)
• European Society of Cardiology (ESC)
• Global Initiative for Chronic Obstructive Lung
Disease (GOLD)
• Infectious Diseases Society of America (IDSA)
• International Society on Thrombosis and
Haemostasis (ISTH)
• National Comprehensive Cancer Network
(NCCN)
• National Health & Medical Research Council
(NHMRC) [Australian guidelines]
• National Institute for Health and Care
Excellence (NICE)
• Pan American Health Organization (PAHO)
• Royal College of Physicians (RCP)
• Scottish Intercollegiate Guidelines Network
(SIGN)
• World Gastroenterology Organisation (WGO)
• World Health Organization (WHO)
44
Showing when research makes
its way into clinical practice
45
Different
artifact types
reveal
different
impacts
46
A research study used for
clinical advice
47
Meta analysis of studies used
to determine clinical practice
48
Researchers contributing to
clinical guidelines
49
Compare like
with like
50
ANALYZE
• By metrics category
• By type of output
• By person
• By grant
• By department
• By institution
51
PlumX: Built for Big Data
• Built for Scale – Our goal is to track all the
world’s researchers, all of their output, and
metrics for each.
51
>18 Million Researchers Worldwide
>120 Million Scholarly Articles
Map artifacts  people 
groups
52
People not Papers
52
53
PlumX Benchmarks
• Gather research output associated with
NIH grants
• Calculate metrics across all 5 categories
• Compare institutions using altmetrics
2012 - 2015
54
PlumX Benchmarks Features
• Select specific institutions to benchmark
• Filter
– By Spending Category
– By Granting Institute and Center
– By Geography
• Rank institutions
– For each category of metrics
• Analyze
55
See Your Clinical Impact
Compared to Peer Institutions
56
57
Online users
expect
metrics
58
59
60
Plum Prints
help
showcase
researchers
61
62
Using metrics
to gain an
advantage in
finding
grants.
63
Big Swings in NIH Funding
• $3 billion
no longer
being
spent on
HIV
research
• $65M
refocused
on ending
AIDS
epidemic
64
State of the Union Address: January 12, 2016
“Let’s make America the country that cures cancer
once and for all.”
65
Vice President Biden’s Memo
“Over the next year, I will lead a dedicated,
combined effort by governments, private
industry, researchers, physicians, patients,
and philanthropies to target investment,
coordinate across silos, and increase
access to information for everyone in the
cancer community,”
66
http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=20&catId=2
4 in 5
people do
NOT
get funded
51,073 Applications
9,241 Awards
18% success rate
NIH: The Competition for
Research $$$ in 2014
67
Finding
Funding
Opportunities
can leverage
metrics
68
69
Search for
HIV Funding
Opportunities
in Health
70
Link directly to the
grant detail
71
Funding
Opportunity
In PlumX
Dashboard
Funding Opps Integrates with
PlumX Dashboards
72
Where we’re heading…
Automatically
Targeting
Funding
Opportunities
XXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
73
Feedback
changes
behavior
74
Metrics that
update in real
time provide a
Feedback Loop
Feedback
loops help
change
behavior.
75
Tools to analyze your stars
• Established researchers
– What are your key researchers doing well?
– How do you measure that?
• Early career researchers
– How are they successfully establishing
themselves?
• Build archetypes by discipline
– What types of outputs are important?
– How should these be measured?
76
Summary
• Altmetrics can be used to uncover and tell
the story of your research.
• We are defining (and expanding)
– Traditional categories of metrics
• Usage
• Citations
– As well as modern metrics
• Captures
• Social Media
• Mentions
77
Summary
• Clinical citation counts are the latest
innovation by Plum Analytics.
• They can help to tell the story about
clinical and translational research.
78
Answer the questions and tell the
stories about your research
Find out more information at:
http://www.plumanalytics.com
79
The PlumX Suite
PlumX Metrics
Get more value from your institutional repository.
PlumX Dashboards
Know what is going on with your research. Without waiting years.
PlumX +Grants
Gain insight about your grants.
PlumX Benchmarks
See how you compare. Now.
PlumX Funding Opportunities
Find the right funding opportunities.

Plum Analytics - Clinical Citations: Telling the Story of Impact

Editor's Notes

  • #15 https://tagul.com/cloud/2
  • #64 http://www.sciencemag.org/news/2015/12/nih-drops-special-10-set-aside-aids-research
  • #66 https://medium.com/@VPOTUS/inspiring-a-new-generation-to-defy-the-bounds-of-innovation-a-moonshot-to-cure-cancer-fbdf71d01c2e#.u33oam4dr
  • #67 # applicants up $ funding available down % submissions accepted down year over year