Overpromise of AI in Drug Discovery

Dr. Acula
Professor,Transfusion Services
10/10/2019
Internal Medicine Grand Rounds
UNM School of Medicine
Albuquerque, NM Funding:Transylvanian Authority
http://druggablegenome.net/
http://datascience.unm.edu/

October is
National Medical
Librarians Month!

1. To discuss about Al, machine learning, and the hype cycle
2. To discuss the knowledge-based classification of proteins
3. To discuss applications of AI/ML to drug discovery
4. Disclosures:
 Consultant for: AstraZeneca, Celgene, Infinity Pharmaceuticals, Novartis,
Sanofi (between 2002 and 2015)
 Honoraria from Pfizer, Genentech
10/10/19 revision

WhichYear is it?
Each culture celebrates NewYear at a different date:
Gregorian & Julian (Russian), Jewish (Rosh Hashana), Chinese
(Xinlian),Tibetan (Lobsar), Gujarati (Diwali), Muslim
(Muharram) etc.
Current year is 2019 or 4717 (Chinese) or 5780 (Jewish)
According to predictions based on the Mayan calendar, the
world ended in December 21, 2012
 Timekeeping is a human endeavor (at least on Earth) that relies
on CONVENTION
 Every other human endeavor does too…

 The way we navigate “through time and space” – or
Medicine - is by observing & using conventions
 Geographical maps are a form of convention…
 So are Disease Ontologies
 Informatics and data science need conventions,
without which there can be little progress
 What do we know?
 When do we know it?

http://flourish.org/upsidedownmap/
North/South and surfaces are represented by convention.
This is the “equal surface” area (and no, Greenland is not as big as Africa!)

The absence of a quantitative language
“is the flaw of biological research”
Y. Lazebnik, Cancer Cell 2002,2:179-182
aka “the more facts we learn the less
we understand”.
When little is known, don’t expect
knowledge to accumulate quickly
Lenat & Feigenbaum, Artif Intell 1991,1173-1182
We rely on confirmation bias, which
opens the door to alternative facts by
tilting the balance of truth
Recall John Ioannidis @ Stanford
(75% of biomedical publications are false)
AI/ML recognize patterns and return correlations.
They will not help us understand, though they can guide hypotheses.

10/07/18 revision http://datascience.unm.edu
Finding therapeutic uses for proteins (target identification) is a time & resource
consuming enterprise.
Understanding the role of Proteins in Disease & Phenotype is a complex process.
IDG KMC uses machines to help us solve this problem.

1. “Machine learning is the study of computer algorithms that improve
automatically through experience.” -- Tom M. Mitchell
2. “Artificial intelligence is the science and engineering of making
computers behave in ways that, until recently, we thought required
human intelligence.” -- Andrew Moore
3. First “AI”-term indexed in MeSH is from 1951 (KH Fletcher
described a tortoise robot). Today we have speech recognition,
automated language translation, self-driving cars, image processing
“AI”, etc.
4. Unlike machine learning, AI is a moving target, as the definition of AI
changes given rapid technological advancements. -- Roberto
Iriondo
9/25/19 revision

Blog link
Blog link
10/10/19 revision

SECRET MESSAGES
I Know
Machine
Learning

Paper link
Paper link
Book byYuval N, Harari link
My colleagues, they study
artificial intelligence; me,
I study natural stupidity.
Amos Tversky
Quote link
Quote credit: Andrew Maynard 5/24/18 revision

10/10/19 revision
AI/ML cannot sift “True Data” from “False Data”
https://en.wiktionary.org/wiki/embiggen

IT
KMC
RFA-RM-16-
026
(DRGC)
GPCR
U24 DK116195:
Bryan Roth, M.D., Ph.D. (UNC)
Brian Shoichet, Ph.D. (UCSF)
Ion
Chan-
nel
U24 DK116214:
Lily Jan, Ph.D. (UCSF)
Michael T. McManus, Ph.D. (UCSF)
Kinase
U24 DK116204:
Gary L. Johnson, Ph.D. (UNC)
RFA-RM-16-
025
(RDOC)
U24 TR002278:
Stephan C. Schürer, Ph.D. (UMiami)
Tudor Oprea, M.D., Ph.D. (UNM)
Larry A. Sklar, Ph.D. (UNM)
RFA-RM-16-
024
(KMC)
U24 CA224260:
Avi Ma’ayan, Ph.D. (ISMMS)
U24 CA224370:
Tudor Oprea, M.D., Ph.D. (UNM)
RFA-RM-18-
011
(CEIT)
U01 CA239106: N Kannan, PhD & KJ Kochut (UGA)
U01 CA239108: PN Robinson, MD PhD (JAX), CJ
Mungall (LBL), T Oprea (UNM)
U01 CA239069: G Wu, PhD (OHSU), PG D’Eustachio
PhD (NYU), Lincoln D Stein, PhD (OICR)
Further information
Email: idg.rdoc@gmail.com
Follow: @DruggableGenome
URLs:
https://druggablegenome.net/
https://commonfund.nih.gov/idg/
IDG Knowledge User-Interface
Email: pharos@mail.nih.gov
Follow: @IDG_Pharos
URL: https://pharos.nih.gov/
4/25/19 revisionG. Rodgers et al., Nature Rev. Drug Discov. 2018, https://www.nature.com/articles/nrd.2017.252

10/20/16 revisionR. Santos & O. Ursu et al., Nature Rev. Drug Discov, 2017, doi:10.1038/nrd.2016.230
Drugs distributed by
ATC codes (levels 1-2).
Concentric rings
indicate ATC levels.
Histograms represent
the number of drugs
distributed per year of
first approval.
Maximum scale: 100.

 Most protein classification schemes are
based on structural and functional criteria.
 For therapeutic development, it is useful to
understand how much and what types of
data are available for a given protein,
thereby highlighting well-studied and
understudied targets.
 Tclin: Proteins annotated as drug targets
 Tchem: Proteins for which potent small
molecules are known
 Tbio: Proteins for which biology is better
understood
 Tdark: These proteins lack antibodies,
publications or Gene RIFs
3/23/18 revisionT. Oprea et al., Nature Rev.Drug Discov. 2018, https://www.nature.com/articles/nrd.2018.14

 6077 human proteins are associated
with at least one Rare Disease.
 Sources: Disease Ontology (RD-slim),
eRAM and OrphaNet
 ~50% agreement (gene level)
 Contrast:Tclin at 3% & Tchem at 7%
overall vs. RD subset: 6.94% Tclin and
14.1% for Tchem.
 20% of the RD proteome is Tclin &
Tchem. This means hope for cures.
 Potentially significant opportunities for
target & drug repurposing.
9/16/19 revisionTambuyzer E, et al. Nature Rev. Drug Discov. 2019 (Analysis, accepted)

IMPC
Unassigned
IMPC in
progress
IMPC
planned
IMPC model
MGI models
Unassigned
Vomoronasal and
taste receptors
Aborted
Other MGI
models
95% of eligible IDG genes
(339/356) have plans,
attempts, or models
Phenotyped knock-out
mice are mapped to
3,431 human genes :
566 genes in
Tclin/Tchem, 2,347
genes in Tbio, 518
genes Tdark
17
28
17
1
63
24
50
79
168
6/14/19 revisionSlide from Steve Murray, Jackson Lab, modified

9/25/19 revision
~35% of the proteins remain
poorly described (Tdark)
~11% of the Proteome (Tclin & Tchem) are currently targeted by
small molecule probes
There may be opportunities for therapies in Rare Diseases

 Philosophers and scientists alike struggle with these questions
…we’re not really equipped to handle contradictions
 Machine learning (binary classification models) reflect our view of the
Universe: “A” or “non-A” – true or false, no alternatives
 We live in a world of relative truths:World Series and Stanley Cup winners
change annually, gravity waves were just recently confirmed, we accept
political polls and weathermen on TV as substitutes for truth…
 While we’re capable to live in the world of half-truths and half-lies, we try to
develop AI systems to distinguish healthy from diseased
 The exact definition of disease is not trivial (same with “health”)
 There are no mathematical models to help discriminate truth from falsehood

 The two leading principles in genetic nosology are pleiotropism and genetic
heterogeneity. Pleiotropism refers to multiple end effects of a single gene.
Genetic heterogeneity refers to the existence of two or more fundamentally
distinct entities with essentially the same clinical picture.
 Nosologists tend to be either lumpers or splitters.To the extent that he pulls
together the multiple features of single gene syndromes, the medical
geneticist is a lumper.To the extent that by various means he identifies
heterogeneity he is a splitter.
Victor A. McKusick, MD
Five decades later, we still have the same problems…
10/10/19 revisionMcKusick V, Perspec Biol Med 1969, 12:298-312

9/22/19 revision
Source:
MONDO
Haendel M, et al. Nature Rev.Drug Discov. 2019 (Commentary, accepted)

We’re revising the
number of RDs from
~7,000 to 10,393 using
Disease Ontology,
OrphaNet, GARD,
NCIT, OMIM and the
Monarch Initiative
MONDO system
10/10/19 revisionHaendel M, et al. Nature Rev.Drug Discov. 2019 (Commentary, accepted)

Tambuyzer E, et al. Nature Rev. Drug Discov. 2019 (Analysis, accepted) 10/08/19 revision

SECRET MESSAGES
DETOUR:
Devour &
Exsanguinate
TOURists

https://pharos-beta.ncats.io/targets/GRIN2A
The IDG KMC tracks more ~10 information
channels for protein-disease associations,
accessible via the Pharos portal.
Our challenge is to harmonize disease
concepts, and to enable computational use:
e.g., GRIN2A with GRIN1 form the
Glutamate NMDA receptor, MoA drug
target for memantine (Alzheimer’s).
The challenge for ML & AI: How to
prioritize targets? i.e., which protein-
disease associations are clinically
actionable?
(involved is not the same as committed)
10/07/18 revision

 IDG KMC2 seeks knowledge gaps
across the five branches of the
“knowledge tree”:
 Genotype; Phenotype; Interactions
& Pathways; Structure & Function;
and Expression, respectively.
 We can use biological systems
network modeling to infer novel
relationships based on available
evidence, and infer new “function”
and “role in disease” data based
on other layers of evidence
 Primary focus on Tdark & Tbio
O. Ursu,T Oprea et al., IDG2 KMC 2/01/18 revision

 a meta-path is a path consisting of
a sequence of relations defined
between different object types
(i.e., structural paths at the meta
level)
 Our metapaths encode type-
specific network topology
between the source node (e.g.,
Protein) and the destination node
(e.g., Disease).
 This approach enables the trans-
formation of assertions/evidence
chains of heterogeneous
biological data types into a ML
ready format.
G. Fu et al., BMC Bioinformatics 2016, 17:160 is an early example for drug-target interactions 10/01/18 revision
Similar assertions or evidence form metapaths (white).
Instances of metapath (paths) are used to determine the strength of the
evidence linking a gene to disease/phenotype/function.

one protein-disease
association at the time
O. Ursu,T Oprea et al., IDG2 KMC 2/01/18 revision
Genes associated with a disease/phenotype are positive examples, whereas genes lacking the same
association are negative examples. The Metapath approach transforms assertions/evidence chains into
classification problems that can be solved using suitably designed machine learning algorithms.

O. Ursu et al., manuscript in preparation
Data source Data type Data points
CCLE Gene expression 19,006,134
GTEx Gene expression 2,612,227
Protein Atlas Gene & Protein expression 949,199
Reactome Biological pathways 303,681
KEGG Biological pathways 27,683
StringDB Protein-Protein interactions 5,080,023
Gene ontology Biological pathways & Gene function 434,317
InterPro Protein structure and function 467,163
ClinVar Human Gene - Disease/Phenotype associations 881,357
GWAS Gene - Disease/Phenotype associations 54,360
OMIM Human Gene - Disease/Phenotype associations 25,557
UniProt Disease Human Gene - Disease/Phenotype associations 5,365
JensenLab DISEASE Gene - Disease associations from text mining 44,829
NCBI Homology Homology mapping of human/mouse/rat genes 70,922
IMPC Mouse Gene - Phenotype associations 2,153,999
RGD Rat Gene - Phenotype associations 117,606
LINCS Drug induced gene signatures 230,111,315
We developed automated
methods for data collection
(TCRD), visualization (Pharos)
and data aggregation.
These aggregated datasets
were used to build machine
learning models for 20+
disease and 73 mouse
phenotype.
Each knowledge graph
contains ~22,000 metapaths
and 284 million path instances.
10/07/18 revision

1/03/19 revision
From: Mark McCarthy <mark.mccarthy@drl.ox.ac.uk>
Sent: Friday, December 7, 2018 11:10 AM
The general summary is that we don’t see any enrichment for T2D associations in either
exome or GWAS data from the predicted gene sets (however we slice them up).
But having that we don’t really see anything in the TRAINING set either: No association in the
exomes, and a weak (just nominal) association in the GWAS data.
To be honest, I think, now we’ve taken a look at it, we’d all question the training set: I had
missed that this came from OMIM, which is simply not a reliable source of information in this
regard, and it’s certainly not something we would ever use to derive a set of “truth set” genes
for a multifactorial trait like T2D as curation of that kind of information within OMIM was never
prioritised. Few of the genes in the training set are ones that we would recognise as having
evidence in favour of a role in diabetes.

1/16/19 revisionML work by Tudor Oprea
Genes 51
Source https://omim.org/entry/125853
AUC 0.72±0.02
Genes 54
Source Causal T2DM transcripts
AUC 0.79±0.01

10/10/19 revision
"The fault, dear Brutus, is not in our stars, / But in ourselves, that we are underlings." *)
The fault is in our Data
…but without prior assumptions, there can be no learning
The hardest part is figuring out what to keep and what to discard
*) W. Shakespeare, Julius Cesar, 1599, Act 1, Scene 2

SECRET MESSAGES
IN GOD WE TRUST.
All others have Data.
Quote attributed to W. Edwards Deming, controversial:
Other attributions: George A. Box and Robert W. Hayden.
Bernhard Fisher, MD has said this to a journalist

 Mackmyra tasked Microsoft and Fourkind to create novel whisky recipes using AI
 From input of 75 recipes,“AI” could generate 70 million combinations.
 Nr 36 on the AI ranked combinations was approved by humans 
https://www.geekwire.com/2019/microsoft-got-creation-worlds-first-whisky-formulated-ai/ 9/22/19 revision

The InSilico Medicine team validated several AI-generated small
molecules, optimized on a complex multi-response landscape
9/25/19 revision
GENTRL (generative tensorial reinforcement learning) is a de novo small-molecule chemistry generator that
optimizes synthetic feasibility, novelty, and biological activity. GENTRL was used to discover potent inhibitors of
discoidin domain receptor 1 (DDR1) kinase in 21 days. Four compounds were active in biochemical assays, and
two were validated in cell-based assays. One candidate demonstrated favorable pharmacokinetics in mice.
A. Zhavoronkov et al., Nature Biotechnol.2019, 37:1038-1040

 Siemens Healthineers have developed two CT
systems dedicated to Radiation Therapy
planning
 Using their competence in Artificial Intelligence,
the Siemens systems use a specialized image
reconstruction to optimize the CT images for
autocontouring, applying a deep learning-
trained contouring algorithm.
 These CT images will enable radiation
oncologists to identify the target tumor and
improve treatment accuracy.
 Credit video: Dorin Comaniciu, SVP, Siemens Healthineers
https://www.siemens-healthineers.com/press-room/press-releases/pr-20190916035shs.html 10/10/19 revision

How long does it take to move from “natural” language processing
to AI-driven large-dataset mining? Klingon, anyone? tlhIngan, vay'?
9/25/19 revision
Tomáš Mikolov (Google), developed an efficient algorithm to compute the
distributed representation of words, Word2Vec. It’s currently used for automatic
translation, spam filtering and speech recognition. Word2vec encodes words
using a distribution of weights across 100s of elements that compose the vectors.
Each element contributes to many words.
T. Mikolov et al.,ICLR 2013

Alexahealth™: Given today’s health status and my calorie budget,
what food should I shop/prepare today?
10/10/19 revision
Expanding on current models, Medicine could migrate towards context-specific
computational reasoning tools (“AMI”) with advanced cognitive computing
capabilities, and as complete sets of data as possible. Such platforms could mine
hospital data in real time taking advantage of –omics, biomarker, biomedical and
EMR data to provide real-time patient services.

SPF 200
Sun Protection
Factors were
invented by
Vampires
The old man in Castle Bran
Dreamed of exsanguinating Stan.
He woke with a fright
In the heat of the sunlight,
But he wasn’t allowed to sun tan

Our review found the diagnostic performance of deep learning models to be
equivalent to that of health-care professionals.
However, a major finding of the review is that few studies presented externally validated
results or compared the performance of deep learning models and health-care
professionals using the same sample.
Additionally, poor reporting is prevalent in deep learning studies, which limits reliable
interpretation of the reported diagnostic accuracy.
10/10/19 revisionLiu X, et al. Lancet Digital Health 2019 1:e271-297

Artificial intelligence methods aren’t good at acquiring “new”
knowledge; they only learn from what is presented to them.
Put differently, artificial intelligence doesn’t ask “why” questions.
Systems don’t operate like the children who persistently question
their parents as they try to understand the world around them.
The system only knows what it was fed. It will not recognize
anything it was not previously made aware of.
Michael Berthold
CEO & Co-founder, KNIME
9/24/19 revision

9/26/19 revision
 Can AI discover new knowledge?
To date, no credible evidence of this has been provided. Chatbots,
winning at chess, GO and Jeopardy! do not count.
AI in medical imaging is “equivalent” to humans but “poor
reporting is prevalent in deep learning studies” (paper in Lancet).
 Alternative facts are just as prevalent in research as in politics.
 People lie. See work by JP Ioannidis, but also notes from Bayer
(Asadullah et al) and Amgen (Begley et al).
If AI processes false data, its output will not be useful.
 Hype (euphemism for “fake news”) remains rampant.
https://www.linkedin.com/pulse/why-ai-ready-drug-discovery-tudor-oprea

9/23/19 revision
Predictivity between different models for the same topic (even using
the same ML methods) are likely to differ due to input variations
High veracity data (“ground truth”) is key to successful AI/ML models
Hardest to predict: Efficacy in Man / Market Success

FUTURE OF CLINICAL INFORMATICS
Dr. Kroth will be the
Founding Chair of the
Department of
Biomedical
Informatics at WMed,
the Stryker School of
Medicine in 2020
Phil Kroth, MD, MS served as Director
of the Clinical Informatics Fellowship
at UNM. We had amazing Fellows,
and we have ACGME Accreditation.
Thank you, Phil.

Overpromise of AI in Drug Discovery

More Related Content

What's hot

Similar to Overpromise of AI in Drug Discovery

Recently uploaded

Overpromise of AI in Drug Discovery