IBM Watson
IBM Watson is a powerful AI platform offering a
variety of tools to help businesses solve complex
problems using artificial intelligence.
Watson included 90 IBM Power 750 servers, each
with four processors for a maximum of 32 logical
cores per processor.
This means that there was a total of 2,880 IBM
Power7 Processor Cores. It was the power of the
2,880 cores that allowed Watson to meet the 3‐
second requirement for delivering an answer to a
question.
In addition, Watson was designed to store its entire
knowledge base in random access memory (RAM)
instead of on disk to further speed up processing
speeds and deliver fast results.
Extremely fast networking technology was included
to help move a lot of data between compute nodes at
fast speeds.
Applications
1. Watson Assistant (Chatbots/Virtual Assistants)
Example: Chatbot for customer service
2. Watson Discovery (Document Search and Analysis)
Example: Legal document analysis
3. Watson for Oncology (Healthcare)
Example: Cancer treatment recommendations
4. Watson Visual Recognition (Image and Video
Analysis)
Example: Product defect detection in manufacturing
5. Watson Natural Language Processing (NLP)
Example: Sentiment analysis for customer feedback
6. Watson Machine Learning (ML)
Example: Predictive maintenance in manufacturing
These examples show how IBM Watson is used across
various industries to improve efficiency, make better
decisions, and provide innovative solutions to business
challenges.
Watson Defined
Overview of Watson
Watson is a cognitive system that integrates:
o Natural Language Processing (NLP)
o Analytics
o Machine Learning techniques
Key Features
Dynamic Learning:
o Watson improves its performance by gaining
insights from each user interaction.
o It becomes smarter as it ingests new
information.
Hypothesis Generation and Evaluation:
o Watson assists professionals in creating
hypotheses from data.
o It helps in accelerating findings and determining
the availability of supporting evidence to solve
problems.
Business Implications
IBM views Watson as a tool to enhance business
outcomes.
It enables natural interaction between humans and
machines, streamlining decision-making processes.
Comparison with Traditional Search
While individuals are used to sophisticated search
engines for information retrieval, Watson employs a
different approach:
o It facilitates data-driven search while leveraging
machine learning and advanced analytics.
o Watson’s architecture, known as DeepQA,
underpins its capabilities, allowing for complex
question answering.
IBM Watson DeepQA Architecture
The architecture is designed to handle complex
queries and generate accurate answers, emphasizing
the integration of multiple analytic components.
Evidence
Answer Sources
Sources
Question Supporting Deep
Candidate
Primary Evidence Evidence
Answer
Search
Generation
Question Query Hypothesis Soft Analysis Hypothesis and Final Merging
Decomposition Generation Filtering Evidence Scoring Synthesis
and Ranking
Trained
Hypothesis Soft Hypothesis and Models
Generation Filtering Evidence Scoring
Answer and
Confidence
IBM Watson DeepQA Architecture Notes
Overview
DeepQA is the software architecture behind IBM
Watson, designed to handle complex question-
answering tasks.
It integrates multiple components to process natural
language and deliver accurate answers based on a
vast knowledge base.
Key Components
1. Question Analysis
o Question Decomposition: Breaks down the
question into manageable parts for better
understanding.
o Query Analysis: Examines the structure and
intent of the question.
2. Answer Generation
o Primary Search: Searches through answer
sources to find potential answers.
o Candidate Answer Generation: Generates
possible answers based on the information
retrieved.
3. Evidence Sources
o Supporting Evidence: Identifies supporting
facts that strengthen potential answers.
o Deep Evidence: Seeks deeper, more nuanced
information to substantiate the candidates.
4. Hypothesis Generation
o Hypothesis Creation: Generates hypotheses
based on the data and potential answers.
o Soft Filtering: Filters out less relevant
hypotheses to focus on more promising ones.
5. Scoring and Ranking
o Hypothesis and Evidence Scoring: Evaluates
each hypothesis against the evidence gathered.
o Final Merging and Ranking: Combines and
ranks hypotheses to determine the best answer
based on confidence levels.
6. Synthesis
o Synthesis of Information: Integrates various
pieces of information to form a cohesive
response.
7. Answer and Confidence
o The architecture outputs the final answer along
with a confidence score to indicate the reliability
of the response.
Additional Concepts
Trained Models: The architecture utilizes models
that have been trained through numerous interactions
to improve accuracy.
Massive Parallelism: Enables the processing of vast
amounts of information simultaneously, enhancing
speed and efficiency.
Summary
The IBM Watson DeepQA architecture is a sophisticated
system designed to analyze, generate, and evaluate
answers to complex questions using natural language
processing and machine learning techniques. Its
structured approach allows for high accuracy and
confidence in responses, making it suitable for various
applications, especially in industries like healthcare and
finance.
Watson’s Software Architecture based on the provided
text:
Overview
Watson's architecture is designed for building
Question-Answering systems and integrates various
algorithmic techniques.
The primary focus during design was on accuracy
and confidence before speed.
DeepQA Architecture
DeepQA is the Natural Language Processing (NLP)
technology used in Watson. Key components include:
o Question Parsing and Classification:
Understanding the structure and type of the
question.
o Question Decomposition: Breaking down
complex questions into simpler components.
o Automatic Source Acquisition and
Evaluation: Automatically gathering relevant
information sources.
o Entity and Relation Detection: Identifying key
entities and their relationships within the data.
o Logical Form Generation: Structuring data
logically for processing.
o Knowledge Representation and Reasoning:
Structuring knowledge for reasoning and
inference.
Technical Standards
Built according to Unstructured Information
Management Architecture (UIMA) standards,
which allows for extreme speed, scalability, and
accuracy across distributed systems.
Core Design Principles of DeepQA
1. Massive Parallelism: Many computer processes
work simultaneously to enhance processing speed
and performance, enabling rapid analysis of vast
information sources.
2. Integration of Probabilistic Question and Content
Analytics: Machine learning models provide correct
answers and estimate relationships and patterns
across multiple domains.
3. Confidence Estimation: Multiple interpretations of a
question are analyzed, continually scoring different
answers to find the most relevant one, enhancing
accuracy.
4. Integration of Shallow and Deep Knowledge:
o Shallow Knowledge: Basic understanding that
provides limited connections between subject
elements.
o Deep Knowledge: A comprehensive
understanding that allows for complex
associations and inferences, critical for
sophisticated problem-solving.
Methodology: AdaptWatson
AdaptWatson is the methodology for developing
and integrating core algorithmic techniques.
It involves:
o Creating core algorithms.
o Measuring and evaluating results.
o Generating new ideas for improvement.
Key roles of AdaptWatson components include:
o Understanding questions.
o Creating confidence levels for answers.
o Evaluating and ranking results.
o Analyzing natural language.
o Identifying sources.
o Generating and finding hypotheses.
o Scoring evidence and answers.
o Merging and ranking hypotheses.
Data Analysis Techniques
Uses machine learning and linear regression to
determine relationships and rank data based on
relevance.
Conclusion
Watson’s architecture combines advanced NLP
techniques, a robust design framework, and sophisticated
algorithmic methodologies to create an effective
Question-Answering system capable of providing
accurate and contextually relevant answers.
