Course Branch Subject Subject Name Semester NOTES

Name Code

B.Tech CS CS-305 ARTIFICIAL INTELLIGENCE 4TH UNIT 4

Introduction
🞂 The planning in Artificial Intelligence is about the decision making tasks performed
by the robots or computer programs to achieve a specific goal.
🞂 The execution of planning is about choosing a sequence of actions with a high
likelihood to complete the specific task.
🞂 Planning is the task of finding a procedural course of action for a declaratively
described system to reach its goals while optimizing overall performance measures.
🞂 Automated planners find the transformations to apply in each given state out of the
possible
transformations for that state.
🞂 In contrast to the classification problem, planners provide guarantees on the solution
quality.
🞂 Automation is an emerging trend that requires efficient automated planning.
🞂 Many applications of planning in industry are, e.g.. robots and autonomous systems,
cognitive assistants, cyber security, service composition, etc.
Introduction
🞂 Requirements of AI Planning Techniques
⮩ When explainability is desired
⮩ When you want to be able to explain why a particular course of action was chosen
⮩ Assignment of responsibility is essential for automation of processes (e.g., autonomous driving,
medical
expert systems)

🞂 In many real life applications, there is a structure of the problem that cannot be learned
with Deep Learning (DL).
🞂 Solving optimization problems with learning is hard, but integrating planning techniques
with
heuristic guidance learned by DL will result in the most famous success stories of AI to day.
⮩ GO player AlphaGO uses planning (monte-carlo tree search) with deep learning (heuristic guidance) to
select the next move
⮩ Cognitive assistant (Samsung) uses knowledge graph, planning, and deep learning to answer
complicated
queries
Components of a Planning System
🞂 Planning refers to the process of computing several steps of a problem-solving
procedure before executing any of them.
🞂 The planning consists of following important steps:
1. Choose the best rule for applying the next rule based on the best available heuristics.
⮩ The most widely used technique for selecting appropriate rules to apply is first to isolate a set
of differences between desired goal state and then to identify those rules that are relevant to reduce
those differences.
⮩ If there are several rules, a variety of other heuristic information can be exploited to choose among them.
2. Apply the chosen rule for computing the new problem state.
⮩ In simple systems, applying rules is easy. Each rule simply specifies the problem state that would
result from its application.
⮩ In complex systems, we must be able to deal with rules that specify only a small part of the
complete
problem state.
⮩ One way is to describe, for each action, each of the changes it makes to the state description.
Components of a Planning System
3. Detect when a solution has been found.
⮩ A planning system has succeeded in finding a solution to a problem when it has found a
sequence of
operators that transform the initial problem state into the goal state.
⮩ How will it know when this has done?
⮩ In simple problem-solving systems, this question is easily answered by a straightforward match
of the state descriptions.
⮩ One of the representative systems for planning systems is predicate logic. Suppose that as a part of
our
goal, we have the predicate P(x).
⮩ To see whether P(x) satisfied in some state, we ask whether we can prove P(x) given the assertions
that
describe that state and the axioms that define the world model.
Components of a Planning System
4. Detect dead ends so that they can be abandoned and the system’s effort is directed in
more fruitful directions.
⮩ As a planning system is searching for a sequence of operators to solve a particular problem, it
must be able to detect when it is exploring a path that can never lead to a solution.
⮩ The same reasoning mechanisms that can use to detect a solution can often use for detecting a dead
end.
⮩ If the search process is reasoning forward from the initial state. It can prune any path that leads to a
state
from which the goal state cannot reach.
⮩ If search process reasoning backward from the goal state, it can also terminate a path either because
it is
sure that the initial state cannot reach or because little progress made.

5. Detect when an almost correct solution has been found.

⮩ The kinds of techniques discussed are often useful in solving nearly decomposable problems.
⮩ One good way of solving such problems is to assume that they are completely decomposable,
proceed to solve the sub-problems separately. And then check that when the sub-solutions combined.
⮩ They do in fact give a solution to the original problem.
Blocks World Problem
🞂 Planning refers to the process of computing several steps of a problem-solving
procedure before executing any of them.
🞂 In order to compare different methods of planning, we should look at all of them in a
single domain.
🞂 The Block World Problem is described as,
⮩ There is a flat surface on which blocks can be placed.
⮩ There are a number of square blocks, all the same size.
⮩ The blocks are labeled with alphabets ‘A’, ‘B’, ‘C’, etc.
⮩ They can be stacked one upon the other.
⮩ There is robot arm that can manipulate the blocks.
⮩ The start state and goal state are given.
Blocks World Problem
🞂 Actions of the robot arm (the robot arm can hold only one block at a time)
⮩ UNSTACK(A, B): Pick up block A from its current position on block B.
⮩ STACK(A, B): Place block A on block B.
⮩ PICKUP(A): Pick up block A from the table and hold it.
⮩ PUTDOWN(A): Put block A down on the table.

🞂 Predicates : In order to specify both the conditions under which an operation may be
performed and the results of performing it, we need the following predicates:
1. ON(A, B): Block A is on Block B.
2. ONTABLES(A): Block A is on the table.
3. CLEAR(A): There is nothing on the top of Block A.
4. HOLDING(A): The arm is holding Block A.
5. ARMEMPTY: The arm is holding nothing.
STRIPS-Based Approach to Robot Control
🞂 It uses the first-order logic and theorem proving to plan strategies from start to goal.
🞂 STRIPS language: “Classical” approach that most planners use, lends itself to efficient
planning
algorithms.
🞂 Environment: office environment with specially colored and shaped objects.
🞂 STRIPS planner: developed for this system to determine the actions of the robot should
take to
achieve goals.
🞂 STRIPS (STanford Research Institute Problem Solver) is a restrictive way to express
states,
actions and goals, but leads to more efficiency.
Robot Problem-Solving Systems (STRIPS)
🞂 ADD List : List of new predicates that the operator causes to become true.
🞂 DELETE List : List of old predicates that the operator causes to become false.
🞂 PRECONDITIONS list contains those predicates that must be true for the operator to be
applied.
🞂 STRIPS style operators for BLOCK World problem are :
• STACK(x, y)
• P: CLEAR(y) Λ HOLDING(x)
• D: CLEAR(y) Λ HOLDING(x)
• A: ARMEMPTY Λ ON(x, y)
• UNSTACK(x, y)
• PICKUP(x)
• P: CLEAR(x) Λ ONTABLE(x) Λ ARMEMPTY
• D: ONTABLE(x) Λ ARMEMPTY
• A: HOLDING(x)
• PUTDOWN(x)
Goal Stack Planning
🞂 Goal Stack Planning is the one of the simplest planning algorithms that is designed to
handle problems which include compound goals.
🞂 It utilizes STRIPS as a formal language for specifying and manipulating the world with
which it is working.
🞂 This approach uses a Stack for plan generation. The stack can contain Sub-goal and
actions described using predicates. The Sub-goals can be solved one by one in any
order.
🞂 It starts by pushing the unsatisfied goals into the stack.
🞂 Then it pushes the individual sub-goals into the stack and its pops an element out of the
stack.
🞂 When popping an element out of the stack the element could be,
⮩ either a predicate describing a situation about our world or
⮩ it could be an action that can be applied to our world under consideration.
Goal Stack Planning
🞂 So, a decision has to be made based on the kind of element we are popping out from the
stack.
🞂 If it is a Predicate, then compares it with the description of the current world, if it is
satisfied or
is already present in our current situation then there is nothing to do because already its true.
🞂 On the contrary if the Predicate is not true then we have to select and push relevant
action
satisfying the predicate to the Stack.
🞂 So after pushing the relevant action into the stack its precondition should also has
to be pushed into the stack.
🞂 In order to apply an operation its precondition has to be satisfied, i.e., the present
situation of the world should be suitable enough to apply an operation.
🞂 For that, the preconditions are pushed into the stack once after an action is pushed.
Goal Stack Planning – Example
🞂 Lets start here with the BLOCK WORLD example, the initial state is our current
description of our world. The Goal state is what we have to achieve.

A C A

Initial State Goal State

ON(B, A)ΛONTABLE(C) ΛONTABLE(A) ON(C,A)ΛON(B,D) ONTABLE(A) ΛONTABLE(D)
ΛONTABLE(D) ΛCLEAR(C) ΛCLEAR(B) Λ ARMEMPTY
ΛCLEAR(D) ΛCLEAR(C) ΛCLEAR(B) Λ ARMEMPTY
Goal Stack Planning – Example
🞂 The following list of actions can be applied to the various situations in the problem.

OPERATORS PRECONDITION DELETE ADD

STACK(A, B): Place block CLEAR(B) Λ HOLDING (A) CLEAR(B) Λ HOLDING (A) ARMEMPTY Λ ON(A,B)
A
on block B.
UNSTACK(A, B): Pick up ON(A,B) Λ CLEAR(A) Λ ON(A,B) Λ ARMEMPTY HOLDING (A) Λ CLEAR(B)
block A from its current ARMEMPTY
position on block B.
PICKUP(A): Pick up block A CLEAR(A) Λ ONTABLE(A) Λ ONTABLE(A) Λ ARMEMPTY HOLDING (A)
from the table and hold it. ARMEMPTY

PUTDOWN(A): Put block A HOLDING (A) HOLDING (A) ONTABLE(A) Λ

down on the table. ARMEMPTY
Goal Stack Planning – Example
Nonlinear Planning using Constraint Posting
🞂 This planning is used to set a goal stack and is included in the search space of all
possible sub- goal orderings. It handles the goal interactions by interleaving method.
🞂 Many problems require an intertwined plan in which multiple sub-problems
worked on simultaneously.
🞂 Such a plan is called nonlinear plan because it is not composed of a linear
sequence of complete sub-plans.
🞂 Non-linear planning may be an optimal solution with respect to plan length (depending
on
search strategy used).
🞂 It takes larger search space, since all possible goal orderings are taken into consideration.
Constraint Posting
🞂 The idea of constraint posting is to build up a plan by incrementally hypothesizing
operators, partial orderings between operators, and binding of variables within operators.
🞂 Constraint posting often comes with Non-Linear Planning. The idea of constraint
posting is to build up a plan incrementally.
🞂 At any given time in the problem-solving process, we may have a set of useful operators
but perhaps no clear idea of how those operators should order with respect to each
other.
🞂 A solution is a partially ordered, partially instantiated set of operators to generate an
actual
plan. And we can convert the partial order into any number of total orders.
Algorithm: Nonlinear Planning
1. Choose a goal 'g' from the goalset
2. If 'g' does not match the state, then
⮩ Choose an operator 'o' whose add-list matches goal g
⮩ Push 'o' on the opstack
⮩ Add the preconditions of 'o' to the goalset
3. While all preconditions of operator on top of opstack are met in state
⮩ Pop operator o from top of opstack
⮩ state = apply(o, state)
⮩ plan = [plan; o]
Hierarchical Planning
🞂 In order to solve hard problems, a problem solver may have to generate long plans.
🞂 It is important to be able to eliminate some of the details of the problem until a solution
that
addresses the main issues is found.
🞂 Then an attempt can be made to fill in the appropriate details.
🞂 Early attempts to do this involved the use of macro operators, in which larger operators
were
built from smaller ones.
🞂 Lower level activities would detail more precise steps for accomplishing the higher level
tasks.
🞂 Instead of having to try out a large number of possible plan ordering, plan hierarchies
limit the ways in which an agent can select and order its primitive operators.
Hierarchical Planning
🞂 Hierarchical Planning describes Hierarchy of actions in terms of major action or minor
action.
🞂 Planning for ”Going to Goa this Christmas”
⮩ Major Steps :
▪ Hotel Booking
▪ Ticket Booking
▪ Reaching Goa
▪ Staying and enjoying there
▪ Coming Back
⮩ Minor Steps :
▪ Take a taxi to reach station / airport
▪ Have candle light dinner on beach
▪ Take photos
Hierarchical Planning
🞂 Hierarchical Planning Properties
⮩ Postpone attempts to solve mere details, until major steps are in place.
⮩ Identify a hierarchy of conditions
⮩ Construct a plan in levels, postponing details to the next level
⮩ Patch higher levels as details become visible
🞂 Hierarchy of conditions reflect the intrinsic difficulty of achieving various conditions.
Intrinsic
difficulty is indicated by criticality value.
🞂 A operation having minimum criticality can be trivially achievable, i.e., the operations
having very less or no precondition. For example : Opening makemytrip.com
🞂 Similarly operation having many preconditions to satisfy will have higher criticality.
🞂 The assignment of appropriate criticality value is crucial to the success of this
hierarchical
planning method.
🞂 Those preconditions that no operator can satisfy are clearly the most critical.
ABSTRIPS : Modified version of STRIPS that incorporates
hierarchical planning
🞂 A better approach is developed in ABSTRIPS systems which is actually planned in a
hierarchy of abstraction spaces, in each of which preconditions at a lower level of
abstraction are ignored.
🞂 ABSTRIPS (Abstraction-Based STRIPS) approach is as follows:
⮩ First solve the problem completely, considering only preconditions whose criticality value is the
highest
possible.
⮩ These values reflect the expected difficulty of satisfying the precondition.
⮩ To do this, do exactly what STRIPS did, but simply ignore the preconditions of lower than peak criticality.
⮩ Once this done, use the constructed plan as the outline of a complete plan and consider preconditions
at the
next-lowest criticality level.
⮩ Augment the plan with operators that satisfy those preconditions.
🞂 Patching in ABSTRIPS : Each level starts with the goal stack that includes the plan
obtained in the higher levels. The last item in the goal stack being the main goal.
🞂 This approach explores the entire plans at one level of detail before it looks at the lower-
level details of any one of them, so, it is called length-first approach.
Reactive Systems
🞂 The idea of reactive systems is to avoid planning altogether, and instead, use the
observable situation as a clue to which one can simply react.
🞂 A reactive system must have an access to a knowledge base of some sort that
describes what actions should be taken under what circumstances.
🞂 A reactive system is very different from the other kinds of planning systems we
have discussed. Because it chooses actions one at a time.
🞂 It does not anticipate and select an entire action sequence before it does the first thing.
🞂 The example is a Thermostat. The job of the thermostat is to keep the temperature
constant inside a room.
🞂 Reactive systems are capable of complex behaviors.
🞂 The main advantage reactive systems have over traditional planners is that they
operate robustly in domains that are difficult to model completely and accurately.
Reactive Systems
🞂 Reactive systems dispense with modeling altogether and base their actions directly on
their perception of the world.
🞂 Another advantage of reactive systems is that they are extremely responsive since they
avoid the combinatorial explosion involved in deliberative planning.
🞂 This makes them attractive for real-time tasks such as driving and walking.
Other Planning Techniques
🞂 Triangle tables
⮩ Provides a way of recording the goals that each operator expected to
satisfy as well as the goals that must
be true for it to execute correctly.
🞂 Meta-planning
⮩ A technique for reasoning not just about the problem solved but also about
the planning process itself.
🞂 Macro-operators
⮩ Allow a planner to build new operators that represent commonly used
sequences of operators.
🞂 Case-based planning:
⮩ Re-uses old plans to make new ones.

• Reproduce an important aspect of 3

intelligent behaviour.
What is learning
• Denotes changes in the system that enable a system to do the same task more
efficiently next time.

• Learning is constructing or modifying representations of what is being

experienced.

• Learning is making useful changes in our minds.

• Learning improves understanding and efficiency

• Discover new things or structures which were previously unknown (data mining,
scientific discovery)
• Reproduce an important aspect of 3
intelligent behaviour.
• Fill in skeletal or incomplete observations or specifications about a domain (this
expands the domain of expertise and lessens the brittleness of the system)

• Build software agents that can adapt to their users or to other software agents

• Reproduce an important aspect of 3

intelligent behaviour.
Learning Systems
• Machine learning systems perform the following iteratively:
➢ Produce a result
➢ Evaluate it against expected result
➢ Tweak a system

• Machine learning systems also discover patterns without prior expected results

• Open box: changes are clearly visible in the knowledge base and clearly
interpretable by the human users.

• Black box: changes done to the system are not readily visible or understandable.

4
Learner Architecture
• Machine learning systems has the four main components:
➢ Knowledge Base (KB):
✓ what is being learnt
✓ Representation of domain
✓ Description and representation of problem space

➢ Performer: does something with the knowledge base to produce results.

➢ Critic: evaluates results produced against expected results

➢ Learner: takes output from the critic and modifies something in the KB or the
performer.

➢ They may also need a problem generator to test performance against.

5
Learning Agent Architecture

6
Learning Examples
Problem Representation Performer (interacts Critic (human player) Learner(elicits new
with human) questions to modify
KBP

Animal guessing Binary decision tree Walk the tree and ask Human feedback Elicit a question from
game associated questions the user and add it to
the binary tree
Playing chess The board layout, game Chain through the rules Who won (credit Increase the weight for
rules, moves to identify move, use assignment problem) some rules and
conflict resolution to decrease for others.
choose one, output the
move.
Categorizing Vector of word frequencies, Apply appropriate A set of human- Modify the weights on
documents corpus of documents functions to identify categorized documents the function and
which category the file improve categorization
belongs to
Fixing computers Frequency matrix of causes Use known symptoms Human input about Update the frequency
and symptoms to identify potential symptoms and cause matrix with actual
causes. observed for a specific symptoms and
case outcomes
Identifying digits in Probability of digits, matrix Input the features for a Human categorized Modify the weights on
Optical Character of pixels, percentage of digit, output probability training set the network of
Recognition light, no: of straight lines that it is one in the set associations
from 0 to 9.

7
Learning Paradigms
Paradigm Description

Rote learning Knowledge engineering: direct entry of rules and facts

Learning by taking Human/system interaction producing explicit mapping
advice
Learning in problem Parameter adjustments
solving
Learning from Using specific examples to reach general conclusions
examples: induction
Explanation based Learn from a single example and later generalize. More analytical and
learning knowledge intensive approach.
Learning through Unsupervised, specific goal not given
discovery
Learning through Determining correspondence between two different representations:
analogy case-based reasoning
Formal learning theory Formal mathematical model of learning
Neural net learning and Evolutionary search techniques, based on an analogy to the “survival
genetic learning of the fittest” 8
Rote Learning
• Rote learning is the basic learning activity.

• It is also called memorization because the knowledge, without any modification is,
simply copied into the knowledge base. Direct entry of rules and facts

• Knowledge base is captured knowledge.

• Traditional approach to develop ontologies.

• Data caching to improve performance.

• As computed values are stored, this technique can save a significant amount of time.

• Rote learning technique can also be used in complex learning systems provided
sophisticated techniques are employed to use the stored values faster and there is a
generalization to keep the number of stored information down to a manageable level.
positions it evaluates in its 9
• Checkers-playing program, for example, uses this technique to learn the board

positions it evaluates in its 9

Rote Learning
• Depends on two important capabilities of complex learning systems:

➢ Organized storage of information: need sophisticated techniques for data

retrieval. It will be much faster than recomputing the
data.

➢ Generalization: The number of distinct objects that might potentially be stored

can be very large. To keep the number of stored objects down to a manageable
levels.

10
 Learning by taking advice
• This type is the easiest and simple way of learning.

• In this type of learning, a programmer writes a program to give some instructions to

perform a task to the computer. Once it is learned (i.e. programmed), the system will
be able to do new things.

• Also, there can be several sources for taking advice such as humans(experts),
internet etc.

• However, this type of learning has a more necessity of inference than rote learning.

• As the stored knowledge in knowledge base gets transformed into an operational

form, the reliability of the knowledge source is always taken
into consideration.

• The programs shall operationalize the advice by turning it into a single or multiple
expressions that contain concepts and actions that the program can use while under
 execution.

• This ability to operationalize knowledge is very critical for learning. This is also a11n
important aspect of Explanation Based Learning (EBL).
Learning in Problem Solving
• When the program does not learn from advice, it can learn by generalizing from its
own experiences.

➢ Learning by parameter adjustment

➢ Learning with macro-operators

➢ Learning by chunking

➢ The unity problem

12
 Learning in Problem Solving
Learning by parameter adjustment

• Here the learning system relies on evaluation procedure that combines information from
several sources into a single summary static.

• For example, the factors such as demand and production capacity may be combined into a
single score to indicate the chance for increase of production.

• But it is difficult to know a priori how much weight should be attached to each factor.

• The correct weight can be found by taking some estimate of the correct settings and then allow
the program modify its settings based on its experience.

• Features that appear to be good predictors of overall success will have their weights
increases, while those that do not will have their weights decreased.

• This type of learning systems is useful when little knowledge is available.

• In game programs, for example, the factors such as piece advantage and mobility are
combined into a single score to decide whether a particular board position is desirable. This
single score is nothing but a knowledge which the program gathered by means of calculation.
13
 Learning in Problem Solving
Learning by parameter adjustment

• Programs do this in their static evaluation functions, in which a variety of factors

are combined into a single score. This function as a polynomial form is given
below:

• The t terms are the values of the features that contribute to the evaluation. The
c terms are the coefficients or weights that are attached to each of these
values. As learning progresses, the c values will change.

• In designing programs it is often difficult to decide on the exact value to give

each weight initially. So the basic idea of idea of parameter
adjustment is to:

increased and bad ones will 14

be decreased.
➢ Start with some estimate of the correct weight settings.
➢ Modify the weight in the program on the basis of accumulated experiences.
➢ Features that appear to be good predictors will have their weights

increased and bad ones will 15

be decreased.
Learning in Problem Solving
Learning by parameter adjustment

• Important factors that affect the performance are:

➢ When should the value of a coefficient be increased and when should it be
decreased
✓ The coefficients of terms that predicted the final outcome accurately
should be increased, while the coefficients of poor predictors should
be decreased.
✓ The problem of appropriately assigning responsibility to each of the
steps that led to a single outcome is known as credit assignment
system.
➢ By how much should be value be changed.
• Learning procedure is a variety of hill-climbing.

• This method is very useful in situations where very little additional knowledge is
available or in programs in which it is combined with more knowledge intensive
methods.
16
Learning in Problem Solving
Learning with Macro-Operators

• Sequence of actions that can be treated as a whole are called macro-operators.

• Once a problem is solved, the learning component takes the computed plan and stores it
as a macro-operator.
• The preconditions are the initial conditions of the problem just solved, and its post
conditions correspond to the goal just achieved.
• The problem solver efficiently uses the knowledge base it gained from its previous
experiences.
• By generalizing macro-operators the problem solver can even solve different problems.
Generalization is done by replacing all the constants in the macro-operators with variables.
• The STRIPS, for example, is a planning algorithm that employed macro-operators in it’s
learning phase. It builds a macro operator MACROP, that contains preconditions, post-
conditions and the sequence of actions. The macro operator will be used in the future
operation.
• The set of problems for which macro-operators are critical are exactly those problems with
non-serializable sub goals.(working on one subgoal will necessarily interfere with the
previous solution to another subgoal).
• One macro operator can produce a small global change in the world, even though the
individual operators that make it up produce many undesirable local changes.

• Domain specific knowledge we need can be learnt in the form of macro operators. 17
Learning in Problem Solving
Learning by chunking

• Chunking is similar to learnig with macro-operators. Generally, it is used by problem solver

systems that make use of production systems.

• A production system consists of a set of rules that are in if-then form. That is given a
particular situation, what are the actions to be performed. For example, if it is raining then
take umbrella.

• Production system also contains knowledge base, control strategy and a rule applier. To
solve a problem, a system will compare the present situation with the left hand side of the
rules. If there is a match then the system will perform the actions described in the right hand
side of the corresponding rule.

• Problem solvers solve problems by applying the rules. Some of these rules may be more
useful than others and the results are stored as a chunk.

• Chunking can be used to learn general search control knowledge.

• Several chunks may encode a single macro-operator and one chunk may participate
in a number of macro sequences.
18
Learning in Problem Solving
Learning by chunking

• Chunks learned in the beginning of problem solving, may be used in the later stage. The
system keeps the chunk to use it in solving other problems.

• Soar is a general cognitive architecture for developing intelligent systems. Soar requires
knowledge to solve various problems. It acquires knowledge using chunking mechanism.
The system learns reflexively when impasses have been resolved. An impasse arises when
the system does not have sufficient knowledge. Consequently, Soar chooses a new
problem space (set of states and the operators that manipulate the states) in a bid to
resolve the impasse. While resolving the impasse, the individual steps of the task plan are
grouped into larger steps known as chunks. The chunks decrease the problem space
search and so increase the efficiency of performing the task.

• In Soar, the knowledge is stored in long-term memory. Soar uses the chunking mechanism
to create productions that are stored in long-term memory. A chunk is nothing but a large
production that does the work of an entire sequence of smaller ones. The productions have
a set of conditions or patterns to be matched to working memory which consists of current
goals, problem spaces, states and operators and a set of actions to perform when the
production fires. Chunks are generalized before storing. When the same impasse occurs
again, the chunks so collected can be used to resolve it.
19
Learning in Problem Solving
The Utility Problem

• The utility problem in learning systems occurs when knowledge learned in an attempt to
improve a system's performance degrades it instead.
• The problem appears in many AI systems, but it is most familiar in speedup learning.
Speedup learning systems are designed to improve their performance by learning control
rules which guide their problem-solving performance. These systems often exhibit the
undesirable property of actually slowing down if they are allowed to learn in an unrestricted
fashion.
• Each individual control rule is guaranteed to have a positive utility (improve performance)
but, in concert, they have a negative utility (degrade performance).
• One of the causes of the utility problem is the serial nature of current hardware. The more
control rules that speedup learning systems acquire, the longer it takes for the system to
test them on each cycle.
• One solution to the utility problem is to design a parallel memory system to eliminate the
increase in match cost. his approach moves the matching problem away from the central
processor and into the memory of the system. These so-called active memories allow
memory search to occur in "nearly constant-time" in the number of data items, relying on
the memory for fast, simple inference and reminding.

20
Learning in Problem Solving
The Utility Problem

• PRODIGY program maintains a utility measure for each control rule. This measure takes
into account the average savings provided by the rule, the frequency of its application and
the cost of matching it.

• If a proposed rule has a negative utility, it is discarded or forgotten.

• If not, it is placed in long term memory with the other rules. It is then monitored during
subsequent problem solving.
• If its utility falls, the rule I discarded.

• Empirical experiments have demonstrated the effectiveness of keeping only those control
rules with high utility.

• Such utility considerations apply to a wide range of learning problems.

21
Learning by Analogy

Learning by analogy means acquiring new knowledge about

an input entity by transferring it from a known similar entity.

Qa=3 Qb=9 I1 I2

I3=I1+I2
Qc=?

Simple Hydraulics Problem Kirchoff's First Law

One may infer, by analogy, that hydraulics laws are similar to Kirchoff's
laws, and Ohm's law.

22
Learning by Analogy

Central intuition supporting learning by analogy:

If two entities are similar in some respects then they
could be similar in other respects as well.

Examples of analogies:

Pressure Drop is like Voltage Drop

A variable in a programming language is like a box.

23
Transformational Analogy
Look for a similar solution and copy it to the
new situation making suitable substitutions
where appropriate.

E.g. Geometry.
If you know about lengths of line
segments and a proof that
certain lines are equal then we
can make similar assertions
about angles.

We know that lines RO = NY and

angles AOB = COD

We have seen that RO + ON = ON + NY -

additive rule.
So we can say that
angles AOB + BOC = BOC + COD

So by a transitive rule line RN = OY 23

So similarly angle AOC = BOD
Derivational Analogy
Two problems share significant aspects if they match within a certain
threshold, according to a given similarity metric.

The solution to the retrieved problem is perturbed incrementally until it

satisfies the requirements of the new problem.

Transformational analogy does not look at how the problem was solved -- it
only looks at the final solution. The history of the problem solution - the steps
involved - are often relevant.

New derivation Old derivation

24
Derivational Analogy
GIVEN: AB = CD GIVEN: <BAC = <DAE

PROVE: AC = BD  = ( AB <- <BAC PROVE: <BAD = <CAE

D CD < - <DAE B C
AC <- <BAD
C BD < - <CAE )
B D
A
A E

AB = CD <BAC = <DAE
BC = BC <CAD = <CAD
AB + BC = BC + CD <BAC + <CAD= <CAD + <DAE
AC = BD <BAD = <CAE

25
Explanation based Learning

26
Explanation based Learning

27
Explanation based Learning

28
Explanation based Learning

29
Explanation based Learning

30
Explanation based Learning

31
Learning by Discovery
An entity acquires knowledge without the help of a teacher.

Theory Driven Discovery - AM (1976)

AM is a program that discovers concepts in elementary
mathematics and set theory.
AM has 2 inputs:
• A description of some concepts of set theory (in LISP form). E.g. set union,
intersection, the empty set.
• Information on how to perform mathematics. E.g. functions.

How does AM work?

AM employs many general-purpose AI techniques:
• A frame based representation of mathematical concepts.
AM can create new concepts (slots) and fill in their
values.
• Heuristic search employed
250 heuristics represent hints about activities that might
 lead to interesting
discoveries.
How to employ functions, create new concepts,
generalisation etc.
• Hypothesis and test based search. 32
• Agenda control of discovery process.
 Learning by Discovery
AM discovered:

• Integers-- it is possible to count the elements of this set and this is an the
image of this counting function -- the integers --
interesting set in its own right.

• Addition-- The union of two disjoint sets and their counting function

• Multiplication-- Having discovered addition and multiplication as laborious

set-theoretic operations more effective descriptions were
supplied by hand.

• Prime Numbers-- factorisation of numbers and numbers with only one factor
were discovered.

• Golbach's Conjecture-- Even numbers can be written as the sum of 2

primes. E.g. 28 = 17 + 11.
• Maximally Divisible Numbers-- numbers with as many factors as possible. A
number k is maximally divisible is k has more factors than
any integer less
than k. E.g. 12 has six divisors 1,2,3,4,6,12. 33
 Learning by Discovery
Data Driven Discovery -- BACON (1981)

Many discoveries are made from observing data obtained

from the world and making sense of it -- E.g. Astrophysics -
discovery of planets, Quantum mechanics - discovery of sub-
atomic particles.

BACON is an attempt at providing such an AI system. BACON

system outline:
• Starts with a set of variables for a problem.
➢ E.g. BACON was able able to derive the ideal gas law. It started with four
variables p - gas pressure, V -- gas volume, n -- molar mass of gas, T -- gas
temperature. Recall pV/nT = k where k is a constant.
• Values from experimental data from the problem are inputted.

• BACON holds some constant and attempts to notice trends in the data.
• Inferences made.

BACON has also been applied to Kepler's 3rd law, Ohm's law,
conservation of 34
momentum and Joule's law.
 Learning by Discovery
Clustering

• Clustering involves grouping data into several new classes.

• It is a common descriptive task where one seeks to identify a finite set of categories
or clusters to describe the data. For example, we may want to cluster houses to find
distribution patterns.

• Clustering is the process of grouping a set of physical or abstract objects into

classes of similar objects.

• A cluster is a collection of data objects that are similar to one another within the same
cluster and are dissimilar to the objects in other clusters. Clustering analysis helps
construct meaningful partitioning of a large set of objects.

35
 Learning by Discovery
Clustering

The task of clustering is to maximize the intra-class similarity

and minimize the interclass
similarity.
• Given N k-dimensional feature vectors, find a "meaningful" partition of the N
examples into c subsets or groups
• Discover the "labels" automatically
• c may be given, or "discovered“
• much more difficult than classification, since in the latter the groups are given, and we
seek a compact description

36
 Learning by Discovery
AutoClass
• AutoClass is a clustering algorithm based upon the Bayesian approach for
determining optimal classes in large datasets.
·
• Given a set X={X1, …, Xn} of data instances Xi with unknown classes, the goal of
Bayesian classification is to search for the best class description that predicts the
data in a model space.
·
• Class membership is expressed probabilistically.

• An instance is not assigned to a unique class, but it has a probability (expressed as

weight values) of belonging to each of the possible classes.

• AutoClass calculates the likelihood of each instance belonging to each class C and
then calculates a set of weights wij=(Ci / SjCj) for each instance.

• Weighted statistics relevant to each term of the class likelihood are calculated for

of every instance for each class and computes the

 estimating the class model.
• The classification step is the most computationally intensive. It computes the we3i7ghts

of every instance for each class and computes the

Formal Learning
Formal learning theory

• Theory of the learnable by Valiant: classifies problems by how difficult they are to learn.

• Formally, a device can learn a concept if it can, given positive and negative examples,
produce an algorithm that will classify future examples correctly with probability 1/h.

• Complexity of learning a function is decided by three factors:

➢ The error tolerance (h)
➢ The number of binary features present in the example (t)
➢ Size of rules necessary to make the discrimination (f)

• If the number of training examples is a polynomial in h,t, f, then the system is said to be
trainable.

• Example: learning feature descriptions

• Mathematical theory will be used to quantify the use of knowledge

38
Formal Learning

39
Other Learning Models
Neural net learning and genetic learning

• Neural networks

• Population genetics and selection

40
Learning in Problem Solving
Neural net learning and genetic learning

• Neural networks

• Population genetics and selection

41
What is learning

42
General Learning Model.
General Learning Model: - AS noted earlier, learning can be accomplished using a number of
different methods, such as by memorization facts, by being told, or by studying examples like
problem solution. Learning requires that new knowledge structures be created from some form of
input stimulus. This new knowledge must then be assimilated into a knowledge base and be
tested in some way for its utility. Testing means that the knowledge should be used in
performance of some task from which meaningful feedback can be obtained, where the feedback
provides some measure of the accuracy and usefulness of the newly acquired knowledge.
General Learning Model

general learning model is depicted in figure 4.1 where the environment has been included as a
part of the overall learner system. The environment may be regarded as either a form of nature
which produces random stimuli or as a more organized training source such as a teacher which
provides carefully selected training examples for the learner component. The actual form of
environment used will depend on the particular learning paradigm. In any case, some
representation language must be assumed for communication between the environment and the
learner. The language may be the same representation scheme as that used in the knowledge base
(such as a form of predicate calculus). When they are hosen to be the same, we say the single
representation trick is being used. This usually results in a simpler implementation since it is not
necessary to transform between two or more different representations.
For some systems the environment may be a user working at a keyboard . Other systems will use
program modules to simulate a particular environment. In even more realistic cases the system
will have real physical sensors which interface with some world environment.

Inputs to the learner component may be physical stimuli of some type or descriptive , symbolic
training examples. The information conveyed to the learner component is used to create and
modify knowledge structures in the knowledge base. This same knowledge is used by the
performance component to carry out some tasks, such as solving a problem playing a game, or
classifying instances of some concept.

given a task, the performance component produces a response describing its action in
performing the task. The critic module then evaluates this response relative to an optimal
response.

Feedback , indicating whether or not the performance was acceptable , is then sent by the critic
module to the learner component for its subsequent use in modifying the structures in the
knowledge base. If proper learning was accomplished, the system’s performance will have
improved with the changes made to the knowledge base.

The cycle described above may be repeated a number of times until the performance of the
system has reached some acceptable level, until a known learning goal has been reached, or until
changes ceases to occur in the knowledge base after some chosen number of training examples
have been observed.

There are several important factors which influence a system’s ability to learn in addition to the
form of representation used. They include the types of training provided, the form and extent of
any initial background knowledge , the type of feedback provided, and the learning algorithms
used.

The type of training used in a system can have a strong effect on performance, much the same as
it does for humans. Training may consist of randomly selected instance or examples that have
been carefully selected and ordered for presentation. The instances may be positive examples of
some concept or task a being learned, they may be negative, or they may be mixture of both
positive and negative. The instances may be well focused using only relevant information, or
they may contain a variety of facts and details including irrelevant data.

There are Many forms of learning can be characterized as a search through a space of possible
hypotheses or solutions. To make learning more efficient. It is necessary to constrain this search
process or reduce the search space. One method of achieving this is through the use of
background knowledge which can be used to constrain the search space or exercise control
operations which limit the search process.

Feedback is essential to the learner component since otherwise it would never know if the
knowledge structures in the knowledge base were improving or if they were adequate for the
performance of the given tasks. The feedback may be a simple yes or no type of evaluation, or it
may contain more useful information describing why a particular action was good or bad. Also ,
the feedback may be completely reliable, providing an accurate assessment of the performance or
it may contain noise, that is the feedback may actually be incorrect some of the time. Intuitively ,
the feedback must be accurate more than 50% of the time; otherwise the system carries useful
information, the learner should also to build up a useful corpus of knowledge quickly. On the
other hand, if the feedback is noisy or unreliable, the learning process may be very slow and the
resultant knowledge incorrect.

Learning Neural Network

Perceptron
• The perceptron an invention of (1962) Rosenblatt was one of the earliest neural network
models.
• Also, It models a neuron by taking a weighted sum of its inputs and sending the output 1
if the sum is greater than some adjustable threshold value (otherwise it sends 0).

Figure: A neuron & a Perceptron

Figure: Perceptron with adjustable threshold

• In case of zero with two inputs g(x) = w0 + w1x1 + w2x2 = 0
 • x2 = -(w1/w2)x1 – (w0/w2) → equation for a line
• the location of the line is determined by the weight w0 w1 and w2
• if an input vector lies on one side of the line, the perceptron will output 1
• if it lies on the other side, the perception will output 0
• Moreover, Decision surface: a line that correctly separates the training instances
corresponds to a perfectly function perceptron.
Perceptron Learning Algorithm
Given: A classification problem with n input feature (x1, x2, …., xn) and two output classes.
Compute A set of weights (w0, w1, w2,….,wn) that will cause a perceptron to fire whenever the
input falls into the first output class.
1. Create a perceptron with n+ 1 input and n+ 1 weight, where the x0 is always set to 1.
2. Initialize the weights (w0, w1,…., wn) to random real values.
3. Iterate through the training set, collecting all examples misclassified by the current set of
weights.
4. If all examples are classified correctly, output the weights and quit.
5. Otherwise, compute the vector sum S of the misclassified input vectors where each vector
has the form (x0, x1, …, Xn). In creating the sum, add to S a vector x if x is an input for
which the perceptron incorrectly fails to fire, but – x if x is an input for which the
perceptron incorrectly fires. Multiply sum by a scale factor η.
6. Moreover, Modify the weights (w0, w1, …, wn) by adding the elements of the vector S to
them.
7. Go to step 3.
• The perceptron learning algorithm is a search algorithm. It begins with a random initial
state and finds a solution state. The search space is simply all possible assignments of real
values to the weights of the perception, and the search strategy is gradient descent.
• The perceptron learning rule is guaranteed to converge to a solution in a finite number of
steps, so long as a solution exists.
• Moreover, This brings us to an important question. What problems can a perceptron
solve? Recall that a single-neuron perceptron is able to divide the input space into two
regions.
• Also, The perception can be used to classify input vectors that can be separated by a
linear boundary. We call such vectors linearly separable.
• Unfortunately, many problems are not linearly separable. The classic example is the XOR
gate. It was the inability of the basic perceptron to solve such simple problems that are
not linearly separable or non-linear.
Genetic Learning
Supervised Learning
Supervised learning is the machine learning task of inferring a function from labeled training
data.
Moreover, The training data consist of a set of training examples.
In supervised learning, each example a pair consisting of an input object (typically a vector) and
the desired output value (also called the supervisory signal).
Training set
A training set a set of data used in various areas of information science to discover potentially
predictive relationships.
Training sets used in artificial intelligence, machine learning, genetic programming, intelligent systems, and statistics.
In all these fields, a training set has much the same role and often used in conjunction with a test set.
Testing set
A test set is a set of data used in various areas of information science to assess the strength and utility of a predictive relationship.
Moreover, Test sets are used in artificial intelligence, machine learning, genetic programming, and statistics. In all these fields, a test set has much the same role.
Accuracy of classifier: Supervised learning
In the fields of science, engineering, industry, and statistics. The accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity’s actual
(true) value.
Sensitivity analysis: Supervised learning
Similarly, Local Sensitivity as correlation coefficients and partial derivatives can only use, if the correlation between input and output is linear.
Regression: Supervised learning
In statistics, regression analysis is a statistical process for estimating the relationships among variables.
Moreover, It includes many techniques for modeling and analyzing several variables. When the focus on the relationship between a dependent variable and one or more independent
variables. More specifically, regression analysis helps one understand how the typical value of the dependent variable (or ‘criterion variable’) changes when any one of the
independent variables varied. Moreover, While the other independent variables held fixed.

47
48
49
50