Artificial Intelligence

BCS515B
According to VTU Syllabus

Module 1
Introduction,
Intelligent Agents

November 6, 2024 1
 Course Outcomes (Course Skill Set)
At the end of the course, the student will be able to:
CO1. Explain the architecture and components of intelligent
agents, including their interaction with the AI environment.
CO2. Apply problem-solving agents and various search strategies
to solve a given problem.
CO3. Illustrate logical reasoning and knowledge representation
using propositional and first-order logic.
CO4. Demonstrate proficiency in representing knowledge and
solving problems using first-order logic.
CO5. Describe classical planning in the context of artificial
intelligence, including its goals, constraints, and applications
in problem-solving.

November 6, 2024 2
 Assessment Details (both CIE and SEE)
• The weightage of Continuous Internal Evaluation (CIE) is 50%
and for Semester End Exam (SEE) is 50%. The minimum
passing mark for the CIE is 40% of the maximum marks (20
marks out of 50) and for the SEE minimum passing mark is
35% of the maximum marks (18 out of 50 marks)
• For the Assignment component of the CIE, there are 25 marks
and for the Internal Assessment Test component, there are 25
marks. (duration 01 hour)
• Any two assignment methods mentioned in the 22OB2.4, if an
assignment is project-based then only one assignment for the
course shall be planned.
• For the course, CIE marks will be based on a scaled-down sum
of two tests and other methods of assessment.
November 6, 2024 3
November 6, 2024 4
 WHAT IS AI?
Thinking Humanly Thinking Rationally
“The exciting new effort to make computers “The study of mental faculties through the
think . . . machines with minds, in the use of computational models.”
full and literal sense.” (Haugeland, 1985) (Charniak and McDermott, 1985)
“[The automation of] activities that we The study of the computations that make
it possible to perceive, reason, and act.”
associate with human thinking, activities (Winston, 1992)
such as decision-making, problem solving,
learning . . .” (Bellman, 1978)

Acting Humanly Acting Rationally

“The art of creating machines that perform “Computational Intelligence is the study
functions that require intelligence of the design of intelligent agents.” (Poole
when performed by people.” (Kurzweil, et al., 1998)
1990)
“The study of how to make computers do “AI . . . is concerned with intelligent
things at which, at the moment, people are behavior in artifacts.” (Nilsson, 1998)
better.” (Rich and Knight, 1991)

Figure 1.1 Some definitions of artificial intelligence, organized into four categories.
System that think like human System that think rationally

November 6, 2024
System that act like human System that act rationally 5
Acting Humanly.
• Turing Test proposed by Alan Turing (1950) was designed as a
thought experiment.
• A computer passes the test if human interrogator, after posing
some questions, cannot tell whether the written responses come
from a person or from computer.

November 6, 2024 6
• The computer would need the following capabilities.
– natural language processing: to communicate in a human
language
– knowledge representation: to store what it knows or hears
– automated reasoning: to answer questions & to draw
conclusions
– machine learning: to adapt to new circumstances and to detect
& extrapolate patterns
• A total Turing test which requires interaction with objects
and people in the real world. To pass the total Turing test,
a robot will need
– Computer vision and speech recognition to perceive the world
– robotics to manipulate objects and move about

These six disciplines compose most of AI.

November 6, 2024 7
Thinking humanly: cognitive modeling
• Cognitive Science - must figure out how human's think. We can learn
about human thought in 3 ways.
– Introspection: trying to catch our own thoughts as they go by
– Psychological experiments: observing a person in action
– Brain imaging: observing the brain action
• If the program’s input-output behavior matches corresponding
human behavior, that is evidence that some of the program’s
mechanism could also be operating in humans.
Example
• Allen Newell and Herbert Simon who developed GPS ( General
Problem Solver ) were not content to have their program solve the
problem correctly.
• They were more concerned with comparing the sequence and timing
of its reasoning steps to those of human subjects solving the same
problems.
November 6, 2024 8
Thinking Rationally. Laws of thought
• The Greek philosopher Aristotle first attempted to codify the right
thinking ( irrefutable reasoning process): what are correct
arguments/thought processes?
• His syllogism provided patterns for argument structures that always
yielded correct conclusions when given correct premises.
Canonical example
• Socrates is a man and all men are mortal and conclude Socrates is
mortal.
• These laws of thought govern the operation of mind: their study initiated
the field called logic.
• In principle programs could solve any solvable problem described in
logical notation.
• This Logicist tradition within AI gave hope to build intelligent systems.
• Logic requires knowledge to understand. The theory of probability
provides the knowledge with uncertain information.
• The theory of probability leads from raw perceptual information to an
understanding of how the world works to predictions about the future.
November 6, 2024 9
Acting rationally: rational agent
• Correct thinking is good but:-
– Sometimes you must do something and there is no
provably correct thing to do
– Sometimes you must react quicker without time for
reasoning
Rational behavior: doing the right thing
– The right thing: that which is expected to maximize
goal achievement, given the available information
– Doesn't necessarily involve thinking
• e.g., blinking reflex - but thinking should be in
the service of rational action
November 6, 2024 10
Rational agents
• An agent is an entity that perceives and acts
• This course is about designing rational agents
• Abstractly, an agent is a function from percept
histories to actions:
[f: P*→ A]
• computational limitations make perfect
rationality unachievable
– design best program for given machine resource
• For any given class of environments and tasks, we
seek the agent (or class of agents) with the best
performance
November 6, 2024 11
 The State of the Art
• What can AI do today? A concise answer is difficult because
there are so many activities in so many subfields.
1. Robotic vehicles: A driverless robotic car named STANLEY
sped through the rough terrain of the Mojave dessert at 22
mph, finishing the 132-mile course first to win the 2005
DARPA Grand Challenge.
• STANLEY is a Volkswagen Touareg outfitted with cameras,
radar, and laser rangefinders to sense the environment and
onboard software to command the steering, braking, and
acceleration.

November 6, 2024 12
2. Speech recognition: A traveler calling United Airlines to book
a flight can have the entire conversation guided by an
automated speech recognition and dialog management
system.
3. Autonomous planning and scheduling: A hundred million
miles from Earth, NASA’s Remote Agent program became the
first on-board autonomous planning program to control the
scheduling of operations for a spacecraft (Jonsson et al.,
2000).
• REMOTE AGENT generated plans from high-level goals
specified from the ground and monitored the execution of
those plans—detecting, diagnosing, and recovering from
problems as they occurred.

November 6, 2024 13
4. Game playing: IBM’s DEEP BLUE became the first computer program to
defeat the world champion in a chess match when it bested Garry Kasparov
by a score of 3.5 to 2.5 in an exhibition match (Goodman and Keene, 1997).
• Kasparov said that he felt a “new kind of intelligence” across the board
from him.
• Newsweek magazine described the match as “The brain’s last stand.” The
value of IBM’s stock increased by $18 billion.
5. Spam fighting: Each day, learning algorithms classify over a billion messages
as spam, saving the recipient from having to waste time deleting what, for
many users, could comprise 80% or 90% of all messages, if not classified
away by algorithms.
6. Logistics planning: During the Persian Gulf crisis of 1991, U.S. forces
deployed a Dynamic Analysis and Replanning Tool, DART (Cross and Walker,
1994), to do automated logistics planning and scheduling for
transportation.
• This involved up to 50,000 vehicles, cargo, and people at a time, and had to
account for starting points, destinations, routes, and conflict resolution
among all parameters.
November 6, 2024 14
• The AI planning techniques generated in hours a plan that
would have taken weeks with older methods.
• The Defense Advanced Research Project Agency (DARPA)
stated that this single application more than paid back
DARPA’s 30-year investment in AI.

7. Robotics: The iRobot Corporation has sold over two million

Roomba robotic vacuum cleaners for home use.
• The company also deploys the more rugged PackBot to Iraq
and Afghanistan, where it is used to handle hazardous
materials, clear explosives, and identify the location of
snipers.

November 6, 2024 15
8. Machine Translation: A computer program automatically
translates from Arabic to English, allowing an English speaker
to see the headline “Ardogan Confirms That Turkey Would Not
Accept Any Pressure, Urging Them to Recognize Cyprus.”
• The program uses a statistical model built from examples of
Arabic-to-English translations and from examples of English
text totaling two trillion words (Brants et al., 2007).
• None of the computer scientists on the team speak Arabic,
but they do understand statistics and machine learning
algorithms.

November 6, 2024 16
 Intelligent Agents

November 6, 2024 17
 AGENTS AND ENVIRONMENTS
• An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that environment
through actuators. This simple idea is illustrated in Figure 2.1.

November 6, 2024 18
• A human agent has eyes, ears, and other organs for sensors
and hands, legs, vocal tract, and so on for actuators.
• A robotic agent might have cameras and infrared range
finders for sensors and various motors for actuators.
• A software agent receives keystrokes, file contents, and
network packets as sensory inputs and acts on the
environment by displaying on the screen, writing files, and
sending network packets.
• We use the term percept to refer to the agent’s perceptual
inputs at any given instant. An agent’s percept sequence is the
complete history of everything the agent has ever perceived.
• We say that an agent’s behavior is described by the agent
function that maps any given percept sequence to an action.
November 6, 2024 19
• We can imagine tabulating the agent function that describes
any given agent; for most agents, this would be a very large
table—infinite, in fact, unless we place a bound on the length
of percept sequences we want to consider.
• The table is, of course, an external characterization of the
agent. Internally, the agent function for an artificial agent will
be implemented by an agent program.
• Example—the vacuum-cleaner world shown in Figure 2.2.
• This particular world has just two locations: squares A and B.
• The vacuum agent perceives which square it is in and whether
there is dirt in the square.

November 6, 2024 20
• It can choose to move left, move right, suck up the dirt, or do
nothing. One very simple agent function is the following: if
the current square is dirty, then suck; otherwise, move to the
other square.
• A partial tabulation of this agent function is shown in Figure 2.3

November 6, 2024 21
November 6, 2024 22
 GOOD BEHAVIOR: THE CONCEPT OF
RATIONALITY

• A rational agent is one that does the right thing—conceptually

speaking, every entry in the table for the agent function is
filled out correctly.
• When an agent is plunked down in an environment, it
generates a sequence of actions according to the percepts it
receives.
• This sequence of actions causes the environment to go
through a sequence of states.
• If the sequence is desirable, then the agent has performed
well. This notion of desirability is captured by a performance
measure that evaluates any given sequence of environment
states.
November 6, 2024 23
Rationality
• What is rational at any given time depends on four things:
• The performance measure that defines the criterion of success.
• The agent’s prior knowledge of the environment.
• The actions that the agent can perform.
• The agent’s percept sequence to date.

This leads to a definition of a rational agent:

• For each possible percept sequence, a rational agent
should select an action that is expected to maximize its
performance measure, given the evidence provided by the
percept sequence and whatever built-in knowledge the
agent has.

November 6, 2024 24
Let us assume the following:
• The performance measure awards one point for each clean
square at each time step, over a “lifetime” of 1000 time steps.
• The “geography” of the environment is known a priori (Figure 2.2)
but the dirt distribution and the initial location of the agent are
not. Clean squares stay clean and sucking cleans the current
square. The Left and Right actions move the agent left and right
except when this would take the agent outside the environment,
in which case the agent remains where it is.
• The only available actions are Left , Right, and Suck.
• The agent correctly perceives its location and whether that
location contains dirt.

We claim that under these circumstances the agent is indeed

rational; its expected performance is at least as high as any other
agent’s.
November 6, 2024 25
Omniscience, learning, and autonomy
• We need to be careful to distinguish between rationality and
omniscience.
• An omniscient agent knows the actual outcome of its actions
and can act accordingly; but omniscience is impossible in
reality.
• Our definition requires a rational agent not only to gather
information but also to learn as much as possible from what it
perceives.
• The agent’s initial configuration could reflect some prior
knowledge of the environment, but as the agent gains
experience this may be modified and augmented.

November 6, 2024 26
Specifying the task environment
• The rationality of the simple vacuum-cleaner agent, we had to
specify the performance measure, the environment, and the
agent’s actuators and sensors.
• We group all these under the heading of the task environment.
• We call this the PEAS (Performance, Environment, Actuators,
Sensors) description.
• Figure 2.4 summarizes the PEAS description for the taxi’s task
environment.

November 6, 2024 27
 performance measure to be considered for
automated taxi
• getting to the correct destination;
• minimizing fuel consumption and wear and tear;
• minimizing the trip time or cost;
• minimizing violations of traffic
• laws and disturbances to other drivers;
• maximizing safety and passenger comfort;
• maximizing profits.

November 6, 2024 28
 The actuators for an automated taxi include those available to a human
driver:
• control over the engine through the accelerator and control over steering
and braking.
• It will need output to a display screen or voice synthesizer to talk back to the
passengers, and
• perhaps some way to communicate with other vehicles,

The basic sensors for the taxi will include one or more controllable video
cameras so that it can see the road;
• it might augment these with infrared or sonar sensors to detect distances to
other cars and obstacles.
• To avoid speeding tickets, the taxi should have a speedometer,
• and to control the vehicle on curves, it should have an accelerometer.
• To determine the mechanical state of the vehicle, it will need the usual array
of engine, fuel, and electrical system sensors,
• a global positioning system (GPS) so that it doesn’t get lost.
• a keyboard or microphone for the passenger to request a destination.
November 6, 2024 29
November 6, 2024 30
 Medical Diagnosis

November 6, 2024 31
 Satellite Image Analysis System

November 6, 2024 32
 Part Picking Robot

November 6, 2024 33
 Properties of task environments
Fully observable vs. partially observable:
• If an agent’s sensors give it access to the complete state of the
environment at each point in time, then we say that the task
environment is fully observable.
• The sensors detect all aspects that are relevant to the choice of action;
relevance, in turn, depends on the performance measure.
• Fully observable environments are convenient because the agent need
not maintain any internal state to keep track of the world.
• An environment might be partially observable because of noisy and
inaccurate sensors or because parts of the state are simply missing from
the sensor data
• Example: a vacuum agent with only a local dirt sensor cannot tell
whether there is dirt in other squares, and an automated taxi cannot see
what other drivers are thinking.
• If the agent has no sensors at all then the environment is unobservable.

November 6, 2024 34
Single agent vs. multiagent:
• For example, an agent solving a crossword puzzle by itself is
clearly in a single-agent environment, whereas an agent
playing chess is in a two agent environment.
Issues to be considered
• We have described how an entity may be viewed as an agent,
but we have not explained which entities must be viewed as
agents.
• The key distinction is whether B’s behavior is best described
as maximizing a performance measure whose value depends
on agent A’s behavior.
• Example, in chess, the opponent entity B is trying to maximize
its performance measure, which, by the rules of chess,
minimizes agent A’s performance measure.
• Thus, chess is a competitive multiagent environment.
November 6, 2024 35
• In the taxi-driving environment, on the other hand, avoiding
collisions maximizes the performance measure of all agents,
so it is a partially cooperative multiagent environment.
• It is also partially competitive because, for example, only one
car can occupy a parking space.
• The agent-design problems in multiagent environments are
often quite different from those in single-agent environments;
• for example, communication often emerges as a rational
behavior in multiagent environments; in some competitive
environments, randomized behavior is rational because it
avoids the pitfalls of predictability

November 6, 2024 36
Deterministic vs. stochastic.
• If the next state of the environment is completely
determined by the current state and the action executed by
the agent, then we say the environment is deterministic;
otherwise, it is stochastic.
• A game can be deterministic even though each agent may be
unable to predict the actions of the others.
• If the environment is partially observable, however, then it
could appear to be stochastic.
• Most real situations are so complex that it is impossible to
keep track of all the unobserved aspects; for practical
purposes, they must be treated as stochastic.
• Taxi driving is clearly stochastic in this sense, because one can
never predict the behavior of traffic exactly; moreover, one’s
tires blow out and one’s engine seizes up without warning.

November 6, 2024 37
Episodic vs. sequential:
• In an episodic task environment, the agent’s experience is
divided into atomic episodes.
• In each episode the agent receives a percept and then
performs a single action. Crucially, the next episode does not
depend on the actions taken in previous episodes.
• Many classification tasks are episodic. For example, an agent
that has to spot defective parts on an assembly line bases each
decision on the current part, regardless of previous decisions;
moreover, the current decision doesn’t affect whether the next
part is defective.
• In sequential environments, on the other hand, the current
decision could affect all future decisions.
• Chess and taxi driving are sequential: in both cases, short-term
actions can have long-term consequences.
• Episodic environments are much simpler than sequential
environments because the agent does not need to think ahead.
November 6, 2024 38
Static vs. dynamic:
• If the environment can change while an agent is deliberating, then we
say the environment is dynamic for that agent; otherwise, it is static.
• Static environments are easy to deal with because the agent need not
keep looking at the world while it is deciding on an action, nor need it
worry about the passage of time.
• Dynamic environments, on the other hand, are continuously asking the
agent what it wants to do; if it hasn’t decided yet, that counts as
deciding to do nothing.
• If the environment itself does not change with the passage of time but
the agent’s performance score does, then we say the environment is
semidynamic.
• Taxi driving is clearly dynamic: the other cars and the taxi itself keep
moving while the driving algorithm dithers about what to do next.
• Chess, when played with a clock, is semidynamic. Crossword puzzles
are static.

November 6, 2024 39
Discrete vs. continuous:
• The discrete/continuous distinction applies to the state of the
environment, to the way time is handled, and to the percepts
and actions of the agent.
• For example, the chess environment has a finite number of
distinct states (excluding the clock).
• Chess also has a discrete set of percepts and actions.
• Taxi driving is a continuous-state and continuous-time
problem: the speed and location of the taxi and of the other
vehicles sweep through a range of continuous values and do
so smoothly over time.
• Taxi-driving actions are also continuous (steering angles, etc.).
Input from digital cameras is discrete, strictly speaking, but is
typically treated as representing continuously varying
intensities and locations.

November 6, 2024 40
Known vs. unknown:
• This distinction refers to the agent’s (or designer’s) state of
knowledge about the “laws of physics” of the environment.
• In a known environment, the outcomes (or outcome
probabilities if the environment is stochastic) for all actions are
given. Obviously, if the environment is unknown, the agent will
have to learn how it works in order to make good decisions.
• It is quite possible for a known environment to be partially
observable—for example, in solitaire card games, I know the
rules but am still unable to see the cards that have not yet
been turned over.
• Conversely, an unknown environment can be fully observable
—in a new video game, the screen may show the entire game
state but I still don’t know what the buttons do until I try them.

November 6, 2024 41
November 6, 2024 42
 THE STRUCTURE OF AGENTS
• The job of AI is to design an agent program that implements
the agent function— the mapping from percepts to actions.
• We assume this program will run on some sort of computing
device with physical sensors and actuators—we call this the
architecture:
agent = architecture + program .
• Obviously, the program we choose has to be one that is
appropriate for the architecture.
• If the program is going to recommend actions like Walk, the
architecture had better have legs.
• The architecture might be just an ordinary PC, or it might be a
robotic car with several onboard computers, cameras, and
other sensors.
November 6, 2024 43
 Agent programs

• The agent programs take the current percept as input from

the sensors and return an action to the actuators.
• Notice the difference between the agent program, which
takes the current percept as input, and the agent function,
which takes the entire percept history.
• The agent program takes just the current percept as input
because nothing more is available from the environment;
• if the agent’s actions need to depend on the entire percept
sequence, the agent will have to remember the percepts.
• For example, Figure 2.7 shows a rather trivial agent program
that keeps track of the percept sequence and then uses it to
index into a table of actions to decide what to do.

November 6, 2024 44
November 6, 2024 45
• It is instructive to consider why the table-driven approach to
agent construction is doomed to failure.
• Let P be the set of possible percepts and let T be the lifetime
of the agent (the total number of percepts it will receive). The
lookup table will contain

entries.
• Consider the automated taxi: the visual input from a single
camera comes in at the rate of roughly 27 megabytes per
second (30 frames per second, 640×480 pixels with 24 bits of
color information). This gives a lookup table with over
10250,000,000,000 entries for an hour’s driving.
• Even the lookup table for chess—a tiny, well-behaved
fragment of the real world—would have at least 10150
entries.
November 6, 2024 46
• The daunting size of these tables means that
(a) no physical agent in this universe will have the
space to store the table,
(b) the designer would not have time to create the
table,
(c) no agent could ever learn all the right table
entries from its experience, and
(d) even if the environment is simple enough to
yield a feasible table size, the designer still has no
guidance about how to fill in the table entries.
November 6, 2024 47
• Despite all this, TABLE-DRIVEN-AGENT does do what
we want: it implements the desired agent function.
• The key challenge for AI is to produce rational
behavior from a smallest program rather than from
a vast table.
• Example, the huge tables of square roots used by
engineers and schoolchildren prior to the 1970s
have now been replaced by a five-line program for
Newton’s method running on electronic calculators.
• Can AI do for general intelligent behavior what
Newton did for square roots? We believe the answer
is yes.
November 6, 2024 48
 Four basic kinds of agent programs that
embody the principles underlying almost all
intelligent systems:
• Simple reflex agents;
• Model-based reflex agents;
• Goal-based agents; and
• Utility-based agents.

November 6, 2024 49
Simple reflex agents
• The simplest kind of agent is the simplereflex
agent. These agents select actions on the basis of
the current percept, ignoring the rest of the percept
history.
• For example, the vacuum agent whose agent
function is tabulated in Figure 2.3 is a simple reflex
agent, because its decision is based only on the
current location and on whether that location
contains dirt.
• An agent program for this agent is shown in Figure
2.8
November 6, 2024 50
November 6, 2024 51
• The vacuum agent program is very small indeed
compared to the corresponding table.
• The most obvious reduction comes from ignoring the
percept history, which cuts down the number of
possibilities from 4T to just 4.
• A further, small reduction comes from the fact that
when the current square is dirty, the action does not
depend on the location.

November 6, 2024 52
• Simple reflex behaviors occur even in more complex
environments.
• In the automated taxi, If the car in front brakes and its
brake lights come on, then you should notice this and
initiate braking.
• Some processing is done on the visual input to
establish the condition we call “The car in front is
braking.”
• Then, this triggers some established connection in the
agent program to the action “initiate braking.”
• We call such a connection a condition–action rule,
written as “ if car-in-front-is-braking then initiate-
braking.”
November 6, 2024 53
• Figure 2.9 gives the structure of this general program in schematic
form, showing how the condition–action rules allow the agent to
make the connection from percept to action.

November 6, 2024 54
• We use rectangles to denote the current
internal state of the agent’s decision process,
and ovals to represent the background
information used in the process.
• The agent program, is shown in Figure 2.10.
• The INTERPRET-INPUT function generates an
abstracted description of the current state
from the percept, and the RULE-MATCH
function returns the first rule in the set of
rules that matches the given state
description.
November 6, 2024 55
November 6, 2024 56
• Simple reflex agents have the admirable property of being simple, but
they turn out to be of limited intelligence.
• The agent in Figure 2.10 will work only if the correct decision can be
made on the basis of only the current percept—that is, only if the
environment is fully observable.
• Even a little bit of unobservability can cause serious trouble.
• For example, the braking rule given earlier assumes that the condition
car-in-front-is-braking can be determined from the current percept—
a single frame of video.
• This works if the car in front has a centrally mounted brake light.
• Older models have different configurations of taillights, brake lights,
and turn-signal lights, and it is not always possible to tell from a single
image whether the car is braking.
• A simple reflex agent driving behind such a car would either brake
continuously and unnecessarily, or, worse, never brake at all.

November 6, 2024 57
Model-based reflex agents
• The most effective way to handle partial observability is for
the agent to keep track of the part of the world it can’t see
now.
• That is, the agent should maintain some sort of internal state
that depends on the percept history and thereby reflects at
least some of the unobserved aspects of the current state.
• For the braking problem, the internal state is not too
extensive— just the previous frame from the camera,
allowing the agent to detect when two red lights at the edge
of the vehicle go on or off simultaneously.
• For other driving tasks such as changing lanes, the agent
needs to keep track of where the other cars are if it can’t see
them all at once.

November 6, 2024 58
• Updating this internal state information as time goes by
requires two kinds of knowledge to be encoded in the agent
program.
• First, we need some information about how the world
evolves independently of the agent—for example, that an
overtaking car generally will be closer behind than it was a
moment ago.
• Second, we need some information about how the agent’s
own actions affect the world—for example, that when the
agent turns the steering wheel clockwise, the car turns to the
right.
• This knowledge about “how the world works”—whether
implemented in simple Boolean circuits or in complete
scientific theories—is called a model of the world.
• An agent that uses such a model is called a model-based
agent.
November 6, 2024 59
• Figure 2.11 gives the structure of the model-based reflex
agent with internal state, showing how the current percept is
combined with the old internal state to generate the updated
description of the current state, based on the agent’s model
of how the world works.
• The agent program is shown in Figure 2.12. The interesting
part is the function UPDATE-STATE, which is responsible for
creating the new internal state description.
• The details of how models and states are represented vary
widely depending on the type of environment and the
particular technology used in the agent design.

November 6, 2024 60
November 6, 2024 61
November 6, 2024 62
• The internal “state” maintained by a model-
based agent does not have to describe “what
the world is like now” in a literal sense.
• Example, the taxi may be driving back home,
and it may have a rule telling it to fill up with
gas on the way home.
• Although “driving back home” may seem to an
aspect of the world state, the fact of the taxi’s
destination is actually an aspect of the agent’s
internal state.

November 6, 2024 63
Goal-based agents
• Knowing something about the current state of the
environment is not always enough to decide what to do.
• For example, at a road junction, the taxi can turn left, turn
right, or go straight on. The correct decision depends on
where the taxi is trying to get to.
• With a current state description, the GOAL agent needs some
sort of goal information that describes situations that are
desirable—for example, being at the passenger’s destination.
• The agent program can combine this with the model to
choose actions that achieve the goal. Figure 2.13 shows the
goal-based agent’s structure.

November 6, 2024 64
November 6, 2024 65
• Sometimes goal-based action selection is straightforward—
• for example, when goal satisfaction results immediately from
a single action.
• Sometimes it will be more tricky—
• For example, when the agent has to consider long sequences
of twists and turns in order to find a way to achieve the goal.
Search and planning are the subfields of AI devoted to finding
action sequences that achieve the agent’s goals.
• Notice that decision making of this kind is fundamentally
different from the condition–
• The reflex agent brakes when it sees brake lights.
• A goal-based agent, in principle, could reason that if the car in
front has its brake lights on, it will slow down.

November 6, 2024 66
• Although the goal-based agent appears less efficient, it is more
flexible because the knowledge that supports its decisions is
represented explicitly and can be modified.
• If it starts to rain, the agent can update its knowledge of how
effectively its brakes will operate; this will automatically cause
all of the relevant behaviors to be altered to suit the new
conditions.
• For the reflex agent, on the other hand, we would have to
rewrite many condition–action rules.
• The goal-based agent’s behavior can easily be changed to go to
a different destination, simply by specifying that destination as
the goal.
• The reflex agent’s rules for when to turn and when to go
straight will work only for a single destination; they must all be
replaced to go somewhere new.
November 6, 2024 67
 Utility-based agents
• Goals alone are not enough to generate high-quality
behaviour in most environments.
• For example, many action sequences will get the taxi to its
destination (thereby achieving the goal) but some are
quicker, safer, more reliable, or cheaper than others.
• Goals just provide a crude binary distinction between
“happy” and “unhappy” states.
• A more general performance measure should allow a
comparison of different world states according to exactly
how happy they would make the agent.
• Because “happy” does not sound very scientific, economists
and computer scientists use the term utility instead.
November 6, 2024 68
• An agent’s utility function is an internalization of the
performance measure.
• If the internal utility function and the external performance
measure are in agreement, then an agent that chooses actions to
maximize its utility will be rational according to the external
performance measure.
• In two kinds of cases, goals are inadequate but a utility-based
agent can still make rational decisions.
• First, when there are conflicting goals, only some of which can be
achieved (for example, speed and safety), the utility function
specifies the appropriate tradeoff.
• Second, when there are several goals that the agent can aim for,
none of which can be achieved with certainty, utility provides a
way in which the likelihood of success can be weighed against
the importance of the goals.
November 6, 2024 69
• The utility-based agent structure appears in Figure 2.14.

November 6, 2024 70
 Learning Agents
• How the agent programs come into being?
• In his famous early paper, Turing (1950) considers the idea of
actually programming his intelligent machines by hand.
• He estimates how much work this might take and concludes
“Some more expeditious method seems desirable.”
• The method he proposes is to build learning machines and
then to teach them.

November 6, 2024 71
November 6, 2024 72
• A learning agent can be divided into four conceptual components, as
shown in Figure 2.15.
• The most important distinction is between the learning element, which
is responsible for making improvements, and the performance
element, which is responsible for selecting external actions.
• The performance element: it takes in percepts and decides on actions.
• The learning element uses feedback from the critic on how the agent is
doing and determines how the performance element should be
modified to do better in the future.
• The critic tells the learning element how well the agent is doing with
respect to a fixed performance standard.
• For example, a chess program could receive a percept indicating that it
has checkmated its opponent, but it needs a performance standard to
know that this is a good thing; the percept itself does not say so.

November 6, 2024 73
• The last component of the learning agent is the problem
generator.
• It is responsible for suggesting actions that will lead to new
and informative experiences.
• Example:
• The performance element consists of collection of knowledge
and procedures the taxi has for selecting its driving actions.
• The taxi goes out on the road and drives, using this
performance element.
• The critic observes the world and passes information along
to the learning element.

November 6, 2024 74
• For example, after the taxi makes a quick left turn across
three lanes of traffic, the critic observes the shocking
language used by other drivers.
• From this experience, the learning element is able to
formulate a rule saying this was a bad action, and the
performance element is modified by installation of the new
rule.
• The problem generator might identify certain areas of
behavior in need of improvement and suggest experiments,
such as trying out the brakes on different road surfaces under
different conditions.

November 6, 2024 75
 How the components of agent programs work

• we can place the representations along an axis of increasing

complexity and expressive power—atomic, factored, and
structured.
• To illustrate these ideas, it helps to consider a particular agent
component, such as the one that deals with “What my
actions do.”
• This component describes the changes that might occur in the
environment as the result of taking an action, and Figure 2.16
provides schematic depictions of how those transitions might
be represented.

November 6, 2024 76
November 6, 2024 77
• In an atomic representation each state of the world is
indivisible—it has no internal structure.
• Consider the problem of finding a driving route from one end
of a country to the other via some sequence of cities.
• For the purposes of solving this problem, it may suffice to
reduce the state of world to just the name of the city we are
in—a single atom of knowledge; a “black box” whose only
discernible property is that of being identical to or different
from another black box.
• The algorithms underlying search and game-playing , Hidden
Markov models , and Markov decision processes all work
with atomic representations.

November 6, 2024 78
• We might need to pay attention to how much gas is in the tank,
our current GPS coordinates, whether or not the oil warning
light is working, how much spare change we have for toll
crossings, what station is on the radio, and so on.
• A factored representation splits up each state into a fixed set
of variables or attributes, each of which can have a value.
• Two different factored states can share some attributes (such as
being at some particular GPS location) and not others (such as
having lots of gas or having no gas); this makes it much easier to
work out how to turn one state into another.
• With factored representations, we can also represent
uncertainty—for example, ignorance about the amount of gas in
the tank can be represented by leaving that attribute blank.
• Areas of AI are based on factored representations, constraint
satisfaction algorithms , propositional logic , planning, Bayesian
networks , and the machine learning algorithms.
November 6, 2024 79
• For many purposes, we need to understand the world as having things in
it that are related to each other, not just variables with values.
• For example, we might notice that a large truck ahead of us is reversing
into the driveway of a dairy farm but a cow has got loose and is
blocking the truck’s path.
• A factored representation is unlikely to be pre-equipped with the
attribute
TruckAheadBackingIntoDairyFarmDrivewayBlockedByLooseCow with
value true or false.
• Instead, we would need a structured representation, in which objects
such as cows and trucks and their various and varying relationships can
be described explicitly. (See Figure 2.16(c).)
• Structured representations underlie relational databases and first-order
logic, first-order probability models, knowledge-based learning and
much of natural language understanding.
• In fact, almost everything that humans express in natural language
concerns objects and their relationships.
November 6, 2024 80
• The axis along which atomic, factored, and structured
representations lie is the axis of increasing expressiveness.
• A more expressive representation can capture, at least as
concisely, everything a less expressive one can capture, plus
some more.
• To gain the benefits of expressive representations while
avoiding their drawbacks, intelligent systems for the real
world may need to operate at all points along the axis
simultaneously.

November 6, 2024 81
 Thank
You

November 6, 2024 82