1

Chapter 2
Intelligent agents

04/26/25
 OUTLINE
2

 Agents and Environment

 Intelligent Agent
 Rational Agent
 The Nature of Environment
 Structure of Intelligent Agents
 Learning Agent

04/26/25
 AGENTS AND
3
ENVIRONMENT
 An agent is anything that perceives
its environment through sensors and
acts upon that environment through
actuators.
 An agent runs in the cycle of
perceiving, thinking, and acting.
 An agent can be:
 Human agent
 Robotic agent
 Software agent 04/26/25
 CONT’D…
4

04/26/25
 CONT’D…
5

Human Agent
Human agent has:

 Eyes, ears, and other organs for sensors

and
 Hands, legs, vocal tract, and so on for
actuators.
Robotic Agent
A robotic agent might have:

 Cameras, infrared range finders, NLP for

sensors and
04/26/25
 Various motors for actuators.
 CONT’D…
6

Software Agent
A software agent receives:

 File contents
 Network packets
 Human input
(keyboard/mouse/touchscreen/voice) as
sensory inputs
It acts on the environment by:


 Writing files
 Sending network packets, and
04/26/25
 Displaying information or generating sounds
 CONT’D…
7

 The world around us is full of agents such

as:
 Thermostat
 Cellphone
 Self-driving car
 Even we are also agents
 Sensor is a device which detects the

change in the environment and sends the

information to other electronic devices.
 An agent observes its environment
through sensors.
04/26/25
 CONT’D…
8

 Actuators are the component of

machines that converts energy into
motion.
 The actuators are only responsible for
moving and controlling a system.
 An actuator can be an electronic
motor, gears, rails, etc.
 The environment could be everything-
the entire universe!
 The term percept refers to the content
an agent’s sensors are perceiving.
04/26/25
 CONT’D…
9

 An agent’s percept sequence is

the complete history of everything
the agent has ever perceived.
 An agent’s choice of action at any
given instant can depend on:
 Its built-in knowledge and
 The entire percept sequence
observed to date,
 But not on anything it hasn’t
perceived. 04/26/25
 CONT’D…
10

 Behavior of Agent is the action that

agent performs after any given sequence
of percepts.
 Mathematically speaking, an agent’s
behavior is described by the agent
function.
 Agent function maps any given percept
sequence (percept histories) to an
action.
 f:P* A
 We can imagine tabulating the agent
function that describes any given
04/26/25
agent.
 CONT’D…
11

 For most agents, this would be a

very large table unless we place a
bound on the length of percept
sequences we want to consider.
 Given an agent to experiment with:
 We can construct this table by trying
out all possible percept sequences
and
 Recording which actions the agent

does in response. 04/26/25

 CONT’D…
12

 The table is an external characterization

of the agent.
 Internally, the agent function for an
artificial agent will be implemented by
an agent program.
 It is important to keep two ideas
distinct:
 The agent function is an abstract
mathematical description.
 The agent program is a concrete
implementation, running 04/26/25
within some
physical system.
 CONT’D…
13

 Let us use the vacuum-cleaner world

(VCW) to illustrate these ideas.

04/26/25
 CONT’D…
14

 It consists of a robotic vacuum-cleaning

agent in a world consisting of squares
that can be either dirty or clean.
 The vacuum agent perceives which
square it is in and whether there is dirt
in the square.
 The agent starts in square A.
 Percepts: location and contents, e.g.,
[A, Dirty]
 Actions: Left, Right, Suck, 04/26/25
NoOp
 CONT’D…
15

 The agent cleans the current square if it is

dirty, otherwise it moves to the other
square.
 A partial tabulation of this agent function is
shown in next slide.
 Intelligent agents perceive their
environment several times before they can
take an action.
 Example: It perceived that room A was clean
and immediately it perceived that room A
was dirty, then it is going to stay there and
04/26/25
suck.
 CONT’D…
16

Partial tabulation of a simple agent function

for the vacuum-cleaner world 04/26/25
 CONT’D…
17

 Vacuum-world agents can be defined

simply by filling in the right-hand
column in various ways.
 What is the right way to fill out the
table?
 In other words, what makes an agent
good or bad, intelligent or stupid?

04/26/25
 INTELLIGENT AGENT
18

 An intelligent agent is an
autonomous entity which act upon
an environment using sensors and
actuators for achieving goals.
 An intelligent agent may learn from
the environment to achieve their
goals.
 A thermostat is an example of an
intelligent agent.
04/26/25
 CONT’D…
19

 There are four main rules for an AI

agent:
 Rule 1: An AI agent must have the
ability to perceive the environment.
 Rule 2: The observation must be
used to make decisions.
 Rule 3: Decision should result in an
action.
 Rule 4: The action taken by an AI
agent must be a relational action.
04/26/25
 GOOD BEHAVIOR: THE
20
CONCEPT OF RATIONALITY
 A rational agent is one that does the
right thing.
 i.e. The table for the agent function is filled
out “correctly.”
 AI has stuck to one of the notions of moral
philosophy called consequentialism.
 Agent’s behavior is evaluated by its
consequences.
 When an agent is plunked down in an
environment, it generates a sequence of
actions according to the percepts it
receives.
04/26/25
 CONT’D…
21

 This sequence of actions causes the

environment to go through a
sequence of states.
 If the sequence is desirable, then the
agent has performed well.
 This notion of desirability is captured
by a performance measure that
evaluates any given sequence of
environment states.
04/26/25
 CONT’D…
22

 As a general rule, it is better to

design performance measures:
 According to what one actually wants
to be achieved in the environment
 Rather than according to how one
thinks the agent should behave.

04/26/25
 CONT’D…
23

 What is rational at any given time

depends on (at least) four things:
I. The performance measure that defines
the criterion of success
II. The agent’s prior knowledge of the
environment
III. The actions that the agent can perform.
IV. The agent’s percept sequence to date.
 This leads to a definition of a rational
agent:
04/26/25
 CONT’D…
24

Definition of Rational Agent:

It is an agent which has clear preference,

models uncertainty, and acts in a way to

maximize its performance measure with all
possible actions.
For each possible percept sequence, a
rational agent should select an action that is
expected to maximize its performance
measure, given the evidence provided by
the percept sequence and whatever built-in
knowledge the agent posses. 04/26/25
 THE NATURE OF
25
ENVIRONMENT
 Task environments are essentially
the “problems” to which rational
agents are the “solutions.”
 The nature of the task environment
directly affects the appropriate
design for the agent program.
 In designing an agent, the first step
must always be to specify the task
environment.
04/26/25
 CONT’D…
26

 A task environment specification

includes:
 Performance measure
 Environment PEAS
 Actuators and
 Sensors
 Example: Let us consider an
automated taxi driver
04/26/25
 CONT’D…
27

Fig 2.2: PEAS description of the task

environment for an automated
04/26/25 taxi
 CONT’D…
28

 What is the performance measure to

which we would like our automated
driver to aspire?
 Desirable qualities include:
 Getting to the correct destination
 Minimizing fuel consumption and wear and
tear
 Minimizing the trip time or cost
 Minimizing violations of traffic laws and
disturbances to other drivers
 Maximizing safety and passenger comfort
 Maximizing profits
 Obviously, some of these goals
04/26/25
conflict,
so tradeoffs will be required.
 CONT’D…
29

 What is the driving environment that

the taxi will face?
 Any taxi driver must deal with a variety
of roads, ranging from rural lanes and
urban alleys to 12-lane freeways.
 The roads contain other traffic,
pedestrians, stray animals, road works,
police cars, puddles, and potholes.
 The taxi must also interact with
potential and actual passengers.
 Obviously, the more restricted the
environment, the easier the design
problem. 04/26/25
 CONT’D…
30

 The actuators for an automated taxi

include those available to a human
driver:
 Control over the engine through the
accelerator
 Control over steering and braking
 Additionally, it will need output to a
display screen or voice synthesizer to talk
back to the passengers
 Perhaps some way to communicate with
other vehicles, politely or otherwise
04/26/25
 CONT’D…
31

 The basic sensors taxi will include:

 One or more video cameras so that it can
see
 Lidar and ultrasound sensors to detect
distances to other cars and obstacles
 Speed meter to avoid speeding tickets
 Accelerometer to control the vehicle
properly, especially on curves
 Engine, fuel, and electrical system
sensors to determine mechanical state of
the vehicle 04/26/25
 CONT’D…
32

 GPS signals so that it doesn’t get lost

 Touchscreen or voice input for the passenger
to request a destination
 Example 2: Basic PEAS elements for a number of
additional agent types
 The examples include physical as well as virtual
environments
 Note that virtual environment can be just as
complex as the “real”:
 For example, a software agent (or software robot
or softbot) that trades on auction and reselling
web sites deals with millions of other users and
billions of object, many with real images.
04/26/25
 CONT’D…
33

Fig 2.3: Examples of agent types and their

04/26/25
PEAS descriptions
 CONT’D…
34

 Task environments vary along

several significant dimensions.
 They can be:
 Fully or partially observable
 Single-agent or multiagent
 Deterministic or nondeterministic
 Episodic or sequential
 Static or dynamic
 Discrete or continuous, and
04/26/25
 CONT’D…
35

FULLY OBSERVABLE VS. PARTIALLY

OBSERVABLE
If an agent’s sensors give it access to the
complete state of the environment at each
point in time, then it is fully observable.
A task environment is effectively fully
observable if the sensors detect all aspects
that are relevant to the choice of action.
Fully observable environments are
convenient because the agent need not
maintain any internal state to keep track of
the world. 04/26/25
 CONT’D…
36

An environment might be partially

observable:
 Because of noisy and inaccurate sensors
or
 Because parts of the state are simply
missing from the sensor data
 For example: A vacuum agent with only
a local dirt sensor cannot tell whether
there is dirt in other squares
 An automated taxi cannot see what
other drivers are thinking. 04/26/25
 CONT’D…
37

 Chess – the board is fully observable,

so are the opponent’s moves
 If the agent has no sensors at all then
the environment is unobservable.
SINGLE-AGENT VS. MULTI-AGENT
 If only one agent is involved in an
environment, and operating by itself
then such an environment is called
single agent environment.
04/26/25
 CONT’D…
38

 For example, an agent solving a

crossword puzzle by itself is clearly
in a single-agent environment
 If multiple agents are operating in an
environment, then such an
environment is called a multi-agent
environment.
 For example, an agent playing chess
is in a two agent environment.
04/26/25
 CONT’D…
39

 In chess, the opponent entity B is

trying to maximize its performance
measure, which, by the rules of
chess, minimizes agent A’s
performance measure.
 Thus, chess is a competitive
multiagent environment.
 In the taxi-driving environment,
avoiding collisions maximizes the
performance measure of all agents.
So it is a partially cooperative
multiagent environment.
04/26/25
 CONT’D…
40

 It is also partially competitive because, for

example, only one car can occupy a parking
space.
 The agent design problems in the multi-
agent environment are different from single
agent environment.
DETERMINISTIC VS. NONDETERMINISTIC
 If the next state of the environment is
completely determined by the current state
and the action executed by the agent(s),
then such environment is called
deterministic environment. 04/26/25
 CONT’D…
41

A nondeterministic environment is
random in nature and cannot be
determined completely by an agent.
 Example:
 Chess- there would be only a few
possible moves for a coin at the current
state and these moves can be
determined.
 Self Driving Cars- the actions of a self-
driving car are not unique, it varies from
time to time. 04/26/25
 CONT’D…
42

 The word stochastic is used by some

as a synonym for “nondeterministic”.
Distinction between the two terms
 A model of the environment is:
 Stochastic if it explicitly deals with
probabilities (e.g., “there’s a 25% chance
of rain tomorrow”) and
 Nondeterministic if the possibilities are
listed without being quantified (e.g.,
“there’s a chance of rain tomorrow”).

04/26/25
 CONT’D…
43

EPISODIC VS. SEQUENTIAL

In an episodic task environment, the
agent’s experience is divided into atomic
episodes.
In each episode the agent receives a
percept and then performs a single action.
Crucially, the next episode does not
depend on the actions taken in previous
episodes.
Many classification tasks are episodic.
04/26/25
 CONT’D…
44

 For example,
 An agent that has to spot defective parts
on an assembly line bases each decision
on the current part, regardless of
previous decisions.
 Moreover, the current decision doesn’t
affect whether the next part is defective.
 In sequential environments, the current
decision could affect all future decisions.
 Chess and taxi driving are sequential.
04/26/25
 CONT’D…
45

 In both cases, short-term actions can

have long-term consequences.
 Episodic environments are much simpler
than sequential environments because
the agent does not need to think ahead.
STATIC VS. DYNAMIC
 If the environment can change itself
while an agent is deliberating then such
environment is called a dynamic
environment.
04/26/25
 CONT’D…
46

 Static environments are easy to deal with

because the agent:
 Need not keep looking at the world while it
is deciding on an action.
 Nor need it worry about the passage of
time.
 Dynamic environments are continuously
asking the agent what it wants to do.
 If the environment itself does not change
with the passage of time but the agent’s
performance score does,04/26/25then such
environment is called semidynamic.
 CONT’D…
47

 Taxi driving is clearly dynamic.

 Chess, when played with a clock, is
semidynamic.
 Crossword puzzles are static.

DISCRETE VS. CONTINUOUS

 The discrete/continuous distinction
applies to:
 The state of the environment
 The way time is handled, and
 The percepts and actions of04/26/25
the agent.
 CONT’D…
48

 Forexample,
 The chess environment has a finite
number of distinct states, discrete set of
percepts and actions.
 Taxi driving is a continuous-state and
continuous-time problem:
 Thespeed and location of the taxi and of
the other vehicles sweep through a range
of continuous values and do so smoothly
over time.
 Taxi-driving actions are also continuous
04/26/25
(steering angles, driving, parking, etc).
 STRUCTURE OF INTELLIGENT
49
AGENTS
 The job of AI is to design an agent
program that implements the agent
function-the mapping from percepts
to actions.
 This program will run on some sort of
computing device with physical sensors
and actuators-agent architecture

AGENT=ARCHITECTURE +
PROGRAM
04/26/25
 CONT’D…
50

 The architecture might be:

 An ordinary PC
 A robotic car with several onboard
computers, cameras, and other
sensors
 In general, the architecture:
 Makes the percepts from the sensors
available to the program,
 Runs the program, and
 Feeds the program’s action choices to
the actuators as they are 04/26/25
generated
 CONT’D…
51

 The agent program has no choice

but to take just the current percept
as input.
 If the agent’s actions need to depend
on the entire percept sequence, the
agent will have to remember the
percepts.
 Designers must construct a table
that contains the appropriate action
for every possible percept04/26/25
sequence.
 CONT’D…
52

Maintains Takes as input

Returns actions
the a percept To be performed
percepts
history

Append the current

percept to the
history of percepts
Lookup table to find
Maintains a table
the action that should
that tells what action
be performed for the
to perform for what 04/26/25
percept sequence
 CONT’D…
53

 The TABLE-DRIVEN-AGENT program

is invoked for each new percept and
returns an action each time.
 It retains the complete percept
sequence in memory.
 It is instructive to consider why the
table-driven approach to agent
construction is doomed to failure.

04/26/25
 CONT’D…
54

 The drawback of table driven agent:

 Huge table
 Takes a long time to construct such
a table
 Even with learning, need a long
time to learn the table entries

04/26/25
 CONT’D…
55

 There are 4 basic kinds of agent

programs that embody the principles
underlying almost all intelligent
systems:
I. Simple reflex agents
II. Model-based reflex agents
III. Goal-based agents and
IV. Utility-based agents
 Each kind of agent program
combines particular components in
particular ways to generate actions.
04/26/25
 CONT’D…
56

1. Simple Reflex Agents

The simplest kind of agents.

These agents select actions on the

basis of the current percepts, ignoring

the rest of the percept history.
For example, the vacuum agent
whose agent function is tabulated on
slide 16 is a simple reflex agent.
 Its decision is based only on the
current location and on whether that
location contains dirt. 04/26/25
 CONT’D…
57

 An agent program for a simple reflex

agent in the two-location vacuum
environment is shown below:

 This program implements the agent

function tabulated in slide04/26/25
16.
 CONT’D…
58

 The simple reflex agent works on

condition-action rule (if-then), which
means it maps the current state to
action.
 Example:
 IF car-in-front-is-braking THEN
initiate-braking
 IF it is dark THEN turn on lights
 Simple reflex agent works only on
fully observable environment.
04/26/25
 CONT’D…
59

Schematic diagram of a simple reflex

agent 04/26/25
 CONT’D…
60

 An agent program for a simple reflex agent

is shown below:

 The INTERPRET-INPUT function generates

an abstracted description of the current
state from the percept.
 The RULEMATCH function returns the first
rule in the set of rules that matches the
given state description. 04/26/25
 CONT’D…
61

2. Model-based Reflex Agents

The Model-based agent can work in a
partially observable environment.
The most effective way to handle
partial observability is for the agent to
keep track of the part of the world it
can’t see now.
The agent should maintain some sort
of internal state that depends on the
percept history. 04/26/25
 CONT’D…
62

 Updating internal state information

as time goes by requires two kinds of
knowledge:
1)Information about how the world
changes over time:
 Divided into two parts:
 The effects of the agent’s actions
 How the world evolves independently
of the agent
 Thiskind of knowledge is called a
transition model of the 04/26/25
world.
 CONT’D…
63

2) Information about how the state of

the world is reflected in the agent’s
percepts.
 This kind of knowledge is called a
sensor model.
Together, the transition model and
sensor model allow an agent to keep
track of the state of the world
An agent that uses such models is
called a model-based agent. 04/26/25
 CONT’D…
64

Schematic diagram of model-based

04/26/25
reflex agent
 CONT’D…
65

A model-based reflex agent

04/26/25
 CONT’D…
66

3. Goal-based Agents
Knowing something about the current
state of the environment is not always
enough to decide what to do.
The agent needs some sort of goal
information that describes situations
that are desirable.
Goal-based agents expand the
capabilities of the model-based agent
by having the "goal" information to
choose actions that achieve the goal.
04/26/25
 CONT’D…
67

A model-based, goal-based
04/26/25
agent
 CONT’D…
68

 Sometimes goal-based action selection is

straightforward.
 For example, when goal satisfaction results
immediately from a single action.
 Sometimes it will be more tricky.
For example, when the agent has to

consider long sequences of twists and turns
in order to find a way to achieve the goal.
 Search and planning are the subfields of
AI devoted to finding action sequences that
achieve the agent’s goals. 04/26/25
 CONT’D…
69

4. Utility-based Agents
Goals alone are not enough to generate
high-quality behavior in most
environments.
Utility-based agents provide an extra
component of utility measurement which
makes them different by providing a
measure of success at a given state.
Utility-based agent acts based not only
goals but also the best way to achieve the
goal. 04/26/25
 CONT’D…
70

04/26/25
 LEARNING AGENT
71

 Any type of agent can be built as a

learning agent (or not).
 A learning agent can be divided into
four conceptual components:
 Learning element
 Performance element
 Critic
 Problem generator

04/26/25
 CONT’D…
72

 The learning element is responsible

for making improvements.
 It uses feedback from the critic on how
the agent is doing and determines how
the performance element should be
modified to do better in the future.
 The performance element is
responsible for selecting external
actions.
 It takes in percepts and decides on
actions. 04/26/25
 CONT’D…
73

 The design of the learning element

depends very much on the design of
the performance element.
 Critic tells the learning element how
well the agent is doing with respect
to a fixed performance standard.
 The critic is necessary because the
percepts themselves provide no
indication of the agent’s success.
04/26/25
 CONT’D…
74

 It is important that the performance

standard be fixed.
 The problem generator is responsible
for suggesting actions that will lead to
new and informative experiences.
 Example: Automated Taxi on city roads
 Learning Element: Formulates goals
 e.g., Learn rules for braking, accelerating,
learn geography of the city
04/26/25
 CONT’D…
75

 Performance Element: consists of

knowledge and procedures for driving
actions.
 e.g., turning, accelerating, braking are
performance element on roads.
 Critic: observes world and passes
information to learning element.
 e.g., quick right turn across three lanes of
traffic, observe reaction of other drivers.
 Problem Generator: Try south city
road. 04/26/25
 CONT’D…
76

A general learning agent

04/26/25