CSE 630: Artificial Intelligence I

Chapter 2: Agents
Jeremy Morris
Spring 2012

1
Overview

 Agents and Environments

 Rationality
 PEAS descriptions
 Environment types
 Agent types

2
Reminder: Agents

 Agent: Entity that perceives and acts

 Rational agents perceive and act rationally
 Agents try to achieve goals given input perceptions
(percepts)
 Functional abstraction: f: Percept*  Action
percept1
Reasoning
percept2 action
Sensors Effectors
percept3
… Knowledge

3
Agent-based design

 Regular programs are procedural

 First, check to see how much is in the bank account
 Debit the account by X dollars
 Return message indicating transaction complete
 Agent-based designs are reactive
 Perceive something, react a certain way
 Similar to GUI-based programming
 Flow comes from outside environment
 Creates reactions within the code

4
Perception-Action Cycle

 Agents interact with the environment over time

 Sensors/effectors do NOT have to be human-
like
 Traffic light: sensor == treadle, effector == light

5
6
Human Activity Detection (Prof. Davis)

 Is the camera an agent?

 No, but it could be part of one

sensors report effectors

reasoning

7
Vacuum World

 Two locations (A & B)

 Some locations have dirt
 Vacuum agent in one place at a time

8
Vacuum World

 Sensors: Location, Dirt?

 Effectors: Wheels, Vacuum

9
Vacuum World

 Percepts: InA/InB, Dirty/NotDirty

 Actions: MoveL, MoveR, Suck, NoOp

10
Autonomy

 A good agent should be autonomous

 Actions depend on percepts
 Actions depend on conclusions drawn from percepts
 Actions should never be predetermined
 Need to encode basic information
 How much?

11
Basic Vacuum Agent Design

while (!done) {
p = getPercept();
a = chooseAction(p);
doAction(a);
done = checkIfDone();
}

12
Designing “chooseAction”

 Could use a lookup table:

Percept Action

[A,clean] MoveRight

[A,dirty] Suck

[B,clean] MoveLeft

[B,dirty] Suck

13
Evaluating the basic agent

 Does this agent perform well?

 Need a performance measure
 Maximize amount of dirt picked up?
 Maximize ratio of dirt to energy expended?
 Maximize ration of dirt to combination of energy and time?
 Selecting a performance measure is not always easy

14
Evaluating the basic agent

 Is this agent rational?

 “For each possible percept sequence, a rational agent
should select an action that is expected to maximize
its performance measure, given the evidence provided
by the percept sequence and whatever built in
knowledge the agent has.”

15
Evaluating the basic agent

 Is this agent rational?

 “For each possible percept sequence, a rational agent
should select an action that is expected to maximize
its performance measure, given the evidence provided
by the percept sequence and whatever built in
knowledge the agent has.”
 Need performance measure
 Need prior knowledge available to agent
 Need actions the agent can perform
 Need agent’s percept sequence up to date

16
Rationality vs. Omniscience

 We can’t expect agents to foresee everything

 Rationality maximizes expected performance
 If power outages are rare, we shouldn’t expect a
vacuum agent to deal with it
 Agents should know what they’ve perceived and
what they’ve done
 May not know the effects of their actions

17
Environments

 Need to characterize the world the agent lives in

 Task environment: PEAS
 Performance Measure
 Environment (of the world)
 Actuators (aka “effectors”)
 Sensors

18
PEAS for Vacuum Agent

 Performance Measure
 Maximize ratio of dirt to energy/time usage
 Environment
 Two squares, A&B, with/without dirt
 Actuators (effectors)
 Wheels, Vacuum
 Sensors
 Location, Dirt Sensor

19
PEAS for Poker Agent

 Performance Measure

 Environment

 Actuators

 Sensors

20
PEAS for Survivor Agent

 Performance Measure

 Environment

 Actuators

 Sensors

21
Properties of task environments

 Observability
 Can the agent get all information from the environment
 Partially observable
 Poker: Only the cards in your hand
 Survivor: Votes cast at tribal council
 Fully observable
 Chess: Positions of all the playing pieces
 Survivor: Who won the immunity challenge

22
Properties of task environments

 Determinism
 Is the state of the world at time t determined
completely by the state of the world at time (t-1) and
the actions of the agent?
 Deterministic?
 Yes
 Stochastic?
 No, not completely predictable
 Strategic?
 Yes – except for the actions of other agents
 Tic-tac-toe, Chess

23
Properties of task environments
 Episodic vs. Sequential
 Episodic: only current percepts needed to decide
 Sequential: need history and current percepts
 Static vs. Dynamic
 Static: time halts while agent makes a decision
 “Turn-based” games
 Dynamic: world moves on while agent decides
 “Real-time” applications/games
 Semi-dynamic: world stands still, but decision time has
an effect on performance score
 Tournament-level Chess

24
Properties of task environments

 Discrete vs. Continuous

 Is time handled in chunks or continuously?
 Turns in chess vs. driving a car
 Single vs. Multi-agent
 Does the agent have to deal with actions of others?
 Are the competitive? Cooperative? Neither?

25
Sample Environments

26
Agent Architecture

 Five different types of agents

 Reflex agent
 Reflex agent with state (model based)
 Goal-based agent
 Utility-based agent
 Learning agent

27
Reflex Agent
Condition-action rules:
if (status==dirty):
return suck
if (location==A):
return moveRight
if (location==B):
return moveLeft

Lookup of action based on condition-action rules

28
Reflex Agent with state
Memory

Model

29
Reflex Agent with state

 What do we mean by “state”?

 Collection of values that (partially) describes the world
 Chess: position of pieces on the board
 Automatic Traffic-light camera: color of light, visual
field of camera
 Agents in partially observable environments can
keep track of unseen portions of the world
 Searching a maze, driving a car
 Like the “memory” of the agent

30
Goal-based Agent

“Look-ahead”

Key difference: tries to predict effect of actions in future

31
Utility-based Agent

 Goal-based Agent: Succeed or not?

 Sometimes, we want something less absolute
 Utility – how “happy” am I with where I am (or
where I’m going)
 Taxi
 Goal: arrive at destination
 Utility: minimize time
 Survivor
 Goal: stay in the game
 Utility: eliminate biggest threat

32
Utility-based Agent

Additional layer of “how happy will I be” on top of goal.

33
 Learning-based Agent
Provides
Feedback

“Agent”
Makes
Improvements

Suggests
Exploratory
Actions

34