Chapter Two:

1. Introduction to Artificial Intelligent

1.1. Goals of AI

 To create expert systems – the systems which exhibit intelligent behaviour, learn,
demonstrate, explain, and advise their users which are called smart programs.
 To implement human intelligence in machines − creating systems that understand, think,
learn, and behave like humans. Shortly, to understand human intelligence better and write
programs that emulate it.
 Make machines smarter
 Understand what intelligence is.
 Make machines more useful.

1.2. What is AI?

AI is an area of computer science that emphasizes the creation of intelligent machines that work
and reacts like humans. It is the simulation of human intelligence processes by machines,
especially computer systems. These processes include learning (the acquisition of information
and rules for using the information), reasoning (using rules to reach approximate or definite
conclusions), and self-correction.

Generally, there are eight definitions of AI, laid out along two dimensions.

 The definitions on top are concerned with thought processes and reasoning, whereas the
ones on the bottom address behavior. The definitions on the left measure success in terms
of fidelity to human performance, whereas the ones on the right measure against an ideal
performance measure, called rationality. A system is rational if it does the “right thing,”
given what it knows.

The above table means that:

a. Systems that think like humans.
b. Systems that think rationally.
c. Systems that act like humans.
d. Systems that act rationally.
1.3. Approaches to AI

The above table can be summarized into the following approaches:

Human-like Rationally
Think (1). Cognitive Science Approach (2). Laws of Thought Approach
Act (3). Turing Test Approach (4). Rational Agent Approach
Most AI work falls into categories (2) and (4).

Historically, all four approaches to AI have been followed, each by different people with
different methods. A human-centered approach must be in part an empirical science, involving
observations and hypotheses about human behaviour. A rationalist approach involves a
combination of mathematics and engineering. The various group has both disparaged and
helped each other. Let us look at the four approaches in more detail.

A. Act Humanly: The Turing Test Approach

The Turing Test (performed by Turing Machine1), was proposed by Alan Turing (1950). A
computer passes the test if a human interrogator, after posing some written questions, cannot
tell whether the written responses come from a person or from a computer. The computer would
need to possess the following capabilities:

 Natural Language Processing (NLP) to enable it to communicate successfully in

English;
 Knowledge Representation to store what it knows or hears;
 Automated Reasoning to use the stored information to answer questions and draw new
conclusions;
 Machine Learning to adapt to new circumstances and to detect and extrapolate patterns.

Turing’s test deliberately avoided direct physical interaction between the interrogator and the
computer, because the physical simulation of a person is unnecessary for intelligence.
However, the so-called total Turing Test includes a video signal so that the interrogator can
test the subject’s perceptual abilities, as well as the opportunity for the interrogator to pass
physical objects “through the hatch.” To pass the total Turing Test, the computer will need

 Computer Vision to perceive objects, and

 Robotics manipulate objects and move about.

B. Think Humanly: The Cognitive Modelling Approach

If we are going to say that a given program thinks like a human, we must have some way of
determining how humans think. We need to get inside the actual workings of human minds.
There are three ways to do this through:

1. Introspection – trying to catch our own thoughts as they go by;

2. Psychological Experiments – observing a person in action; and
3. Brain Imaging – observing the brain in action.
Once we have a sufficiently precise theory of the mind, it becomes possible to express the
theory as a computer program. If the program’s input-output behaviour matches corresponding
human behaviour, that is evidence that some of the program’s mechanisms could also be
operating in humans.

The interdisciplinary field of cognitive science2 brings together computer models from AI and
experimental techniques from psychology to construct precise and testable theories of the
human mind. Real cognitive science is necessarily based on experimental investigation of
actual humans or animals.

C. Thinking Rationally: The “Laws of Thought” Approach

“Right Thinking” – is first codified by Aristotle. His syllogisms provided patterns for argument
structures that always yielded correct conclusions when given correct premises.

Example 1.1:

 “Socrates is a man; all men are mortal; therefore, Socrates is mortal.”

These laws of thought were supposed to govern the operation of the mind; their study
initiated the field called logic.

There are two main obstacles to this approach.

i. It is not easy to take informal knowledge and state it in the formal terms required by
logical notation, particularly when the knowledge is less than 100% certain.
ii. There is a big difference between solving a problem “in principle” and solving it in
practice. The “in principle” – solve any solvable problem described in logical notation,
if no solution exists, the program might loop forever. This so-called logicist tradition
within an AI hope to create Intelligent systems.

D. Acting Rationally: The Rational Agent Approach

Cognitive Science – The field of science concerned with cognition; includes parts of
cognitive psychology, linguistics, computer science, cognitive neuroscience, and philosophy
of mind.

An agent is just something that acts (agent comes from the Latin agree, to do). All computer
programs do something, but computer agents are expected to do more:

 Operate autonomously,
 Perceive their environment,
 Persist over a prolonged time period,
 Adapt to change, and
 Create and pursue goals.

A rational agent is one that acts so as to achieve the best outcome or when there is uncertainty,
the best-expected outcome. In the “laws of thought” approach to AI, the emphasis was on
correct inferences. Making correct inferences is sometimes part of being a rational agent
because one way to act rationally is to reason logically to the conclusion that a given action
will achieve one’s goals and then act on that conclusion. On the other hand, the correct
inference is not all rationality.

All the skills needed for the Turing Test also allow an agent to act rationally. Knowledge
representation and reasoning enable agents to reach good decisions. We need to be able to
generate comprehensible sentences in natural language to get by in a complex society. We need
learning not only for erudition but also because it improves our ability to generate effective
behaviour.

The rational-agent approach has two advantages over the other approaches.

a. It is more general than the “laws of thought” approach because the correct inference is just
one of several possible mechanisms for achieving rationality.
b. It is more amenable to scientific development than approaches based on human behaviour
or human thought. Human behaviour, on the other hand, is well adapted to one specific
environment and is defined by, well, the sum total of all the things that humans do.

Perfect rationality – always doing the right thing – isn’t feasible in complicated
environments.

There are two categories of AI. They are

 Weak AI – is also known as Narrow AI. It is an AI system that is designed and trained
for a particular task.

Example: – Virtual Personal Assistants such as Apple’s Siri.

 Strong AI – it is also known as Artificial General Intelligence. It is an AI system with

generalized human cognitive abilities. When presented with an unfamiliar task, it is able
to find a solution without human intervention.

1.4. The Foundation of AI

1.There are basically

1. Philosophy

 Can formal rules be used to draw valid conclusions?

 How does the mind arise from a physical brain?
 Where does knowledge come from?
 How does knowledge lead to action?

Aristotle (384–322 B.C.) was the first to formulate a precise set of laws governing the rational
part of the mind. He developed an informal system of syllogisms for proper reasoning, which
in principle allowed one to generate conclusions mechanically, given initial premises. Thomas
Hobbes (1588–1679) proposed that reasoning was like numerical computation, that “we add
and subtract in our silent thoughts.”

Ren´e Descartes (1596–1650) gave a strong advocate of the power of reasoning in

understanding the world, a philosophy now called rationalism. But Descartes was also a
 proponent of dualism. He held that there is a part of the human mind (or soul or spirit) that is
outside of nature, exempt from physical laws. Animals, on the other hand, did not possess this
dual quality; they could be treated as machines. An alternative to dualism is materialism, which
holds that the brain’s operation according to the laws of physics constitutes the mind. Free will
is simply the way that the perception of available choices appears to the choosing entity.

The empiricism movement, starting with Francis Bacon’s (1561–1626) Novum Organum, is
characterized by a dictum of John Locke (1632–1704): “Nothing is in the understanding, which
was not first in the senses.” David Hume’s (1711–1776) A Treatise of Human Nature (Hume,
1739) proposed what is now known as the principle of induction: that general rules are acquired
by exposure to repeated associations between their elements. Building on the work of Ludwig
Wittgenstein (1889–1951) and Bertrand Russell (1872–1970), the famous Vienna Circle, led
by Rudolf Carnap (1891–1970), developed the doctrine of logical positivism. This doctrine
holds

that all knowledge can be characterized by logical theories connected, ultimately, to

observation sentences that correspond to sensory inputs; thus, logical positivism combines
rationalism and empiricism. The confirmation theory of Carnap and Carl Hempel (1905–1997)
attempted to analyse the acquisition of knowledge from experience.

The final element in the philosophical picture of the mind is the connection between knowledge
and action because intelligence requires action as well as reasoning.

2. Mathematics

 What are the formal rules to draw valid conclusions?

 What can be computed?
 How do we reason with uncertain information?

Philosophers required a level of mathematical formalization in three fundamental areas:

I. Logic – George Boole (1815–1864), worked out the details of propositional, or Boolean,
logic (Boole, 1847). In 1879, Gottlob Frege (1848–1925) extended Boole’s logic to
include objects and relations, creating the first-order logic. Alfred Tarski (1902–1983)
introduced a theory of reference that shows how to relate the objects in logic to objects in
the real world.
II. Computation – The first nontrivial algorithm is thought to be Euclid’s algorithm for
computing the greatest common divisors. Tractability is the understanding of computation
via decidability and computability to solve a problem by dividing it into sub-problems
within a given time. It the time required to solve a given problem grows exponentially (not
polynomial), it is called intractable and cannot be solved in a given time. Such an
intractable problem can be solved by reducing the time via the theory of NP-
Completeness.
III. Probability – is first framed by the Italian Gerolamo Cardano (1501–1576) in terms of
the possible outcomes of gambling events. In 1654, Blaise Pascal (1623–1662), in a letter
to Pierre Fermat (1601–1665), showed how to predict the future of an unfinished gambling
game and assign average payoffs to the gamblers. Probability quickly became an
invaluable part of all the quantitative sciences, helping to deal with uncertain
measurements and incomplete theories.
3. Economics

 How should we make decisions so as to maximize payoff?

 How should we do this when others may not go along?
 How should we do this when the payoff may be far in the future?

The science of economics got its start in 1776 when Scottish philosopher Adam Smith (1723–
1790) published An Inquiry into the Nature and Causes of the Wealth of Nations. It doesn’t
merely mean of money, but also it makes people choices that lead to preferred outcomes. It is
used to provide a preferred outcome or utility. It is just like a game, which means that the
actions of one player can significantly affect the utility of another (either positively or
negatively).

4. Neuroscience

 How do brains process information?

Neuroscience is the study of the nervous system, particularly the brain. Aristotle wrote, “Of
all the animals, man has the largest brain in proportion to his size.” Generally, a collection of
simple cells can lead to thought, action, and consciousness, or, in the pithy words of John
Searle (1992), brains cause minds.

The comparison of supercomputers, personal computers,s and the human brain since 2008.

Mysticism means that minds operate in some mystical realm that is beyond physical science.
Brains and digital computers have somewhat different properties. From the above table,
computers have a cycle time that is a million times faster than a brain.

5. Psychology

 How do humans and animals think and act?

The origins of scientific psychology are usually traced to the work of the German physicist
Hermann von Helmholtz (1821–1894) and his student Wilhelm Wundt (1832–1920). The
behaviourism movement is used between humans and animals considering the measures of
percept (stimulus) and actions (response).

6. Computer Engineering

 How can we build an efficient computer?

For artificial intelligence to succeed, we need two things: intelligence and an artifact. The
computer has been the artifact of choice. The first operational computer was the
electromechanical Heath Robinson, built in 1940 by Alan Turing’s team for a single purpose:
deciphering German messages.

7. Control Theory and Cybernetics

 How can artifacts operate under their own control?

Ktesibios of Alexandria (c. 250 B.C.) built the first self-controlling machine: a water clock
with a regulator that maintained a constant flow rate. This invention changed the definition of
what an artifact could do. Previously, only living things could modify their behaviour in
response to changes in the environment.

Modern control theory, especially the branch known as stochastic optimal control, has as its
goal the design of systems that maximize an objective function over time. This roughly
matches our view of AI: designing systems that behave optimally.

8. Linguistics

 How does language relate to thought?

Modern linguistics and AI, then, were “born” at about the same time, and grew up together,
intersecting in a hybrid field called computational linguistics or natural language processing.
Much of the early work in knowledge representation (the study of how to put knowledge into
a form that a computer can reason with) was tied to language and informed by research in
linguistics, which was connected in turn to decades of work on the philosophical analysis of
language.

1.5. Bit History of AI

 1940s – the invention of the programmable digital computer, the Atanasoff Berry
Computer (ABC). This specific invention inspired scientists to move forward with the
idea of creating an “electronic brain,” or an artificially intelligent being. In the same
year, Alan Turing, a mathematician among other things, proposed a test that measured
a machine’s ability to replicate human actions to a degree that was indistinguishable
from human behaviour.
 1949 – Computer scientist Edmund Berkeley’s book “Giant Brains: Or Machines That
Think” noted that machines have increasingly been capable of handling large amounts
of information with speed and skill. He went on to compare machines to a human brain
if it were made of “hardware and wire instead of flesh and nerves,” describing machine
ability to that of the human mind, stating that “a machine, therefore, can think.”
 1950 – Alan Turing published “Computing Machinery and Intelligence,” which
proposed the idea of The Imitation Game – a question that considered if machines can
think. This proposal later became The Turing Test, which measured machine (artificial)
intelligence. Turing’s development tested a machine’s ability to think as a human
would. The Turing Test became an important component in the philosophy of artificial
intelligence, which discusses intelligence, consciousness, and ability in machines.

 1956 – is the birth of Artificial Intelligent. American computer scientist John

McCarthy organized the Dartmouth Conference, at which the term ‘Artificial
Intelligence was first adopted. Therefore, McCarthy is the father of AI. He also
developed the LISP programming language.

 1959 – Samuel coined the term “machine learning” when speaking about programming
a computer to play a game of chess better than the human who wrote its program.
 1966 – Shakey the Robot, developed by Charles Rosen with the help of 11 others, was
the first general-purpose mobile robot, also known as the “first electronic person.”
 1970 – WABOT-1, the first anthropomorphic robot, was built in Japan at Waseda
University. Its features included moveable limbs, ability to see, and ability to converse.
 1977 – Director George Lucas’ film Star Wars is released. The film features C-3PO, a
humanoid robot who is designed as a protocol droid and is “fluent in more than seven
million forms of communication.”
 1980 – WABOT-2 was built at Waseda University. This inception of the WABOT
allowed the humanoid to communicate with people as well as read musical scores and
play music on an electronic organ.
 1990s – a fifth generation computer to advance of machine learning developed by
Japanese government. AI enthusiasts believed that soon computers would be able to
carry on conversations, translate languages, interpret pictures, and reason like people.
 1997 – Computer scientists Sepp Hochreiter and Jürgen Schmidhuber developed Long
Short- Term Memory (LSTM), a type of a recurrent neural network (RNN) architecture
used for handwriting and speech recognition.
 1998 – Dave Hampton and Caleb Chung invented Furby, the first “pet” toy robot for
children.
 1999 – In line with Furby, Sony introduced AIBO (Artificial Intelligence RoBOt), a
robotic pet dog crafted to “learn” by interacting with its environment, owners, and other
AIBOs. Its features included the ability to understand and respond to 100+ voice
commands and communicate with its human owner.
 2000 – Professor Cynthia Breazeal developed Kismet, a robot that could recognize and
simulate emotions with its face. It was structured like a human face with eyes, lips,
eyelids, and eyebrows.
 2000 – Honda releases ASIMO, an artificially intelligent humanoid robot.
 2004 – NASA’s robotic exploration rovers Spirit and Opportunity navigate Mars’
surface without human intervention.
 2006 – Oren Etzioni (computer science professor), Michele Banko, and Michael
Cafarella (computer scientists), coined the term “machine reading,” defining it as
unsupervised autonomous understanding of text.

 2010 – Microsoft launched Kinect for Xbox 360, the first gaming device that tracked
human body movement using a 3D camera and infrared detection.
 2011 – Apple released Siri, a virtual assistant on Apple iOS operating systems. Siri uses
a natural-language user interface to infer, observe, answer, and recommend things to its
human user. It adapts to voice commands and projects an “individualized experience”
per user.
 2016 – A humanoid robot named Sophia is created by Hanson Robotics. She is known
as the first “robot citizen.” What distinguishes Sophia from previous humanoids is her
likeness to an actual human being, with her ability to see (image recognition), make
facial expressions, and communicate through AI.
 2016 – Google released Google Home, a smart speaker that uses AI to act as a “personal
assistant” to help users remember tasks, create appointments, and search for
information by voice.
 2017 – The Facebook Artificial Intelligence Research lab trained two “dialog agents”
(chatbots) to communicate with each other in order to learn how to negotiate. However,
as the chatbots conversed, they diverged from human language (programmed in
English) and invented their own language to communicate with one another –
exhibiting artificial intelligence to a great degree.
  2018 – Google developed BERT, the first “bidirectional, unsupervised language
representation that can be used on a variety of natural language tasks using transfer
learning.”
 2018 – Samsung introduced Bixby, a virtual assistant. Bixby’s functions include Voice,
where the user can speak to and ask questions, recommendations, and suggestions;
Vision, where Bixby’s “seeing” ability is built into the camera app and can see what
the user sees (i.e. object identification, search, purchase, translation, landmark
recognition); and Home, where Bixby uses app-based information to help utilize and
interact with the user (e.g. weather and fitness applications.)

1.6. State of the Art

The state-of-the-art deals with the applications of AI. Some of the AI applications are

 Robotic Vehicles – A driverless robotic car named STANLEY sped through the rough
terrain of the Mojave Desert at 22 mph, finishing the 132-mile course first to win the
2005 DARPA Grand Challenge. STANLEY is a Volkswagen Touareg outfitted with
cameras, radar, and laser rangefinders to sense the environment and onboard software
to command the steering, braking, and acceleration (Thrun, 2006).
 Speech Recognition – Some intelligent systems are capable of hearing and
comprehending the language in terms of sentences and their meanings while a human
talks to it. It can handle different accents, slang words, noise in the background, changes
in human noise due to cold, etc.

 Autonomous Planning and Scheduling – A hundred million miles from Earth,

NASA’s Remote Agent program became the first onboard autonomous planning
program to control the scheduling of operations for a spacecraft (Jonsson et al., 2000).
REMOTE AGENT generated plans from high-level goals specified from the ground
and monitored the execution of those plans—detecting, diagnosing, and recovering
from problems as they occurred.
 Game Playing – AI plays a crucial role in strategic games such as chess, poker, tic-tac-
toe, etc., where machines can think of a large number of possible positions based on
heuristic knowledge.
 Spam Fighting – Each day, learning algorithms classify over a billion messages as
spam, saving the recipient from having to waste time deleting what, for many users,
could comprise 80% or 90% of all messages, if not classified away by algorithms.
 Logistics Planning – During the Persian Gulf crisis of 1991, U.S. forces deployed a
Dynamic Analysis and Re-planning Tool, DART (Cross and Walker, 1994), to do
automated logistics planning and scheduling for transportation. This involved up to
50,000 vehicles, cargo, and people at a time, and had to account for starting points,
destinations, routes, and conflict resolution among all parameters.
 Robotics – Robots are able to perform the tasks given by a human. They have sensors
to detect physical data from the real world such as light, heat, temperature, movement,
sound, bumps, and pressure. They have efficient processors, multiple sensors, and huge
memory, to exhibit intelligence. In addition, they are capable of learning from their
mistakes and they can adapt to a new environment.
 Machine Translation – A computer program automatically translates from Arabic to
English, allowing an English speaker to see the headline “Erdogan Confirms That
Turkey Would Not Accept Any Pressure, Urging Them to Recognize Cyprus.”
 Natural Language Processing − It is possible to interact with a computer that
understands the natural language spoken by humans.
 Expert Systems − There are some applications that integrate machines, software, and
special information to impart reasoning and advising. They provide explanations and
advice to the users.
 Machine (Systems) Vision − These systems understand, interpret, and comprehend
visual input on the computer. For example,

 A spying airplane takes photographs, which are used to figure out spatial information
or map of the areas.
o Doctors use a clinical expert system to diagnose the patient.
o Police use computer software that can recognize the face of criminals with the
stored portrait made by forensic artists.
 Handwriting Recognition − The handwriting recognition software reads the text
written on paper with a pen or on screen by a stylus. It can recognize the shapes of the
letters and convert them into editable text.
1.7. Summary

Programming Without AI Programming With AI

A computer program without AI can answer A computer program with AI can answer
the specific questions it is meant to solve. the generic questions it is meant to solve.

AI programs can absorb new modifications by

putting highly independent pieces of information
Modifications in the program lead to changes
together. Hence you can modify even a minute
in its structure.
piece of information of a program without
affecting its structure.

Modification is not quick and easy. It may lead

Quick and easy program modification.
to affecting the program adversely.
 Chapter Two:
2. Intelligent Agents

2.1. Introduction

The concept of rationality can be applied to a wide variety of agents operating in any
imaginable environment. We begin by examining agents, environments, and the coupling
between them. The observation that some agents behave better than others leads naturally to
the idea of a rational agent one that behaves as well as possible. How well an agent can behave
depends on the nature of the environment; some environments are more difficult than others.
We give a crude categorization of environments and show how the properties of an
environment influence the design of suitable agents for that environment.

2.2. Agents and Environments

An agent is anything that can be viewed as perceiving its environment through sensors and
acting upon that environment through actuators.

Example 2.1:

Fig. 2.1. Agents interact with environments through sensors and actuators.

A human agent has eyes, ears, and other organs for sensors and hands, legs, vocal tract, and
so on for actuators.
A robotic agent might have cameras and infrared range finders for sensors and various motors
for actuators.

A software agent has Keystrokes, file contents, received network packages which act as
sensors and displays on the screen, files, sent network packets acting as actuators.
packets as sensory inputs and acts on the environment by displaying on the screen, writing file,
and sending network packets.

The term percept to refer to the agent’s perceptual inputs at any given instant. An
agent’s percept sequence is the complete history of everything the agent has ever perceived.
In general, an agent’s choice of action at any given instant can depend on the entire percept
sequence observed to date, but not on anything it hasn’t perceived. An agent’s behaviour is
described by the agent function that maps any given percept sequence to an action.

Given an agent to experiment with, we can, in principle, construct a table by trying out all
possible percept sequences and recording which actions the agent does in response. The table
is an external characterization of the agent. Internally, the agent function for an artificial agent
will be implemented by an agent program. The agent function is an abstract mathematical
description; the agent program is a concrete implementation, running within some physical
system.

Example: The vacuum-cleaner world. It

has two locations: squares A and B. It is
shown here below the figure.

2.3. Acting of Intelligent Agents

(Rationally)

A rational agent is one that does the right

thing. When an agent is plunked down in an
environment, it generates a sequence of actions according to the percepts it receives. This
sequence of actions causes the environment to go through a sequence of states. If the sequence
is desirable, then the agent has performed well. This notion of desirability is captured by
a performance measure that evaluates any given sequence of environment states.
Obviously, there is not one fixed performance measure for all tasks and agents; typically, a
designer will devise one appropriate to the circumstances.

As a general rule, it is better to design performance measures according to what one actually
wants in the environment, rather than according to how one thinks the agent should behave.

Rationality:

What is rational at any given time depends on four things:

 The performance measure defines the criterion of success.

 The agent’s prior knowledge of the environment.
 The actions that the agent can perform.
 The agent’s percept sequence to date. This leads to a definition of a rational agent:

 For each possible percept sequence, a rational agent should select an action that is expected
to maximize its performance measure, given the evidence provided by the percept sequence
and whatever built-in knowledge the agent has.

Omniscience, Learning, and Autonomy:

And omniscient agent knows the actual outcome of its actions and can act accordingly, but
omniscience is impossible in reality. Rationality maximizes expected performance, while
performance, while perfection maximizes actual performance. Rationality does not require
omniscience, then because the rational choice depends only on the percept sequence to date.

Doing actions in order to modify future percepts – sometimes called information gathering –
is an important part of rationality. A rational agent is not only required to gather information
but also to learn as much as possible from what it perceives.

To the extent that an agent relies on the prior knowledge of its designer rather than on its own
percepts, we say that the agent lacks autonomy. A rational agent should be autonomous – it
should learn what it can to compensate for partial or incorrect prior knowledge.
After the sufficient experience of its environment, the behaviour of a rational agent can become
effectively independent of its prior knowledge. Hence, the incorporation of learning allows one
to design a single rational agent that will succeed in a vast variety of environments.

The Nature of Environments

The flavour of the task environment directly affects the appropriate design for the agent
program that provides solutions for problems. These flavours are:

A. Specifying the Task Environment:

A group of the performance measure, the environment, and the agent’s actuators and sensors
specified are called the task environment. Acronymically minded as PEAS (Performance,
Environment, Actuators, Sensors). In designing an agent, the first step must always be to
specify the task environment as fully as possible.

Example 2.2:

 Consider an automated taxi driver.

Fig. 2.2. PEAS description of the task environment for an automated taxi.
What are the desirable qualities of PEAS?

B. Properties of Task Environment

 Fully Observable vs. Partially Observable – if an agent’s sensors give it access to the
complete state of the environment at each point in time, then is called fully observable.
A task environment is effectively fully observable if the sensors detect all aspects that are
relevant to the choice of action; relevance, in turn, depends on the performance measure.
 Fully observable environments are convenient because the agent need not maintain any
internal state to keep track of the world. An environment might be partially
observable because of noisy and inaccurate sensors or because parts of the state are simply
missing from the sensor data – for example, a vacuum agent with only a local dirt sensor
cannot tell whether there is dirt in other squares, and an automated taxi cannot see what
other drivers are thinking. If the agent has no sensors at all then the environment is
unobservable.
 Single Agent vs. Multiagent – an agent solving a crossword puzzle by itself is clearly in
a single-agent environment, whereas an agent playing chess is in a two-agent environment.
In chess, the opponent entity B is trying to maximize its performance measure, which, by
the rules of chess, minimizes agent A’s performance measure. Thus, chess is a competitive
multiagent environment.

In the taxi-driving environment, on the other hand, avoiding collisions maximizes the
performance measure of all agents, so it is a partially cooperative multiagent environment.

 Deterministic vs. Stochastic – If the next state of the environment is completely

determined by the current state and the action executed by the agent, then we say the
environment is deterministic; otherwise, it is stochastic. In principle, a fully observable,
deterministic environment, while if the environment is partially observable, however,
then it could appear to be stochastic. Taxi driving is stochastic because one can never
predict the behaviour of traffic exactly. The vacuum world is deterministic, but variations
can include stochastic elements such as randomly appearing dirt and an unreliable suction

mechanism. We say an environment is uncertain if it is not fully observable or not

deterministic.

The word “stochastic” generally implies that uncertainty about outcomes is quantified in terms
of probabilities; a nondeterministic environment is one in which actions are characterized by
their possible outcomes, but no probabilities are attached to them. Nondeterministic
environment descriptions are usually associated with performance measures that require the
agent to succeed for all possible outcomes of its actions.
  Competitive vs Collaborative - An agent is said to be in a competitive environment
when it competes against another agent to optimize the output. The game of chess is
competitive as the agents compete with each other to win the game which is the output.
An agent is said to be in a collaborative environment when multiple agents cooperate
to produce the desired output.
When multiple self-driving cars are found on the roads, they cooperate with each other to
avoid collisions and reach their destination which is the output desired.

 Episodic vs. Sequential – In an episodic task environment, the agent’s experience is

divided into atomic episodes. In each episode the agent receives a percept and then
performs a single action. Crucially, the next episode does not depend on the actions taken
in previous episodes. In sequential environments, on the other hand, the current decision
could affect all future decisions. Chess and taxi driving are sequential: in both cases,
short-term actions can have long-term consequences. Episodic environments are much
simpler than sequential environments because the agent does not need to think ahead.
 Static vs. Dynamic – If the environment can change while an agent is deliberating, then
we say the environment is dynamic for that agent; otherwise, it is static. Static
environments are easy to deal with because the agent need not keep looking at the world
while it is deciding on an action, nor need it worry about the passage of time. Dynamic
environments, on the other hand, are continuously asking the agent what it wants to do;
if it hasn’t decided yet, that counts as deciding to do nothing. If the environment itself
does not change with the passage of time but the agent’s performance score does, then
we say the environment is semi-dynamic. Taxi driving is clearly dynamic: the other cars
and the taxi itself keep moving while the driving algorithm dithers about what to do next.
Chess, when played with a clock, is semi-dynamic. Crossword puzzles are static.
 Discrete vs. Continuous – they are applied to the state of the environment, to the
way time is handled, and to the percepts and actions of the agent. For example, the chess
environment has a finite number of distinct states (excluding the clock). Taxi driving is a
continuous-state and continuous-time problem: the speed and location of the taxi and of
the other vehicles sweep through a range of continuous values and do so smoothly over
time. Input from digital cameras is discrete.
 Known vs. Unknown – it refers to both the environment and agent’s (or designer’s) state
of knowledge about the “laws of physics” of the environment. In a known environment,
the outcomes (or outcome probabilities if the environment is stochastic) for all actions
are given. Obviously, if the environment is unknown, the agent will have to learn how it
works in order to make good decisions. For a known environment to be partially
 observable like, in solitaire card games, I know the rules but am still unable to see the
cards that have not yet been turned over. Conversely, an unknown environment can be
fully observable – in a new video game, the screen may show the entire game state but
you still don’t know what the buttons do until you try them.

2.4. Structure of Intelligent Agents

Behaviour is the action that is performed after any given sequence of percepts. The job of AI
is to design an agent program that implements the agent function – the mapping from percepts
to actions. We assume this program will run on some sort of computing device with physical
sensors and actuators – we call this the architecture:

agent = architecture + program

The program we choose has to be one that is appropriate for the architecture. If the program is
going to recommend actions like Walk, the architecture had better have legs. The architecture
might be an ordinary PC, or it might be a robotic car with several onboard computers, cameras,
and other sensors. Generally, the architecture makes the percepts from the sensors available to
the program, runs the program, and feeds the program’s action choices to the actuators as they
are generated.

 Agent Programs

It takes the current percept as input from the sensors, the agent function, which takes the entire
percept history and return an action to the actuators. What is the difference between agent
program and agent functions? The agent program takes the current percept as input, while the
agent function takes the entire percept history.

2.5. Agent Program Types

There are basically five basic kinds of agent programs that embody the principles underlying
almost all intelligent systems:

A. Simple Reflex Agent

It is the simplest kind of agent. These agents select actions on the basis of the current percept,
ignoring the rest of the percept history.
Example-1-:

 The vacuum agent whose agent function is based only on the current location and on
whether that location contains dirt because its decision is based only on the current
location and on whether that location contains dirt of figure 2.2. above. An agent
program for this agent is shown here below.

Example-2.3:

Consider an automated taxi driver. If the car in front brakes and its brake lights come on, then
you should notice this and initiate braking. In other words, some processing is done on the
visual input to establish the condition we call “The car in front is braking.” Then, this triggers
some established connection in the agent program to the action “initiate braking.” We call such
a connection a condition–action rule is written as

if car-in-front-is-braking then initiate-braking.

The figure below gives the structure of the general program in schematic form, showing how
the condition–action rules allow the agent to make the connection from percept to action.
 Fig. 2.3. Schematic diagram of a simple reflex agent.

Fig. 2.4. A simple reflex agent that acts according to a rule whose condition matches the
current state.

The rectangles are used to denote the current internal state of the agent’s decision process, and
ovals to represent the background information used in the process. The INTERPRET-INPUT
function generates an abstracted description of the current state from the percept, and the
RULE- MATCH function returns the first rule in the set of rules that matches the given state
description. The description in terms of “rules” and “matching” is purely conceptual.

The agent in Figure 2.7. above will work only if the correct decision can be made on the basis
of only the current percept – that is, only if the environment is fully observable.

B. Model-Based Reflex Agent

The most effective way to handle partial observability is for the agent to keep track of the part
of the world it can’t see now. That is, the agent should maintain some sort of internal state that
depends on the percept history and thereby reflects at least some of the unobserved aspects of
the current state. Shortly is knowledge-base.

Example:

 The automated taxi driver (In case of car braking problem).

The Figure below gives the structure of the model-based reflex agent with an internal state,
showing how the current percept is combined with the old internal state to generate the updated
description of the current state, based on the agent’s model of how the world works.
 Fig. 2.5. A model-based reflex agent.

The agent program of model-based agent is shown in figure below.

Fig. 2.6. A model-based reflex agent. It keeps track of the current state of the world,
using an internal model. It then chooses an action in the same way as the reflex agent.

The interesting part is the function UPDATE-STATE, which is responsible for creating the
new internal state description. The details of how models and states are represented vary
widely depending on the type of environment and the particular technology used in the agent
design.

C. Goal-Based Agent

Knowing something about the current state of the environment is not always enough to
decide what to do. It is an expansion of a model-based reflex agent.

Example:
  At a road junction, the taxi can turn left, turn right, or go straight on. The correct
decision depends on where the taxi is trying to get to. In other words, as well as a current
state description, the agent needs some sort of goal information that describes situations
that are desirable for example, being at the passenger’s destination. The agent program
can combine this with the model (the same information as was used in the model-based
reflex agent) to choose actions that achieve the goal. The figure below shows the goal-
based agent’s structure.

Fig. 2.7. A model-based, goal-based agent. It keeps track of the world state as well as a
set of goals it is trying to achieve and chooses an action that will (eventually) lead to the
achievement of its goals.
Sometimes goal-based action selection is straightforward.

Example 2.4:

When goal satisfaction results immediately from a single action. Sometimes it will be more-
tricky. For example, when the agent has to consider long sequences of twists and turns in order
to find a way to achieve the goal. Search and planning are the subfields of AI devoted to
finding action sequences that achieve the agent’s goals.

In this, decision-making involves consideration of the future – both “What will happen if I do
such-and-such?” and “Will that make me happy?” It is less efficient; it is more flexible because
the knowledge that supports its decisions is represented explicitly and can be modified.
 D. Utility-Based Agent

An agent’s utility function is essentially an internalization of the performance measure. If the

internal utility function and the external performance measure are in agreement, then an agent
that chooses actions to maximize its utility will be rational according to the external
performance measure.

It has two kinds of cases in terms of flexibility and learning to make rational decisions.

1. When there are conflicting goals, only some of which can be achieved (for example,
speed and safety), the utility function specifies the appropriate tradeoff.
2. When there are several goals that the agent can aim for, none of which can be achieved
with certainty, the utility provides a way in which the likelihood of success can be
weighed against the importance of the goals.

Partial observability and stochasticity are ubiquitous in the real world, and so, therefore, is
decision-making under uncertainty. Technically speaking, a rational utility-based agent
chooses the action that maximizes the expected utility of the action outcomes – that is, the
utility the agent expects to derive, on average, given the probabilities and utilities of each
outcome. An agent that possesses an explicit utility function can make rational decisions with
a general-purpose algorithm that does not depend on the specific utility function being
maximized. The utility-based agent structure appears in the figure below.
 Fig. 2.8. A model-based, utility-based agent.

A utility-based agent has to model and keep track of its environment, tasks that have involved
a great deal of research on perception, representation, reasoning, and learning.

E. Learning Agent

Each kind of agent program combines particular components in particular ways to generate
actions. We convert all these agents to improve the performance of their components so as to
generate better actions which is called learning agents. A learning agent can be divided into

four conceptual components as shown in the figure below.

Fig 2.9. A general learning agent

The learning element is responsible for making improvements, while the performance element
is responsible for selecting external actions. The performance element is what we have
previously considered being the entire agent: it takes in percepts and decides on actions. The
learning element uses feedback from the critic on how the agent is doing and determines how
the performance element should be modified to do better in the future.

The design of the learning element depends very much on the design of the performance
element. When trying to design an agent that learns a certain capability, the first question is not
“How am I going to get it to learn this?” but “What kind of performance element will my agent
need to do this once it has learned how?” Given an agent design, learning mechanisms can be
constructed to improve every part of the agent.

The critic tells the learning element how well the agent is doing with respect to a fixed
performance standard. The critic is necessary because the percepts themselves provide no
indication of the agent’s success.

Example:
A chess program could receive a percept indicating that it has checkmated its opponent, but it
needs a performance standard to know that this is a good thing; the percept itself does not say
so. It is important that the performance standard be fixed. Conceptually, one should think of it
as being outside the agent altogether because the agent must not modify it to fit its own
behaviour.

Th problem generator is responsible for suggesting actions that will lead to new and
informative experiences. The point is that if the performance element had its way, it would
keep doing the actions that are best, given what it knows. But if the agent is willing to explore
a little and do some perhaps suboptimal actions in the short run, it might discover much better
actions for the long run. The problem generator’s job is to suggest these exploratory actions.
This is what scientists do when they carry out experiments. Galileo did not think that dropping
rocks from the top of a tower in Pisa was valuable in itself. He was not trying to break the rocks
or to modify the brains of unfortunate passers-by. His aim was to modify his own brain by
identifying a better theory of the motion of objects.

Let us consider an automated taxi to summarize the learning agent.

The performance element consists of whatever collection of knowledge and procedures the taxi
has for selecting its driving actions. The taxi goes out on the road and drives, using this
performance element. The critic observes the world and passes information along to the
learning element. For example, after the taxi makes a quick left turn across three lanes of traffic,
the critic observes the shocking language used by other drivers. From this experience, the
learning element is able to formulate a rule saying this was a bad action, and the performance
element is modified by the installation of the new rule. The problem generator might identify
certain areas of behaviour in need of improvement and suggest experiments, such as trying out
the brakes on different road surfaces under different conditions.
The forms of learning in the preceding paragraph do not need to access the external
performance standard – in a sense, the standard is the universal one of making predictions that
agree with the experiment. The situation is slightly more complex for a utility-based agent that
wishes to learn utility information. For example, suppose the taxi-driving agent receives no tips
from passengers who have been thoroughly shaken up during the trip. The external
performance standard must inform the agent that the loss of tips is a negative contribution to
its overall performance; then the agent might be able to learn that violent manoeuvres do not
contribute to its own utility. In a sense, the performance standard distinguishes part of the
incoming percept as a reward (or penalty) that provides direct feedback on the quality of the
agent’s behaviour. Hard-wired performance standards such as pain and hunger in animals can
be understood in this way.

Generally, learning in intelligent agents can be summarized as a process of modification of

each component of the agent to bring the components into closer agreement with the available
feedback information, thereby improving the overall performance of the agent.

2.6. Important Concepts and Terms

How do the Components of Agent Programs Work?

To understand how the components of the agent program work, it is important to categorize them
into the following parts as shown in the figure below.

Fig. 2.13 Three ways to represent states and the transitions between them.
(a) Atomic representation: a state (such as B or C) is a black box with no internal structure; (b)
Factored representation: a state consists of a vector of attribute values; values can be Boolean, real-
valued, or one of a fixed set of symbols.

 Structured representation: a state includes objects, each of which may have attributes of
its own as well as relationships to other objects.

In an atomic representation, each state of the world is indivisible – it has no internal structure.
Consider the problem of finding a driving route from one end of a country to the other via some
sequence of cities. For the purposes of solving this problem, it may suffice to reduce the state of
the world to just the name of the city we are in – a single atom of knowledge; a “black box” whose
only discernible property is that of being identical to or different from another black box.

Example 2.5:

 Search, and Game-Playing.

A factored representation splits up each state into a fixed set of variables or attributes, each of
which can have a value. While two different atomic states have nothing in common – they are just
different black boxes – two different factored states can share some attributes (such as being at
some particular GPS location) and not others (such as having lots of gas or having no gas); this
makes it much easier to work out how to turn one state into another. With factored representations,
we can also represent uncertainty – for example, ignorance about the amount of gas in the tank can
be represented by leaving that attribute blank.

Example 2.6:

 Constraint Satisfaction Problem, Propositional Logic, and Machine Learning.

For many purposes, we need to understand the world as having things in it that are related to each
other, not just variables with values. For example, we might notice that a large truck ahead of us is
reversing into the driveway of a dairy farm but a cow has got loose and is blocking the truck’s path.

A structured representation is pre-equipped with the attribute

TruckAheadBackingIntoDairyFarmDrivewayBlockedByLooseCow with value true or false, in
which objects such as cows and trucks and their various and varying relationships can be
described explicitly.

Example 2.7:

 Relational Databases, First-Order Logic, First-Order Probability Models, Knowledge-

Based Learning and Natural Language Understanding.

In fact, almost everything that humans express in natural language concerns objects and their
relationships. The axis expressiveness representation increases and become more complex from
atomic to structured representation.