Intelligent Agents

Introduction to Agents
• An agent is anything that can be viewed as perceiving its environment through
sensors and actuators.
• A human agent has eyes, ears, and other organs for sensors and hands, legs, vocal
tract, and so on for actuators.
• A robotic agent might have cameras and infrared range finders for sensors and
various motors for actuators.
• A software agent receives keystrokes, file contents, and network packets as
sensory inputs and acts on the environment by displaying on the screen, writing
files, and sending network packets.
• We use the term percept to refer to the agent’s perceptual inputs at any given
instant.
Introduction to Agents
• An agent’s percept sequence is the complete history of everything the agent has
ever perceived.
• In general, an agent’s choice of action at any given instant can depend on the
entire percept sequence observed to date, but not on anything it hasn’t perceived.
• In terms of mathematics, an agent’s behavior is described by the agent function
that maps any given percept sequence to an action.
• We can formulate the agent function as tabulation and this table might be a very
large table-infinite, in fact , unless we place a bound on the length of percept
sequences we want to consider.
• We will construct the agent table by trying out all possible percept sequences and
recording which actions does the agent do in response.
Introduction
to Agents
• Internally agent function
will be implemented by
an agent program.
• The agent function is an
abstract mathematical
description.
• The agent program is a
concrete implementation
running within the
physical system.
Sample Percept Sequence
Sample Agent Program
Good Behavior- concept of Rationality
• A rational agent is one that does the right thing—conceptually speaking, every
entry in the table for the agent function is filled out correctly.
• Obviously, doing the right thing is better than doing the wrong thing, but what
does it mean to do the right thing?
• When an agent is plunked down in an environment, it generates a sequence of
actions according to the percepts it receives.
• This sequence of actions causes the environment to go through a sequence of
states. If the sequence is desirable, then the agent has performed well.
• This notion of desirability is captured by a performance measure that evaluates
any given sequence of environment states.
• Performance measure is not same for all tasks and agents; it depends on how a
designer will be devised based on circumstances in the application.
Good Behavior- concept of Rationality
• We might propose to measure performance by the amount of dirt cleaned up in a
single eight-hour shift.
• A rational agent can maximize this performance measure by cleaning up the dirt,
then dumping it all on the floor, then cleaning it up again, and so on.
• A more suitable performance measure would reward the agent for having a clean
floor.
• As a general rule, it is better to design performance measures according to what
one actually wants in the environment, rather than according to how one thinks
the agent should behave.
Rationality
What is rational at any given time depends on four things:
• The performance measure that defines the criterion of success.
• The agent’s prior knowledge of the environment.
• The actions that the agent can perform.
• The agent’s percept sequence to date

Definition of Rational Agent

For each possible percept sequence, a rational agent should select an action that is
expected to maximize its performance measure, given the evidence provided by the
percept sequence and whatever built-in knowledge the agent has.
Rationality
• Consider the simple vacuum-cleaner agent that cleans a square if it is dirty and
moves to the other square if not; this is the agent function tabulated in above
mentioned Figure in earlier slides.
• Is this a rational agent? That depends! First, we need to say what the performance
measure is, what is known about the environment, and what sensors and actuators the
agent has.
Let us assume the following:
• The performance measure awards one point for each clean square at each time step,
over a “lifetime” of 1000 time steps.
• The “geography” of the environment is known a priori but the dirt distribution and
initial location of the agent is not known.
• The Left and Right actions move the agent left and right except when this would take
the agent outside the environment, in which case the agent remains where it is.
Rationality
• The only available actions are Left, Right, and Suck.
• The agent correctly perceives its location and whether that location contains dirt.
• The same agent would be irrational under different situations.
• For example, once all the dirt is cleaned up, the agent will oscillate needlessly
back and forth; if the performance measure includes a penalty of one point for
each movement left or right, the agent will fare poorly.
• A better agent for this case would do nothing once it is sure that all the squares are
clean. If clean squares can become dirty again, the agent should occasionally
check and re-clean them if needed.
• If the geography of the environment is unknown, the agent will need to explore it
rather than stick to squares A and B.
Omniscience, Learning, Autonomy
• An omniscient agent knows the actual outcomes of it’s actions and can act
accordingly but it is impossible in reality.
• Consider the following example: I am walking along the Champs Elys´ees one
day and I see an old friend across the street. There is no traffic nearby and I’m not
otherwise engaged, so, being rational, I start to cross the street. Meanwhile, at
33,000 feet, a cargo door falls off a passing airliner, and before I make it to the
other side of the street I am flattened. Was I irrational to cross the street?
• The above example shows rationality is not the same as perfection.
• Rationality maximizes expected performance, while perfection maximizes actual
performance.
• For example, if an agent does not look both ways before crossing a busy road,
then its percept sequence will not tell it that there is a large truck approaching at
high speed.
Omniscience, Learning, Autonomy
• A rational agent should be autonomous-it should learn what it can to compensate
for partial or incorrect prior knowledge.
• For example, a vacuum-cleaning agent that learns to foresee where and when
additional dirt will appear will do better than one that does not.
• When the agent has had little or no experience, it would have to act randomly
unless the designer gave some assistance.
• After sufficient experience of its environment, the behavior of a rational agent can
become effectively independent of its prior knowledge.
• Incorporation of learning allows us to design a single rational agent that can be
applied in wide variety of environments.
Omniscience, Learning, Autonomy
• First, it would not be rational to cross the road given this uninformative percept
sequence.
• Second, a rational agent should choose the “looking” action before stepping into
the street, because looking helps maximize the expected performance.
• Doing actions in order to modify future percepts—sometimes called information
gathering—is an important part of rationality.
• Our definition requires a rational agent not only to gather information but also
to learn as much as possible from what it perceives.
• The agent’s initial configuration could reflect some prior knowledge of the
environment, but as the agent gains experience this may be modified and
augmented.
The nature of Environments

• When designing a rational agent, we must think about task environments which
are essentially the problems to which rational agents are the solutions.
• Specifying the task environment
• PEAS(Performance, Environment, Actuators, Sensors)
• In designing an agent, the first step should always be to specify the task
environment in a precise manner.
The nature of Environments
The nature of Environments

• Performance measure- getting to the correct destination; minimizing fuel

consumption; minimizing the trip time or cost; minimizing violations of traffic
laws and disturbances to other drivers; maximizing safety and passenger comfort;
maximizing profits.
• Environment- Any taxi driver must deal with a variety of roads, ranging from
rural lanes and urban alleys to 12-lane freeways. The roads contain other traffic,
pedestrians, stray animals, road works, police cars, puddles, and patholes. The taxi
must also interact with potential and actual passengers.
• Actuators for an automated Taxi-control over the engine through the accelerator
and control over steering and braking. It will need output to a display screen or
voice synthesizer to talk back to the passengers, and perhaps some way to
communicate with other vehicles, politely or otherwise.
The nature of Environments

• Sensors- Video cameras so that it can see the road. Infrared or sonar sensors to
detect distances to other cars and obstacles.
• Speedometers for controlling the speed of vehicles
• Electrical system sensors to know the mechanical state of the vehicle, fuel
consumption, air pressure and seat adjustments.
• GPS is needed to show the path
• Keyboard or microphone for the passenger to request a destination.
The nature of Environments
Properties of Task Environments
Fully Observable
• If an agent’s sensors give it access to the complete state of the
environment at each point of time, then it is called fully observable.
• A task environment is effectively fully observable if the sensors detect
all aspects that are relevant to the choice of action; relevance, in turn,
depends on the performance measure.
• Fully observable environments are convenient because the agent need
not maintain any internal state to keep track of the world.
Properties of Task Environments
Partially Observable
• An environment might be partially observable because of noisy and
inaccurate sensors or because parts of the state are simply missing
from the sensor data.
• For example, a vacuum agent with only a local dirt sensor cannot tell
whether there is dirt in other squares, and an automated taxi cannot see
what other drivers are thinking.
• If the agent has no sensors at all then the environment is
Unobservable
Single agent vs Multiple agents
• For example, an agent solving a crossword puzzle by itself is clearly in a single-
agent environment, whereas an agent playing chess is in a two-agent environment.
• Up to now, we have described how an entity be viewed as an agent, but we
haven’t explained which entities must be viewed as agents.
• Competitive Agent- For example, in chess, the opponent entity B is trying to
maximize its performance measure, which, by the rules of chess, minimizes agent
A’s performance measure. Therefore, chess is a competitive multi agent
environment.
• Cooperative Agent- In taxi driving, avoiding collisions maximizes the
performance measure of all agents, so it is partially Cooperative Multi agent.
Deterministic vs Stochastic
• If the next state of the environment is completely determined by the current state
and the action executed by the agent, then we say the environment is
deterministic; otherwise, it is stochastic.
• An agent need not worry about uncertainty in a fully observable, deterministic
environment.
• If the environment is partially observable, however, then it could appear to be
stochastic.
• Taxi driving is clearly stochastic in this sense, because one can never predict the
behavior of traffic exactly; moreover, one’s tires blow out and one’s engine seizes
up without warning.
• We say an environment is uncertain if it is not fully observable or not
deterministic.
Episodic vs Sequential
• In an episodic task environment, the agent’s experience is divided into atomic
episodes.
• In each episode the agent receives a percept and then performs a single action.
Crucially, the next episode does not depend on the actions taken in previous
episodes.
• Many classification tasks are episodic.
• Ex: An agent that has to spot defective parts on an assembly line bases each
decision on the current part, regardless of previous decisions; moreover, the
current decision doesn’t affect whether the next part is defective.
Episodic vs Sequential
• In sequential environments, on the other hand, the current decision could affect all
future decisions.
• Chess and taxi driving are sequential: in both cases, short-term actions can have
long-term consequences.
Static vs Dynamic
• If the environment can change while an agent is deliberating, then we say the
environment is dynamic for that agent; otherwise, it is static.
• Ex: Taxi driving is clearly dynamic: the other cars and the taxi itself keep moving
while the driving algorithm dithers about what to do next.
• Static environments are easy to deal with because the agent need not keep
looking at the world while it is deciding on an action, nor need it worry about the
passage of time. Ex: Cross word Puzzles
• Semi dynamic - If the environment itself does not change with the passage of
time but the agent’s performance score does, then it is semi dynamic. Ex: Chess
played with a clock.
Discrete vs Continuous
• The discrete/continuous distinction applies to the state of the environment, to the
way time is handled, and to the percepts and actions of the agent.
• Ex: Chess environment is a discrete environment which consists of discrete set of
percepts and actions.
• Taxi driving is a continuous-state and continuous-time problem.
Known vs Unknown
• In a known environment, the outcomes for all actions are given.
• In an Unknown environment the agent will have to learn how it works in order to
make good decisions.
• Note: The distinction between known and unknown environments is not the same
as the one between fully and partially observable environments.
• Ex: For a known environment to be partially observable- for example, in solitaire
card games, I know the rules but am still unable to see the cards that have not yet
been turned over.
• An unknown environment can be fully observable—in a new video game, the
screen may show the entire game state but I still don’t know what the buttons do
until I try them.
Sample Task Environments
Structure of Agents
• The job of AI is to design an agent program that implements agent function-
mapping from percepts to actions.
• Agent program needs an architecture to run which consists of Sensors and
Actuators,
Agent = architecture + Program

• The architecture might be just an ordinary PC, or it might be a robotic car with
several onboard computers, cameras, and other sensors.
• In general, the architecture makes the percepts from the sensors available to the
program, runs the program, and feeds the program’s action choices to the
actuators as they are generated.
Structure of Agents
• Agent Program takes the current percept as input and while agent function takes
the entire history.
• The agent program takes just the current percept as input because nothing more is
available from the environment;
• If the agent’s actions need to depend on the entire percept sequence, the agent
will have to remember the percepts.
• Why the table-driven approach to agent construction is doomed to failure?
• Let P be the set of possible percepts and let T be the lifetime of the agent (the
total number of percepts it will receive).
• The lookup table will contain entries.
Structure of Agents
• Consider the automated taxi: the visual input from a single camera comes in at the
rate of roughly 27 megabytes per second (30 frames per second, 640 × 480 pixels
with 24 bits of color information).
• This gives a lookup table with over 10^250,000,000,000 entries for an hour’s
driving.
• Even the lookup table for chess—a tiny, well-behaved fragment of the real world
—would have at least 10^150 entries.
• The daunting size of these tables (the number of atoms in the observable universe
is less than 1080) means that
(a) no physical agent in this universe will have the space to store the table
(b) the designer would not have time to create the table
(c) no agent could ever learn all the right table entries from its experience
Structure of Agents
(d) even if the environment is simple enough to yield a feasible table size, the
designer still has no guidance about how to fill in the table entries.
• The key challenge for AI is to find out how to write programs that, to the extent
possible, produce rational behavior from a smallish program rather than from a
vast table.
• Ex: The huge tables of square roots used by engineers and schoolchildren prior to
the 1970s have now been replaced by a five-line program for Newton’s method
running on electronic calculators.
Types of Agents
Agents can be grouped into five classes based on their degree of perceived
intelligence and capability. All these agents can improve their performance and
generate better action over the time. These are given below:
• Simple Reflex Agent
• Model-based reflex agent
• Goal-based agents
• Utility-based agent
• Learning agent
Simple Reflex Agent
• The Simple reflex agents are the simplest agents. These agents take decisions on
the basis of the current percepts and ignore the rest of the percept history.
• These agents only succeed in the fully observable environment.
• The Simple reflex agent does not consider any part of percepts history during their
decision and action process.
• The Simple reflex agent works on Condition-action rule, which means it maps the
current state to action. Such as a Room Cleaner agent, it works only if there is dirt
in the room.
Simple Reflex Agent

• Problems for the simple reflex agent design approach:

• They have very limited intelligence
• They do not have knowledge of non-perceptual parts of the current state
• Mostly too big to generate and to store.
• Not adaptive to changes in the environment.
Simple Reflex Agent
Model-based Reflex Agent
• The Model-based agent can work in a partially observable environment and track
the situation.
A model-based agent has two important factors:
• Model: It is knowledge about "how things happen in the world," so it is called a
Model-based agent.
• Internal State: It is a representation of the current state based on percept history.
• These agents have the model, "which is knowledge of the world" and based on the
model they perform actions.
• Updating the agent state requires information about:
• How the world evolves
• How the agent's action affects the world.
Model-based Reflex Agent
Goal-based Agents
• The knowledge of the current state environment is not always sufficient to decide
for an agent to what to do.
• The agent needs to know its goal which describes desirable situations.
• Goal-based agents expand the capabilities of the model-based agent by having the
"goal" information.
• They choose an action, so that they can achieve the goal.
• These agents may have to consider a long sequence of possible actions before
deciding whether the goal is achieved or not. Such considerations of different
scenario are called searching and planning, which makes an agent proactive.
Goal-based Agent
Utility-based Agents
• These agents are similar to the goal-based agent but provide an extra component
of utility measurement which makes them different by providing a measure of
success at a given state.
• Utility-based agent act based not only goals but also the best way to achieve the
goal.
• The Utility-based agent is useful when there are multiple possible alternatives,
and an agent has to choose in order to perform the best action.
• The utility function maps each state to a real number to check how efficiently
each action achieves the goals.
Utility-based Agent
Learning Agents
• A learning agent in AI is the type of agent which can learn from its past experiences, or it
has learning capabilities.
• It starts to act with basic knowledge and then able to act and adapt automatically through
learning.
• A learning agent has mainly four conceptual components, which are:
• Learning element: It is responsible for making improvements by learning from environment
• Critic: Learning element takes feedback from critic which describes that how well the agent
is doing with respect to a fixed performance standard.
• Performance element: It is responsible for selecting external action.
• Problem generator: This component is responsible for suggesting actions that will lead to
new and informative experiences.
• Hence, learning agents are able to learn, analyze performance, and look for new ways to
improve the performance.
Learning
Agents