One way of dealing with this conservative
nature of depth limited search
is to improve upon the arbitrary zero
value return for non terminal states.
In heuristic search this is accomplished
by
applying a heuristic evaluation function
to non-terminal states.
Such functions are based on features of
states, and so
they can be computed without examining the
entire game tree.
The implementation
of fixed depth heuristic search is easy.
Now at the top here, we have the
implementation of simple depth-limited
search that we saw earlier.
And at the bottom, we have
the implementation of heuristic
fixed-depth search.
The only difference is that we've replaced
the default value, zero, with the result
of
calling the evaluation subroutine,
evalfun, on the
state being considered whenever the depth
limit's exceeded.
Tough part of the implementation is
figuring out how to evaluate non-terminal
states.
Fortunately, examples of heuristic
functions are bound.
For example, in chess, we often use piece
count to compare states.
With the idea that in the absence of
immediate threats,
having more material is generally better
than having less material.
And similarly, we sometimes use board
control, with
the idea that having control of the center
of the board is more valuable than
controlling
the edges or the corners of the board.
The downside of using heuristic functions
is
that they're not necessarily guaranteed to
be successful.
They may work in many cases, but they can
occasionally fail.
That happens, for example, in chess, when
a player is
check-mated, even though he has more
material and better board control.
Still, games often admit heuristics that
are useful in
a sense that they, work more often than
not.
while for specific games such as chess,
programmers are able to build
in these value evaluation functions.
This is unfortunately not possible for
general game playing,
since the rules of the game are not known
in advance, rather the game player itself
must analyze
the game in order to find useful
evaluation functions.
Now in an upcoming lesson, we'll discuss
how to find such heuristics automatically.
That said, there are some heuristics for
game
playing that have arguable merit across
all games.
And in this section, we're going to
examine some of these heuristics.
And along the way, we'll show how to
build a game player that utilizes these
heuristics.
Okay, mobility is one such heuristic.
Mobility is a measure of the number of
things a player can do.
This could be the number of actions
available in a given state,
or the number of actions available in
steps away from the given state.
Or it could be the number of states
reachable within n steps, from the given
state.
I that, that can be different from the
number of actions since
sometimes different actions can refer,
can, can lead to the same state.
a simple limitation of the mobility
heuristic is shown here.
the method simply computes the number of
actions that are legal in the given state.
And returns as value the percentage of all
possible actions represented by
this set of legal actions, that's all
possible actions in all states.
Focus is another heuristic.
It's the opposite of mobility.
It's a measure of the narrowness of the
search base.
Sometimes it's good to focus, to cut down
on the number of possibilities to be
considered.
Usually it's better to restrict an
opponent's moves, rather than keeping
one's own, rather so that one keeps one's
own options open.
And a simple limitation of the focus
heuristic can be implemented
as shown here.
It's a dual of mobility, again we divide
the the
number of legal actions in a state by the
total number
of actions available in any state but,
rather than returning
that percentage as a, as result, we
subtract it from 100.
Goal proximity is another heuristic, a
little bit different from the previous
two.
It's a measure of how similar a given
state is to a desirable goal terminal
state.
Now, there are various ways this can be
computed.
One common method is to count how many
propositions are true in the
current state and are also true in a
terminal state with sufficient utility.
The difficulty of implementing this method
unfortunately is that
is it is obtaining the set of desirable
terminal states.
With which the current state can be
compared.
It's not so easy.
Another alternative is to use the goal
value of
a state as a measure of progress toward
the goal.
With the idea being that the goal value of
a non-terminal state, even though it's
not terminal, however the higher it is,
the closer one is to the actual goal.
of course, this is not always true.
However, many game, games, the goal values
are indeed monotonic.
Meaning that the values do increase with
proximity to the goal.
If you're trying to get out of a cave, and
you
see some light down one corridor, and not
down the other.
You might want to go to the one that has
light.
Moreover, it's sometimes possible to
compute this
by simple examination of the game
description.
Using methods which we can describe in
later lessons.
Now, none of these heuristics is
guaranteed to work
in all games, but all have strengths in
some games.
To deal with this fact, some designers of
GGP players, have opted
to use a weighted combination of
heuristics in place of a single heuristic.
Equation shown here is a typical formula.
Each fi here is a heuristic function such
as your mobility
or focus or goal proximity, and wi is the
corresponding weight.
Of course, there's no way of knowing in
advance
what the weight should be, but sometimes,
playing a few
instances of the game, for example, during
the start clock
of the game, can suggest weights for the
various heuristics.
As mentioned earlier, depth-limited search
is not guaranteed to succeed in all cases.
Failing is never good, however, it's
particularly
embarrassing in situations where just a
little
more search would have revealed
significant changes
in the player's circumstances for better
or worse.
In the research literature this is often
called a horizon problem.
So an example, consider, chess, a
situation, and a situation where the
players
are exchanging pieces, with white
capturing black's pieces and vice versa.
Now, imagine cutting off the search at an
arbitrary depth, say at two captures each.
At this point, white might believe it has
an advantage, since it has more material.
However, if the very next move by black is
a
capture of the white queen, this
evaluation could be misleading.
A common solution to this problem is to
forgo the fixed depth planet in
favor of one that is itself dependent on
the current state of the affairs.
Searching deeper in some areas of the
tree, searching less deep in other areas.
Of the tree.
Here's an implementation of the
variable, what's called variable-depth
heuristic search.
this version of maxscore differs from
fixed depth heuristic search in
that there is a subroutine here called
expfun that is called
to determine whether the current state and
or depth meets in appropriate condition.
If so the tree expansion continues,
otherwise the player terminates the
expansion and
simply returns the resulting, result of
applying
its evaluation function to the
non-terminal state.
the challenge in variable depth search
is finding an appropriate definition for
expfun.
One common technique is to focus on
differentials of heuristic functions.
For example, a significant change in
mobility
or goal proximity might indicate that
further search
is warranted whereas actions that do not
lead to dramatic changes might be less
important.
In chess, a good example of this is to
look for what's called quiescence.
It is a state where there are no immediate
captures.