Okay, so we've seen that propositional
nets can
be used to advantage in searching game
trees.
The other main use of propnets is in
analyzing games.
For example, by analyzing the propnet for
a
game, it's sometimes possible for a player
to
detect structure that allows him to
actually decrease
the size of the game tree in significant
ways.
Game decomposition, or so-called
factoring, is a good example of this.
A compound game is a single game
consisting
of two or more individual games glued
together.
The state of the compound game is a
composition of the states of the
individual games.
On each step of a compound game the
players perform actions in each of the
individual games.
The compound game is over when either one
or all of the individual games are over.
Depending on the type of the compound
game.
Now using the techniques we have seen thus
far, compound games can be quite expensive
to play.
Unless a player recognizes that there are
independent subgames involved, it's likely
to search a game tree that's far larger
than it needs to be.
For example, if one subgame has branching
factor a,
and a second has branching factor b, then
the
branching factor of the joint game is a
times
b, one choice of each of the b actions
for each of the a actions.
Then the fringe of the game tree at depth
d is likely
to be something like (a x b) raised to the
power d.
Now this is wasteful if the two games are
truly independent, in that case there are
two trees, one
with branching factor a and one with
branching factor b
and the total size of the fringe of these
trees.
At depth d, should be only a to the d plus
b to the d, significantly smaller.
Double Tic Tac Toe is an example.
On each move, players play on one of the
two boards.
First to win on either board wins the
overall game.
If the game is left unfactored, the
branching factors are quite large.
There are 81 possibilities in that first
move.
Nine for each board, and then they're 64
in a second and so forth.
However, the games are factored, the
branching factors are much more modest.
Game factoring is the process of
discovering independent games inside of
larger games.
Once discovered, game players can use
these factors
to play the individual games independently
of each other.
And those cut down on the combinatory cost
of playing such games.
Turns out that it's often easier to
discover independent subgames
using the propnet representation for
games, rather than other representations.
Now in this segment, we'll look at
some elementary techniques for factoring
games using propnets.
We'll first talk about discovering factors
with completely independent subgames.
We'll then talk about factoring with
interdependent termination and rewards.
And after that, we talk about factoring
with interdependent actions as well.
And finally we'll discuss briefly the
conditional factors.
That is factors that appear only
in certain states of the games but not in
all states.
Okay, so let's begin our discussion of
factoring
with a simple case of a compound game.
consisting of multiple and completely
independent subgames.
Only one of which is relevant to the
overall game.
Without factoring, the player is likely to
consider actions of
all of the subgames, when only one of
those subgames matters.
Now, okay, now this is admittedly a very
special case.
It doesn't arise often,
and it can be solved by means other than
factoring.
But not by the method we've seen thus far.
Nevertheless, it's worth considering
because of it prepares
us for factoring more complicated sorts of
games.
Okay, Multiple Buttons and Lights is an
example of a game of this sort.
The overall game consists of multiple
copies of mul-, buttons and lights, and
each copy of buttons and lights, there are
three base propositions for lights.
And three actions,
the buttons.
Pushing the first button in each group
toggles the first light in that group.
Pushing the second button in each group
interchanges
the first and second lights in that group.
And pushing the third button in each group
interchanges the second and third lights
in that group.
Initially, all of the lights are off.
The goal is to turn on all of the lights
in the middle group.
The setting of the others doesn't matter.
Now here's a propnet for the game.
Notice that there are three disjoined
parts of the propnet.
One portion for the first group of buttons
and lights, a second
portion for the second group, and a third
portion for the third group.
And note that the goal and termination
conditions for the overall
game are based entirely on the lights in
the second group.
Looking at the propnet for this game, it's
easy to
see that the game is of this very special
structure.
The propnet consists entirely of three
completely disconnected subnets, one for
each subgame.
Finding factors in situations like this is
really easy, it can
be computed in time that's polynomial in
the size of the propnet.
And having found these factors, the
solution is really simple.
The player just keeps the essential
component and discards the other two.
Now, note that this technique can be
applied equally well to multiple player
games.
For example, Multiple Tic Tac Toe, that is
three games of
Tic Tac Toe glued together, in which only
the middle game matters.
Propnet from Multiple Tic Tac Toe is
similar
to the propnet from Multiple Buttons and
Lights.
And it's possible to find the structure in
Multiple Tic
Tac Toe just as easily as from Multiple
Buttons and Lights.
Okay, now let's consider a slightly more
complicated case.
Namely, compound games with interdependent
termination.
We can assume interdependent goals as well
but I'm going
to ignore goals for now and just focus on
termination.
As with the case of independent subgames,
entirely independent subgames actions are
petitioned over distinct subgames and
there
are no incoming connections between those
subgames.
The main difference is that the
termination condition for the overall game
can depend on more than one and perhaps
all of the subgames.
Now in games of this sort, the termination
of the overall game
can be defined as any boolean combination
of conditions in the individual games.
In the case where the combinations are
disjunction
then the game is said to be disjunctive.
In the case where the combinations is
conjunct the game is said to be
conjunctive.
As a simple example of a factorable game
with disjunctive termination, consider
Best Buttons and Lights.
In multiple buttons and lights, only one
group of buttons and lights matters.
In Best Buttons and Lights, the compound
games
terminate, compound game terminates,
whenever any group terminates.
Not just the one in the middle.
This gives the player the freedom to play
whichever subgame
it likes and rest assured that if it
succeeds on
that subgame, it succeeds on the overall
game with the same score.
Okay here's a propnet for Best Buttons and
Lights.
It's similar to the propnet for Multiple
Buttons and Lights, except that the goals
and termination are based disjunctively on
all three subgames.
The good news is that we can
extend the technique, this discussed, in
just discussed.
Let's consider the simply disjunctive
case.
If the connective leading to a termination
is a disjunction as it is
in this case with its inputs, in terms
supplied by nodes in different subgames,
and we simply cut off the overall
termination condition, and and, and make
the termination of each of the individual
subnets a termination condition for that
subnet.
And we
can repeat this process as long as we only
encounter disjunctions.
If the game is truly disjunctive, this
will eventually
lead to a separable propnet like the ones
shown here.
Now, if the process does succeed in
factoring the propnet in this way, then
the
player simply picks one of the subgames,
and
proceeds as with the case of independent
subgames.
Alternatively and better, the player tries
playing all of
the subgames, and picks the one with the
best score.
Or, at least that's the basic idea.
Unfortunately, it's not quite that simple.
There's a problem that does not arise
in the case of completely independent
subgames.
The player may choose one subgame, find
a winning strategy, and begin executing
that strategy.
Unfortunately in the course of execution
on the
chosen subgame, one of the other subgames
may terminate.
Terminating the game as a whole before
the strategy in the chosen subgame is
complete.
This can lead to a lower
score than a player might have otherwise
expected.
The solution to this problem is to check
each of the
subgames for terminations when no actions
are played in that subgame.
The player takes the shortest time period
and
plays each of the other subgames with that
as a step limit and takes the one
that provides the best result within that
step limit.
For the subgame with the shortest
termination, if there are no
other subgames with that step limit, the
player tries the next
shortest step limit and so forth.
Although in this approach the player
searches all of the sub-games more than
once, this is usually a lot less expensive
that searching the game tree with
the unfactored game because it's not, not
cross multiply the branching factors of
the independent games as it would if it
did not use the game's factors.
And once again, it's possible to extend
this technique to
multi-player games, but this has to be
done with some care.
Okay, finally, let's look at compound
games with interdependent actions,
interdependent, interdependent actions.
By interdependence of actions, we mean
those actions have effects
on more than one of the subgames of the
compound game.
Okay, now, at first blush, it might seem
that
games of this sort are just plain not
factorable.
However, that's not necessarily true.
Under certain circumstances, even with
interdependent
actions, it's possible to identify factors
and use those factors to
search the game tree more efficiently then
without considering the factors.
Okay here's an example, this is called
Joint Buttons and Lights.
As with the other buttons and lights
variants
that we've seen so far, lights in this
game
are organized in terms of three, however
unlike the
previous variance, buttons are not
associated in specific groups,
instead each button has effects on all
three groups.
Button aaa toggles the first light in each
group.
Button aab toggles the first light in the
first group and the first light in the
second
group and interchanges the values of the
first and
second light in the third group, and so
forth.
So we have all possible combinations of
the actions.
In the preceding games.
Although all of the buttons in this game
affect all of the subgames.
And the game is still factorable with just
three actions per factor.
The reason is that there's one button in
the compound
game for each combination of actions in
the other two subgames.
Thus the game trees for each subgame can
be searched independently of each other
and the actions chosen can be reassembled
into overall actions of the compound game.
For example, if action a is
chosen in the first and second subgames,
and action c is chosen in
the third subgame, then action aac can be
performed in the compound game.
Okay, now recognizing when this can be
done is not easy.
But it is doable.
There are just basically two steps, and
the first step the
player groups actions into equivalence
classes of actions for each subgame.
And in the second step, the player
determines whether these
equivalence classes satisfy what is called
the lossless join condition.
Finding equivalence classes is done by
looking at the propnet for each subgame.
Two actions are equivalent if they are
used
in the same way in the propnet for that
subgame.
For example, if each action is input to an
and gate
with a same other input, and if the
outputs are connected by
an or gate, then the effects of one action
can not be
distinguished from the effects of the
other action, hence they are equivalent.
The process of finding equivalence classes
is repeated for each potential subgame.
In general, the equivalence classes for
each subgame will be different.
In fact, as we'll see, in order for
the game to be factorable, they must be
different.
Once a player has equivalence classes for
each potential subgame, and
then checks whether those equivalence
classes are independent of each other.
Criterion is simple.
Each equivalence class in one potential
subgame must have a
non-empty intersection with each
equivalence
class of every other potential subgame.
If this is true, then these
partitions pass the so-called lossless
join criterion.
More details in the notes.
If
a player finds equivalence classes for
various two for potential subgames.
And they pass this lossless join tests.
And the player can factor the games into
subgames.
In order to benefit from the factoring,
the player must modify each propnet so
that the individual actions are replaced
with
the equivalence classes of which they are
members.
This cuts down on the number of possible
actions to consider, otherwise the
branching factor of the
game trees for the subgames will be just
as
large as the branching factor for the
overall game.
Once this is done, the player can then
search the game trees
for the different subgames to select
actions to perform in each subgame.
They can then find an action in the
compound game for the particular
combination of subgame actions.
This is always doable provided that the
partitioning of action satisfies that
lossless joins property.
the player then performs any action in the
intersection of the equivalence classes
chosen in this process.
Okay, again, check out the notes for more
details on, on factoring games with
interdependent actions.
Finally, let's look at one more case,
namely, conditional factoring.
That is factoring in cases where a game is
not factorable to start out, but over time
becomes factorable.
Here's an example.
It's called Linked Tic Tac Toe.
Two games of Tic Tac Toe connect,
connected by
a single square that, that connects the
two subgames.
The goal of the game as a whole is for the
player to get two lines a row, column, or
diagonal of that player's mark, with at
least two of
the marks residing in a specific Tic Tac
Toe domain.
Diagonals through the middle, don't count,
diagonals don't count.
When in each turn the player in control
can place two marks either one in each, in
each distinct Tic Tac Toe domain, or one
mark in the Tic Tac Toe domain, and one
in the center square.
Now suppose the state of the game is as
shown here.
Once it is not possible like this to
achieve
a row utilizing the center square, the
only possible
solution lie in the domains of the Tic Tac
Toe games that are conjoined by the center
square.
The states of these two Tic Tac Toe games
can be considered
independently to find the remaining
optimal
moves for the duration of the game.
Only the game trees for each
Tic Tac Toe game, modular the center
square, need
to be searched to determine the remaining
optimal moves.
Given the current state, the game can best
be factored into two independent subgames
at this point.
However, for the initial state, this game
cannot
be factored into sub, independent games
because the shared
middle square intertwines the two domains
as it's
relevant to the satisfaction goals in the,
both subgames.
While this game is not factorable, in
general, it is factorable
contingent upon entering a state in which
no
row through the middle is possible for
either player.
So the game is s-, said to be contingently
factorable since it
reduces to independent simultaneous
subgames, given
that it enters in this certain state.
It's contingent upon this state.
The raw computational benefit acquired
from recognizing contingent factors like
these is less than that of recognizing
initial factors because
it requires that the game be in a specific
state.
However, the re-, relative computational
savings are still
the same because the number of accessible
fringe nodes
reduces to the sum of the remaining
accessible fringe
nodes of the individual games rather than
their product.
Okay, at this point in time, there
are no established techniques for
discovering and exploiting
cases of contingent, conditional
independence, although there is
some work that is being done right now.
However, it seems likely that some of
these techniques
that we have already discussed can be
further adapted
or these new techniques that are being
discussed may
turn out to have power in finding
conditional factors.
Okay, game factoring is complicated, as
you
probably have guessed but it is really
worthwhile.
You can significantly decrease the cost of
game playing.
Or not the only source of propnet based
game analysis that is worthwhile.
For example, propnets can be used to find
bottlenecks.
That is, games structured in a series
where the
player must win the first game before
playing subsequent games.
Propnets can be used to eliminate dead
states, that
is states that cannot lead to an
acceptable solution.
And which can, therefore, be pruned from
further consideration.
And propnets can also be used to detect
certain kinds of monotonicities, that is,
cases where
higher goal values on non-terminal states
correlate with
progress toward terminal states with
acceptable goal values.
And there's a lot more research to be done
in this area.