BTW, have you written anything about your move that I might cite here? Or, if it’s not already out there, could I convince you to write something about the larger personal context of your work — like Meehan on Tale-Spin and Turner on Minstrel?

Meanwhile, in a later section I’m going to get into things like Oz, The Expressivator, and Facade — so will at least approach some of the work that leads up to Alex’s on behavior trees.

That said, while I’m interested in the article you mention (and will check it out), the main point of the section of this chapter I’ll post on Monday is that, when we’re building media, I don’t think we should be too concerned about how well our AI models match theories of human cognition. Still, since I’m mentioning the scruffy argument here, it would probably make sense here to footnote the fact that there’s some evidence that supports viewing Strips as a model of some kinds of human thought.

It’s been a while since I’ve read your F.E.A.R. papers. Do you say anything specific about the implementation of squad behaviors? If not, would you be willing to say a little here?

Thanks for the pointer to your GOAP list. I’m guessing it will get even longer before the paper version of Expressive Processing is in the hands of readers, but I think I’ll still add a footnote about how the approach is finding traction with other creators.

I agree with Richard in principal that the separation between planning and executing is not a perfect solution, and is not the way real humans work, but in the case of an FPS, FEARs planner should be able to handle the cases Richard brought up. As Noah suggested, an AI running from Zorgs could repeatedly replan running to new destinations until all options are exhausted, at which point he could cower in fear. This is almost exactly how AI in FEAR handle grenades — run away if you can find a route, and crouch for cover from the blast otherwise.

But what is important to note is that the motivation to pursue a planning system for FEAR was really to make authoring complex behaviors more comprehensible by human designers. By breaking behavior in bite-sized modular chunks, and offloading low level action sequencing to the AI characters themselves, we freed ourselves to focus on the higher level of complexity — squad behaviors.

The problem is that STRIPS is not a hierarchical planner. When we think about squads, we need a plan that can handle multiple agents acting in parallel. More recent developments in planning include Hierarchical Task Network (HTN) planners, which may be well suited for games. Since FEAR was my first foray into planning, I chose to focus on formalizing planning for individuals, rather than biting off more than I could chew.

But the next idea I’m trying to get at in this section (which clearly needs some reworking) is this one: In an authored microworld (like a game) can we create a situation in which that gap won’t matter to the audience?

For example, take the three routes to the vehicle. If the planner says “Get to the vehicle” then the executor can choose the wrong route, due to lack of information. If the planner says “Get to the vehicle via this route” then the executor can choose the wrong method of movement (e.g., choosing skiing, when the whole reason to move is that the snow is almost completely melted). And so on. The problem can always appear at the level where the planner hands off any meaningful decision to the executor.

In the everyday world, we’ll always have more in the world than the planner can handle. But in a simplified, authored microworld, shouldn’t we be able to plan “all the way down”? That’s to say, what if the planner chooses to leave and chooses which route to take — and the world involves no choice between skiing and walking? In this case, it seems like the NPC/AI can start to execute something that made sense during planning, then perhaps find out it doesn’t make sense, then re-plan, and the result is perfectly appropriate character behavior (e.g., starting to take route B, finding it too dangerous, visibly registering surprise, taking a moment to re-plan, turning around and going another way).

For this to work, of course, we need to enable it. We could try to give the executor all the information about what would invalidate the plan (e.g., Planex knows what the state of the world is supposed to be before the execution of each stage of the plan) and then the executor can run until something invalidates the plan. Or, alternately, we could re-plan after anything important happens (e.g., the NPC makes the first move down the path and there’s an explosion, or some Zorgs come in from an unexpected direction, or new information comes in from a teammate, or the character actually makes it to the vehicle, or…). My thought, in this section, is that F.E.A.R. points to something like this second option, doing quite well for its limited characters and fictional world.

But our conversation so far has me concerned. Is there something seriously flawed about this reasoning? Or is there a wider point that this way of posing things misses? Or, perhaps, is the problem that I’m not stating things clearly? I appreciate your thoughts on this.

Suppose the planner tells you to get to your vehicle. There are three paths to your vehicle, A B and C. Both B and C are more time-efficient, but are infested with Zorg guards. A is less time-efficient, but safe. Now if your executor (the router) has no understanding that it has to avoid Zorg guards, then it will choose B or C. Once it starts executing, it will replan, perhaps it will try again and choose C this time. No matter how frequently it replans, the router (which is by hypothesis blind to the planner’s reasons) will continue to choose B or C (because they are nearer to the vehicle by hypothesis). So the agent will oscillate foolishly between B and C and never get to his vehicle via A. This is so, no matter how frequently you re-plan. The problem is deeper: the router doesn’t understand what the executor knows. (Of course, we can “fix” this problem by telling the router/executor to avoid the last n failed routes, but that is to cover up the problem, rather than to solve it. The real solution is to allow the executor to have access to all the information that the planner has access to).

That said, shouldn’t we make a distinction between continual replanning (as in your Memento post), replanning after each compound action and/or when a plan is invalidated (as I understand F.E.A.R.), and replanning after each execution step (as in Agre’s quote from the Strips creators)?

Obviously, it’s not specified what replanning after each execution step would mean in the context of media. Following Phoebe Sengers, I’d be inclined to define an “execution step” for an NPC/AI as something like “an action we have reason to believe the audience has perceived.” And I would want to imagine the current action context (what I’m doing now, and why, and what I was doing before) to be part of what informs the planning process. But maybe this kind of approach has pitfalls I’m not sufficiently acknowledging.

Meanwhile, I’m going to go back and look again at your three video posts on “Behavior Trees for Next-Gen Game AI” (starting here, for those following the discussion).

It makes things harder to debug, as you get the “memento” effect:
http://aigamedev.com/essays/memento-debugging-planner

I think it’s also very wasteful, and potentially a source of oscillation unless you have good safeguards in place. There have been a few projects in robotics and planning that integrate planning and execution, but I hope to experiment with these ideas myself in my Game::AI++ project… (See AiGameDev.com for details. :)

Given these conditions, it’s okay for a character to have cover invalidated, run to different cover, and then have that invalidated — the character will probably get killed before they bounce back and forth a couple more times and look to be dithering. (As your example character might, running back and forth between guard Zorgs and the cluster of Zorgs.)

The hard question is the next step, beyond F.E.A.R. (as it were).

For Agre the next step is to approach AI a different way, based on his ideas about how people operate intelligently in our contingent world — a matter, as I mention in this section, of improvisation.

But when we’re making media, like games, I don’t think we should worry too much about how real people operate in the everyday world. So our question is different, though still very challenging.

To return to your example, are you sure that re-planning after each execution step (as in Agre’s quote from the Strips creators) wouldn’t work here? If the planner decides to run to a safe place (away from the Zorg guards), then starts to run to the identified place (only to find a cluster of Zorgs), then (because a step has taken place) re-plans to find a different Zorg-free place to run (or take a different action, if the preconditions for running away can no longer be met, since no Zorg-free location is accessible) wouldn’t this potentially produce both appropriate and legible NPC/AI behavior? In fact, we could even add something to the system that causes the character to act increasingly at wit’s end the more quickly plans are being invalidated. Or it could open the powerful characterization device of showing how characters respond to pressure, with each character responding in their own way to a string of necessary re-planning, in a manner maximally legible to the audience — from exaggerated frightened dithering to cool calculation.

But back to the larger question about the effectiveness of re-planning. Does it all, perhaps, come down to the size of the execution steps after which re-planning occurs?

The really deep problem here is, as you say, that planning and plan execution are separate. Re-planning won’t make this problem go away – in bad cases, it will just make the plan executor appear to be stuttering.

The problem is that plan execution is typically a self-contained routine which has no access to the data of the planner. The planner can weigh up as many options as it likes, but then, once it has made its decision, the only piece of data that is passed to the plan executor is that decision. The reasons for the decision, and all the other ambient issues which the planner considered, are lost to the executor. But all that the player directly sees is the movement of the executor. All that information wasted!

A concrete example: the planner decides to run away from the guard because the guard is a Zorg, and Zorgs are scary. It passes its decision to the plan executor who routes in the other direction. Unfortunately, this route goes through a cluster of Zorgs! The problem here, to reiterate to the point of tedium, is that the plan executor (the router) does not have access to the information (avoid Zorgs!) which prompted the decision (to avoid the guard).