http://www.codeasart.com/poetry/darwin.html

The idea is the computer shows you two poems, and you choose which of the two you like. The poems that get chosen are then “bred” with each other, to create new poems that are then served out for more voting. If it were somehow possible to find out the characteristics of surviving poems, maybe the action of “breeding” the poems could be made more intelligent.

Even when automated (by running strategies against one another according to rules and seeing which ones persist) evolutionary methods are the indeed the “opposite” of machine learning. Machine learning systems improve their performance over time, while evolutionary systems, considered individually, don’t change. It’s only by selecting particular systems, the winners, that such an approach results in overall improvement.

Isn’t that like saying that we want individual ants to learn, while ignoring the fact that an anthill, the community of ants, already learns plenty in the course of its 15 year lifetime? Surely that’s a bit of a waste?

Yes, I read Steven Johnson’s Emergence and loved the stories about the anthills! I had had no idea that anthills go through puberty.

Evolutionary approaches run a large number of programs against one another to select the one that best accomplishes the task. Typically, these programs that participate do not learn, as learning is defined in ML. They simply don’t improve their performance on a task based on experience, so they don’t learn.

Although you can draw the boundary of “computer program” at different places, it’s not at all a waste to distinguish these approaches. It’s the type of precision that allows us to make advances in the field, just as scholars of new media writing benefit from distinguishing ergodic and non-ergodic texts so they can develop specific techniques for understanding specific works. If we don’t understand this distinction in the evolutionary/machine learning case, it will be much harder, for instance, to understand the work that’s been done on combining machine learning approaches with evolutionary ones.

Learning clearly makes sense for long-term, open-ended experiences with intelligent computer characters. Learning takes time, and as you interact with a character, it could alter its attitudes and associations to other characters and objects in the world, make new connections, etc., which results in altering its behavior. For example, in Creatures, interaction over time modifies connections in each Norn’s neural net. Petz and Babyz can learn associations by way of positive and negative reinforcement. Far more ambitiously, one could imagine characters that learn new brand-new behaviors altogether.

But when thinking about ways to innovate and improve more tightly-plotted interactive stories, the idea of applying learning techniques hasn’t seemed as obviously useful to me as, say, new reactive planning techniques, NLP, or knowledge representation. Again, learning takes time; without creating stories that last a long time (weeks?), I’m not sure that enough could be learned during the play of an interactive story to have much effect. Unless it’s a short story designed to be played over and over for some reason, e.g., Facade; then theoretically you could alter subsequent plays with what you learned in previous plays.

Even if you find a good design reason for using learning, how do you implement it? How does learning “hook” into the system exactly? I think there’s a tension between top-down and bottom-up approaches here. Bottom-up approaches to characters and worlds, e.g. Creatures, the Sims, run open-ended simulations in which (fragments of) what we call “narrative” may or may not emerge over time. Whereas more top-down, authored approaches (e.g., some IF, drama managers) explicitly cause narratives to occur. From what I understand, learning techniques are more easily applied to bottom-up approaches, because they are more synthetic by nature; the bottom-up rules of a simulation are excellent “hooks” to apply learning. That is, make a small change in a rule or neural net weight, see big changes in behavior. But, IMO, bottom-up approaches, by their open-ended nature, run the risk of long streches of dullness where nothing much is happening — a problem if you’re trying to keep things interesting and dramatic. (I talk about this a bit in this essay from a few years back.)

“when is learning useful?” is a key question for desginers, and one reason I was looking for examples to refer to. Another issue is “what is supposed to learn?” The systems mentioned so far involve an agent within a simulated environment (a virtual baby on the desktop, or creature in the Creatures world, or creature in the Black & White world) and a situation in which the agent is learning. What about just having the entire program, considered as a whole, learn to improve its performance on a task such as “elict text input from the interactor”?

Perhaps you’ve answered that already: this would at least suggest less direct control over the experience for the designer/author, making this less welcoming of an approach for most people…

Sure, I’m with you — having the entire program as a whole improve its performance on a task, such as eliciting text input from the interactor, would be really great. That answers the “useful” question. Then I think, how would learning hook into that task? Ideally the “task”, whatever it is, would have to have some sort of bottom-up-ish rules that accomplish its behavior, some sort of hooks, e.g., at a minimum, a set of parameters that can be tweaked, that a learning process could modify, to cause changes in behavior. That is, to make learning possible, you’d have to originally code the task itself in such a way to expose hooks to a learning process. You couldn’t write that task’s code in the “normal” top-down way.

Or, perhaps more ambitiously, one could imagine (and I’m sure examples have been built) implementing tasks, such as eliciting text from a user, in a kind of top-down, simple, clean script or plan-like structure for controlling behavior. Then, you separately write AI that can truly reason about those scripts / plans / behaviors, and can actually either modify them, or write new ones. AI code that is smart enough to write new code. I believe case-based reasoning is an example of something that at least modifies already-written control structures, in an attempt to create new ones.

Michael is well read up on this kind of research, when he’s back from Cosign he can hopefully point to some examples for us. Or, maybe some of our readers can point to examples?