Research - Action Selection

Action Selection / sub-symbolic Society of Mind

My main interest in AI is in large, sub-symbolic, decentralised "Society of Mind" models of agent minds.

My PhD thesis is on the "Action Selection" problem - the problem of run-time choice between multiple parallel, competing, conflicting and overlapping goals, that must respond to unpredictable and passing opportunities in the world - in other words the "Operating System" for a multiple-goal autonomous creature. This is about the overall architecture of a mind - how does an animal, mobile robot or software agent decide which goals to pursue at a given moment, when to interrupt, and when to opportunistically divert, in response to events in the world? The thesis contains a Survey of Action Selection. For a shorter introduction see the SAB-96 paper.

Publications

Action Selection publications

My PhD "family tree" (Who supervised who)

Video demo of W-learning.

My PhD

My PhD thesis argues for a self-organising such "Operating System", based on an internal currency. One way of generating such an internal currency is by numeric techniques such as Reinforcement Learning (RL). The thesis contains an Introduction to Reinforcement Learning.

This is a strongly decentralised model of mind, with internal tension and competition between selfish behaviors. I introduce an algorithm called "W-learning", whereby different parts of the mind modify their behaviour based on whether or not they are succeeding in getting the body to execute their actions. The thesis sets W-learning in context among the different ways of exploiting RL numbers for the purposes of Action Selection. It is a "Minimize the Worst Unhappiness" strategy.

My model of mind is radically decentralised, with the goals set up in free competition against each other. You can view it as Multiple Minds in the same Body. It is a numeric (non-symbolic) Society of Mind which functions as an individual-driven liberal democracy.

A radically decentralised mind
made up of 3 agents competing against each other for control of the body.

Methods that resolve competition (organise action selection)
without reference to a global reward.

Their political analogs.

The sub-symbolic Society of Mind

Where this is headed is towards a complex, overlapping, competing, sub-symbolic Society of Mind based on Reinforcement Learning.

The general form of a Society of Mind based on Reinforcement Learning.
This is not a hierarchy.

My model of mind can be seen as a primitive "Joycean machine" (see Dennett's Consciousness Explained).
It may be of interest that my grandfather knew Joyce (though he was not a fan).

It is ironic that I have come up with such a seething, competitive, multi-processor model of mind, since this is not the way I personally seem to work at all! Any of my friends will tell you that I am a mono-processor, who works on one thing at a time, devoting lots of concentration to it and being immune to interrupts. Of course my model can produce such a serial result - with all the frustrated competition still going on below the conscious level.

Minds should not have a Single Thread of Control

One way of considering this "multiple-minds" view of AI is as follows:

Movie demo of W-learning in the Ant World problem

Movie demo of the House Robot problem