Last time we talked about robot control architectures, and about the importance of time-scale. We said that the capability to look ahead distinguished different architectures, because it requires time to compute, and therefore directly impacts the time-scale of the controller.
Today we are going to talk about reactive control, a class of architectures which operate on a short time-scale and do not look ahead.
Before we move to reactive control, we will cover another key distinguishing feature between architectures: representation of the world/environment, also called world modeling.
So far, everything your robot needed to know was immediately available by sensing its own behavior and/or the environment (distance to the wall, position, velocity, etc.). However, some tasks and architectures involve storing information about the environment internally, in the form of an internal representation of the environment.
For example, while exploring a maze, a robot may want to remember a sequence of moves it has made (e.g., "left, left, right, straight, right, left"), so it can back-track and find its way. Thus, the robot is constructing a representation of its path through the maze. The robot can also build a map of the maze, by drawing it using exact lengths of corridors and distances between walls, etc. This is also a representation of its environment, a model of the world. If two robots are working together, and one is much slower than the other, if the fast robot remembers/learns that the other is always slower, that is also a type of a model of the world, in this case, a model of the other robot.
There are numerous aspects of the world that a robot can represent/model, and numerous ways in which it can do it, including:
spatial: metric or topological (maps, navigable spaces, structures)
objects (instances of detectable things in the world)
actions (outcomes of specific actions on the self and environment)
self/ego (stored proprioception, self-limitations, etc.)
intentional (goals, intended actions, plans)
symbolic (abstract encoding of state/information)
etc...
The amount and type of representation or modeling used by a robot is critically related to the type of control architecture it is using. Some models are very elaborate; they take a long time to construct and are therefore kept around possibly throughout the lifetime of the robot's task (for example detailed metric maps). Others may be relatively quickly constructed and transient, used quickly and discarded or updated (for example the next few steps in a short plan, the immediate goal, etc.).
How long it takes to construct/build a model is an important aspect of the robot's controller. Similarly, how long it takes to use it is equally important.
Consider maps again: it takes a long time to construct an accurate and detailed metric map, because it requires exploring and measuring the environment. Furthermore, it takes time to use such a map as well (even if it took no time to construct it, but it was given to the robot by the designer); one must find the free/navigable spaces in the map, and then search through those to find the best path to the goal.
Similarly, any internal model can require time to construct and be used, and these timing requirements directly affect the time-scale of the controller.
A control architecture can make it easy or difficult to store internal models (just as a programming language can make it more or less convenient to build and store structures) and manipulate them, i.e., compute with them.
Food for thought:
Why might you not want to store and use internal models?
Can you use deliberative control without having some internal representation?
You might wonder how internal state, i.e., information a robot system keeps around, relates to representation. In principle, any internal state is a form of representation. What matters, as mentioned above, is the form and function of that representation. The reason two different terms are employed is as follows: state refers to the "status" of the system itself, whereas "representation" refers to arbitrary information that may be contained in the system.
Reactive control is based on a tight loop connecting the robot's sensors with its effectors. Purely reactive systems do not use any internal representations of the environment, and do not look ahead: they react to the current sensory information.
Thus, reactive systems use a direct mapping between sensors and effectors, and minimal, if any, state information. They consist of collections of rules that map specific situations to specific actions.
If a reactive system divides its perceptual world into mutually exclusive or unique situations, then only one of those situations can be triggered by any sensory input at any one time, and only one action will be activated as a result. This is the simplest form of a reactive control system.
It is often too difficult to split up all possible situations this way, or it may require unnecessary encoding. Consider the case of multiple sensors: to have mutually-exclusive sensory inputs, the controller must encode rules for all possible sensory combinations. There is an exponential number of those (Food for Thought: exponential in what?). This is, in fact, the robot's entire sensory space (as we defined earlier in the semester). This space then needs to be mapped to all possible actions (the action space), resulting in the complete control space for that robot.
Although this mapping is done while the system is being designed, i.e., not at run-time, it can be very tedious, and it results in a large look up table which takes space to encode/store in a robot, and can take time to search, unless some clever parallel look up technique is used.
In general, this complete mapping is not used in hand-designed reactive systems. Instead, specific situations trigger appropriate actions, and default actions are used to cover all other cases. Human designers can effectively reduce the sensory space to only the inputs/situations that matter, map those to the appropriate actions, and thus greatly simplify the control system.
Food for thought:
How would you put a reactive system together
automatically, i.e., how would you learn one? Would you be able
to automatically reduce the input space in a similar way?
If the rules are not triggered by mutually-exclusive conditions, more than one rule can be triggered in parallel, resulting in two or more different actions being output by the system. Deciding among multiple actions or behaviors is called arbitration, and is in general a difficult problem.
Arbitration can be done based on:
a fixed priority hierarchy (processes have pre-assigned priorities)
a dynamic hierarchy (process priorities change at run-time)
learning (process priorities may be initialized or not, and are learned at run-time, once or repeatedly/dynamically)
If a reactive system needs to support parallelism, i.e., the ability to execute multiple rules at once, the underlying programming language must have the ability to multi-task, i.e., execute several processes/pieces of code in parallel. The ability to multi-task is critical in reactive systems: if a system cannot monitor its sensors in parallel, but must go from one to another in sequence, it may miss some event, or at least the onset of an event, thus failing to react in time.
Now that we understand the building blocks of a reactive system (reactive rules coupling sensors and effectors, i.e., situations and actions), we need to consider principled ways of organizing reactive controllers. We will start with the best known reactive control architecture, the Subsumption Architecture, introduced by Rod Brooks at MIT in 1985.
The following are the guiding principles of the architecture:
systems are built from the bottom up
components are task-achieving actions/behaviors (not functional modules)
components can be executed in parallel
components are organized in layers, from the bottom up
lowest layers handle most basic tasks
newly added components and layers exploit the existing ones
each component provides and does not disrupt a tight coupling between sensing and action
there is no need for internal models: "the world is its own best model"
Here is a rough image of how the system works: If we number the layers from 0 up, we can assume that the 0th layer is constructed, debugged, and installed first. As layer 1 is added, layer 0 continues to function, but may be influenced by layer 1, and so on up. If layer 1 fails, layer 0 is unaffected. When layer 1 is designed, layer 0 is taken into consideration and utilized, i.e., its existence is subsumed, thus the name of the architecture. Layer 1 can inhibit the outputs of layer 0 or suppress its inputs.
Subsumption systems grow from the bottom up, and layers can keep being added, depending on the tasks of the robot. How exactly layers are split up depends on the specifics of the robot, the environment, and the task. There is no strict recipe, but some solutions are better than others, and most are derived empirically.
The inspiration behind the Subsumption Architecture is the evolutionary process, which introduces new competencies based on the existing ones. Complete creatures are not thrown out and new ones created from scratch; instead, solid, useful substrates are used to build up to more complex capabilities.
The original Subsumption Architecture was implemented using a language based on finite state machines (FSMs) augmented with a very small amount of state (AFSMs), themselves implemented in Lisp. (As we pointed out, the underlying language can be anything, but we will see how the structure of FSMs will be convenient for incremental system building.) An AFSM can be in one state at a time, can receive one or more inputs, and send one or more outputs AFSMs are connected by communication wires, which pass input and output messages between them.
Thus, a Subsumption Architecture controller, using the above programming language, is a network of AFSMs. The network can be divided into layers. Once a certain competence is achieved, such as for example moving around without collisions, that can be labeled as layer 0.
The next layer, say one that looks for objects and collects them, can then be added on top of layer 0, and called layer 1. The use of layers is meant to modularize the reactive system, so it is not good form to put a large number of connections between the layers, so that they are strongly coupled. This would violate the modularity of the system. However, a higher layer can always use a lower one, so layer 1 certainly can and should be coupled with layer 0, by using suppression and inhibition of the appropriate inputs and outputs.
Coupling between layers, and even between AFSMs, need not be through the system itself (i.e., not through explicit communication wires), but through the world. For example, one Subsumption robot you'll read about (Herbert) had a layer that looked around for soda cans, and when it picked one up, it tucked the arm in. Another one headed to the can depository whenever it sensed that the arm was tucked in. Notice that there is no internal wire between the AFSMs that achieve can finding, grabbing, arm tucking, and going home, yet the events are all executed in proper sequence, because the respective parts of the control system interact and activate each other through the world.
This brings us back to a key principle of the Subsumption Architecture, use the world as its own best model. The idea is that if the world can provide the information directly (through sensing), it is best to get it that way, rather than to store it internally in a representation (which may be large, slow, expensive, and outdated).
Food for thought:
How are goals encoded and achieved in reactive
systems?