There are many tasks that require learning a temporal sequence of events. These problems can be broken into 3 distinct types of tasks:
Some of the methods that are used include
One of the simplest ways of performing sequence recognition because conventional backpropagation algorithms can be used.
Downsides: Memory is limited by length of tapped delay line. If a large number of input units are needed then computation can be slow and many examples are needed.
A simple extension to this is to allow non-uniform sampling:
where wi is the integer delay assoicated with component i. Thus if there are n input units, the memory is not limited simply the previous n timesteps.
Another extension that deals is for each "input" to really be a convolution of the original input sequence.
In the case of the delay line memories:
Other variations for c are shown graphically below:
This figure is taken from "Neural Net Architectures for Temporal Sequence Processing", by Mike Moser.
[Top] [Next: Recurrent Networks I] [Back to the first page]