Despite the success of connectionist systems to model some aspects of cognition, critics argue that the lack of symbol processing makes them inadequate for modelling
high-level cognitive tasks which require the representation and processing of hierarchical structures. In this thesis we investigate four mechanisms for encoding hierarchical structures in distributed representations that are suitable for processing in
connectionist systems: Tensor Product Representation, Recursive Auto-Associative
Memory (RAAM), Holographic Reduced Representation (HRR), and Binary Spatter
Code (BSC). In these four schemes representations of hierarchical structures are either
learned in a connectionist network or constructed by means of various mathematical
operations from binary or real-value vectors.
It is argued that the resulting representations carry structural information without being themselves syntactically structured. The structural information about a represented
object is encoded in the position of its representation in a high-dimensional representational space. We use Principal Component Analysis and constructivist networks to
show that well-separated clusters consisting of representations for structurally similar
hierarchical objects are formed in the representational spaces of RAAMs and HRRs.
The spatial structure of HRRs and RAAM representations supports the holistic yet
structure-sensitive processing of them. Holistic operations on RAAM representations
can be learned by backpropagation networks. However, holistic operators over HRRs,
Tensor Products, and BSCs have to be constructed by hand, which is not a desirable situation. We propose two new algorithms for learning holistic transformations of HRRs
from examples. These algorithms are able to generalise the acquired knowledge to
hierarchical objects of higher complexity than the training examples. Such generalisations exhibit systematicity of a degree which, to our best knowledge, has not yet been
achieved by any other comparable learning method.
Finally, we outline how a number of holistic transformations can be learned in parallel and applied to representations of structurally different objects. The ability to distinguish and perform a number of different structure-sensitive operations is one step
towards a connectionist architecture that is capable of modelling complex high-level
cognitive tasks such as natural language processing and logical inference.