Associative memory is a concept that originated from the field of psychology, and in that original sense means recalling a previously experienced item by thinking of something that is linked with it, thus invoking the association. Associative memory or associative storage is a data-storage technique by which a location is identified by its informational content rather than by names, addresses, or relative positions, and from which the data may be retrieved. As with many areas where neural networks are highly efficient, associative memory is yet another form of optimization problem, in this case finding the best (optimal) match given partial information.
An associative memory is fundamentally different from a digital computer memory. A conventional computer stores information by assigning addresses, which identify the physical locations where the data will be stored in hardware, such as a sector or track on a hard drive. When the central processor requires a piece of data, it issues an instruction to read the data at a particular address. The address itself contains no information about the nature of the data stored there.
Now reflect for a moment about your own memories. If you think of a particular friend, you will remember many facts – name, age, hair color, height, job, hobbies, schooling, family, house, shared experiences and so on. These facts are somehow combined to form the memory of the individual. There is no notion of storage address in the way you retrieve such information from your memory. Instead pieces of the information itself are used in place of an address.
Fruit flies and garden slugs have associative memories. The fact that such relatively simple nervous systems display the phenomenon suggests that it must be a natural, almost spontaneous property of biological neural networks. Mathematical models of associative memory were developed in the 1970s. The concept of the computational energy surface provides a means to understand and study associative-memory circuits built of saturable amplifiers.
How would one make a collective-decision circuit behave like an associative memory? Think of a space of many Cartesian coordinates in which each axis is labeled with some attribute a person might have. One axis might refer to height, one to sailing experience, one to the first name of the individual, one to city of residence and so on. Any point in the space describes the characteristics of a hypothetical possible individual. Each of your friends is represented by a particular point in the space. Because you have very few friends compared with the set of all possible individuals, if you put a mark at the position of each of the people you know, you will have marked a very few points in a large space. When someone gives you partial information about a person – for example, color of hair and weight but not name – this describes an approximate location in the space of possible people. The idea of an associative memory is to find the friend who best matches the partial data.
A collective-decision circuit such as the one described for the task-assignment problem could perform as an associative memory if the computational energy surface can be shaped to have valleys, or stable points, at the places that correspond to particular memories. A pattern of input voltages corresponding to a partial memory would be supplied to the amplifiers and the circuit would then follow a trajectory to the bottom of a local valley in the computational energy terrain and read out the output state of the amplifiers as the stored memory. Unlike the task-assignment circuit, in which the connections are highly regular because of the simple global rules that constrain the problem, in an associative memory the connections are irregular and the stable points are scattered somewhat at random because the memories need not have any particular relationship among themselves. Therefore, to construct an associative memory one must find connections between amplifiers such that the many desired memories are represented simultaneously by the circuit’s stable states.
Further Reading:
An Associative Memory
No comments:
Post a Comment