The most impressive abilities of associative memory are its capability of fault tolerance and associated retrieval. In an ordinary memory of a conventional computer, every piece of the stored information has a specific address. One must know the complete address to retrieve particular information. By contrast, in an associative memory one needs to know only partial information relative to the contents for accomplishing a complete recall. In a hetero-associative memory, by inputing a partial pattern of one category, an associated pattern of another category can be obtained. It is well known that this can be achieved by an optical holographical method, in which two associated patterns are utilized to the record interconnection weights at the Fourier plane. However, due to the grating degeneracy, the number of the associated pairs is limited because of the serious cross-talk between different pairs. In the paper, we present our studies on the properties of a hetero-associative memory of multiple-pairs. We use a phase modulated liquid crystal television (LCTV) as the reference pattern to record holographic memories and we propose a novel technique applied to generate phase only interconnection weights. The results show that the cross-talk between different pairs of memories can be significantly reduced by the phase only filter technique.