The greatest increases in speed brought about by parallel processing have been achieved using highly parallel SIMD computers such as the CLIP4, DAP, MPP, Connection Machine CM-2, and MasPar MP-1. The computations have been primarily in applications such as image processing, graphics, fluid flow simulation, and others using 2D and 3D systems of equations. The range of applications is currently limited by factors such as lack of addressing autonomy in each processing element, an I/O bottleneck, insufficient data-moving capacity, and inadequate arithmetic capability. Some of these limitations could be overcome by improving the memory systems of such architectures. Advances in memory technology now permit the integration of a great deal of logic circuitry in memory chips, with the potential to greatly increase the flexibility of the memory subsystem. Some recent developments in memories are discussed, and proposals are made to enhance the memory systems of highly parallel SIMD computers.