Near-DRAM Accelerated Matrix Multiplications

General matrix multiplication (GeMM) is a fundamental computing operation underpinning various applications in machine learning, data science, computer graphics, and scientific simulations. However, its performance on von Neumann computers, such as CPUs and GPUs, is bottlenecked due to challenges such as insufficient memory access bandwidth, low cache efficiency, hardware under-utilization, and massive power consumption. The performance of GPUs suffers drastically for complex sequential analysis often occurring along with GeMMs in various big data pipelines. Furthermore, GeMM-optimized Tensor cores available in modern Nvidia GPUs offer no extensibility to non-GeMM tasks. Various Near-Memory Computing (NMC) architectures have recently been explored to alleviate the data-intensive nature of analyses such as GeMM. This work evaluates the performance potential of GeMM using NMC through clusters of simple interconnected processing cores on a stacked DRAM platform. The organized design shows efficiency comparable to high-end Nvidia GPUs while consuming lower power and being highly extensible to various non-GeMM logically complex workloads.