A 512Gb In-Memory-Computing 3D-NAND Flash Supporting Similar-Vector-Matching Operations on Edge-AI Devices

Similar-vector-matching (SVM) applications for unstructured vectors that are generated via machine-learning methods, such as face search and audio texturing from a dataset for access control systems, are frequently operated on edge devices, as depicted in Fig. 7.5.1. The SVM operation [1]-[3] typically comprises of (1) in the offline phase, the extracted raw vectors (V_RAW) are obtained from machine learning approaches and stored in non-volatile NAND Flash; (2) in the online phase, a processor request V_RAW data from edge storage; (3) the entire V_RAW dataset is moved from storage to the processor; (4) the processor scores the similarities between an input query and each candidate V_RAW and provide a best match. However, the large-amount data movement across the memory hierarchy consumes a large amount of energy (E_MEM), while also resulting in a long search-latency (t_SR) for SVM operations. The entire V_RAW dataset includes a large amount of invalid data. To reducing data movement will lower E_MEM and t_SR; edge storage with nonvolatile computing-in-memory (nvCIM) support for similarity computation (vector-vector multiplication (VVM) for cosine similarity) is required to reduce the V_RAW dataset to a small candidate size. However, there are challenges in leveraging 3D NAND for VVM operations: (1) a low-readout accuracy when there is a large amount of current summation by using the wide range V_t-level of cells (e.g., 1^st to 4^th V_t-level of TLC cell) and (2) the large readout power consumption required to achieve a constant settling time against a wide range of summation currents for the possible data-patterns.