Neuhash: Hash-based Architectures for Efficient Neuromorphic Visual Processing

The convergence of artificial intelligence (AI) and vision sensors in the Internet of Things (IoT) promises to enable many applications ranging from handphones with visual recognition to smart glasses, smart cameras for drones, etc. The significant challenge for this is the simultaneous requirement of high frame rate and low size, weight, and power (SWaP) for many applications, such as eye tracking or collision avoidance in drones. A high frame rate produces copious data that invariably requires considerable energy and latency for processing. This proposal aims to overcome these challenges using novel architectures, integrated circuits, and neuromorphic vision sensors. Neuromorphic cameras have pixels that produce data asynchronously only if there is sufficient change in temporal contrast, thus compressing data at the source and allowing fast readout of active pixels since the whole frame does not need access. However, this new format of asynchronous, sparse event data requires redesigning algorithms,architectures, and hardware of the following processor. We propose two solutions that exploit the sparsity of event data.In the first architecture for low-level image processing, we propose to use a hashingbashed bloom filter (BF) method from computer architecture to map the output of the high-dimensional sensor array to a smaller-sized memory, exploiting the sparsity of input data. Conventional BFs suffer from the need to reset the memory to reduce false detections frequently. To this end, we propose to extend the BF to a second dimension with the possibility of a partial reset of the oldest rows. In the second architecture for high-level object recognition, we propose to use spiking neural networks (SNN) with in-memory computing to accelerate its execution. Using analog locally-sensitive hash (LSH) functions, neurons most likely to fire in the following layers can be estimated, and the remaining can be turned off. This leads to energy savings and an increase in throughput since the slow serial operation of neural state update can be reduced in proportion to sparsity. Both methods provide a completely new degree of freedom to study approximationresource tradeoffs for vision systems. The first method is expected to yield >10X reduction in memory and ~5X reduction in energy for memory access, while the second is estimated to improve energy-efficiency and throughput of SNNs by ~ 4-10X. We expect the outcomes of this project to enable many new revolutionary vision applications in the future.