Adaptive In-Sensor Computing for Enhanced Feature Perception and Broadband Image Restoration

Traditional imaging systems struggle in weak or complex lighting environments due to their fixed spectral responses, resulting in spectral mismatches and degraded image quality. To address these challenges, a bioinspired adaptive broadband image sensor is developed. This innovative sensor leverages a meticulously designed type-I heterojunction alignment of 0D perovskite quantum dots (PQDs) and 2D black phosphorus (BP). This configuration enables efficient carrier injection control and advanced computing capabilities within an integrated phototransistor array. The sensor's unique responses to both visible and infrared (IR) light facilitate selective enhancement and precise feature extraction under varying lighting conditions. Furthermore, it supports real-time convolution and image restoration within a convolutional autoencoder (CAE) network, effectively countering image degradation by capturing spectral features. Remarkably, the hardware responsivity weights perform comparably to software-trained weights, achieving an image restoration accuracy of over 85%. This approach offers a robust and versatile solution for machine vision applications that demand precise and adaptive imaging in dynamic lighting environments. © 2024 Wiley-VCH GmbH.