Theoretical analysis and image reconstruction for multi-bit quanta image sensors

A new breed of photon-counting sensors, called quanta image sensor (QIS), enables the detection of light with precision represented by the number of photons arriving within a time period. However, most existing analysis results on QIS systems are formulated for the single-bit case only. Directly extending the existing single-bit analysis to the multi-bit case leads to the situation that the variance of the estimated exposure is greater than the Cramér-Rao bound (CRB). Also, the existing dynamic range analysis leads to a strange situation that the maximum exposure level is not a function of the spatiotemporal jot kernel size. This paper formally analyzes the properties of multi-bit QIS (MBQIS) systems. It derives the log likelihood function of the received photon counts in a spatiotemporal jot kernel, and introduces the concept of the probability of all jots being saturated. From the likelihood function result, we can obtain a maximum likelihood (ML) estimate for the exposure level and present an image construction algorithm, namely ML multi-bit (MLM). Since the estimate is ML based, the variance of the estimated exposure achieves the CRB asymptotically and the MLM is an asymptotically unbiased estimator. Also, based on the Fisher information concept, this paper derives the CRB on the variance of the estimated exposure. Hence the CRB given by this paper can be considered as a performance indicator for all algorithms. From the jot saturation analysis result, we can accurately formulate the relationship between dynamic range and spatiotemporal kernel size. Specifically, with our two analysis results, we can model the relationships between sensor design parameters and performance metrics (variance of the estimated exposure and the dynamic range). Since the two analysis results are independent of the construction algorithms used, they give us some guidelines to design a QIS system. In addition, this paper empirically studies the effect of the readout Gaussian noise. Finally, to demonstrate another application of the likelihood analysis result, we develop an enhanced version of MLM, namely MLM with denoising (MLMDN), based on the proposed likelihood function and the regularization concept.