Joint rate-distortion optimization for simultaneous texture and deep feature compression of facial images

The explosion of surveillance cameras in smart cites and the increasing demand of low latency visual analysis have pushed the horizon from the traditional image/video compression to feature compression. Due to the recent advances of face recognition, we investigate the simultaneous compression of facial images and deep features, which is demonstrated to be beneficial in terms of the whole system performance including visual quality and recognition accuracy. Herein, we propose the Texture-Feature-Quality-Index (TFQI) to measure the ultimate utility of the facial images based on automatic visual analysis and monitoring. Furthermore, based on TFQI, a bit allocation scheme is proposed to optimally allocate the given bits for images and features, such that the overall coding performance can be optimized. The proposed scheme is validated using the standard face verification benchmark, Labeled Faces in the Wild (LFW). Better rate-TFQI and rate-Accuracy performance compared to the traditional texture coding can be achieved, especially in the scenario of low bit-rate coding.