Utility-Aware Image Quality Perception for Trustworthy Recognition

Abstract

Recognition systems in unconstrained environments are highly susceptible to variations in input quality, which directly threaten recognition reliability and trustworthiness. As a critical component in ensuring the robustness of recognition systems, utility-aware image quality perception has become an essential yet challenging task for trustworthy recognition. The core premise of utility-aware image quality perception is to predict the quality of the input image in a way that is directly aligned with its utility for deployed recognition models. This thesis focuses on exploring utility-aware quality perception for trustworthy recognition, addressing the critical research gaps through four primary parts: 1) A domain adaptation framework to mitigate ethnic quality bias in Face Image Quality Assessment (FIQA); 2) A confidence-calibrated method that leverages multi-modal quality factors to overcome fitting bottlenecks in FIQA model training; 3) A quality-aware feature refinement framework that uses quality priors to optimize deep recognition models; and 4) The pioneering creation of a synthetic dataset and a novel multi-reference quality characterization method for privacy-preserving FIQA.

In the first part, we propose the Ethnic-Quality-Bias Mitigating (EQBM) framework, which is the first attempt in FIQA to address the quality bias across different ethnic groups. Specifically, to overcome the feature-scaling challenge inherent in transferring regression-based tasks, we adopt the Likert-scale quality probability distributions as source-domain annotations. Furthermore, to effectively reduce source risk and enhance transfer learning, we design an easy-to-hard training scheduler guided by inter-domain uncertainty and intra-domain quality margins, along with the ranking-based domain-adversarial network. Experimental results show that our framework significantly mitigates ethnic quality bias.

In the second part, we identify a fitting bottleneck issue in quality-fitting-based FIQA methods, which indiscriminately treat the confidence of accurate and inaccurate quality anchors. We present Confidence-Calibrated Face Image Quality Assessment (CLIB-FIQA), a novel approach that establishes a synergistic interplay between quality anchors and objective quality factors (e.g. blur, pose, occlusion). Built upon a vision-language alignment model, our method leverages the joint distribution of multiple quality factors to facilitate more robust quality fitting. Furthermore, we propose a confidence calibration mechanism that corrects the quality distribution by exploiting these objective factors during training, alleviating the problem of over-trust in inaccurate anchors and leading to superior performance across diverse datasets.

In the third part, we shift focus from assessing quality to refining recognition features directly. We propose a quality-aware feature refinement framework based on dedicated quality priors derived from recognition performance. Through a novel quality self-distillation solution, we are able to refine deep recognition features for low-quality images during training. We demonstrate that this framework significantly boosts recognition performance for face recognition and person re-identification tasks, showing impressive generalization capability and seamless integration with existing approaches without added deployment costs.

In the fourth part, to address the pressing privacy concerns that have halted the release of real-world FIQA datasets, we undertake a pioneering initiative to establish a Synthetic dataset for FIQA (SynFIQA). We validate the hypothesis that accurate quality labels can be derived from quality priors across diverse domains in a quality-controllable generation process. This involves tailoring the generation of reference and degraded samples using stable diffusion, 3D facial parameters, and custom text prompts. Furthermore, we propose a novel quality characterization method that examines relationships across multiple reference representations, including recognition embedding, spatial, and visual-language domains, to acquire the necessary annotations for fitting FIQA models. Extensive experiments confirm the validity of our synthetic data and the advantages of our characterization method.

Overall, this thesis advances the field of utility-aware image quality perception from four key perspectives: 1) It enhances the generalization and fairness of FIQA through a novel domain adaptation framework that mitigates ethnic bias. 2) It improves the reliability of FIQA model training by confidence calibration based on multi-modal quality factors. 3) It expands the scope of quality awareness by demonstrating how quality priors can directly refine features within recognition models themselves. 4) It pioneers a privacy-preserving pathway for future FIQA research by establishing the first dedicated synthetic dataset and a multi-reference quality characterization method. Extensive experimental results verify the effectiveness and superiority of all the proposed schemes in building trustworthy recognition.

Date of Award	8 May 2026
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Shiqi WANG (Supervisor) & Tak Wu Sam KWONG (External Co-Supervisor)