Evaluating Perceptual Color Preferences in Smartphone Photography: Dataset and Challenges

In smartphone image signal processing (ISP), different parameter settings can yield diverse color renditions, even when images have similar color accuracy and aesthetic quality. A key yet underexplored question is: which rendition does a specific user or demographic prefer? This is difficult to answer due to the subjective nature of preference. Existing assessments focus on color fidelity or aesthetics using visibly degraded images, limiting their ability to capture subtle color preferences in similar image sets. Averaged metric predictions further obscure individual perceptual differences. To address these gaps, we present the Smartphone Photography Color Preference (SPCP) dataset-the largest of its kind-designed to evaluate color preferences arising from ISP-induced variations. The SPCP dataset comprises 12,000 images derived from 1,000 diverse scenes, with each scene rendered into 12 distinct variants. These variants include (i) real-world captures from six flagship smartphones and (ii) synthetic images generated through systematic variation of key ISP parameters. To obtain reliable ground-truth annotations, we conduct a large-scale psychophysical study involving 20 subjects under controlled laboratory conditions. Subjects perform exhaustive pairwise comparisons among the 12 variants for each scene, yielding fine-grained human preference data. Using this dataset, we identify three key challenges in modeling color preferences and outline the corresponding desiderata for the development of effective computational color preference models. The dataset is publicly available at: https://huggingface.co/datasets/zwx8981/SPCP_dataset.

© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.