Revealing Visual Cognition with AI Simulator: Hierarchical Attention Entropy Derived from Artificial Neural Network

Leveraging the structural similarities between artificial neural networks and human visual processing, this study proposes a novel approach to approximate hierarchical visual complexity perception with the layered attention matrices of transformer-based models. Using the vision encoder of CLIP, a transformer-based multimodal model, we propose Hierarchical Attention Entropy (HAE) to simulate stagewise brain activation during visual perception. Our initial results from a social media dataset reveal that early-stage attention entropy (low-level and high-level AE) positively correlates with user engagement, while late-stage entropy (high-level AE) shows a negative relationship. This method contributes a cognitively inspired data augmentation approach and offers new insights into how hierarchical visual processing influences user behavior online.