LLaVA-Oil Painting Appreciation: A Vision-Language Model for Enhancing Understanding of Oil Paintings Through AI-Driven Analysis and Conversation

Oil painting has been a dominant art form since the 15th century, known for its profound impact on artistic expression. While exhibitions offer people unprecedented access to masterpieces, many viewers lack the formal artistic training or understanding of oil painting found hard to appreciate the intricate techniques and historical context of these works. This paper introduces LLaVA-Oil Painting Appreciation (LLaVA-OPA), a vision-language conversational assistant designed to help individuals engage more deeply with oil paintings. Leveraging the LLaVA-1.6 model and fine-tuning it using LoRA, LLaVA-OPA is trained on a curated dataset of oil paintings, supplemented by expert annotations from art scholars and professionals. The model identifies key elements such as brushwork, composition, and style, offering insightful, accessible feedback to users. By incorporating GPT-4 for generating open-ended questions and responses, LLaVA-OPA creates a conversational experience that allows users, regardless of prior art knowledge, to explore the complexities of oil paintings in an engaging manner. Initial evaluations demonstrate that LLaVA-OPA excels in providing tailored, expert-like analysis while maintaining user-friendly interactions. Future research will focus on deepening the model's understanding by integrating information about artists' creative inspirations and historical context, enabling even more nuanced and meaningful conversations in various art settings, including exhibitions. © 2024 IEEE.