Explainable Fine-tuning Data Selection for Large Language Models

Instruction fine-tuning is indispensable in making pre-trained large language models (LLMs) effectively follow human instructions and achieve specific purposes. This process involves refining pre-trained models using datasets that consist of instruction-response pairs, bridging the gap between the model's inherent capabilities and user expectations. Despite being important, attributing, selecting, and constructing fine-tuning data are still cumbersome, which requires high-quality instructions and intense computation resources.Although recent studies have managed to identify high-quality fine-tuning data for LLMs, there are still several challenges: 1) lacking adaptive attribution metrics suited for different LLMs; 2) lacking effective approaches to compositely attribute instruction data; 3) lacking efficient schemes to examine data selection strategies and explain the selected instructions.To tackle the aforementioned challenges, we will conduct systematic investigations on the explainable fine-tuning data attribution for LLMs. Generally, our project consists of three major tasks.1) We plan to develop attribution metrics adapted to various LLMs. We will develop an LLM-aware metric adaptation scheme that calibrates the attribution metric tailored to LLMs' characteristics. Furthermore, we will design dataset-level attribution metrics to ensure fair and comprehensive assessments for instruction-tuning datasets regarding complexity and diversity.2) We plan to compositely attribute and select the instruction data through advanced techniques. We will resort to mutual information to measure the complementarity of selected instruction data and further enhance the composition. Moreover, we will leverage the Wasserstein distance between datasets to balance the proximity and diversity of selected data.3) We plan to incorporate the efficient examination of selection strategies and explainability into the instruction tuning. We will design a retraining-free scheme with the influence function to efficiently evaluate candidate data selection strategies. Furthermore, we will investigate explainable instruction tuning with the computationally efficient Shapley value, which will enhance the trustworthiness of selected instructions. With the support from industrial collaborators, our outcomes will establish a practical foundation for the analysis of data attribution in LLMs, provide effective data attribution techniques, enhance the training efficiency of supervised fine-tuning, and push forward the scientific frontier of this research area.