视觉指令微调（VIT）已成为使多模态大语言模型（MLLMs）能够熟练遵循用户指令的关键技术。然而，对于高质量指令微调数据的属性以及其自动化选择框架的理解仍存在显著差距。为了解决这一问题，我们提出了MLLM-Selector，这是一种通过权衡必要性和多样性来识别VIT中有价值数据的自动化方法。我们的过程从VIT数据池中随机采样一个子集以微调预训练模型，从而创建一个具有初步指令跟随能力的种子模型。然后，利用该种子模型，计算VIT数据池中每个样本的必要性得分，以确定对提升模型性能至关重要的样本。研究结果强调了在数据选择中混合必要性和多样性的的重要性，这促使我们开发了MLLM-Selector，一种融合必要性评分与策略性采样的方法以实现更优的数据精炼。实验结果显示，在相同实验条件下，MLLM-Selector在某些基准测试中仅使用不到1%的数据就超过了LLaVA-1.5，并且在使用少于50%的数据时，在所有验证的基准测试中始终表现出色。

Visual instruction tuning (VIT) has emerged as a crucial technique for enabling multi-modal large language models (MLLMs) to follow user instructions adeptly. Yet, a significant gap persists in understanding the attributes of high-quality instruction tuning data and frameworks for its automated selection. To address this, we introduce MLLM-Selector, an automated approach that identifies valuable data for VIT by weighing necessity and diversity. Our process starts by randomly sampling a subset from the VIT data pool to fine-tune a pretrained model, thus creating a seed model with an initial ability to follow instructions. Then, leveraging the seed model, we calculate necessity scores for each sample in the VIT data pool to identify samples pivotal for enhancing model performance. Our findings underscore the importance of mixing necessity and diversity in data choice, leading to the creation of MLLM-Selector, our methodology that fuses necessity scoring with strategic sampling for superior data refinement. Empirical results indicate that within identical experimental conditions, MLLM-Selector surpasses LLaVA-1.5 in some benchmarks with less than 1% of the data and consistently exceeds performance across all validated benchmarks when using less than 50%.