本研究探讨了训练数据质量与数量对小规模语言模型（SLMs）性能的相对影响，利用TinyStories数据集进行实证分析。通过对数据集在大小（原大小的25%和50%）以及复制率（控制在25%、50%、75%和100%）方面的变化进行了分析。模型性能基于验证损失、准确率和困惑度指标进行评估。结果表明，训练数据质量对小规模语言模型的整体性能具有更重要的影响，尤其是在本次实验的规模下。适度的数据复制对模型准确率有积极影响（25%复制时准确率提升0.87%），且未显著增加困惑度（从0%到25%复制时困惑度仅增加0.52%），但过度复制则导致明显的性能下降（100%复制时准确率下降40%）。这项探索的意义不仅限于模型性能本身；训练大规模模型带来了巨大的经济和计算负担，这对组织、个人以及全球公众，尤其是发展中国家而言可能难以承受。此外，大规模训练带来的能源消耗也引发了环境问题。理解数据质量和数量的相对重要性可以实现人工智能技术的民主化，使先进的模型更易获取且更具可持续性。

This study investigates the relative impact of training data quality versus quantity on the performance of small language models (SLMs), utilizing the TinyStories dataset for empirical analysis. Analysis of dataset variations with respect to size (25% and 50% of the original size) and duplication (controlled rates of 25%, 50%, 75%, and 100%) were performed. Model performance was evaluated based on the validation loss, accuracy, and perplexity metrics. Results indicate training data quality plays a more significant role in the overall performance of SLMs, especially given scale of this experiment. Minimal duplication positively impacted model accuracy (+0.87% increase in accuracy at 25% duplication) without significantly increasing perplexity (+0.52% increase going from 0% to 25% duplication) but excessive duplication led to pronounced performance degradation (-40% drop in accuracy at 100% duplication). The implications of this exploration extend beyond just model performance; training large-scale models imposes significant financial and computational burdens, which can be prohibitive for organizations, individuals, and the public at large, especially in developing countries. Additionally, the energy consumption associated with large-scale training raises environmental concerns. Understanding the relative importance of data quality versus quantity could democratize AI technology, making advanced models more accessible and sustainable for all.