摘要 Abstract

金融领域的大规模语言模型(LLMs)在推动金融任务及特定领域的应用方面具有巨大潜力,但其发展受到语料稀缺、弱多模态能力以及评估范围狭窄等问题的限制,难以满足真实世界的应用需求。为解决这些问题,我们提出了\textit{Open-FinLLMs},这是首个开源的多模态金融领域大模型系列,旨在处理文本、表格、时间序列以及图表等多种数据形式下的多样化任务,并在零样本、少样本及微调设置下表现出色。该系列包括基于520亿标记全面预训练的FinLLaMA模型,以及通过57.3万条金融指令进行微调的FinLLaMA-Instruct模型;此外,FinLLaVA模型还通过143万个多模态调优对齐样本增强了跨模态推理能力。我们在14项金融任务、30个数据集以及四种多模态任务中,采用零样本、少样本和监督微调设置对Open-FinLLMs进行了全面评估,并引入了两个新的多模态评估数据集。结果表明,Open-FinLLMs在金融自然语言处理、决策制定以及多模态任务中超越了先进的金融领域模型如GPT-4,展示了其应对现实世界挑战的巨大潜力。为了促进学术界与工业界之间的创新与合作,我们在OSI认可的许可协议下公开了所有代码(https://anonymous.4open.science/r/PIXIU2-0D70/B1D7/LICENSE)和模型。

Financial LLMs hold promise for advancing financial tasks and domain-specific applications. However, they are limited by scarce corpora, weak multimodal capabilities, and narrow evaluations, making them less suited for real-world application. To address this, we introduce \textit{Open-FinLLMs}, the first open-source multimodal financial LLMs designed to handle diverse tasks across text, tabular, time-series, and chart data, excelling in zero-shot, few-shot, and fine-tuning settings. The suite includes FinLLaMA, pre-trained on a comprehensive 52-billion-token corpus; FinLLaMA-Instruct, fine-tuned with 573K financial instructions; and FinLLaVA, enhanced with 1.43M multimodal tuning pairs for strong cross-modal reasoning. We comprehensively evaluate Open-FinLLMs across 14 financial tasks, 30 datasets, and 4 multimodal tasks in zero-shot, few-shot, and supervised fine-tuning settings, introducing two new multimodal evaluation datasets. Our results show that Open-FinLLMs outperforms afvanced financial and general LLMs such as GPT-4, across financial NLP, decision-making, and multi-modal tasks, highlighting their potential to tackle real-world challenges. To foster innovation and collaboration across academia and industry, we release all codes (https://anonymous.4open.science/r/PIXIU2-0D70/B1D7/LICENSE) and models under OSI-approved licenses.