Financial Sentiment Analysis with Large Language Models
DOI:
https://doi.org/10.61173/gvntfv56Keywords:
Financial Sentiment Analysis, Large Language Models (LLMs), Parameter-Efficient Fine-TuningAbstract
Financial sentiment analysis is vital for applications such as market prediction and risk management. While domain-specific models like (Financial Bidirectional Encoder Representations from Transformers) FinBERT are widely used, their limited scalability constrains performance across diverse financial texts. This paper investigates the effectiveness of large language models (LLMs) with parameter-efficient fine-tuning strategies. We fine-tune Llama-3.1-8B and Owen-3-8B using LoRA and QLoRA, and evaluate them on Financial PhraseBank and FiOASA datasets. Experiments show that LLMs consistently outperform FinBERT, achieving up to 88.9% accuracy on PhraseBank and 81.7% accuracy with 0.74 macro-Fl on FiQA-SA LoRA yields stronger performance, especially on minority classes, while QLoRA maintains comparable accuracy with significantly reduced memory cost. Moreover, Qwen-3 outperforms Llama-3.1 on noisy microblogs, benefiting from its Mixture-of-Experts (MoE) architecture, which enhances efficiency and diversity through conditional computation. These findings confirm that parameter-efficient fine-tuned LLMs provide both accuracy and efficiency, and represent strong alternatives to domain-specific models in financial sentiment analysis.
References
[1] Araci D T. FinBERT: Financial sentiment analysis with pretrained language models. arXiv preprint arXiv:1908.10063, 2019.
[2] Inserte P, Rodriguez P, Nakhlé M, Qader R, Caillaut G, Liu J. Large language model adaptation for financial sentiment analysis. FinNLP-2, 2023.
[3] Meta AI. Llama 3.1: Advancing open foundation models. Technical Report, Meta AI, 2024.
[4] Alibaba Cloud. Qwen 3.0: Scaling multilingual open LMs. Technical Report, Alibaba Group, 2024.
[5] Devlin J, Chang M W, Lee K, Toutanova K. BERT: Pretraining of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, 1: 4171– 4186.
[6] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 2017: 5998–6008.
[7] Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, Bashlykov N, Batra S, Bhargava A, Bhosale S, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
[8] Zhang S, Roller S, Goyal N, Artetxe M, Chen M, Chen S, Dewan C, et al. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.
[9] Biderman S, Schoelkopf H, Anthony Q, Bradley H, Ohlson N, Black S. Pythia: A suite for analyzing large language models Dean&Francis Xinyu Cheng across training and scaling. Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202, 2023: 4370–4385.
[10] Bai J, Dai Z, Dong L, Zhang W, Zhang X, Zhang S, Huang S, et al. Qwen technical report: Open foundation and chat models by Alibaba Cloud. arXiv preprint arXiv:2309.16609, 2023.
[11] Hu E J, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
[12] Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient finetuning of quantized LLMs. arXiv preprint arXiv:2305.14314, 2023.
[13] Manning C D, Raghavan P, Schütze H. Introduction to Information Retrieval. Cambridge, UK: Cambridge University Press, 2008.
[14] Malo P, Simha A, Korhonen P, Wallenius J, Takala P. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 2014, 65(4): 782–796.
[15] Maia M, Handschuh S, Freitas A, Davis B, McDermott R, Zarrouk M, Balahur A. WWW’18 open challenge: Financial opinion mining and question answering. Companion Proceedings of The Web Conference 2018 (WWW’18 Companion), 2018: 1941–1942.
[16] Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Rush A M. Transformers: State-of-the-art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020: 38–45.
[17] Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient finetuning of quantized LLMs. arXiv preprint arXiv:2305.14314, 2023.
[18] Hu E J, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
[19] Kraus M, Feuerriegel S. Sentiment analysis based on rhetorical structure theory: Learning deep neural networks from discourse trees. Expert Systems with Applications, 2017, 118: 65–79.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 by the authors.
This work is licensed under a Creative Commons Attribution 4.0 International License.
