Unveiling the AI Potential: FinBert beats ChatGPT in Financial Text Analytics

There are several aspects in a study conducted by JPMorgan and Queen's University where FinBert,  an LLM fine-tuned on financial domain data, specific terminology, language structures, and concepts demonstrated its superiority over ChatGPT in the context of financial text analytics. 

Gpt-3.5-turbo and GPT-4 with 8k tokens were compared with FinBert, and tested on Arithmetic Reasoning, News Classification Sentiment Analysis and Named Entity Recognition. 

FinBert outperforms ChatGPT in sentiment analysis tasks related to financial texts. Sentiment analysis in finance requires understanding nuanced expressions and the impact of news on investors. 

FinBert, being specifically designed for the financial domain, does not require extensive adaptation or fine-tuning to perform well in these tasks. In Few-shot Learning: ChatGPT's performance on various tasks required more extensive prompts. 

FinBert outperforms ChatGPT and even competes with human experts in arithmetic reasoning. This indicates that FinBert is a highly specialised model for financial tasks, while ChatGPT, might not reach the level of expertise demonstrated by FinBert in the financial domain.

In tasks such as financial named entity recognition (NER) and sentiment analysis, where a deep well of domain-specific knowledge is essential, ChatGPT and GPT-4 struggles. Their inability to grasp the intricacies of financial terminologies becomes evident.

Comparative analysis puts both the models against fine-tuned models tailored for the financial sector, like FinBert and FinQANet. The outcome underscores the fact that, while these LLMs hold potential, they are not yet on par with their specialised counterparts.

This study paves the way for further enhancements.

The gap between these state-of-the-art generative language models and domain-specific proficiency remains, but it also presents a promising opportunity for refinement.

Rajiv Shah, a machine learning engineer at Hugging Face said on linkedin, “A domain-specific model like FinBERT is more accurate for finance tasks than GPT-4”.