Unveiling the AI Potential: FinBert beats ChatGPT in Financial Text Analytics

Gpt-3.5-turbo and GPT-4 with 8k tokens were compared with FinBert, and tested on various tasks ranging from everyday math to complex calculations
Listen to this story

There are several aspects in a study conducted by JPMorgan and Queen’s University where FinBert,  an LLM fine-tuned on financial domain data, specific terminology, language structures, and concepts demonstrated its superiority over ChatGPT in the context of financial text analytics. 

Gpt-3.5-turbo and GPT-4 with 8k tokens were compared with FinBert, and tested on Arithmetic Reasoning, News Classification Sentiment Analysis and Named Entity Recognition. 

FinBert outperforms ChatGPT in sentiment analysis tasks related to financial texts. Sentiment analysis in finance requires understanding nuanced expressions and the impact of news on investors. 

FinBert, being specifically designed for the financial domain, does not require extensive adaptation or fine-tuning to perform well in these tasks. In Few-shot Learning: ChatGPT’s performance on various tasks required more extensive prompts. 

FinBert outperforms ChatGPT and even competes with human experts in arithmetic reasoning. This indicates that FinBert is a highly specialised model for financial tasks, while ChatGPT, might not reach the level of expertise demonstrated by FinBert in the financial domain.

In tasks such as financial named entity recognition (NER) and sentiment analysis, where a deep well of domain-specific knowledge is essential, ChatGPT and GPT-4 struggles. Their inability to grasp the intricacies of financial terminologies becomes evident.

Comparative analysis puts both the models against fine-tuned models tailored for the financial sector, like FinBert and FinQANet. The outcome underscores the fact that, while these LLMs hold potential, they are not yet on par with their specialised counterparts.

This study paves the way for further enhancements.

The gap between these state-of-the-art generative language models and domain-specific proficiency remains, but it also presents a promising opportunity for refinement.

Rajiv Shah, a machine learning engineer at Hugging Face said on linkedin, “A domain-specific model like FinBERT is more accurate for finance tasks than GPT-4”. 

Download our Mobile App

Pranav Kashyap
As a financial journalist, Pranav covers various beats from Technology, Telecom and Companies. He follows the money trail, and dives deep into how companies utilize their funds. He loves to travel and is an aviation geek.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Recent Stories

Our Upcoming Events

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox