Financial consulting companies must definitely know the cost of using generative AI. That is why they are building their own customised LLMs for chatbots and other purposes to make sense of documents and not solely rely on expensive offerings by others.

The latest joiner in the LLM development league is JPMorgan, a financial consulting firm which deals in investment banking, commercial banking and other financial services. The firm has introduced DocLLM, a generative language model designed for multimodal document understanding.

It stands out as a lightweight extension to LLMs for analysing enterprise documents, spanning forms, invoices, reports, and contracts that carry intricate semantics at the intersection of textual and spatial modalities. 

Generalised models are sub-par

OpenAI’s ChatGPT Plus also allows users to scan and analyse documents. The feature is also available in ChatGPT Enterprise. But these models have been criticised for their privacy concerns, which makes enterprises and consulting companies, with all the financial data, scared to use them.

Moreover, GPT and other AI models have been experiencing hiccups when it comes to analysing documents. Most recently, the models were not able to analyse SEC filings, resulting in major backlash for the models. 

The top-performing configuration AI model, specifically OpenAI’s GPT-4-Turbo, achieved a mere 79% accuracy when equipped with the capability to analyse almost an entire filing in conjunction with the posed question. Frequently, the models exhibited reluctance to respond or would generate inaccurate information—or hallucinate—which did not align with the details found in SEC filings.

Anand Kannappan, co-founder of Patronus AI, a company which evaluates the security of AI models, expressed dissatisfaction with this level of performance, deeming it “absolutely unacceptable”. He emphasised on the necessity for a significantly higher accuracy rate to make the technology truly effective in automated and production-ready applications.

These discoveries underscore the difficulties that AI models encounter as major corporations, particularly those in regulated sectors like finance and consulting, strive to integrate state-of-the-art technology into their operations, whether for customer service or research purposes.

Thus, developing own models

These inaccuracies and securities were one of the reasons why BloombergGPT was launched, specifically for finance. It has been helping people make sense of financial documents, reports, and invoices. This also highlights the need for open source models, when it comes to dealing with financial information, where DocLLM definitely shines. 

 JPMorgan is making DocLLM open for other users as well. The two versions of DocLLM, one with 1 billion parameters is built on top of Falcon-1B architecture, and 7 billion parameter models are built on Llama2-7B. Being open source, the model provides safety and security to its users.

Similarly, KPMG had internally developed a system based on OpenAI models and called it KaiChat to aid its staff with exclusive data. PwC is set to invest $1 billion over three years to advance generative AI in its US operations, collaborating with Microsoft and OpenAI to automate tasks in tax, audit, and consulting.

EY leverages generative AI, integrating tax laws into an AI system for instant responses through a ChatGPT-like interface, particularly for tasks like payroll queries.

In August, McKinsey embraced the potential of LLMs with the launch of “Lilli”, designed to streamline and enhance the utilisation of the firm’s vast knowledge base. Wells Fargo also introduced Fargo in 2022, a virtual assistant powered by Google Cloud’s AI, for providing a personalised, convenient, and simple banking experience.

In October last year, Deloitte launched DARTbot, an internal chatbot for enhancing efficiency of Deloitte’s 18,000 US Audit & Assurance professionals.

But can they compete against OpenAI?

When OpenAI launched GPT-4, the much hyped BloombergGPT for the financial field slowly stopped gaining traction. Adi Polak said that models such as GPT-5 coming up soon can possibly outperform JPMorgan’s DocLLM, as they would be better at specialised and generalised tasks combined. 

“This could become the go-to model for document intelligence tasks, saving companies time and money. For example, insurance firms can automate claim assessments, while banks can speed loan processing,” said a user on X. To this, Polak replied that it would require a lot of fine tuning. 

Whenever OpenAI releases new features to ChatGPT, it gets blamed for affecting startups and others doing the same. When it introduced ‘Upload many types of documents’ this new ‘multimodal’ update, according to many, was expected to kill hundreds of startups. Some of the popular names include ChatPDF, AskYourPDF, and PDF.ai, and many more, which were basically wrappers of OpenAI’s models.

But for the time being, it is clear that consulting companies building their own LLMs for financial planning and decisions is better than relying on other offerings.