Active Hackathon

Optimising Enterprise Data Warehousing Project costs in Banks

Optimising Enterprise Data Warehousing Project costs in BanksBanks embarking on Enterprise Data Warehouse (EDW) projects have of late been on the increase. This is an impact of India’s primary regulator the RBI, asking Public Sector Banks to implement a Central Data Repository, which is actually another avatar of a Data Warehouse (DW). While a DW does deliver great benefits, costs have been escalating irrationally. Not so long ago, large Banks (in the 3000 to 5000 branches league) were able to implement an EDW by investing around INR 50 crores including the cost of about 5 years of post-implementation support. Project costs today for the very same exercise have gone up by a whopping 100%. Besides the cost factor, most Banks are dissatisfied with their DW.

This makes one ask ‘is it really worth spending an astronomical sum for a DW?’, and more importantly ‘What is the true value Banks derive out of a DW exercise’? The second question that naturally comes up is ‘Is it not possible to get the same (or better) benefits at a lower cost?’ To understand the state of affairs in depth, let’s first take a quick look at the very purpose of establishing an EDW.


Sign up for your weekly dose of what's up in emerging technology.

Banks typically expect the EDW to enable –

  • Regulatory reports
  • Ad-hoc reports
  • MIS and Dashboards
  • Analysis of Business trends
  • Product Gaps and opportunities
  • Analysis of Delivery Channel Transactions, identifying new initiatives
  • Performance Tracking v/s Budget
  • Customer Analysis and Pricing
  • Cross selling
  • Promotional Campaigns
  • Exposure and Risk Management
  • ALM
  • AML
  • Social Media Analytics

Each Bank must dispassionately examine its specific requirements and demand only required functionalities. Also, these requirements should be in phases, with each phase not exceeding six months. The DW project delivery schedule should be clearly defined. Wish lists and good-to-haves should be excluded in phase one. While Data ETL should be a combined activity, the phases of Data modelling and implementation of application for each of the requirements should be clearly defined in vendor agreements. The licensing fees and implementation charges should be paid accordingly. It does not make sense to pay licensing fees for the applications and Database requirements from day one, if the usage comes much later. If necessary activating the processors can be synchronised with the processing requirements.

Agreements and contracts must clearly state that service continuation of the vendors would be subject to satisfactory performance and timely achievement of agreed milestones. This gives Banks the freedom to drop the System Integrator (SI) if they fail to deliver on time. Also, the cost of each project leg should be clearly defined so that if the SI exits midway, the dues do not become controversial. Banks must also ensure diligent documentation of deliveries versus payments made to the SI/vendor.

Let’s also examine a traditional anomaly that provides safe haven to most EDW vendors. As a standard practice almost all Core Banking vendors refrain from providing the Data Dictionary for their systems. This enables them to continually demand their ‘pound of flesh’ from unsuspecting clients. This issue has to be taken up by the Banking fraternity along with the Indian Banks Association (IBA) to ensure that documentation of data, which rightfully belongs to the Banks, is provided to them by the Core Banking vendors and other system vendors at no cost. Interestingly, this is a standard practice the world over. If this practice is not corrected on the home turf, Indian Banks will only continue to bleed.

With requirements defined, what is needed to achieve the desired objective would be a good ETL software, a good Data Mining software, a good OLAP software and a few other tools (depending on the expectations from the EDW). Apart from these, Banks require hardware, that can store up to 10 TB or more of data (for a Bank with about 5000 branches; and this can vary depending on the number of applications in a Bank) and can address the processing requirements for all applications, the people cost for creating the DW Data Models, applications and support.

Banks must also take care to ensure that the cost of Business Continuity / Disaster Recovery planning is kept low. The number of user licenses also must be kept to the minimum. For instance, every Branch or Regional Manager needn’t have a user license and they should only be information Dashboard users. They can access reports relating to the other areas from shared folders. Care should be taken to ensure that only data relevant to the required applications defined by the Bank are extracted and stored.

A third area, where costs can be controlled, is manpower expenses. Many vendors operating in India do not have high quality manpower to support critical activities such as Data Modelling. So activities take longer and Banks keep getting billed. In this scenario, vendors take cover under the ‘unavailability of Data Dictionary’ justification. It would be prudent therefore to go for a fixed cost implementation with a clearly defined and phased delivery schedule. For support, Banks must train their own teams to develop and run ad-hoc reports and Analytics with limited support. There should be a provision in the RFP to reduce vendor support, if necessary or discontinue support, if found unsatisfactory.

A final word of caution: Vendors, especially some of the prominent ‘ivy league’ ones, are adroit at getting agreements signed, which are generally tilted in their favour. Exceptional care must be taken while signing agreements to ensure the Bank’s interest. In fact it may make sense to add the following sentence in vendor agreements – “In the event of a conflict between the RFP, the Purchase Order and the implementation agreement, conditions in the RFP (and responses to the same) will prevail.”

Besides bringing down expenses, these considerations primarily protect the Bank’s interests. If the tools are selected prudently, the cost for a Bank with 4000 to 5000 branches can be brought down by at least 50%.

More Great AIM Stories

AK Mohanty
AK Mohanty currently serves as a Strategic Advisor at iCreate. He was previously the Head of IT at the Central Bank of India from 2009 to 2012. Earlier in his career, he was the Head of IT - Foreign Offices at the State Bank of India, where he was one of the key members spearheading the domestic implementation of B@ncs24 and the implementation of Finacle Core Banking Solution in their foreign offices. He also piloted the implementation of Core Banking and several strategic IT projects at the Central Bank of India. With an extensive experience spanning 18 years in IT across all domains, AK Mohanty is a ‘practical’ Banker with over a decade of rich experience in heading several Bank branches and regions.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM

The curious case of Google Cloud revenue

Porat had earlier said that Google Cloud was putting in money to make more money, but even with the bucket-loads of money that it was making, profitability was still elusive.

Global Parliaments can do much more with Artificial Intelligence

The world is using AI to enhance the performance of its policymakers. India, too, has launched its own machine learning system NeVA, which at the moment is not fully implemented across the nation. How can we learn and adopt from the advancement in the Parliaments around the world? 

Why IISc wins?

IISc was selected as the world’s top research university, trumping some of the top Ivy League colleges in the QS World University Rankings 2022