Data is an integral part of modern businesses. Organisational structure plays a critical role in enabling the organisation’s success and building a data-driven culture. Over the years, companies have experimented with different organisational structures. In this article, we will discuss why organisational structure is critical to a company’s success.

Data team structure

Typically, a data team has analytics, data engineering, ML engineering, data science, and R&D divisions. Click here for a deep dive.

Analytics: Drives insights, democratises data through reporting and dashboarding etc. The analytics teams can quickly react to changes in trends and are constantly nudging stakeholders to make data-driven decisions.

Data engineering: Ingests and stores data. The team makes data consumable for downstream consumers and establishes data instrumentation processes. 

ML engineering: Provides infrastructure for the models to deploy in batch and real-time. 

R&D: Responsible for building long term capabilities for the businesses. 

Data science: Builds AI solutions and is responsible for generating value from AI solutions by collaborating with key stakeholders. 

Horizontal vs vertical 

Companies follow both horizontal and vertical structures.

For instance, data engineering, ML engineering, and R&D come under horizontal structures. Data pipelines are common to all organisations. The same is the case with ML engineering infra that is scalable across multiple data science models. R&D looks at platform level problems, such as computer vision, building language capabilities, improving allocation engines in the organisation, etc. 

On the other hand, data science and business analytics can be vertical or horizontal.

Types of organisational structure 

Central Teams (horizontal) 

Applies only to the data science and business analytics team. The sub-teams could be organised functionally or across different product or business functions. 

Functional: Central teams for data science 

In most organisations, the objective and key results (OKR), hiring plan, delivery and performance management are data driven. The data team works with the product and business teams to generate outputs. 

The data team reports to a central leader. The organisation structure is based on functional specialisation or different platforms and products of the company. For instance, the team could be divided into – function sub-teams and vertical sub-teams. 

NLP capabilities: 

• Customer feedback analysis: NLP allows companies to understand customer data at scale and in real-time. 
• Text analysis of customer feedback data: NLP is used to understand customer satisfaction drivers.

• Bot for customer service: Many companies today are using NLP tools and chatbots to improve product recommendations. 
• Product extraction and catalogue services: NLP techniques are used for product extraction and building catalogues. 
• Search and recommendations: Searching for the products based on user queries and recommending items matching personal preferences. 

Computer vision 

• Image classification: Different applications including putting the images into categories and detecting the objects within the images. 
• Image validation: Identifying the patterns of objects in the images and matching them with the database. 
• Image captioning: Real-time assigning or labelling of the objects present in the image. 

Financial models 

• Fraud scores for onboarding: Identifying the potential risks in the beginning itself whenever a customer enters the system. 
• Score for cash-on-delivery: Identifying the customers who are not trustworthy and providing personalised payment options for them accordingly. 
• Credit score – for post-payment options: Identifying the default-risk customers well in advance.

Marketing models

• Targeted campaigns: Knowing the customers choices and offering them personalised product ads and recommendations. 
• Channel optimisation: Enhance the customer experience, increase interactions, and drive incremental revenue within a single marketing channel or platform.
• Optimisation of loyalty programme: Giving customers discounts and coupons based on their purchasing behaviour. 

Optimisation 

• Driver allocation: This involves assigning the right order to the right driver. 
• Warehouse allocation: Choosing the right warehouse to optimise delivery time. 
• Expected time of delivery: The key to a better customer experience in hyperlocal delivery is the transparency in order tracking. The optimisation team solves this problem based on the items in the order. 
• Supply forecasting: Forecast the supply for optimal inventory planning.

Data extractions 

• When a vendor uploads the information of a product for sale, the data team ensures the extracted data is cleaned and ready for customer consumption. 

Functional central team for business analytics

The business analytics teams covers business intelligence, dashboarding, visualisation, and analysis and insights. 

Business intelligence team 

It is responsible for ensuring the underlying data marts are consumable by downstream users. The downstream uses could be dashboarding, visualisation etc. In some cases they could also be products of  business users who can directly query the database. The BI teams are responsible for denormalizing the data and keep it ready for consumption.

Dashboarding team

The dashboarding team works closely with non-data stakeholders and enables monitoring via reports and dashboards. The teams work hand in glove with the business teams to produce accurate dashboards. 

Visualisation team

A large amount of data thrown into a tabular dashboard or very complex changes are not in a consumable format. The visualisation team can create meaningful visualisations on top of the underlying data. The team is key in democratising data for the entire company

Analysis and insights team

The team is responsible for generating critical hypotheses that could explain the company’s key performance metrics. The team is critical in making the day-to-day operational decisions of the organisation. 

Pros and cons 

• Central teams eliminate redundancies
• Bandwidth for team members to specialise in their areas of expertise
• Deliver high performance
• Specialist teams can lead to fatigue
• Lack of full stackability can slow down career growth. 
• Multiple vertical teams breeds confusion and poor accountability. 

Vertical central teams

In vertical central teams, the organisation is hard-aligned to a data functional leader. However, the sub teams are organised facing the vertical product organisations. Each subteam is responsible for the delivery of a specific product organisation within the company.

The data team owns the OKR and deliverables and dictates headcount, planning, deliverables, and performance. The product teams act as consultants in performance management. When there is redundancy, the data team irons out engagement issues between the PODs. 

The projects in each of the functions can also come from the data team. The bandwidth to experiment is not tied to product or business requirements. 

Vertical central teams for data science 

Let’s take an example of a hyper-local grocery delivery business. As shown below, data science teams are involved in various tasks, starting from collecting data via sign-ups to the discovery of products and streamlining order management, inventory, post order management, customer experience, research, etc. 

Ideally, in verticalisation, business analytics teams dedicate each of the PODs for analysis, dashboarding and KPI metrics. 

Pros and cons 

• Outcome-aligned teams 
• Long term work happens inside the data team in the research 
• Redundancy can go unchecked without proper governance 
• Certain initiatives from the data team can be left unused 
• The workers might feel stuck

POD

This way of structuring comes with a single-threaded leader. More or less, it is quite similar to ‘vertical’ function teams. The differences include – 

• OKR is owned by product 
• Headcount asks come from product/business teams
• Performance management budget comes from product/business teams
• POD owners are driving deliverables 
• Data comes into purely hire headcounts and managing technical challenges faced by the team 
• The day-to-day team interacts with respective POD owners 
• Redundancy between PODs resolved by the product

Pros and cons 

• Extremely outcome focussed
• Razor-sharp business impacts
• Predictive models may not be well-governed
• Lack of technical mentorship
• Long term initiatives and research will remain on the back burner

Hybrid

Hybrid follows a moderation ‘POD’ level structure. Here, data is divided into PODs like central ‘vertical’ teams. However, the data leader owns the hiring and performance process in consultation with the product or business teams. As shown below, the data leader has a dotted line reporting to the product or business. 

Data is responsible for its own deliverables. Product/business teams owns the OKRs, and data will provide insights into the OKR process. It will also have a view or ask about products. However, it will independently manage teams within the POD structure. 

Data drives the outcomes and has more independence to suggest and drive long term initiatives. Hence, data can be managed by a separate R&D team providing visibility to product and business. 

Pros and cons 

• Extremely outcome focussed
• Well plugged into business/product decisions
• Limited redundancy compared to pod structure
• Enables long term initiatives 
• Long term initiatives are limited by-product or business understanding.
• Not data first

Startup vs organisational structure

Early-stage startups should start with central functional organisations as the requirements are unclear. Data-centric organisations could move to central ‘vertical organisations,’ aka hybrid. Mature setups could move to a ‘hybrid’ structure. Pure ‘POD’ of setups when there is an absence of senior data leaders in the organisations should be in a transitional phase gravitating towards ‘Hybrid’ setups. 

This article is written by a member of the AIM Leaders Council. AIM Leaders Council is an invitation-only forum of senior executives in the Data Science and Analytics industry. To check if you are eligible for a membership, please fill out the form here.