Key takeaways
- Netflix serves over 125 million members globally, generating vast amounts of data from user interactions across numerous devices.
- The company's data infrastructure produces over a trillion events daily, necessitating complex data management and storage solutions.
- Building an accurate data lineage system is crucial for Netflix, as it helps track data movement through ingestion, conformance, and consumption stages.
- Data lineage enhances the visibility of data pipelines, allowing engineers to improve reliability and efficiency in data handling.
With over 125 million members spread across 191 countries, Netflix has an uphill task of providing high-quality services to all its members. Every time a member interacts with client applications which are on over 250 million devices, data points are generated. These interactions result in over 200 billion daily data points.
Netflix’s data infrastructure generates over a trillion events per day and stores over 100 PB of data. This data has to be collected, stored and should be ready for further analysis. The data infrastructure gets complex with more data and various data ingestion patterns.
The Netflix engineering team has a complex job of building an accurate data lineage system to map out data repositories, dashboards, ad-hoc queries and other such data-artefacts.
The data life cycle consists of three main stages:
- Ingestion
- Conformance
- Consumption




