What Does It Take To Maintain Cost-Efficient Open-Source Platforms For Data Hungry Organisations

The term “open-source” was first coined in 1998. Since then, open-source software has generated billions of dollars. Moreover, the paradigm of open-source practices has been a blessing for modern-day big data systems. For instance, a decade ago, when Apache Spark was still in its nascence, companies like Yahoo, Amazon, and Intel helped improve the project.  Spark enabled data storage in the memory subsystems of the thousands of servers it pulls together. In contrast, Hadoop allowed seamless data storage on hard disks and reduced the time taken to call the data. Today, many big data companies are founded on these open-source projects. But with scale comes cost. Ride-hailing company Uber, which has accumulated tonnes of data over the years, claims to have a cost-efficient way of