Key takeaways
- LinkedIn has transformed the relationship between productivity and social networking for millennials.
- The platform enables connections that were previously difficult for professionals to establish.
- LinkedIn effectively manages vast amounts of user data while ensuring minimal latency.
- The company's approach combines conventional strategies with in-house tools for improved machine learning outcomes.
- The process of building machine learning models at LinkedIn follows standard procedures of data collection, processing, training, and testing.

Building ML Models To Scale
The blueprint of a machine learning model would more or less consist of the same procedures; data collection, processing, training and testing the models and so on. Firms seeking ML transition can get an idea of how to scale their productivity by taking a look at LinkedIn’s model:- The major part of data or to be more precise, the most crucial data with respect to LinkedIn is based on the kind of jobs liked, jobs saved and connections made. So, recommending jobs to an individual and calculating the probability of a job posting being checked are one of the few important features of the dataset.
- At LinkedIn, the ML team proceeds by building a domain-specific language (DSL) and then a Jupyter notebook to integrate the select features and for parameter tuning.
- Most of the model training occurs offline where the ML teams train and retrain the models every few hours. For this, they avail the services of Hadoop. LinkedIn’s own Pro-ML training service is updated with newer model types for hyperparameter tuning. This training service leverages Azkaban and Spark to ensure that there is no missing input data.
Structure Of Pro-ML

Key Ideas
- Leverage and improve best-of-breed components from our existing code base to the maximum extent feasible.
- Use an agile-inspired strategy; making one product line better at a time.
- To enable services hosting the models to be independently upgraded without breaking their downstream or upstream services.
- Enable new technologies to be A/B testable in production.




