Netflix Achieves 4.7x Faster LLM Post-Training With New Distributed Platform

Netflix has overhauled its large language model post-training infrastructure, publishing a detailed engineering account on its blog of how it scaled fine-tuning and reinforcement learning workflows to production. 

The effort led to a 4.7x improvement in training throughput on highly skewed datasets. The company said the work reflects a shift away from ad-hoc scripts toward a standardised, distributed training system.

Post-training is the phase in which pre-trained foundation models are adapted to real-world product needs. At Netflix, this includes personalisation, recommendation, and search. 

“At Netflix scale, post-training quickly becomes an engineering problem as much as a modelling one: building and operating complex data pipelines, coordinating distributed state across multi-node GPU clusters, and orchestrating workflows that interleave training and inference,” the company said.

According to Netflix, engineers say the biggest challenge is not model design, but operating these workflows at scale. A key bottleneck is uneven sequence lengths in training data. 

“In FSDP-style training, long-tail sequences create stragglers: faster workers end up waiting at synchronisation points for the slowest batch, lowering utilisation.”

While offline bin-packing can mitigate this, the company noted it “can add substantial preprocessing latency and make it harder to keep datasets fresh,” which pushed engineers to look for a more dynamic solution.

Netflix instead built an asynchronous, on-the-fly sequence packing system. Samples are streamed from storage and packed dynamically in memory while GPUs are already training.

The packing runs on CPUs in parallel with GPU execution. This reduces padding waste and keeps batch shapes consistent. “For our most skewed dataset, on-the-fly packing improved the effective token throughput by up to 4.7x.”

The post-training framework runs on Netflix’s internal ML compute platform, Mako. It relies heavily on open-source tools such as PyTorch, Ray, and vLLM.

In addition, Netflix provides a library of standardised components across four areas: data, model, compute, and workflow. Developers define jobs using configuration files and choose recipes such as supervised fine-tuning (SFT), direct preference optimisation (DPO), reinforcement learning (RL), or knowledge distillation.

Supporting RL required a major architectural change. Netflix explained that “on-policy RL changes the shape of the system,” because training depends on data generated by the current policy. 

RL involves multiple interdependent stages—including rollout generation, reward scoring, reference model inference, and policy optimisation—that must be coordinated explicitly. “Individual sub-stages can each be implemented as SPMD (single program, multiple data) workloads, but the end-to-end algorithm needs explicit coordination.”

Netflix also standardised on Hugging Face formats for loading and saving models and tokenisers to avoid training–serving mismatches. Internally, however, the company uses its own optimised model implementations. This enables features such as memory-efficient loss computation, consistent Model FLOPS Utilisation tracking, and uniform LoRA support.

The engineers also uncovered a low-level performance trap. Certain vocabulary sizes can cause GPU kernels to fall back to slower implementations. Netflix now automatically pads vocabularies to multiples of 64 to avoid this, preventing up to 3x slowdowns in the language model head.

Netflix said the result is a managed post-training system that scales from a single node to hundreds of GPUs. It allows researchers to focus on model behaviour rather than distributed systems plumbing. The company views this foundation as critical for future agentic, reasoning-heavy, and multimodal models.