NVIDIA Unveils Enhanced NeMo Framework, Improves LLM Training on H200 GPU

NVIDIA has updated its NeMo framework and enhanced Large Language Model (LLM) training on their H200 GPU. These developments target developers and researchers in AI, particularly those working with AI Foundation Models such as Llama 2 and Nemotron-3. 

The new NeMo framework, now cloud-native, supports a wider range of model architectures and utilises advanced parallelism techniques for efficient training. The H200 GPU specifically improves performance for the Llama 2 model, offering significant advancements over previous versions.

Announced on December 04 and now accessible globally, these tools serve various applications, from academic research to industry use. 

The updates aim to meet the increasing demand for better training performance in complex and diverse LLMs. They focus on accelerating training processes, improving efficiency, and expanding model capabilities, crucial for models requiring extensive computation.

The enhancements include mixed-precision implementations, optimised activation functions, and improved communication efficiency. The H200 GPU achieves up to 836 TFLOPS per GPU, significantly increasing training throughput. 

The introduction of Fully Sharded Data Parallelism and Mixture of Experts architecture optimizes model training and capacity. Reinforcement learning from human feedback is enhanced with TensorRT-LLM, supporting larger models and improving performance.

For those interested, NVIDIA offers the NeMo framework as an open-source library, a container on NGC, and as part of NVIDIA AI Enterprise. Additional resources such as GTC sessions, webinars, and SDKs are available for further engagement with NVIDIA's AI tools.