Musk Reveals Grok 4.8’s Pre-Training Stack is Written in C++ by ‘Humans’, Not AI

Elon Musk, who leads SpaceXAI, the company behind the Grok family of AI models, said its pre-training software is now an internally developed stack written in C and C++.

He made the disclosure in a post on X, where he said Grok 4.8, a 2.5-trillion-parameter model, is expected to finish training this week and begin reinforcement learning. “Our pre-training software is now an internally developed stack in C and C++,” Musk wrote. 

He said the Starlink software team wrote the software and “was not written by AI”. “AI is not yet good enough for extremely high-performance software (but it will be)”  

“Our reinforcement learning stack is still somewhat of a legacy pile of many different third-party software systems,” he wrote, adding that it would be fixed “in due time”.

The comments prompted discussion among software engineers about the trade-offs between custom, low-level training infrastructure, and higher-level frameworks. 

Patrick C Toulme, a software engineer at Google, wrote on X that SpaceXAI’s use of C++ could be helping it produce models quickly.

“The MFU on this full C++ stack is part of the reason xAI is churning out so many models,” Toulme wrote, referring to model FLOPs utilisation (MFU), a measure of how effectively training hardware is being used. 

He speculated that xAI’s implementation could contain “inline PTX”—NVIDIA’s low-level GPU assembly language—and suggested that the stack might have been generated with the help of stronger AI models.

“The need for higher-level frameworks for [production] training is rapidly going away,” Toulme added. “They can’t compete with agent-written C stacks.”

Musk also revealed that the new stack uses assembly and direct calls to hardware accelerators in NVIDIA’s streaming multiprocessors, allowing the software to target GPU hardware more closely.