China's Chitu to Challenge Dependence on NVIDIA Chips

NVIDIA leads the AI chip market. However, due to US hardware restrictions on China and the rise of non-NVIDIA GPUs, some businesses seek to lessen their reliance on NVIDIA products.

A team associated with China’s Tsinghua University is joining the effort by releasing a new open-source AI framework, Chitu, made available under the Apache-2.0 license.

Chitu is a high-performance inference framework for large language models, focusing on efficiency, flexibility, and availability. With its initial release, it supports various mainstream large language models, including DeepSeek, LLama series, Mixtral, and more.

The report by South China Morning Post states that the framework can operate on chips made in China, challenging the supremacy of NVIDIA’s Hopper series graphics processing units (GPUs). The information is referenced from a joint statement on Friday by the startup Qingcheng.AI and a team led by Zhai Jidong, computer science professor at Tsinghua University.

The startup was founded in 2023 by Jidong, and his students from Tsinghua University, with him serving as chief scientist. The report states that it is backed by Beijing’s municipal fund for the AI industry.

The framework’s GitHub page mentioned, “We not only focus on the popular NVIDIA GPUs, but pay special attention to all kinds of hardware environments, including legacy GPUs, non-NVIDIA GPUs and CPUs. We aim to provide a versatile framework to encounter the diverse deploying requirements.”

The report mentions that the company claims to have achieved a 315% increase in model inference speed while reducing GPU usage by 50% compared to foreign open source frameworks. This is as per the test with DeepSeek-R1 using NVIDIA’s A800 GPUs.

The team states that the framework is ready and deployed for real-world production.