Article complete

Zhipu AI’s GLM-5.1 Becomes Top Model on SWE-Bench Pro, Beats GPT-5.4, Claude Opus 4.6

The Chinese AI model was only trained on 100,000 Huawei Ascend without relying on NVIDIA chips.

A dragon sculpture against a red background filled with code text.

Key takeaways

  • Zhipu AI's GLM-5.1 model is an open-source competitor, surpassing GPT-5.4 and Claude Opus 4.6 on SWE-Bench Pro.
  • The model was trained exclusively on 100,000 Huawei Ascend GPUs, without using NVIDIA chips.
  • GLM-5.1 demonstrates sustained performance during extended coding sessions, improving productivity and problem-solving capabilities.
  • In tests, the model significantly enhanced query throughput and achieved a notable speedup in GPU kernel optimisation tasks.
  • The improvements in GLM-5.1 are attributed to a combination of incremental tuning and structural changes.

Takeaways drafted with AI from the article text.

Beijing-based AI company Zhipu AI has launched a new model, GLM-5.1, an open-source model for agentic software engineering.

Zhipu AI released GLM-5.1 under the MIT License and is available through developer platforms and local deployment frameworks, including vLLM and SGLang.

The model was only trained on 100,000 Huawei Ascend GPUs with no reliance on NVIDIA chips. 

The Chinese company said GLM-5.1 can sustain performance over extended coding sessions. The model scored 58.4, beating GPT-5.4 and Claude Opus 4.6 on SWE-Bench Pro. It also showed gains over its predecessor, GLM-5, on benchmarks such as NL2Repo and Terminal-Bench 2.0.

Image

“We’ve found that the model handles ambiguous problems with better judgement and stays productive over longer sessions. It breaks complex problems down, runs experiments, reads results, and identifies blockers with real precision,” the company said in a blog.

“GLM-5.1 sustains optimisation over hundreds of rounds and thousands of tool calls. The longer it runs, the better the result,” it added.

In one test involving vector database optimisation, the model was run across more than 600 iterations. It improved query throughput from prior benchmarks of 3,547 queries per second (QPS) to 21,500 QPS while maintaining a recall above 95%. 

The company described the improvement pattern as incremental tuning followed by structural changes. 

In a second evaluation using GPU kernel optimisation tasks, GLM-5.1 achieved a 3.6x speedup across 50 problems. The model continued improving over longer runs, while earlier versions plateaued. However, Claude Opus 4.6 remained ahead with a 4.2x speedup.

The third scenario tested open-ended application development. The model was tasked with building a Linux-style desktop environment without predefined metrics. Over an eight-hour run, it expanded from a basic interface to a multi-application system including a file browser, terminal, and system tools.

Earlier, in February, the company launched GLM-5, which delivered results close to Claude Opus 4.5 on coding benchmarks and outperformed Gemini 3 Pro in select evaluations.



Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

AI Models · 5 stories
  1. Anthropic Launches Claude Sonnet 4.6 with 1M Token Context Window
  2. You are here: Zhipu AI’s GLM-5.1 Becomes Top Model on SWE-Bench Pro, Beats GPT-5.4, Claude Opus 4.6
  3. Hugging Face Crosses $100 Million in Annual Run Rate
  4. Pathway Claims 11x Lower AI Reasoning Costs With 150M-Parameter Model
  5. DeepSeek Launches V4.1-Flash to Make Large-Scale AI Inference Cheaper
About the author

Tech Journalist

Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.