Zhipu AI’s GLM-5.1 Becomes Top Model on SWE-Bench Pro, Beats GPT-5.4, Claude Opus 4.6

Beijing-based AI company Zhipu AI has launched a new model, GLM-5.1, an open-source model for agentic software engineering.

Zhipu AI released GLM-5.1 under the MIT License and is available through developer platforms and local deployment frameworks, including vLLM and SGLang.

The model was only trained on 100,000 Huawei Ascend GPUs with no reliance on NVIDIA chips. 

The Chinese company said GLM-5.1 can sustain performance over extended coding sessions. The model scored 58.4, beating GPT-5.4 and Claude Opus 4.6 on SWE-Bench Pro. It also showed gains over its predecessor, GLM-5, on benchmarks such as NL2Repo and Terminal-Bench 2.0.

“We’ve found that the model handles ambiguous problems with better judgement and stays productive over longer sessions. It breaks complex problems down, runs experiments, reads results, and identifies blockers with real precision,” the company said in a blog.

“GLM-5.1 sustains optimisation over hundreds of rounds and thousands of tool calls. The longer it runs, the better the result,” it added.

In one test involving vector database optimisation, the model was run across more than 600 iterations. It improved query throughput from prior benchmarks of 3,547 queries per second (QPS) to 21,500 QPS while maintaining a recall above 95%. 

The company described the improvement pattern as incremental tuning followed by structural changes. 

In a second evaluation using GPU kernel optimisation tasks, GLM-5.1 achieved a 3.6x speedup across 50 problems. The model continued improving over longer runs, while earlier versions plateaued. However, Claude Opus 4.6 remained ahead with a 4.2x speedup.

The third scenario tested open-ended application development. The model was tasked with building a Linux-style desktop environment without predefined metrics. Over an eight-hour run, it expanded from a basic interface to a multi-application system including a file browser, terminal, and system tools.

Earlier, in February, the company launched GLM-5, which delivered results close to Claude Opus 4.5 on coding benchmarks and outperformed Gemini 3 Pro in select evaluations.