Article complete

Alibaba Cloud Launches its New Qwen2.5 Series

The results show that Qwen2.5-Coder-32B-Instruct is strongly aligned with human preferences.

Alibaba Cloud Launches its New Qwen2.5 Series

Key takeaways

  • Alibaba Cloud unveils Qwen2.5-Coder-32B-Instruct, a significant upgrade for coding applications.
  • The Qwen2.5 series aims to advance Open CodeLLMs and includes six model sizes.
  • Notable features include strong alignment with human preferences and improved coding capabilities.
  • Industry experts showcase innovative projects created entirely using the Qwen2.5 model.

Takeaways drafted with AI from the article text.

On 11 November, Alibaba Cloud released its new version of the open foundation Qwen model— Qwen2.5-Coder-32B-Instruct. The company refers to it as a ‘family of coder models’ and not just a model that can code. 

This model is a significant upgrade from its predecessor, CodeQwen1.5. Ahsen Khaliq, the ML growth lead at Hugging Face, took to LinkedIn to express and said he created a tic-tac-toe game and noted that it was similar to Claude's artefacts.

The architecture of Qwen2.5-Coder is spread across six different model sizes: 0.5B, 1.5B, 3B, 7B, 14B, and 32B parameters. While all sizes share the same architecture in terms of head size, they differ in several other key aspects. 

Binyuan Hui, core maintainer at Alibaba Qwen, took to X to share an an interesting game he created with Qwen2.5-Coder.

Training and Data

According to the official research published earlier this year, high-quality, large-scale, and diverse data is necessary for building pre-trained models. With this in mind, the Qwen team with Alibaba group developed a dataset called Qwen2.5-Coder-Data, which includes five primary data types: Source Code Data, Text-Code Grounding Data, Synthetic Data, Math Data, and Text Data.

Following the file-level pretraining, the company proceeded to repo-level pretraining to improve the model’s long-context capabilities. In this phase, the context length is expanded from 8,192 tokens to 32,768 tokens, and the base frequency of RoPE is adjusted from 10,000 to 1,000,000.

Results

The Qwen2.5-Coder series has set a new standard in open-source coding models, particularly with its flagship, Qwen2.5-Coder-32B-Instruct. This model excels in code generation, matching the capabilities of GPT-4o on benchmarks like EvalPlus, LiveCodeBench, and BigCodeBench. 

Source: Qwen official blog
Source: Qwen official blog

Beyond generating code, Qwen2.5-Coder-32B-Instruct is adept at code repair, helping developers identify and fix errors efficiently. On the Aider benchmark, it achieved a score of 73.7, comparable to GPT-4o's performance. 

The model’s strength lies in its understanding of code execution, enabling accurate predictions of inputs and outputs. It supports over 40 programming languages, scoring 65.9 on the McEval benchmark, and leads in code repair tasks, scoring 75.2 on the MdEval benchmark. 

Additionally, to see how well Qwen2.5-Coder-32B-Instruct matches human coding preferences, a test called Code Arena was conducted. Using a simple comparison with GPT-4o, the team measured which model performed better in each example. The results show that Qwen2.5-Coder-32B-Instruct is strongly aligned with human preferences.

Source: Qwen official blog
Source: Qwen official blog

Qwen2.5-Coder with Cursor

Even though code assistants are now widely utilised, most still depend on closed-source models. Alibaba and the Qwen team aim for Qwen2.5-Coder to offer developers a robust, open-source alternative. Below is an example of Qwen2.5-Coder in action within Cursor.

Published

Corrections: AIM corrects errors of fact and notes the change on the story. Spotted one? Tell the editors. Read our editorial standards.

More in this story

Alibaba · 5 stories
  1. You are here: Alibaba Cloud Launches its New Qwen2.5 Series
  2. Alibaba Launches New Qwen Reasoning Model; To Release iOS and Android App for Qwen Chat
  3. China Offers Cheaper Power to Local AI Firms Ditching NVIDIA Chips
  4. Alibaba’s Qwen-MT Promises Smarter, Cheaper Translations Across 92 Languages
  5. Alibaba Unveils New Qwen AI Smart Glasses to Compete With Meta
About the author

Technology Journalist

Covering deep and emerging tech: semiconductors, quantum, robotics, space tech, drones and GCCs. Connect via socials below or email:

All stories
Belamy · Daily

The day’s AI news, before your first meeting.