Meta Releases 816B-Parameter Open-Weight Model With 1M Context Window

Meta has released the weights for Muse Spark Open, an 816-billion-parameter open-weight model built for coding, software engineering and multi-step reasoning. The model has 42 billion active parameters and a context window of one million tokens.

The company said the model is available through inference platforms including Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter and Together AI. Developers can download the weights through Hugging Face, Amazon SageMaker AI, Google Cloud Model Garden and Microsoft Foundry Models.

“Today, we’re releasing the weights for Muse Spark Open, the most capable US open-weight model to date,” Meta said in its announcement on October 15.

The model is natively multimodal and supports agentic workflows, allowing it to carry out tasks involving multiple steps and tools. Developers can customise it using PyTorch and run it through inference engines including SGLang, vLLM, TensorRT-LLM, Hugging Face Transformers and llama.cpp.

Meta has optimised Muse Spark Open to run locally on two Apple Mac Studio computers equipped with M5 Ultra chips, using Apple's MLX machine learning framework.

According to the company, each Mac Studio can provide up to 512GB of unified memory, allowing two machines to accommodate the model's weights across a combined memory capacity of approximately 1TB.

Meta tested the model on several on-device workflows using Muse Code, its coding tool powered by Muse Spark Open. These included fixing bugs, adding features and building an iOS application using Ice Cubes, an open-source Mastodon client.

The company also demonstrated the model operating DaVinci Resolve, where it reviewed footage frame by frame to identify suitable clips and crop them into vertical videos.

Meta said the model can run these workflows entirely on-device, allowing developers to perform coding and media-editing tasks without relying on cloud inference for those demonstrations.

Muse Spark Open includes custom GPU kernels optimised for inference through SGLang. Meta said the kernels delivered speedups ranging from 1.03 times to 3.95 times in kernel-level tests on NVIDIA and AMD GPUs.

The company also reported a 3% end-to-end performance improvement on AMD MI350 GPUs.

The model supports several hardware platforms, including AMD, Apple, Intel and NVIDIA silicon. Developers can further fine-tune it using PyTorch's TorchTitan framework.

Meta has released the weights under its Muse Model License, which includes limited commercial terms. The licence allows many developers to use the model without charge, although users will need to review its conditions before deploying it commercially.