Elon Musk’s AI startup, xAI has introduced Grok-1.5V, a  first-generation multimodal model. In addition to its strong text capabilities, Grok can process a wide variety of visual information, including documents, diagrams, charts, screenshots, and photographs.

Grok-1.5V will be available soon to early testers and existing Grok users.

Grok-1.5V’s notable feature is its ability to understand real-world spatial concepts, surpassing other models in the RealWorldQA benchmark—an important measure of a model’s practical grasp of physical environments.

In a comparative analysis against leading models like GPT-4V, Claude 3 Sonnet, Claude 3 Opus, and Gemini Pro 1.5, Grok-1.5V shows competitive advantages across several benchmarks, highlighting its versatility and strength.

One of Grok-1.5V’s standout features is its ability to translate complex visual information into executable code. For example, when given a flowchart depicting a guessing game, Grok-1.5V easily converts it into Python code, showcasing its practical application in problem-solving scenarios.

Looking forward, the developers of Grok-1.5V anticipate significant improvements in multimodal capabilities across images, audio, and video, signaling a promising path towards building beneficial Artificial General Intelligence (AGI) that comprehensively understands and interacts with the universe.

Grok-1.5V follows the recent introduction of Grok-1.5 by xAI, featuring enhanced reasoning capabilities and a context length of 128,000 tokens. Grok-1.5 boasts notable improvements, particularly in coding and math-related tasks. It beats Mistral Large on various benchmarks including MMLU, GSM8K and HumanEval.