How Moreh is Making AI Software Better with AMD

AMD is rising rapidly up the AI market ever since it announced its MI300X and the software updates for ROCm. It has been building partnerships with various AI companies for testing out and delivering its products. One such company led us to Korean-based Moreh. 

One of the problems with AMD GPUs earlier was that the software stack was not well established, but now the company has ROCm, its alternative to CUDA. But still, it was not well suited for large GPU clusters. “Our software enables people to use AMD GPUs without any more codes, they just can run their or larger language models without any further engineering,” Junghwan Lim, head of AI group at Moreh, told AIM in an exclusive interview.

Lim has also worked as a data scientist at PUBG Corporation and has also worked with Samsung in South Korea. Currently, Lim is focused on developing better software for AI workloads at Moreh and making language models smaller for better efficiency.

AMD is being embraced way too much, and that’s good

Moreh’s flagship AI software, known as MoAI, is positioned similarly to NVIDIA's CUDA but boasts compatibility with existing machine learning frameworks like Meta's PyTorch and Google's TensorFlow, and even OpenAI’s Triton. Now, the company is helping AMD boost up its ROCm performance.

In August, Moreh announced that it has been using AMD MI250 for the longest time and it is outperforming NVIDIA. According to Moreh, AMD's MI250 Instinct accelerator, when powered by the MoAI platform, achieved 116% higher GPU throughput than NVIDIA's A100.

“We have been using more than 400 MI250 GPUs along with a few MI300X for training AI models,” said Lim. The company also plans to buy more MI300X from AMD in the future. 

“If anyone wants to use AMD GPUs, they can come to us without any code, and just use our software to run on them,” said Lim. This is somewhat similar to what Lamini has been doing in partnership with AMD, but Lim says that there is still a difference in the requirement of codes, as companies can also use their existing GPUs to run their models. 

A key aspect of Moreh's success lies in its software being built on AMD GPU infrastructure, showcasing performance that surpasses NVIDIA GPUs in AI model development. The MoAI platform, a comprehensive software offering, is not tied to a specific hardware vendor and supports various device backends, including AMD GPUs.

“Every configuration or any technique that the customer wants to apply should be easily applied without the need of any programmer. That is what we are aiming to do, and that is what differentiates us,” Lim said.

“If you are building an AI model on a thousand GPUs, there can be issues such as a GPU malfunction because of hardware or software issues,” explained Lim. “The training suddenly stops and it takes hours, or sometimes days to start it again.” He explains that Moreh is also building software and devising techniques to reduce this by parallelising computers to different fragments. 

Large language models for the win

Moreh has also finished training its own LLM for the Korean language which consists of 221 billion parameters and wishes to make upcoming models open source. 

“We are developing something similar to GPT or Gemini, but it is going to be open source.” Lim says that the current model is too big to open source, so Moreh is also planning to release smaller models soon. “Our models would include the code, weights, inference code, and everything else,” Lim highlighted about the recent trend in open source models which come with some or the other restrictions. 

In October, AMD and Korean telecommunications (KT) invested a $22 million series B round in Moreh, bringing the valuation of the startup to $30 million. The company also projected its revenue to reach $30 million by the end of 2023. 

KT has also bought one of the largest AMD GPU cluster in the world and are also building AI models using them. “We are supporting all those clusters and cloud systems for KT,” said Lim. KT is starting to focus on GPU cloud provider business, also providing APIs for language models, but which would just be focused on Korean language, not English.

KT, which has been working with Moreh since 2021, claims that Moreh's technology has demonstrated superior performance compared to NVIDIA's DGX, specifically in terms of speed and GPU memory capacity. 

“People believe that AMD GPUs are not that suited for machine learning, but the company has been increasingly proving them wrong,” concluded Lim.