Now Build ChatGPT On Your Own Device

Since OpenAI has not open-sourced the code for ChatGPT, replicating the chatbot is a herculean task, and even the big-tech are struggling. But, AI startup Colossal-AI has found a way to build your own ChatGPT with less computing resources. 

Towards this goal, the company has leveraged a PyTorch-based implementation that covers all three stages from pre-training, reward model training, and reinforcement learning. They offer a demo version of the training process that requires only 1.62 GB of GPU memory and can be done on a single consumer-grade GPU, with 10.3x growth on one GPU model capacity. 

Check out the GitHub repository here.

Colossal-AI said that compared to the original PyTorch, the single-machine process is 7.7 times faster and a single-GPU inference can be 1.42 times faster, which is achievable on a single line of code. For fine-tuning, users can increase the capacity of the model by up to 3.7 times with one line of code on a single GPU while running at a high speed. 

The original PyTorch implementation typically requires a 780 million parameter model on A100 80GB, which costs $14,999. Colossal-AI, on the other hand, boosts it to a single GPU by 10.3 times to 8 billion parameters. 

There are multiple versions available of a single-GPU scale, a multiple-GPUs scale on a single node, along with a 175-billion parameter scale. Developers can also import OPT, GPT-3, and BLOOM pre-trained language models from Hugging Face. 

Learn more about the process by checking out the documentation and code.