The text generator has risen in the writers’ industry because who doesn’t need an ‘assistant’ that can handle your work efficiently and professionally. Especially with the rise of Generative Pretrained Transformer 3 (GPT 3) created by Open AI which is a powerful AI writing tool. The unique thing about GPT-3 is that it is the first-ever program that can learn how to write like a human without needing to be taught by a computer scientist. Such cutting edge technology doesn’t come for free. So, Eleuther AI came up with a solution GPT-Neo and GPT-J. In this article, we will be talking about GPT-Neo and will show how an essay can be written with it in just 5 lines of code. Following are the topics to be covered.
Table of contents
- About GPT-Neo
- Generating Text with GPT-Neo
Let’s start with talking about GPT-Neo
About GPT-Neo
GPT Neo is an open-source alternative to GPT 3. It is an open-source model trained like GPT 3, an autoregressive transformer using the mesh library. GPT-Neo has 3 versions.
- With 125 million parameters,
- With 1.3 billion parameters which are equivalent to GPT-3 Babbage
- With 2.7 billion parameters
Eleuther AI has also developed another alternative GPT-J which is the most powerful open source text generator with 6 billion parameters.
CPU and GPU usage
GPT-J needs around 25GB of GPU VRAM as well as many CPU threads to run. On CPUs, GPT-J runs slowly, so it is simpler to run it on a GPU. This does not fit in most of the existing NVIDIA GPUs that come with 8GB or 16GB of VRAM. In addition to these requirements, it is impossible to test GPT-J and GPT-Neo, let alone use them reliably for inference in production with high availability and scalability in mind. But thanks to cloud computing we can run these NLP models smoothly.
Let’s generate some text using GPT-Neo in python.
Are you looking for a complete repository of Python libraries used in data science, check out here.
Generating Text with GPT-Neo
In this article, we will be using a predefined pipeline that will make it easier to access the outcomes. This pipeline is built by hugging face using the Eleuther AI GPT-Neo and with help of this, we can generate a paragraph just in three lines of code.
Installing the required packages:
Need to install the PyTorch library for multidimensional tensors and mathematical operators.
- PyTorch is an open-source python library based on Torch which is developed on C. Pytorch is developed and maintained by Facebook AI. In simple words, it is a NumPy with GPU.
! pip install torch torchvision torchaudio
Need to install the transformers
- A transformer in NLP is an architecture that can handle long-range dependencies while solving sequence to sequence tasks without using sequence-aligned RNNs or convolutions.
! pip install transformers
Download the pipeline for text generation:
from transformers import pipeline
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B')
Here using GTP-Neo 1.3 billion parameters since the 2.7 billion parameter version needed 20GB of VRAM and it takes much more time for text generation.
Parameters for model:
prompt = 'The rise of Natural Language Processing' res = generator(prompt, max_length=300, do_sample=True, temperature=0.5) res1= generator(prompt, max_length=300, do_sample=True, temperature=0.7) res2= generator(prompt, max_length=300, do_sample=True, temperature=0.9)
Parameters description:
- Prompt is where you need to input the title for the paragraph or essay.
- Max_length is the total number of words to be in the paragraph.
- Temperature is the randomness of words. It varies between 0 to 1.
As in the above line of code, I used three different generators with different randomness to see the efficiency of the transformer.
Output:
The output is in a dictionary format so extract the text from this dictionary using the below line of code.
print(res[0]['generated_text'])
print("-------------------------------------------------------------------------")
print(res1[0]['generated_text'])
print("-------------------------------------------------------------------------")
print(res2[0]['generated_text'])
Output from the generator with the randomness of 0.5.
In this result, we can observe that the same line is being repeated multiple times.
Output from the generator with the randomness of 0.7.
In this result, some sentences were not making any sense.
Final generator with randomness 0.9.
This is preferably the best one out of all these and the sentences were making sense when read.
Final Words
GPT-Neo is a powerful alternative to GPT 3, and with the implementation, got a glimpse of its great potential. It is a fabulous assistant to kick start the writing process. With the help of hugging face pipeline, the experience of GPT-Neo becomes better and in just three lines of code get a jump start on the topic. With a hands-on implementation of this concept in this article, we could write an essay with the help of artificial intelligence.
