Using Hugging Face’s Transformers Library
Hugging Face is a repository that provides open-source pre-trained language models ready to load and use for NLP tasks like text generation, translation, and sentiment analysis. It is powered by its centerpiece library: Transformers, which offers seamless integration with popular Python libraries like PyTorch, JAX, and TensorFlow. Best of all: they are free to use and require minimal setup, making AI development accessible to everyone.
Let’s start this practical tour by installing the transformers library in a new notebook of your own:
!pip install transformers
We will now load a model from Hugging Face, specifically GPT-2 for text generation. When loading pre-trained models, we normally need to load not only the model itself, but also a compatible tokenizer responsible for splitting text inputs into logical language units called tokens (roughly equivalent to words in most cases) before being passed to the language model for processing it and generating follow-up text.
The following code imports the necessary packages and initializes the language model and tokenizer.
from transformers import GPT2LMHeadModel, GPT2Tokenizer
model_name = "gpt2"
model = GPT2LMHeadModel.from_pretrained(model_name)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
Next we define a prompt for the model, which is typically a query, question, or request to which the language model will try to generate a response. Some less sophisticated models limit themselves to generating follow-up words that continue the input sequence or prompt, for instance to continue a tale that starts with “Once upon a time”. In our case, we will try asking GPT-2 what is the capital of Japan and … fingers crossed.
prompt = "What is the capital of Japan?"
inputs = tokenizer.encode(prompt, return_tensors="pt")
output = model.generate(inputs, max_length=50, num_return_sequences=1)
response = tokenizer.decode(output[0], skip_special_tokens=True)
print(response)
Getting technical, in the above code you may have noticed that there are two processing steps taking place: encoding and decoding. The input sequence (the prompt) must be encoded into a numerical vector representation understandable by the model, and after it generates the raw (numerical) response, it is decoded back into text to make it understandable by us.
This is the decoded response:
The capital of Japan is Tokyo. It is the capital of Japan. It is the capital of Japan. It is the capital of Japan. It is the capital of Japan. It is the capital of Japan
Well, at least it gave the right answer! These pre-trained models are comparatively smaller, more manageable for testing environments, and not as absurdly powerful and high-performing as ChatGPT ones, hence it is not surprising they might be more limited in their text generation behavior. In particular, since we specified a maximum response length of 50 tokens, the model seems to prioritize the need to provide a response that closely matches that length, rather than keeping it short, simple, and logical.
Time to look at another example and introduce another framework.