From the course: AI Text Summarization with Hugging Face
Join today to access over 24,700 courses taught by industry experts.
Fine-tuning the T5 small model - Hugging Face Tutorial
From the course: AI Text Summarization with Hugging Face
Fine-tuning the T5 small model
Just like we separately access the tokenizer to preprocess our data, let's separately load the model which we plan to fine-tune. And I do this using AutoModelForSeq2SeqLM. This is the class used to load language models with the encoder-decoder architecture, which is the case for our T5 model. We use the from_pretrained method, specify our model name, and this will give us the pre-trained model that we'll fine-tune. We'll now fine-tune this model on our CNN Daily Mail training data. Remember, we are working with a small sample of the original dataset. We'll use the tokenizer to preprocess the samples that we feed in to fine-tune the model. And we do this within the preprocess function. The input to the preprocess function are the examples that need to be preprocessed. Remember that every input text article needs to be prefixed with summarize:, which is why on line 6, for each document in the example articles we add the prefix in and those are our inputs. We then tokenize these inputs…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.
