From the course: AI Text Summarization with Hugging Face
Unlock this course with a free trial
Join today to access over 24,700 courses taught by industry experts.
A brief introduction to Transformers - Hugging Face Tutorial
From the course: AI Text Summarization with Hugging Face
A brief introduction to Transformers
Now, transformers are an advanced and complex neural network architecture used in great effect in natural language processing tasks. Here in this movie, we'll briefly understand how transformers work. In the previous movie, we understood the role of attention in sequence-to-sequence models, and transformers use attention or self-attention to detect how different parts of the input sequence are related to one another, even when the relationship is very subtle. The basic transformer architecture is made up of two components; the encoder and the decoder. Now, this seems to imply that transformers are sequence-to-sequence models, but that's not necessarily the case. Sequence-to-sequence models use both parts of a transformer architecture, the encoder as well as the decoder. As we've discussed earlier, text summarization is performed using sequence-to-sequence models. The encoder in the transformer architecture receives an input and builds a representation of the input, and the entire…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.