From the course: AI Text Summarization with Hugging Face

Unlock this course with a free trial

Join today to access over 24,700 courses taught by industry experts.

A brief introduction to Transformers

A brief introduction to Transformers - Hugging Face Tutorial

From the course: AI Text Summarization with Hugging Face

A brief introduction to Transformers

Now, transformers are an advanced and complex neural network architecture used in great effect in natural language processing tasks. Here in this movie, we'll briefly understand how transformers work. In the previous movie, we understood the role of attention in sequence-to-sequence models, and transformers use attention or self-attention to detect how different parts of the input sequence are related to one another, even when the relationship is very subtle. The basic transformer architecture is made up of two components; the encoder and the decoder. Now, this seems to imply that transformers are sequence-to-sequence models, but that's not necessarily the case. Sequence-to-sequence models use both parts of a transformer architecture, the encoder as well as the decoder. As we've discussed earlier, text summarization is performed using sequence-to-sequence models. The encoder in the transformer architecture receives an input and builds a representation of the input, and the entire…

Contents