From the course: Advanced RAG Applications with Vector Databases

Unlock the full course today

Join today to access over 24,700 courses taught by industry experts.

Ways to do multimodal RAG

Ways to do multimodal RAG

From the course: Advanced RAG Applications with Vector Databases

Start my 1-month free trial Buy for my team

Ways to do multimodal RAG

“

- [Instructor] Let's look at some ways we can do multimodal RAG. The basic idea behind multimodal RAG is the same as the basic idea behind multimodal data. So there's two main ways to do multimodal RAG. One, we can use one multimodal model that can handle multiple types of data. Two, we can use multiple models, usually one for each type of data that we want to work with. Let's look at how the ways to do multimodal RAG breakdown. We can use one multimodal embedding model. This will be a model that can process multiple types of data, usually images and text. We can use multiple embedding models and multiple search modes. These will be models for different types of data, and we'll need to route each type of data to the right model for storing the data and searching the data. Third, we can just do a combination of the two ways to do multimodal RAG and use multiple multimodal models. While this is totally acceptable to do, this is extremely uncommon. The canonical way to build a multimodal…

Contents