From the course: AWS Certified Machine Learning Engineer Associate (MLA-C01) Cert Prep

Unlock this course with a free trial

Join today to access over 24,700 courses taught by industry experts.

Multi-model vs. multi-container endpoints

Multi-model vs. multi-container endpoints

From the course: AWS Certified Machine Learning Engineer Associate (MLA-C01) Cert Prep

Start my 1-month free trial Buy for my team

Multi-model vs. multi-container endpoints

“

(gentle music) - [Instructor] Hello, guys. In today's lesson, we're going to talk about the multi-model and the multi-container deployments. So what is a multi-model endpoint? A multi-model endpoint allows a single SageMaker endpoint and a container image to dynamically load and serve multiple models, which optimizes both the cost and the efficiency. So you're sharing a single set of compute resources across multiple models, which significantly reduces the cost, and you're minimizing the complexity of managing individual endpoints for each model. So for the use cases, it could be suitable for when you have a large number of model that use the same machine learning framework, and they share similar code or logical flows. For example, you could deploy one model per user for personalized recommendations while hosting all the models on the same endpoint, and it could serve multiple variations of a language model tailored to different regions like English, French, or Spanish. So how does…

Contents