From the course: Rust LLMOps

Unlock this course with a free trial

Join today to access over 24,700 courses taught by industry experts.

Invoke an LLM on an AWS G5 instance, part 1

Invoke an LLM on an AWS G5 instance, part 1 - Rust Tutorial

From the course: Rust LLMOps

Invoke an LLM on an AWS G5 instance, part 1

- So this week, I'm going to dive into a interesting scenario here where you can use the package management system in Rust to invoke large language models. So I think everybody's talking about larger language models and how cool they are, but what's interesting is, you'll see people giving these really complex workflows, applications, all this complexity around running a large language model. But because of the beauty of Rust and the cargo package management system, you don't have to do anything. You can just actually use the cargo system to invoke models. I'm going to do this with huggingface candle and I'm going to do it on an AWS GPU. That's a very powerful GPU. So let's go ahead and take a look at how we would do that. So first up here, I've got huggingface candle. Let's just look at what this does. So this is a minimalistic ML framework for Rust. And you can see, the code is really simple. But the biggest thing here, and I think this is why it's so exciting, and I would say…

Contents