# Deploy Meta Llama 3.1 405B on Google Cloud Vertex AI

DevFeed: [Deploy Meta Llama 3.1 405B on Google Cloud Vertex AI](<https://devfeed.tech/articles/deploy-meta-llama-3-1-405b-on-google-cloud-vertex-ai-7336.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/llama31-on-vertex-ai>)

Author: Alvaro Bartolome; Philipp Schmid; Simon Pagezy; Jeff Boudier

Published: 2024-08-19T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [llama](<https://devfeed.tech/topics/llama.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Meta](<https://devfeed.tech/topics/meta.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Containers](<https://devfeed.tech/topics/containers.md>)

Tags: [container](<https://devfeed.tech/tags/container.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llama](<https://devfeed.tech/tags/llama.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [vertex](<https://devfeed.tech/tags/vertex.md>), [vertex-ai](<https://devfeed.tech/tags/vertex-ai.md>)

## AI overview

This tutorial explains how to programmatically deploy the FP8-quantized Meta Llama 3.1 405B model on Google Cloud Vertex AI using Text Generation Inference and Hugging Face Deep Learning Containers. It covers deployment on an A3 node with eight NVIDIA H100 GPUs, alternative deployment paths, and the memory considerations for running the model.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.